{
  "id": 125529,
  "title": "A way to debug memory problem",
  "url": "/competitions/bengaliai-cv19/discussion/125529",
  "author_name": "Qishen Ha",
  "post_date": "2020-01-11T14:09:09.684000",
  "votes": 7,
  "comment_count": 5,
  "views": 0,
  "content": "<p>If you got <code>Submission Scoring Error</code> or <code>Submission CSV Not Found</code> when submitting kernels, it may be out of memory (OOM) problem.</p>\n\n<p>As we cannot access to the runtime log so it's hard to debug.</p>\n\n<p>I've got an idea to debug it in this case.</p>\n\n<h3><strong>Load training data in the prediction notebook instead of test data and run it to see what didn't work.</strong></h3>\n\n<p>Becasue the private test data has around 200k samples which is just as big as training data.</p>",
  "messages": [
    {
      "id": 716287,
      "postDate": "2020-01-11T14:09:09.683Z",
      "content": "<p>If you got <code>Submission Scoring Error</code> or <code>Submission CSV Not Found</code> when submitting kernels, it may be out of memory (OOM) problem.</p>\n\n<p>As we cannot access to the runtime log so it's hard to debug.</p>\n\n<p>I've got an idea to debug it in this case.</p>\n\n<h3><strong>Load training data in the prediction notebook instead of test data and run it to see what didn't work.</strong></h3>\n\n<p>Becasue the private test data has around 200k samples which is just as big as training data.</p>",
      "rawMarkdown": "If you got `Submission Scoring Error` or `Submission CSV Not Found` when submitting kernels, it may be out of memory (OOM) problem.\n\nAs we cannot access to the runtime log so it's hard to debug.\n\nI've got an idea to debug it in this case.\n\n### **Load training data in the prediction notebook instead of test data and run it to see what didn't work.**\n\nBecasue the private test data has around 200k samples which is just as big as training data.",
      "votes": 7
    },
    {
      "id": 717989,
      "postDate": "2020-01-13T21:04:36.783Z",
      "content": "<p>Same for me. I spend my whole GPU quota with just 2 commits. Maybe the memory of Notebooks was silently reduced to 4GB?</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F712087%2F1d90cc2f95db53663732317b3e465923%2Fqutoa_error.png?generation=1578949345414339&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "Same for me. I spend my whole GPU quota with just 2 commits. Maybe the memory of Notebooks was silently reduced to 4GB?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F712087%2F1d90cc2f95db53663732317b3e465923%2Fqutoa_error.png?generation=1578949345414339&amp;alt=media)\n\n",
      "votes": 2,
      "replies": [
        {
          "id": 718094,
          "postDate": "2020-01-14T02:02:07.340Z",
          "content": "<p>You mean you are debugging oom problem for 30h ？... 😂</p>",
          "rawMarkdown": "You mean you are debugging oom problem for 30h ？... 😂"
        },
        {
          "id": 718213,
          "postDate": "2020-01-14T07:24:47.887Z",
          "content": "<p>Hm, rather something like this <a href=\"https://www.kaggle.com/c/tensorflow2-question-answering/discussion/120419\">https://www.kaggle.com/c/tensorflow2-question-answering/discussion/120419</a></p>",
          "rawMarkdown": "Hm, rather something like this https://www.kaggle.com/c/tensorflow2-question-answering/discussion/120419"
        },
        {
          "id": 718221,
          "postDate": "2020-01-14T07:29:50.290Z",
          "content": "<p>hmmmm, that's a terrible bug 😨\nI see you are scoring 0.0614, so maybe you need this: <a href=\"https://www.kaggle.com/c/bengaliai-cv19/discussion/123632\">https://www.kaggle.com/c/bengaliai-cv19/discussion/123632</a></p>",
          "rawMarkdown": "hmmmm, that's a terrible bug 😨\nI see you are scoring 0.0614, so maybe you need this: https://www.kaggle.com/c/bengaliai-cv19/discussion/123632"
        }
      ]
    },
    {
      "id": 716412,
      "postDate": "2020-01-11T17:24:59.313Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 717989,
      "author_name": "See--",
      "author_url": "",
      "post_date": "2020-01-13T21:04:36.783000",
      "content": "<p>Same for me. I spend my whole GPU quota with just 2 commits. Maybe the memory of Notebooks was silently reduced to 4GB?</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F712087%2F1d90cc2f95db53663732317b3e465923%2Fqutoa_error.png?generation=1578949345414339&amp;alt=media\" alt=\"\"></p>",
      "votes": 2,
      "replies": [
        {
          "id": 718094,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2020-01-14T02:02:07.340000",
          "content": "<p>You mean you are debugging oom problem for 30h ？... 😂</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 718213,
          "author_name": "See--",
          "author_url": "",
          "post_date": "2020-01-14T07:24:47.887000",
          "content": "<p>Hm, rather something like this <a href=\"https://www.kaggle.com/c/tensorflow2-question-answering/discussion/120419\">https://www.kaggle.com/c/tensorflow2-question-answering/discussion/120419</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 718221,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2020-01-14T07:29:50.290000",
          "content": "<p>hmmmm, that's a terrible bug 😨\nI see you are scoring 0.0614, so maybe you need this: <a href=\"https://www.kaggle.com/c/bengaliai-cv19/discussion/123632\">https://www.kaggle.com/c/bengaliai-cv19/discussion/123632</a></p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 716412,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-01-11T17:24:59.313000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "716287": "If you got `Submission Scoring Error` or `Submission CSV Not Found` when submitting kernels, it may be out of memory (OOM) problem.\n\nAs we cannot access to the runtime log so it's hard to debug.\n\nI've got an idea to debug it in this case.\n\n### **Load training data in the prediction notebook instead of test data and run it to see what didn't work.**\n\nBecasue the private test data has around 200k samples which is just as big as training data.",
    "717989": "Same for me. I spend my whole GPU quota with just 2 commits. Maybe the memory of Notebooks was silently reduced to 4GB?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F712087%2F1d90cc2f95db53663732317b3e465923%2Fqutoa_error.png?generation=1578949345414339&amp;alt=media)\n\n",
    "716412": ""
  }
}