{
  "id": 386370,
  "title": "Notebook threw an Exception. How to debug?",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/386370",
  "author_name": "",
  "post_date": "2023-02-12T19:00:32.844283700Z",
  "votes": 2,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>I'm having trouble debugging what's causing the \"Notebook threw an exception\" error<br>\nMy commit passes without errors but when I submit the notebook it throws an error in less than a minute. How do I debug what's causing it? I exhausted all my submissions for today, but what's even more frustrating ideas how to tackle this problem. Can I get the exception code, line number or anything at all where this exception is taking place?<br>\nHow is the submission run different from the commit run (except for the larger number of samples)?</p>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "2141436",
      "postDate": "02/12/2023 19:00:32",
      "content": "<p>Hi everyone,</p>\n<p>I'm having trouble debugging what's causing the \"Notebook threw an exception\" error<br>\nMy commit passes without errors but when I submit the notebook it throws an error in less than a minute. How do I debug what's causing it? I exhausted all my submissions for today, but what's even more frustrating ideas how to tackle this problem. Can I get the exception code, line number or anything at all where this exception is taking place?<br>\nHow is the submission run different from the commit run (except for the larger number of samples)?</p>\n<p>Thanks!</p>",
      "rawMarkdown": "Hi everyone,\n\nI'm having trouble debugging what's causing the \"Notebook threw an exception\" error\nMy commit passes without errors but when I submit the notebook it throws an error in less than a minute. How do I debug what's causing it? I exhausted all my submissions for today, but what's even more frustrating ideas how to tackle this problem. Can I get the exception code, line number or anything at all where this exception is taking place?\nHow is the submission run different from the commit run (except for the larger number of samples)?\n\nThanks!",
      "votes": null
    },
    {
      "id": "2141613",
      "postDate": "02/12/2023 23:40:09",
      "content": "<p>I found one difference in submission compared with commit. The nan in submission variables is different from the nan in commit variables. Pandas can fix both nan in commit and submission using fillna. But numpy/python wasn't able to fill the nan in submission using nan_to_num function. </p>",
      "rawMarkdown": "I found one difference in submission compared with commit. The nan in submission variables is different from the nan in commit variables. Pandas can fix both nan in commit and submission using fillna. But numpy/python wasn't able to fill the nan in submission using nan_to_num function.",
      "votes": null
    },
    {
      "id": "2142332",
      "postDate": "02/13/2023 13:09:58",
      "content": "<p>I think two issues may cause this:</p>\n<ol>\n<li>Random GPU assignment: sometimes you get 2 T4 GPUs and sometimes you get P100 GPU. It would be good to check which instance you get for running the submissions.</li>\n<li>Check for memory leaks: Make sure that your code does not have any memory leaks that could cause the kernel to run out of memory. This can be done by monitoring the memory usage of the kernel while the code is running.</li>\n</ol>",
      "rawMarkdown": "I think two issues may cause this:\n1. Random GPU assignment: sometimes you get 2 T4 GPUs and sometimes you get P100 GPU. It would be good to check which instance you get for running the submissions.\n2. Check for memory leaks: Make sure that your code does not have any memory leaks that could cause the kernel to run out of memory. This can be done by monitoring the memory usage of the kernel while the code is running.",
      "votes": null
    },
    {
      "id": "2143986",
      "postDate": "02/14/2023 17:00:44",
      "content": "<p>Giba, I think you were right and it has to do with nan treatment. I initially used scikit learn LableEncoder, but decided to treat NANs first  with Pandas and then apply label encoder and it worked.</p>",
      "rawMarkdown": "Giba, I think you were right and it has to do with nan treatment. I initially used scikit learn LableEncoder, but decided to treat NANs first  with Pandas and then apply label encoder and it worked.",
      "votes": null
    },
    {
      "id": "2188818",
      "postDate": "03/20/2023 02:39:17",
      "content": "<p>Hi, l have faced the same problem. But l have already used Pandas to treat NANs first, but my notebook still threw exception during submission, but commit run show no error. l'm quite confused…</p>",
      "rawMarkdown": "Hi, l have faced the same problem. But l have already used Pandas to treat NANs first, but my notebook still threw exception during submission, but commit run show no error. l'm quite confused...",
      "votes": null
    },
    {
      "id": "2189747",
      "postDate": "03/20/2023 18:13:20",
      "content": "<p>Mine was related to the processing of either text or text_fqid. I don't remember what exact steps I was taking though. Something related to either scikit learn label encoder or one hot encoder.</p>",
      "rawMarkdown": "Mine was related to the processing of either text or text_fqid. I don't remember what exact steps I was taking though. Something related to either scikit learn label encoder or one hot encoder.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2141613,
      "author_name": "titericz",
      "author_url": "",
      "post_date": "02/12/2023 23:40:09",
      "content": "<p>I found one difference in submission compared with commit. The nan in submission variables is different from the nan in commit variables. Pandas can fix both nan in commit and submission using fillna. But numpy/python wasn't able to fill the nan in submission using nan_to_num function. </p>",
      "votes": null,
      "replies": [
        {
          "id": 2143986,
          "author_name": "sakvaua",
          "author_url": "",
          "post_date": "02/14/2023 17:00:44",
          "content": "<p>Giba, I think you were right and it has to do with nan treatment. I initially used scikit learn LableEncoder, but decided to treat NANs first  with Pandas and then apply label encoder and it worked.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2188818,
              "author_name": "feiyidong",
              "author_url": "",
              "post_date": "03/20/2023 02:39:17",
              "content": "<p>Hi, l have faced the same problem. But l have already used Pandas to treat NANs first, but my notebook still threw exception during submission, but commit run show no error. l'm quite confused…</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2189747,
                  "author_name": "sakvaua",
                  "author_url": "",
                  "post_date": "03/20/2023 18:13:20",
                  "content": "<p>Mine was related to the processing of either text or text_fqid. I don't remember what exact steps I was taking though. Something related to either scikit learn label encoder or one hot encoder.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2142332,
      "author_name": "giranntu",
      "author_url": "",
      "post_date": "02/13/2023 13:09:58",
      "content": "<p>I think two issues may cause this:</p>\n<ol>\n<li>Random GPU assignment: sometimes you get 2 T4 GPUs and sometimes you get P100 GPU. It would be good to check which instance you get for running the submissions.</li>\n<li>Check for memory leaks: Make sure that your code does not have any memory leaks that could cause the kernel to run out of memory. This can be done by monitoring the memory usage of the kernel while the code is running.</li>\n</ol>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2141436": "Hi everyone,\n\nI'm having trouble debugging what's causing the \"Notebook threw an exception\" error\nMy commit passes without errors but when I submit the notebook it throws an error in less than a minute. How do I debug what's causing it? I exhausted all my submissions for today, but what's even more frustrating ideas how to tackle this problem. Can I get the exception code, line number or anything at all where this exception is taking place?\nHow is the submission run different from the commit run (except for the larger number of samples)?\n\nThanks!",
    "2141613": "I found one difference in submission compared with commit. The nan in submission variables is different from the nan in commit variables. Pandas can fix both nan in commit and submission using fillna. But numpy/python wasn't able to fill the nan in submission using nan_to_num function.",
    "2142332": "I think two issues may cause this:\n1. Random GPU assignment: sometimes you get 2 T4 GPUs and sometimes you get P100 GPU. It would be good to check which instance you get for running the submissions.\n2. Check for memory leaks: Make sure that your code does not have any memory leaks that could cause the kernel to run out of memory. This can be done by monitoring the memory usage of the kernel while the code is running.",
    "2143986": "Giba, I think you were right and it has to do with nan treatment. I initially used scikit learn LableEncoder, but decided to treat NANs first  with Pandas and then apply label encoder and it worked.",
    "2188818": "Hi, l have faced the same problem. But l have already used Pandas to treat NANs first, but my notebook still threw exception during submission, but commit run show no error. l'm quite confused...",
    "2189747": "Mine was related to the processing of either text or text_fqid. I don't remember what exact steps I was taking though. Something related to either scikit learn label encoder or one hot encoder."
  },
  "source": "meta"
}