{
  "id": 338161,
  "title": "Kaggle Kernel Restarts Randomly",
  "url": "/competitions/amex-default-prediction/discussion/338161",
  "author_name": "",
  "post_date": "2022-07-19T13:10:18.373663900Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hey everyone, I have been facing an issue recently. I have been using the integer dataset (thanks to <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a>) and been careful to load only enough data to stay well within the RAM limit. <strong>On loading only the <code>train.parquet</code> file, the RAM usage is approximately 5.8 GBs</strong>. </p>\n<p>However, after around 5 mins of usage or so, the <strong>RAM randomly spikes to a full</strong> and restarts with the message \"<strong>Your notebook tried to allocate more memory than is available. It has restarted</strong>\". I end up losing all my work. </p>\n<p>I am not even sure if this problem has anything to do with this competition. Is this something other people have also been facing? </p>\n<p><strong>EDIT</strong>: I am still unable to figure out a cause to this but it does seem to have gone away for now. I can now run the exact same notebook with nothing changed and not have the notebook crash. </p>",
  "messages": [
    {
      "id": "1862103",
      "postDate": "07/19/2022 13:10:18",
      "content": "<p>Hey everyone, I have been facing an issue recently. I have been using the integer dataset (thanks to <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a>) and been careful to load only enough data to stay well within the RAM limit. <strong>On loading only the <code>train.parquet</code> file, the RAM usage is approximately 5.8 GBs</strong>. </p>\n<p>However, after around 5 mins of usage or so, the <strong>RAM randomly spikes to a full</strong> and restarts with the message \"<strong>Your notebook tried to allocate more memory than is available. It has restarted</strong>\". I end up losing all my work. </p>\n<p>I am not even sure if this problem has anything to do with this competition. Is this something other people have also been facing? </p>\n<p><strong>EDIT</strong>: I am still unable to figure out a cause to this but it does seem to have gone away for now. I can now run the exact same notebook with nothing changed and not have the notebook crash. </p>",
      "rawMarkdown": "Hey everyone, I have been facing an issue recently. I have been using the integer dataset (thanks to @raddar) and been careful to load only enough data to stay well within the RAM limit. **On loading only the `train.parquet` file, the RAM usage is approximately 5.8 GBs**. \n\nHowever, after around 5 mins of usage or so, the **RAM randomly spikes to a full** and restarts with the message \"**Your notebook tried to allocate more memory than is available. It has restarted**\". I end up losing all my work. \n\nI am not even sure if this problem has anything to do with this competition. Is this something other people have also been facing? \n\n**EDIT**: I am still unable to figure out a cause to this but it does seem to have gone away for now. I can now run the exact same notebook with nothing changed and not have the notebook crash.",
      "votes": null
    },
    {
      "id": "1862382",
      "postDate": "07/19/2022 16:32:01",
      "content": "<blockquote>\n  <p>However, after around 5 mins of usage or so</p>\n</blockquote>\n<p>What are you doing during these 5 minutes? </p>",
      "rawMarkdown": ">However, after around 5 mins of usage or so\n\nWhat are you doing during these 5 minutes?",
      "votes": null
    },
    {
      "id": "1862442",
      "postDate": "07/19/2022 17:35:39",
      "content": "<p>That's the thing - I am not doing anything memory intensive at all, just basic code like <code>df.head()</code> or <code>df.shape</code>. It seems whatever I do, it shuts down in approx. 5 mins. </p>",
      "rawMarkdown": "That's the thing - I am not doing anything memory intensive at all, just basic code like `df.head()` or `df.shape`. It seems whatever I do, it shuts down in approx. 5 mins.",
      "votes": null
    },
    {
      "id": "1862623",
      "postDate": "07/19/2022 21:09:26",
      "content": "<p><a href=\"https://www.kaggle.com/abhisheksharma1998\" target=\"_blank\">@abhisheksharma1998</a> Maybe you can share your notebook. Basic code should work without big problems.</p>",
      "rawMarkdown": "abhisheksharma1998 Maybe you can share your notebook. Basic code should work without big problems.",
      "votes": null
    },
    {
      "id": "1862625",
      "postDate": "07/19/2022 21:16:57",
      "content": "<p>If the code takes 5 minutes, i suspect you are doing more than just <code>df.head()</code> and <code>df.shape</code>. I suggest you look at what code is running in those 5 minutes and make sure that code is not using too much memory.</p>",
      "rawMarkdown": "If the code takes 5 minutes, i suspect you are doing more than just `df.head()` and `df.shape`. I suggest you look at what code is running in those 5 minutes and make sure that code is not using too much memory.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1862382,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "07/19/2022 16:32:01",
      "content": "<blockquote>\n  <p>However, after around 5 mins of usage or so</p>\n</blockquote>\n<p>What are you doing during these 5 minutes? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1862442,
          "author_name": "abhisheksharma1998",
          "author_url": "",
          "post_date": "07/19/2022 17:35:39",
          "content": "<p>That's the thing - I am not doing anything memory intensive at all, just basic code like <code>df.head()</code> or <code>df.shape</code>. It seems whatever I do, it shuts down in approx. 5 mins. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1862623,
          "author_name": "aliabdin1",
          "author_url": "",
          "post_date": "07/19/2022 21:09:26",
          "content": "<p><a href=\"https://www.kaggle.com/abhisheksharma1998\" target=\"_blank\">@abhisheksharma1998</a> Maybe you can share your notebook. Basic code should work without big problems.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1862625,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "07/19/2022 21:16:57",
          "content": "<p>If the code takes 5 minutes, i suspect you are doing more than just <code>df.head()</code> and <code>df.shape</code>. I suggest you look at what code is running in those 5 minutes and make sure that code is not using too much memory.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1862103": "Hey everyone, I have been facing an issue recently. I have been using the integer dataset (thanks to @raddar) and been careful to load only enough data to stay well within the RAM limit. **On loading only the `train.parquet` file, the RAM usage is approximately 5.8 GBs**. \n\nHowever, after around 5 mins of usage or so, the **RAM randomly spikes to a full** and restarts with the message \"**Your notebook tried to allocate more memory than is available. It has restarted**\". I end up losing all my work. \n\nI am not even sure if this problem has anything to do with this competition. Is this something other people have also been facing? \n\n**EDIT**: I am still unable to figure out a cause to this but it does seem to have gone away for now. I can now run the exact same notebook with nothing changed and not have the notebook crash.",
    "1862382": ">However, after around 5 mins of usage or so\n\nWhat are you doing during these 5 minutes?",
    "1862442": "That's the thing - I am not doing anything memory intensive at all, just basic code like `df.head()` or `df.shape`. It seems whatever I do, it shuts down in approx. 5 mins.",
    "1862623": "abhisheksharma1998 Maybe you can share your notebook. Basic code should work without big problems.",
    "1862625": "If the code takes 5 minutes, i suspect you are doing more than just `df.head()` and `df.shape`. I suggest you look at what code is running in those 5 minutes and make sure that code is not using too much memory."
  },
  "source": "meta"
}