{
  "id": 391505,
  "title": "Why RAM is not getting cleaned up?",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/391505",
  "author_name": "",
  "post_date": "2023-03-01T17:56:22.701041200Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>The dataset in this competition is pretty huge so after creating a new variable I delete any unused variables and do garbage collection using gc module. But even after doing this the RAM usage is not getting cleared. Why is that? So how do remove the used variables and free ram.<br>\nI know it is a very basic question but it would be helpful if someone points it out. Thanks in advance</p>",
  "messages": [
    {
      "id": "2164623",
      "postDate": "03/01/2023 17:56:22",
      "content": "<p>The dataset in this competition is pretty huge so after creating a new variable I delete any unused variables and do garbage collection using gc module. But even after doing this the RAM usage is not getting cleared. Why is that? So how do remove the used variables and free ram.<br>\nI know it is a very basic question but it would be helpful if someone points it out. Thanks in advance</p>",
      "rawMarkdown": "The dataset in this competition is pretty huge so after creating a new variable I delete any unused variables and do garbage collection using gc module. But even after doing this the RAM usage is not getting cleared. Why is that? So how do remove the used variables and free ram.\nI know it is a very basic question but it would be helpful if someone points it out. Thanks in advance",
      "votes": null
    },
    {
      "id": "2164903",
      "postDate": "03/01/2023 21:24:38",
      "content": "<p>using <code>gc.collect()</code> should do the job if there is <strong>unassigned allocated memory</strong> in RAM, I also suggest you also see <a href=\"https://www.kaggle.com/code/curiosity30/reduce-memory-2gb-500mb\" target=\"_blank\">this</a> notebook to reduce the size of the data in RAM</p>",
      "rawMarkdown": "using `gc.collect()` should do the job if there is **unassigned allocated memory** in RAM, I also suggest you also see [this](https://www.kaggle.com/code/curiosity30/reduce-memory-2gb-500mb) notebook to reduce the size of the data in RAM",
      "votes": null
    },
    {
      "id": "2165220",
      "postDate": "03/02/2023 04:07:46",
      "content": "<p>Try to create functions for each step (data_preparation, model_development, etc.). This way can help you. You will not save variables used inside functions, only returns</p>",
      "rawMarkdown": "Try to create functions for each step (data_preparation, model_development, etc.). This way can help you. You will not save variables used inside functions, only returns",
      "votes": null
    },
    {
      "id": "2166842",
      "postDate": "03/03/2023 04:02:39",
      "content": "<p>I think processing the training individually, saving the results then processing the testing may help. Also similar to what <a href=\"https://www.kaggle.com/alexanderkireev\" target=\"_blank\">@alexanderkireev</a> said, you may write your code in functions.</p>",
      "rawMarkdown": "I think processing the training individually, saving the results then processing the testing may help. Also similar to what @alexanderkireev said, you may write your code in functions.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2164903,
      "author_name": "janmpia",
      "author_url": "",
      "post_date": "03/01/2023 21:24:38",
      "content": "<p>using <code>gc.collect()</code> should do the job if there is <strong>unassigned allocated memory</strong> in RAM, I also suggest you also see <a href=\"https://www.kaggle.com/code/curiosity30/reduce-memory-2gb-500mb\" target=\"_blank\">this</a> notebook to reduce the size of the data in RAM</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2165220,
      "author_name": "alexanderkireev",
      "author_url": "",
      "post_date": "03/02/2023 04:07:46",
      "content": "<p>Try to create functions for each step (data_preparation, model_development, etc.). This way can help you. You will not save variables used inside functions, only returns</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2166842,
      "author_name": "mohammad2012191",
      "author_url": "",
      "post_date": "03/03/2023 04:02:39",
      "content": "<p>I think processing the training individually, saving the results then processing the testing may help. Also similar to what <a href=\"https://www.kaggle.com/alexanderkireev\" target=\"_blank\">@alexanderkireev</a> said, you may write your code in functions.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2164623": "The dataset in this competition is pretty huge so after creating a new variable I delete any unused variables and do garbage collection using gc module. But even after doing this the RAM usage is not getting cleared. Why is that? So how do remove the used variables and free ram.\nI know it is a very basic question but it would be helpful if someone points it out. Thanks in advance",
    "2164903": "using `gc.collect()` should do the job if there is **unassigned allocated memory** in RAM, I also suggest you also see [this](https://www.kaggle.com/code/curiosity30/reduce-memory-2gb-500mb) notebook to reduce the size of the data in RAM",
    "2165220": "Try to create functions for each step (data_preparation, model_development, etc.). This way can help you. You will not save variables used inside functions, only returns",
    "2166842": "I think processing the training individually, saving the results then processing the testing may help. Also similar to what @alexanderkireev said, you may write your code in functions."
  },
  "source": "meta"
}