Hello, I am having memory problems therefore I have used del, howevere the RAM usage does not get reduced. I have used this code https://stackoverflow.com/questions/24455615/python-how-to-display-size-of-all-variables to list all the variables but the size listed does not match the one in Kaggle RAM. How can I check the size of the objects and del them properly?
\nRegards and thank you for you help
", "messages": [ { "id": "2721222", "postDate": "03/28/2024 22:10:26", "content": "Hello, I am having memory problems therefore I have used del, howevere the RAM usage does not get reduced. I have used this code https://stackoverflow.com/questions/24455615/python-how-to-display-size-of-all-variables to list all the variables but the size listed does not match the one in Kaggle RAM. How can I check the size of the objects and del them properly?
\nRegards and thank you for you help
", "rawMarkdown": "Hello, I am having memory problems therefore I have used del, howevere the RAM usage does not get reduced. I have used this code [https://stackoverflow.com/questions/24455615/python-how-to-display-size-of-all-variables](url) to list all the variables but the size listed does not match the one in Kaggle RAM. How can I check the size of the objects and del them properly?\n\nRegards and thank you for you help", "votes": null }, { "id": "2721916", "postDate": "03/29/2024 09:40:12", "content": "I encountered similar problem and discovered that problem occurs when I output some dataframe in notebook without passing it to some function like print or display (so simply placing in cell: df). In such situation some objects (with size corresponding to size of dataframe) appear in memory without possibility to clear memory even after removing dataframe and calling gc.collect(). The problem is solved with a command like display(df). I haven't delved much into the mechanics of such situations, maybe it's related to how Jupyter notebooks work.
", "rawMarkdown": "I encountered similar problem and discovered that problem occurs when I output some dataframe in notebook without passing it to some function like print or display (so simply placing in cell: df). In such situation some objects (with size corresponding to size of dataframe) appear in memory without possibility to clear memory even after removing dataframe and calling gc.collect(). The problem is solved with a command like display(df). I haven't delved much into the mechanics of such situations, maybe it's related to how Jupyter notebooks work.", "votes": null }, { "id": "2725345", "postDate": "03/31/2024 13:56:22", "content": "Thank you very much for your help, I have tried to momment all the \"printing\" cells to reduce disk space consumption. It seems to work a bit better. Thank you again.
", "rawMarkdown": "Thank you very much for your help, I have tried to momment all the \"printing\" cells to reduce disk space consumption. It seems to work a bit better. Thank you again.", "votes": null }, { "id": "2768978", "postDate": "04/23/2024 05:29:45", "content": "apart from other solutions in this thread, you can try and reuse the same variables so that it overwrites the memory and doesn't allocate more memory than you would need.
\nSeen this to be effective, do let me know if that worked @rubgac
", "rawMarkdown": "apart from other solutions in this thread, you can try and reuse the same variables so that it overwrites the memory and doesn't allocate more memory than you would need.\n\nSeen this to be effective, do let me know if that worked @rubgac", "votes": null } ], "comments": [ { "id": 2721916, "author_name": "andreynesterov", "author_url": "", "post_date": "03/29/2024 09:40:12", "content": "I encountered similar problem and discovered that problem occurs when I output some dataframe in notebook without passing it to some function like print or display (so simply placing in cell: df). In such situation some objects (with size corresponding to size of dataframe) appear in memory without possibility to clear memory even after removing dataframe and calling gc.collect(). The problem is solved with a command like display(df). I haven't delved much into the mechanics of such situations, maybe it's related to how Jupyter notebooks work.
", "votes": null, "replies": [] }, { "id": 2725345, "author_name": "rubgac", "author_url": "", "post_date": "03/31/2024 13:56:22", "content": "Thank you very much for your help, I have tried to momment all the \"printing\" cells to reduce disk space consumption. It seems to work a bit better. Thank you again.
", "votes": null, "replies": [] }, { "id": 2768978, "author_name": "luciferisback", "author_url": "", "post_date": "04/23/2024 05:29:45", "content": "apart from other solutions in this thread, you can try and reuse the same variables so that it overwrites the memory and doesn't allocate more memory than you would need.
\nSeen this to be effective, do let me know if that worked @rubgac
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2721222": "Hello, I am having memory problems therefore I have used del, howevere the RAM usage does not get reduced. I have used this code [https://stackoverflow.com/questions/24455615/python-how-to-display-size-of-all-variables](url) to list all the variables but the size listed does not match the one in Kaggle RAM. How can I check the size of the objects and del them properly?\n\nRegards and thank you for you help", "2721916": "I encountered similar problem and discovered that problem occurs when I output some dataframe in notebook without passing it to some function like print or display (so simply placing in cell: df). In such situation some objects (with size corresponding to size of dataframe) appear in memory without possibility to clear memory even after removing dataframe and calling gc.collect(). The problem is solved with a command like display(df). I haven't delved much into the mechanics of such situations, maybe it's related to how Jupyter notebooks work.", "2725345": "Thank you very much for your help, I have tried to momment all the \"printing\" cells to reduce disk space consumption. It seems to work a bit better. Thank you again.", "2768978": "apart from other solutions in this thread, you can try and reuse the same variables so that it overwrites the memory and doesn't allocate more memory than you would need.\n\nSeen this to be effective, do let me know if that worked @rubgac" }, "source": "meta" }