I get this error by running a kernel, after loading the train_sample and the test of this competition. It seems it is a memory error, though I didn't do much operation on my data before the kernel breaks: only a few groupby and computing some basic functions on those groupie such as sum() and count().\nHas anyone an idea of why it happens while I didn't even load the training set ? If so, are there ways to solve this issue ?\nThank you in advance ;)
", "messages": [ { "id": "310461", "postDate": "04/07/2018 15:56:51", "content": "I get this error by running a kernel, after loading the train_sample and the test of this competition. It seems it is a memory error, though I didn't do much operation on my data before the kernel breaks: only a few groupby and computing some basic functions on those groupie such as sum() and count().\nHas anyone an idea of why it happens while I didn't even load the training set ? If so, are there ways to solve this issue ?\nThank you in advance ;)
", "rawMarkdown": "I get this error by running a kernel, after loading the train_sample and the test of this competition. It seems it is a memory error, though I didn't do much operation on my data before the kernel breaks: only a few groupby and computing some basic functions on those groupie such as sum() and count().\nHas anyone an idea of why it happens while I didn't even load the training set ? If so, are there ways to solve this issue ?\nThank you in advance ;)", "votes": null }, { "id": "310472", "postDate": "04/07/2018 16:47:41", "content": "You ran out of memory. The Kaggle kernel limits you to 16 Gb and I do not think that it sufficient to process the entire data set that we are given in this competition. You either have to process it locally, on your computer (if it has enough memory) or chop it in pieces and then process each piece separately.
", "rawMarkdown": "You ran out of memory. The Kaggle kernel limits you to 16 Gb and I do not think that it sufficient to process the entire data set that we are given in this competition. You either have to process it locally, on your computer (if it has enough memory) or chop it in pieces and then process each piece separately.", "votes": null }, { "id": "310477", "postDate": "04/07/2018 17:01:26", "content": "@Antoine Gruet It is a memory issue as the train dataset is on the large side. This kernel may be a good starting point for you to explore further.
\n\nAn unsolicited advice: if this is the first time you are looking at this competition, consider that it has only month left to go. That is certainly enough time to learn lots of things, but may not be if you wish to compete.
", "rawMarkdown": "Antoine Gruet It is a memory issue as the train dataset is on the large side. [__This kernel__](https://www.kaggle.com/yuliagm/how-to-work-with-big-datasets-on-16g-ram-dask) may be a good starting point for you to explore further.\n\nAn unsolicited advice: if this is the first time you are looking at this competition, consider that it has only month left to go. That is certainly enough time to learn lots of things, but may not be if you wish to compete.", "votes": null }, { "id": "310482", "postDate": "04/07/2018 17:11:41", "content": "Thank you, I will process it locally in different pieces then.
", "rawMarkdown": "Thank you, I will process it locally in different pieces then.", "votes": null }, { "id": "310483", "postDate": "04/07/2018 17:11:57", "content": "Thank you for the advice !
", "rawMarkdown": "Thank you for the advice !", "votes": null } ], "comments": [ { "id": 310472, "author_name": "graf10a", "author_url": "", "post_date": "04/07/2018 16:47:41", "content": "You ran out of memory. The Kaggle kernel limits you to 16 Gb and I do not think that it sufficient to process the entire data set that we are given in this competition. You either have to process it locally, on your computer (if it has enough memory) or chop it in pieces and then process each piece separately.
", "votes": null, "replies": [ { "id": 310482, "author_name": "antoinegruet", "author_url": "", "post_date": "04/07/2018 17:11:41", "content": "Thank you, I will process it locally in different pieces then.
", "votes": null, "replies": [] } ] }, { "id": 310477, "author_name": "tilii7", "author_url": "", "post_date": "04/07/2018 17:01:26", "content": "@Antoine Gruet It is a memory issue as the train dataset is on the large side. This kernel may be a good starting point for you to explore further.
\n\nAn unsolicited advice: if this is the first time you are looking at this competition, consider that it has only month left to go. That is certainly enough time to learn lots of things, but may not be if you wish to compete.
", "votes": null, "replies": [ { "id": 310483, "author_name": "antoinegruet", "author_url": "", "post_date": "04/07/2018 17:11:57", "content": "Thank you for the advice !
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "310461": "I get this error by running a kernel, after loading the train_sample and the test of this competition. It seems it is a memory error, though I didn't do much operation on my data before the kernel breaks: only a few groupby and computing some basic functions on those groupie such as sum() and count().\nHas anyone an idea of why it happens while I didn't even load the training set ? If so, are there ways to solve this issue ?\nThank you in advance ;)", "310472": "You ran out of memory. The Kaggle kernel limits you to 16 Gb and I do not think that it sufficient to process the entire data set that we are given in this competition. You either have to process it locally, on your computer (if it has enough memory) or chop it in pieces and then process each piece separately.", "310477": "Antoine Gruet It is a memory issue as the train dataset is on the large side. [__This kernel__](https://www.kaggle.com/yuliagm/how-to-work-with-big-datasets-on-16g-ram-dask) may be a good starting point for you to explore further.\n\nAn unsolicited advice: if this is the first time you are looking at this competition, consider that it has only month left to go. That is certainly enough time to learn lots of things, but may not be if you wish to compete.", "310482": "Thank you, I will process it locally in different pieces then.", "310483": "Thank you for the advice !" }, "source": "meta" }