{
  "id": 91739,
  "title": "Not enough memory",
  "url": "/competitions/LANL-Earthquake-Prediction/discussion/91739",
  "author_name": "",
  "post_date": "2019-05-08T14:07:34.803168300Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>when i try to use pandas of python3 to load train.csv,the 16G memory is not enough, even if it is just loading</p>",
  "messages": [
    {
      "id": "528760",
      "postDate": "05/08/2019 14:07:34",
      "content": "<p>when i try to use pandas of python3 to load train.csv,the 16G memory is not enough, even if it is just loading</p>",
      "rawMarkdown": "when i try to use pandas of python3 to load train.csv,the 16G memory is not enough, even if it is just loading",
      "votes": null
    },
    {
      "id": "528767",
      "postDate": "05/08/2019 14:19:49",
      "content": "<p>try this:\ntrain = pd.read_csv('../input/train.csv', dtype={'acoustic_data': np.int16, 'time_to_failure': np.float32})</p>",
      "rawMarkdown": "try this:\ntrain = pd.read_csv('../input/train.csv', dtype={'acoustic_data': np.int16, 'time_to_failure': np.float32})",
      "votes": null
    },
    {
      "id": "529268",
      "postDate": "05/09/2019 14:41:53",
      "content": "<p>Check out my discussion on how to load train using feather:</p>\n\n<p><a href=\"https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/90330\">https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/90330</a></p>",
      "rawMarkdown": "Check out my discussion on how to load train using feather:\n\nhttps://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/90330",
      "votes": null
    },
    {
      "id": "529538",
      "postDate": "05/10/2019 06:43:07",
      "content": "<p>Try reading data in chunks, for example like described in the tread <a href=\"https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/80250\">A framework for fast feature extraction</a></p>",
      "rawMarkdown": "Try reading data in chunks, for example like described in the tread [A framework for fast feature extraction](https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/80250)",
      "votes": null
    },
    {
      "id": "530063",
      "postDate": "05/11/2019 15:57:17",
      "content": "<p>I've addressed this problem in a kernel <a href=\"https://www.kaggle.com/friedchips/how-to-reduce-the-training-data-to-400mb\">here</a>. Hope this helps!</p>",
      "rawMarkdown": "I've addressed this problem in a kernel [here](https://www.kaggle.com/friedchips/how-to-reduce-the-training-data-to-400mb). Hope this helps!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 528767,
      "author_name": "greenwing1985",
      "author_url": "",
      "post_date": "05/08/2019 14:19:49",
      "content": "<p>try this:\ntrain = pd.read_csv('../input/train.csv', dtype={'acoustic_data': np.int16, 'time_to_failure': np.float32})</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 529268,
      "author_name": "teeyee314",
      "author_url": "",
      "post_date": "05/09/2019 14:41:53",
      "content": "<p>Check out my discussion on how to load train using feather:</p>\n\n<p><a href=\"https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/90330\">https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/90330</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 529538,
      "author_name": "alexfir",
      "author_url": "",
      "post_date": "05/10/2019 06:43:07",
      "content": "<p>Try reading data in chunks, for example like described in the tread <a href=\"https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/80250\">A framework for fast feature extraction</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 530063,
      "author_name": "friedchips",
      "author_url": "",
      "post_date": "05/11/2019 15:57:17",
      "content": "<p>I've addressed this problem in a kernel <a href=\"https://www.kaggle.com/friedchips/how-to-reduce-the-training-data-to-400mb\">here</a>. Hope this helps!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "528760": "when i try to use pandas of python3 to load train.csv,the 16G memory is not enough, even if it is just loading",
    "528767": "try this:\ntrain = pd.read_csv('../input/train.csv', dtype={'acoustic_data': np.int16, 'time_to_failure': np.float32})",
    "529268": "Check out my discussion on how to load train using feather:\n\nhttps://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/90330",
    "529538": "Try reading data in chunks, for example like described in the tread [A framework for fast feature extraction](https://www.kaggle.com/c/LANL-Earthquake-Prediction/discussion/80250)",
    "530063": "I've addressed this problem in a kernel [here](https://www.kaggle.com/friedchips/how-to-reduce-the-training-data-to-400mb). Hope this helps!"
  },
  "source": "meta"
}