{
  "id": 333734,
  "title": "running out of memory for loading data ",
  "url": "/competitions/amex-default-prediction/discussion/333734",
  "author_name": "",
  "post_date": "2022-06-28T01:48:46.516477300Z",
  "votes": 3,
  "comment_count": 6,
  "views": 0,
  "content": "<p>hello,<br>\nI am running out of memory when trying to load the training data. <br>\nAny help of how to deal with this?</p>",
  "messages": [
    {
      "id": "1835670",
      "postDate": "06/28/2022 01:48:46",
      "content": "<p>hello,<br>\nI am running out of memory when trying to load the training data. <br>\nAny help of how to deal with this?</p>",
      "rawMarkdown": "hello,\nI am running out of memory when trying to load the training data. \nAny help of how to deal with this?",
      "votes": null
    },
    {
      "id": "1835689",
      "postDate": "06/28/2022 02:20:12",
      "content": "<p>I recommend checking out discussion forums. So many resources are available for this case. <br>\none of them by checking out this <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/328514\" target=\"_blank\">discussion</a></p>",
      "rawMarkdown": "I recommend checking out discussion forums. So many resources are available for this case. \none of them by checking out this [discussion](https://www.kaggle.com/competitions/amex-default-prediction/discussion/328514)",
      "votes": null
    },
    {
      "id": "1835693",
      "postDate": "06/28/2022 02:27:25",
      "content": "<p>thank you :) </p>",
      "rawMarkdown": "thank you :)",
      "votes": null
    },
    {
      "id": "1835859",
      "postDate": "06/28/2022 06:40:16",
      "content": "<p>I had the same issue. I locally uploaded the data into colab for execution. The data would not be available after the session though. You could upload a zipped version of the file on drive and mount the drive for using it on colab.</p>",
      "rawMarkdown": "I had the same issue. I locally uploaded the data into colab for execution. The data would not be available after the session though. You could upload a zipped version of the file on drive and mount the drive for using it on colab.",
      "votes": null
    },
    {
      "id": "1835948",
      "postDate": "06/28/2022 08:40:05",
      "content": "<p>The discussion is filled with solutions for this problem. Many of us are facing this issue. </p>\n<p>As a general advice, I would ask you to np.iinfo and np.finfo to downcast certain columns and then save the file with feather/ parquet format.</p>",
      "rawMarkdown": "The discussion is filled with solutions for this problem. Many of us are facing this issue. \n\nAs a general advice, I would ask you to np.iinfo and np.finfo to downcast certain columns and then save the file with feather/ parquet format.",
      "votes": null
    },
    {
      "id": "1836221",
      "postDate": "06/28/2022 13:22:01",
      "content": "<p>Check out e.g: <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/330754\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/330754</a></p>",
      "rawMarkdown": "Check out e.g: https://www.kaggle.com/competitions/amex-default-prediction/discussion/330754",
      "votes": null
    },
    {
      "id": "1836626",
      "postDate": "06/28/2022 21:35:57",
      "content": "<p>thank you all.</p>",
      "rawMarkdown": "thank you all.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1835689,
      "author_name": "naiborhujosua",
      "author_url": "",
      "post_date": "06/28/2022 02:20:12",
      "content": "<p>I recommend checking out discussion forums. So many resources are available for this case. <br>\none of them by checking out this <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/328514\" target=\"_blank\">discussion</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1835693,
          "author_name": "saramantach",
          "author_url": "",
          "post_date": "06/28/2022 02:27:25",
          "content": "<p>thank you :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1835859,
      "author_name": "arniragha",
      "author_url": "",
      "post_date": "06/28/2022 06:40:16",
      "content": "<p>I had the same issue. I locally uploaded the data into colab for execution. The data would not be available after the session though. You could upload a zipped version of the file on drive and mount the drive for using it on colab.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1835948,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "06/28/2022 08:40:05",
      "content": "<p>The discussion is filled with solutions for this problem. Many of us are facing this issue. </p>\n<p>As a general advice, I would ask you to np.iinfo and np.finfo to downcast certain columns and then save the file with feather/ parquet format.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1836221,
      "author_name": "aliabdin1",
      "author_url": "",
      "post_date": "06/28/2022 13:22:01",
      "content": "<p>Check out e.g: <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/330754\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/330754</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1836626,
      "author_name": "saramantach",
      "author_url": "",
      "post_date": "06/28/2022 21:35:57",
      "content": "<p>thank you all.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1835670": "hello,\nI am running out of memory when trying to load the training data. \nAny help of how to deal with this?",
    "1835689": "I recommend checking out discussion forums. So many resources are available for this case. \none of them by checking out this [discussion](https://www.kaggle.com/competitions/amex-default-prediction/discussion/328514)",
    "1835693": "thank you :)",
    "1835859": "I had the same issue. I locally uploaded the data into colab for execution. The data would not be available after the session though. You could upload a zipped version of the file on drive and mount the drive for using it on colab.",
    "1835948": "The discussion is filled with solutions for this problem. Many of us are facing this issue. \n\nAs a general advice, I would ask you to np.iinfo and np.finfo to downcast certain columns and then save the file with feather/ parquet format.",
    "1836221": "Check out e.g: https://www.kaggle.com/competitions/amex-default-prediction/discussion/330754",
    "1836626": "thank you all."
  },
  "source": "meta"
}