{
  "id": 334956,
  "title": "Reading Gigabytes size files in Notebook",
  "url": "/competitions/amex-default-prediction/discussion/334956",
  "author_name": "",
  "post_date": "2022-07-04T01:55:35.736225100Z",
  "votes": 6,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hi,<br>\nWhile reading the files (train &amp; test) as csv, it throws memory error. How can i read it? is there any way that i can read it as perquet. Will that be readable considering memory concern here?<br>\nI am not sure on this. Please let me know.</p>",
  "messages": [
    {
      "id": "1842426",
      "postDate": "07/04/2022 01:55:35",
      "content": "<p>Hi,<br>\nWhile reading the files (train &amp; test) as csv, it throws memory error. How can i read it? is there any way that i can read it as perquet. Will that be readable considering memory concern here?<br>\nI am not sure on this. Please let me know.</p>",
      "rawMarkdown": "Hi,\nWhile reading the files (train & test) as csv, it throws memory error. How can i read it? is there any way that i can read it as perquet. Will that be readable considering memory concern here?\nI am not sure on this. Please let me know.",
      "votes": null
    },
    {
      "id": "1842467",
      "postDate": "07/04/2022 03:31:21",
      "content": "<p>This <a href=\"https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format\" target=\"_blank\">dataset</a> from <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a> is very useful. </p>",
      "rawMarkdown": "This [dataset](https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format) from @raddar is very useful.",
      "votes": null
    },
    {
      "id": "1842469",
      "postDate": "07/04/2022 03:42:54",
      "content": "<p><a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886</a></p>",
      "rawMarkdown": "https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886",
      "votes": null
    },
    {
      "id": "1842570",
      "postDate": "07/04/2022 05:50:54",
      "content": "<p>One Kaggle user named <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a> has solved this problem and has uploaded the dataset as a parquet file on Kaggle. You could use pandas.read_parquet and refer to his data-set to solve the problem. </p>\n<p>The below discussion thread could also be useful for you- <br>\n<a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886</a></p>",
      "rawMarkdown": "One Kaggle user named @raddar has solved this problem and has uploaded the dataset as a parquet file on Kaggle. You could use pandas.read_parquet and refer to his data-set to solve the problem. \n\nThe below discussion thread could also be useful for you- \nhttps://www.kaggle.com/competitions/amex-default-prediction/discussion/334886",
      "votes": null
    },
    {
      "id": "1842703",
      "postDate": "07/04/2022 08:15:27",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/ravi20076\" target=\"_blank\">@ravi20076</a>  looks like everyone has suggested the same :).</p>",
      "rawMarkdown": "Thanks @ravi20076  looks like everyone has suggested the same :).",
      "votes": null
    },
    {
      "id": "1842704",
      "postDate": "07/04/2022 08:15:45",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> </p>",
      "rawMarkdown": "Thanks @cdeotte",
      "votes": null
    },
    {
      "id": "1842705",
      "postDate": "07/04/2022 08:16:22",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/momoseoyama\" target=\"_blank\">@momoseoyama</a> .</p>",
      "rawMarkdown": "Thanks @momoseoyama .",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1842467,
      "author_name": "momoseoyama",
      "author_url": "",
      "post_date": "07/04/2022 03:31:21",
      "content": "<p>This <a href=\"https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format\" target=\"_blank\">dataset</a> from <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a> is very useful. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1842705,
          "author_name": "nknarendra7",
          "author_url": "",
          "post_date": "07/04/2022 08:16:22",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/momoseoyama\" target=\"_blank\">@momoseoyama</a> .</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1842469,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "07/04/2022 03:42:54",
      "content": "<p><a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1842704,
          "author_name": "nknarendra7",
          "author_url": "",
          "post_date": "07/04/2022 08:15:45",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1842570,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "07/04/2022 05:50:54",
      "content": "<p>One Kaggle user named <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a> has solved this problem and has uploaded the dataset as a parquet file on Kaggle. You could use pandas.read_parquet and refer to his data-set to solve the problem. </p>\n<p>The below discussion thread could also be useful for you- <br>\n<a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1842703,
          "author_name": "nknarendra7",
          "author_url": "",
          "post_date": "07/04/2022 08:15:27",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/ravi20076\" target=\"_blank\">@ravi20076</a>  looks like everyone has suggested the same :).</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1842426": "Hi,\nWhile reading the files (train & test) as csv, it throws memory error. How can i read it? is there any way that i can read it as perquet. Will that be readable considering memory concern here?\nI am not sure on this. Please let me know.",
    "1842467": "This [dataset](https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format) from @raddar is very useful.",
    "1842469": "https://www.kaggle.com/competitions/amex-default-prediction/discussion/334886",
    "1842570": "One Kaggle user named @raddar has solved this problem and has uploaded the dataset as a parquet file on Kaggle. You could use pandas.read_parquet and refer to his data-set to solve the problem. \n\nThe below discussion thread could also be useful for you- \nhttps://www.kaggle.com/competitions/amex-default-prediction/discussion/334886",
    "1842703": "Thanks @ravi20076  looks like everyone has suggested the same :).",
    "1842704": "Thanks @cdeotte",
    "1842705": "Thanks @momoseoyama ."
  },
  "source": "meta"
}