{
  "id": 399036,
  "title": "Problems about reading in dataset in kaggle notebook",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/399036",
  "author_name": "",
  "post_date": "2023-04-02T06:35:28.216862100Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi all, I am a new learner of machine learning and I would like to try to work on this problem.</p>\n<p>I added the data into my kaggle notebook, then tried to use pandas to read the train.csv file as a dataframe but seems that the data is too big(4.7GB) and the notebook is unable to read it and it just suggest me to upgrade to GCP notebook. Is that normal?</p>",
  "messages": [
    {
      "id": "2205994",
      "postDate": "04/02/2023 06:35:28",
      "content": "<p>Hi all, I am a new learner of machine learning and I would like to try to work on this problem.</p>\n<p>I added the data into my kaggle notebook, then tried to use pandas to read the train.csv file as a dataframe but seems that the data is too big(4.7GB) and the notebook is unable to read it and it just suggest me to upgrade to GCP notebook. Is that normal?</p>",
      "rawMarkdown": "Hi all, I am a new learner of machine learning and I would like to try to work on this problem.\n\nI added the data into my kaggle notebook, then tried to use pandas to read the train.csv file as a dataframe but seems that the data is too big(4.7GB) and the notebook is unable to read it and it just suggest me to upgrade to GCP notebook. Is that normal?",
      "votes": null
    },
    {
      "id": "2206010",
      "postDate": "04/02/2023 06:54:11",
      "content": "<p>Here you can see how to read 953 MB train file <a href=\"https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb\" target=\"_blank\">https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb</a></p>",
      "rawMarkdown": "Here you can see how to read 953 MB train file https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb",
      "votes": null
    },
    {
      "id": "2206033",
      "postDate": "04/02/2023 07:21:29",
      "content": "<blockquote>\n  <p>Here you can see how to read 953 MB train file <a href=\"https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb\" target=\"_blank\">https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb</a></p>\n</blockquote>\n<p>Great. Didn't expect that changing the data formats of each columns results such a large reduction in size.. </p>",
      "rawMarkdown": "> Here you can see how to read 953 MB train file https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb\n\nGreat. Didn't expect that changing the data formats of each columns results such a large reduction in size..",
      "votes": null
    },
    {
      "id": "2208384",
      "postDate": "04/04/2023 02:25:36",
      "content": "<p>It this competition, if you create a new notebook it is restricted to 8GB of RAM, however there is a trick to get 32GB RAM posted <a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/396979\" target=\"_blank\">here</a></p>",
      "rawMarkdown": "It this competition, if you create a new notebook it is restricted to 8GB of RAM, however there is a trick to get 32GB RAM posted [here][1]\n\n[1]: https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/396979",
      "votes": null
    },
    {
      "id": "2211203",
      "postDate": "04/05/2023 22:06:05",
      "content": "<p>Excuse me, sir.  I have just have a question related to the size of the data.</p>\n<p>Assuming we have loaded in the data successfully, does the size of the dataframe affect the running time of processing data, training models afterwards? If yes, is the effect significant or not?</p>\n<p>Thanks.</p>",
      "rawMarkdown": "Excuse me, sir.  I have just have a question related to the size of the data.\n\nAssuming we have loaded in the data successfully, does the size of the dataframe affect the running time of processing data, training models afterwards? If yes, is the effect significant or not?\n\nThanks.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2206010,
      "author_name": "vadimkamaev",
      "author_url": "",
      "post_date": "04/02/2023 06:54:11",
      "content": "<p>Here you can see how to read 953 MB train file <a href=\"https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb\" target=\"_blank\">https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 2206033,
          "author_name": "lunaayase1993",
          "author_url": "",
          "post_date": "04/02/2023 07:21:29",
          "content": "<blockquote>\n  <p>Here you can see how to read 953 MB train file <a href=\"https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb\" target=\"_blank\">https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb</a></p>\n</blockquote>\n<p>Great. Didn't expect that changing the data formats of each columns results such a large reduction in size.. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2211203,
          "author_name": "lunaayase1993",
          "author_url": "",
          "post_date": "04/05/2023 22:06:05",
          "content": "<p>Excuse me, sir.  I have just have a question related to the size of the data.</p>\n<p>Assuming we have loaded in the data successfully, does the size of the dataframe affect the running time of processing data, training models afterwards? If yes, is the effect significant or not?</p>\n<p>Thanks.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2208384,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "04/04/2023 02:25:36",
      "content": "<p>It this competition, if you create a new notebook it is restricted to 8GB of RAM, however there is a trick to get 32GB RAM posted <a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/396979\" target=\"_blank\">here</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2205994": "Hi all, I am a new learner of machine learning and I would like to try to work on this problem.\n\nI added the data into my kaggle notebook, then tried to use pandas to read the train.csv file as a dataframe but seems that the data is too big(4.7GB) and the notebook is unable to read it and it just suggest me to upgrade to GCP notebook. Is that normal?",
    "2206010": "Here you can see how to read 953 MB train file https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb",
    "2206033": "> Here you can see how to read 953 MB train file https://www.kaggle.com/code/vadimkamaev/reading-data-953-mb\n\nGreat. Didn't expect that changing the data formats of each columns results such a large reduction in size..",
    "2208384": "It this competition, if you create a new notebook it is restricted to 8GB of RAM, however there is a trick to get 32GB RAM posted [here][1]\n\n[1]: https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/396979",
    "2211203": "Excuse me, sir.  I have just have a question related to the size of the data.\n\nAssuming we have loaded in the data successfully, does the size of the dataframe affect the running time of processing data, training models afterwards? If yes, is the effect significant or not?\n\nThanks."
  },
  "source": "meta"
}