{
  "id": 51715,
  "title": "Why I am not able to load to load the data on kernel?",
  "url": "/competitions/talkingdata-adtracking-fraud-detection/discussion/51715",
  "author_name": "",
  "post_date": "2018-03-12T13:24:19.011890900Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>My question to other kagglers regarding this challenge is that, why i am not able to load 8gb (approximately) data on the environment with 17.2 GB of ram. I think the data is loading into ram only?? I may be wrong... i am not clear with system's specification. I am attaching a snap where in the bottom right it is mentioned the memory sizes</p>\n\n<p>. </p>",
  "messages": [
    {
      "id": "294708",
      "postDate": "03/12/2018 13:24:19",
      "content": "<p>My question to other kagglers regarding this challenge is that, why i am not able to load 8gb (approximately) data on the environment with 17.2 GB of ram. I think the data is loading into ram only?? I may be wrong... i am not clear with system's specification. I am attaching a snap where in the bottom right it is mentioned the memory sizes</p>\n\n<p>. </p>",
      "rawMarkdown": "My question to other kagglers regarding this challenge is that, why i am not able to load 8gb (approximately) data on the environment with 17.2 GB of ram. I think the data is loading into ram only?? I may be wrong... i am not clear with system's specification. I am attaching a snap where in the bottom right it is mentioned the memory sizes\n\n.",
      "votes": null
    },
    {
      "id": "295182",
      "postDate": "03/13/2018 07:50:13",
      "content": "<p>This is a very good tutorial on how to deal with this : <a href=\"https://www.kaggle.com/yuliagm/talkingdata-eda-plus-time-patterns\">https://www.kaggle.com/yuliagm/talkingdata-eda-plus-time-patterns</a></p>",
      "rawMarkdown": "This is a very good tutorial on how to deal with this : https://www.kaggle.com/yuliagm/talkingdata-eda-plus-time-patterns",
      "votes": null
    },
    {
      "id": "295418",
      "postDate": "03/13/2018 16:14:51",
      "content": "<p>Dear @Asparuh my question is why i the dataset is not loading. it's size is 8gb and the ram(memory) allocated for kernel is 17.2 GB. So it should be loaded. Are there any other processes running on ram ? This is what i want to ask </p>",
      "rawMarkdown": "Dear @Asparuh my question is why i the dataset is not loading. it's size is 8gb and the ram(memory) allocated for kernel is 17.2 GB. So it should be loaded. Are there any other processes running on ram ? This is what i want to ask",
      "votes": null
    },
    {
      "id": "295420",
      "postDate": "03/13/2018 16:17:06",
      "content": "<p>True, I misunderstood the question. Hope somebody clarifies the issue that you raise.</p>",
      "rawMarkdown": "True, I misunderstood the question. Hope somebody clarifies the issue that you raise.",
      "votes": null
    },
    {
      "id": "295696",
      "postDate": "03/14/2018 02:54:26",
      "content": "<p>This kernel : <a href=\"https://www.kaggle.com/mapodoufu/baseline-the-channel-is-important-for-download\">https://www.kaggle.com/mapodoufu/baseline-the-channel-is-important-for-download</a> \nLoad all data just fine. Make sure you provide the correct datatypes, otherwise unefficient datatype chossing will be used and your data will use so much memory. </p>\n\n<p>The 8 giga bytes is text representation of the data. When you open it using pandas or whatever they need to pass each data to a variables such as int or float or string, and the total memory needed will be totally different with what you observed when its still in text (csv) form. </p>",
      "rawMarkdown": "This kernel : https://www.kaggle.com/mapodoufu/baseline-the-channel-is-important-for-download \nLoad all data just fine. Make sure you provide the correct datatypes, otherwise unefficient datatype chossing will be used and your data will use so much memory. \n\nThe 8 giga bytes is text representation of the data. When you open it using pandas or whatever they need to pass each data to a variables such as int or float or string, and the total memory needed will be totally different with what you observed when its still in text (csv) form.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 295182,
      "author_name": "asparuhhristov",
      "author_url": "",
      "post_date": "03/13/2018 07:50:13",
      "content": "<p>This is a very good tutorial on how to deal with this : <a href=\"https://www.kaggle.com/yuliagm/talkingdata-eda-plus-time-patterns\">https://www.kaggle.com/yuliagm/talkingdata-eda-plus-time-patterns</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 295418,
          "author_name": "blasteraj",
          "author_url": "",
          "post_date": "03/13/2018 16:14:51",
          "content": "<p>Dear @Asparuh my question is why i the dataset is not loading. it's size is 8gb and the ram(memory) allocated for kernel is 17.2 GB. So it should be loaded. Are there any other processes running on ram ? This is what i want to ask </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 295420,
          "author_name": "asparuhhristov",
          "author_url": "",
          "post_date": "03/13/2018 16:17:06",
          "content": "<p>True, I misunderstood the question. Hope somebody clarifies the issue that you raise.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 295696,
      "author_name": "muhammadalfiansyah",
      "author_url": "",
      "post_date": "03/14/2018 02:54:26",
      "content": "<p>This kernel : <a href=\"https://www.kaggle.com/mapodoufu/baseline-the-channel-is-important-for-download\">https://www.kaggle.com/mapodoufu/baseline-the-channel-is-important-for-download</a> \nLoad all data just fine. Make sure you provide the correct datatypes, otherwise unefficient datatype chossing will be used and your data will use so much memory. </p>\n\n<p>The 8 giga bytes is text representation of the data. When you open it using pandas or whatever they need to pass each data to a variables such as int or float or string, and the total memory needed will be totally different with what you observed when its still in text (csv) form. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "294708": "My question to other kagglers regarding this challenge is that, why i am not able to load 8gb (approximately) data on the environment with 17.2 GB of ram. I think the data is loading into ram only?? I may be wrong... i am not clear with system's specification. I am attaching a snap where in the bottom right it is mentioned the memory sizes\n\n.",
    "295182": "This is a very good tutorial on how to deal with this : https://www.kaggle.com/yuliagm/talkingdata-eda-plus-time-patterns",
    "295418": "Dear @Asparuh my question is why i the dataset is not loading. it's size is 8gb and the ram(memory) allocated for kernel is 17.2 GB. So it should be loaded. Are there any other processes running on ram ? This is what i want to ask",
    "295420": "True, I misunderstood the question. Hope somebody clarifies the issue that you raise.",
    "295696": "This kernel : https://www.kaggle.com/mapodoufu/baseline-the-channel-is-important-for-download \nLoad all data just fine. Make sure you provide the correct datatypes, otherwise unefficient datatype chossing will be used and your data will use so much memory. \n\nThe 8 giga bytes is text representation of the data. When you open it using pandas or whatever they need to pass each data to a variables such as int or float or string, and the total memory needed will be totally different with what you observed when its still in text (csv) form."
  },
  "source": "meta"
}