{
  "id": 543011,
  "title": "running out of ram memory while training",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/543011",
  "author_name": "Prarabdha Srivastava",
  "post_date": "2024-10-28T08:13:34.078000",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>so i have just started working on this problem, and for now i am just trying to train a simple xg boost model on thsi dataset but i am running out of cpu memory.</p>\n<p>so i just wanted know if you guys are also facing same issue and i should work locally or there is some efficient way to handle this data</p>",
  "messages": [
    {
      "id": 3030166,
      "postDate": "2024-10-28T08:54:47.670Z",
      "content": "<p>Training locally is a good option-  avoid Kaggle kernels for this competition <a href=\"https://www.kaggle.com/prarabdhasrivastava\" target=\"_blank\">@prarabdhasrivastava</a> </p>",
      "rawMarkdown": "Training locally is a good option-  avoid Kaggle kernels for this competition @prarabdhasrivastava ",
      "votes": 1,
      "replies": [
        {
          "id": 3085969,
          "postDate": "2025-01-01T20:49:17.290Z",
          "content": "<p>Then what is the point of even having this competition on Kaggle, if we require external resources for training? Clearly, anyone with abundant resources can easily load all the datasets and train a model.</p>",
          "rawMarkdown": "Then what is the point of even having this competition on Kaggle, if we require external resources for training? Clearly, anyone with abundant resources can easily load all the datasets and train a model.",
          "votes": -1
        }
      ]
    },
    {
      "id": 3030142,
      "postDate": "2024-10-28T08:13:34.080Z",
      "content": "<p>so i have just started working on this problem, and for now i am just trying to train a simple xg boost model on thsi dataset but i am running out of cpu memory.</p>\n<p>so i just wanted know if you guys are also facing same issue and i should work locally or there is some efficient way to handle this data</p>",
      "rawMarkdown": "so i have just started working on this problem, and for now i am just trying to train a simple xg boost model on thsi dataset but i am running out of cpu memory.\n\nso i just wanted know if you guys are also facing same issue and i should work locally or there is some efficient way to handle this data",
      "votes": 1
    },
    {
      "id": 3030155,
      "postDate": "2024-10-28T08:36:55.107Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3030166,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2024-10-28T08:54:47.670000",
      "content": "<p>Training locally is a good option-  avoid Kaggle kernels for this competition <a href=\"https://www.kaggle.com/prarabdhasrivastava\" target=\"_blank\">@prarabdhasrivastava</a> </p>",
      "votes": 1,
      "replies": [
        {
          "id": 3085969,
          "author_name": "Hotson Honet",
          "author_url": "",
          "post_date": "2025-01-01T20:49:17.290000",
          "content": "<p>Then what is the point of even having this competition on Kaggle, if we require external resources for training? Clearly, anyone with abundant resources can easily load all the datasets and train a model.</p>",
          "votes": -1,
          "replies": []
        }
      ]
    },
    {
      "id": 3030155,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-10-28T08:36:55.107000",
      "content": "",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3030166": "Training locally is a good option-  avoid Kaggle kernels for this competition @prarabdhasrivastava ",
    "3030142": "so i have just started working on this problem, and for now i am just trying to train a simple xg boost model on thsi dataset but i am running out of cpu memory.\n\nso i just wanted know if you guys are also facing same issue and i should work locally or there is some efficient way to handle this data",
    "3030155": ""
  }
}