{
  "id": 365842,
  "title": "what is the best way to deal with this size of data ?",
  "url": "/competitions/deepfake-detection-challenge/discussion/365842",
  "author_name": "",
  "post_date": "2022-11-13T11:51:02.643670900Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>1- How can I deal with this volume of data?<br>\n2- Do I have to download the data and upload it again to google drive in order to deal with it in colab, or is there a simpler way?</p>",
  "messages": [
    {
      "id": "2027972",
      "postDate": "11/13/2022 11:51:02",
      "content": "<p>1- How can I deal with this volume of data?<br>\n2- Do I have to download the data and upload it again to google drive in order to deal with it in colab, or is there a simpler way?</p>",
      "rawMarkdown": "1- How can I deal with this volume of data?\n2- Do I have to download the data and upload it again to google drive in order to deal with it in colab, or is there a simpler way?",
      "votes": null
    },
    {
      "id": "2028319",
      "postDate": "11/13/2022 19:16:48",
      "content": "<p>Hello <a href=\"https://www.kaggle.com/seifzein\" target=\"_blank\">@seifzein</a>, No, you can use the Kaggle API directly inside of Colab, you still will have to download a copy of the data locally, but because they are all Google servers, the transfer should be quick. From there, you need to use Generator to read this massive amount of information.</p>",
      "rawMarkdown": "Hello @seifzein, No, you can use the Kaggle API directly inside of Colab, you still will have to download a copy of the data locally, but because they are all Google servers, the transfer should be quick. From there, you need to use Generator to read this massive amount of information.",
      "votes": null
    },
    {
      "id": "3107208",
      "postDate": "01/26/2025 06:25:50",
      "content": "<p>hello <a href=\"https://www.kaggle.com/cv13j0\" target=\"_blank\">@cv13j0</a> i am new to this, i want to know how can i download this chunks of data locally in today's date</p>",
      "rawMarkdown": "hello @cv13j0 i am new to this, i want to know how can i download this chunks of data locally in today's date",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2028319,
      "author_name": "cv13j0",
      "author_url": "",
      "post_date": "11/13/2022 19:16:48",
      "content": "<p>Hello <a href=\"https://www.kaggle.com/seifzein\" target=\"_blank\">@seifzein</a>, No, you can use the Kaggle API directly inside of Colab, you still will have to download a copy of the data locally, but because they are all Google servers, the transfer should be quick. From there, you need to use Generator to read this massive amount of information.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3107208,
          "author_name": "meetrajpal",
          "author_url": "",
          "post_date": "01/26/2025 06:25:50",
          "content": "<p>hello <a href=\"https://www.kaggle.com/cv13j0\" target=\"_blank\">@cv13j0</a> i am new to this, i want to know how can i download this chunks of data locally in today's date</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2027972": "1- How can I deal with this volume of data?\n2- Do I have to download the data and upload it again to google drive in order to deal with it in colab, or is there a simpler way?",
    "2028319": "Hello @seifzein, No, you can use the Kaggle API directly inside of Colab, you still will have to download a copy of the data locally, but because they are all Google servers, the transfer should be quick. From there, you need to use Generator to read this massive amount of information.",
    "3107208": "hello @cv13j0 i am new to this, i want to know how can i download this chunks of data locally in today's date"
  },
  "source": "meta"
}