{
  "id": 126496,
  "title": "Access to full dataset data via GCP (and other cloud providers)?",
  "url": "/competitions/deepfake-detection-challenge/discussion/126496",
  "author_name": "",
  "post_date": "2020-01-17T23:15:30.747233600Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>There seems to be Kaggle API access to the 4 GB limited dataset, however the full zip (dfdc_train_all.zip) seems to only be able to be locally downloaded (remote wget does not work).</p>\n\n<p>I need to transfer this to a GCP bucket, however I do NOT want to download the entire 470 GB to my PC and then upload it to the bucket ))</p>\n\n<p>Any advice on how to pull the entire dataset from the Ubuntu bash shell on a GCP instance?</p>\n\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "721941",
      "postDate": "01/17/2020 23:15:30",
      "content": "<p>There seems to be Kaggle API access to the 4 GB limited dataset, however the full zip (dfdc_train_all.zip) seems to only be able to be locally downloaded (remote wget does not work).</p>\n\n<p>I need to transfer this to a GCP bucket, however I do NOT want to download the entire 470 GB to my PC and then upload it to the bucket ))</p>\n\n<p>Any advice on how to pull the entire dataset from the Ubuntu bash shell on a GCP instance?</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "There seems to be Kaggle API access to the 4 GB limited dataset, however the full zip (dfdc_train_all.zip) seems to only be able to be locally downloaded (remote wget does not work).\n\nI need to transfer this to a GCP bucket, however I do NOT want to download the entire 470 GB to my PC and then upload it to the bucket ))\n\nAny advice on how to pull the entire dataset from the Ubuntu bash shell on a GCP instance?\n\nThanks!",
      "votes": null
    },
    {
      "id": "721962",
      "postDate": "01/18/2020 00:05:45",
      "content": "<p>Yes please!</p>",
      "rawMarkdown": "Yes please!",
      "votes": null
    },
    {
      "id": "721970",
      "postDate": "01/18/2020 00:30:10",
      "content": "<p>Hi\nFound a solution by searching )))</p>\n\n<p><a href=\"https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121194\">https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121194</a></p>\n\n<p>Good luck!</p>",
      "rawMarkdown": "Hi\nFound a solution by searching )))\n\nhttps://www.kaggle.com/c/deepfake-detection-challenge/discussion/121194\n\nGood luck!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 721962,
      "author_name": "debanga",
      "author_url": "",
      "post_date": "01/18/2020 00:05:45",
      "content": "<p>Yes please!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 721970,
      "author_name": "eaustin",
      "author_url": "",
      "post_date": "01/18/2020 00:30:10",
      "content": "<p>Hi\nFound a solution by searching )))</p>\n\n<p><a href=\"https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121194\">https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121194</a></p>\n\n<p>Good luck!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "721941": "There seems to be Kaggle API access to the 4 GB limited dataset, however the full zip (dfdc_train_all.zip) seems to only be able to be locally downloaded (remote wget does not work).\n\nI need to transfer this to a GCP bucket, however I do NOT want to download the entire 470 GB to my PC and then upload it to the bucket ))\n\nAny advice on how to pull the entire dataset from the Ubuntu bash shell on a GCP instance?\n\nThanks!",
    "721962": "Yes please!",
    "721970": "Hi\nFound a solution by searching )))\n\nhttps://www.kaggle.com/c/deepfake-detection-challenge/discussion/121194\n\nGood luck!"
  },
  "source": "meta"
}