{
  "id": 181580,
  "title": "Connecting to google colab",
  "url": "/competitions/osic-pulmonary-fibrosis-progression/discussion/181580",
  "author_name": "",
  "post_date": "2020-09-09T12:13:29.101851600Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hey all,<br>\nHas anybody tried downloading the dataset to google colab?</p>\n<p>I have tried  the well known method:</p>\n<p>! pip install -q kaggle<br>\nfrom google.colab import files<br>\nfiles.upload() #and upload kaggle.json<br>\n! mkdir ~/.kaggle<br>\n! cp kaggle.json ~/.kaggle/<br>\n! chmod 600 ~/.kaggle/kaggle.json<br>\n! kaggle competitions download -c osic-pulmonary-fibrosis-progression</p>\n<p>However, only the files, without the directories, are downloading. (I read that a zip file is supposed to be downloaded, but that's not the case for me)<br>\nSo the dicom files with the same names replace each other and everything gets mixed up.</p>\n<p>Does anybody have a solution to downloading the data to colab with the directories intact?<br>\nOr perhaps as an alternative, a way to 'copy' data of size &gt;5GB from a kaggle kernel to a colab environment? (I figured I could just zip the whole stuff and download the zip file from colab, but the compressed file is over 5GB)</p>",
  "messages": [
    {
      "id": "1004030",
      "postDate": "09/09/2020 12:13:29",
      "content": "<p>Hey all,<br>\nHas anybody tried downloading the dataset to google colab?</p>\n<p>I have tried  the well known method:</p>\n<p>! pip install -q kaggle<br>\nfrom google.colab import files<br>\nfiles.upload() #and upload kaggle.json<br>\n! mkdir ~/.kaggle<br>\n! cp kaggle.json ~/.kaggle/<br>\n! chmod 600 ~/.kaggle/kaggle.json<br>\n! kaggle competitions download -c osic-pulmonary-fibrosis-progression</p>\n<p>However, only the files, without the directories, are downloading. (I read that a zip file is supposed to be downloaded, but that's not the case for me)<br>\nSo the dicom files with the same names replace each other and everything gets mixed up.</p>\n<p>Does anybody have a solution to downloading the data to colab with the directories intact?<br>\nOr perhaps as an alternative, a way to 'copy' data of size &gt;5GB from a kaggle kernel to a colab environment? (I figured I could just zip the whole stuff and download the zip file from colab, but the compressed file is over 5GB)</p>",
      "rawMarkdown": "Hey all,\nHas anybody tried downloading the dataset to google colab?\n\nI have tried  the well known method:\n\n! pip install -q kaggle\nfrom google.colab import files\nfiles.upload() #and upload kaggle.json\n! mkdir ~/.kaggle\n! cp kaggle.json ~/.kaggle/\n! chmod 600 ~/.kaggle/kaggle.json\n! kaggle competitions download -c osic-pulmonary-fibrosis-progression\n\nHowever, only the files, without the directories, are downloading. (I read that a zip file is supposed to be downloaded, but that's not the case for me)\nSo the dicom files with the same names replace each other and everything gets mixed up.\n\nDoes anybody have a solution to downloading the data to colab with the directories intact?\nOr perhaps as an alternative, a way to 'copy' data of size >5GB from a kaggle kernel to a colab environment? (I figured I could just zip the whole stuff and download the zip file from colab, but the compressed file is over 5GB)",
      "votes": null
    },
    {
      "id": "1007293",
      "postDate": "09/12/2020 03:49:19",
      "content": "<p>I have. I did some hacking to recreate the directories in Kaggle. Although, I am having trouble with getting the full list of participants. doing the <code>kaggle competitions files</code> command from the API only gives me a handful of IDs.</p>",
      "rawMarkdown": "I have. I did some hacking to recreate the directories in Kaggle. Although, I am having trouble with getting the full list of participants. doing the `kaggle competitions files` command from the API only gives me a handful of IDs.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1007293,
      "author_name": "kitangladmatu",
      "author_url": "",
      "post_date": "09/12/2020 03:49:19",
      "content": "<p>I have. I did some hacking to recreate the directories in Kaggle. Although, I am having trouble with getting the full list of participants. doing the <code>kaggle competitions files</code> command from the API only gives me a handful of IDs.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1004030": "Hey all,\nHas anybody tried downloading the dataset to google colab?\n\nI have tried  the well known method:\n\n! pip install -q kaggle\nfrom google.colab import files\nfiles.upload() #and upload kaggle.json\n! mkdir ~/.kaggle\n! cp kaggle.json ~/.kaggle/\n! chmod 600 ~/.kaggle/kaggle.json\n! kaggle competitions download -c osic-pulmonary-fibrosis-progression\n\nHowever, only the files, without the directories, are downloading. (I read that a zip file is supposed to be downloaded, but that's not the case for me)\nSo the dicom files with the same names replace each other and everything gets mixed up.\n\nDoes anybody have a solution to downloading the data to colab with the directories intact?\nOr perhaps as an alternative, a way to 'copy' data of size >5GB from a kaggle kernel to a colab environment? (I figured I could just zip the whole stuff and download the zip file from colab, but the compressed file is over 5GB)",
    "1007293": "I have. I did some hacking to recreate the directories in Kaggle. Although, I am having trouble with getting the full list of participants. doing the `kaggle competitions files` command from the API only gives me a handful of IDs."
  },
  "source": "meta"
}