Hi all,
\nI would like to ask how to download the data. It's so big that bigger than 120 GB. Are yo directly downloading to your local or is there any other solution?
\nThanks in advance.
", "messages": [ { "id": 1387366, "postDate": "2021-07-14T06:11:50.860Z", "content": "Hi all,
\nI would like to ask how to download the data. It's so big that bigger than 120 GB. Are yo directly downloading to your local or is there any other solution?
\nThanks in advance.
", "rawMarkdown": "Hi all,\n\nI would like to ask how to download the data. It's so big that bigger than 120 GB. Are yo directly downloading to your local or is there any other solution?\n\nThanks in advance.", "votes": 5 }, { "id": 1388057, "postDate": "2021-07-14T16:07:42.510Z", "content": "An other approach would be to make tfrecords out of data and use kaggle-tpu
\nHow to make tfrecords:
\nhttps://www.kaggle.com/ryanholbrook/walkthrough-building-a-dataset-of-tfrecords
\nData converted to png has been already uploaded:
\nhttps://www.kaggle.com/jonathanbesomi/rsna-miccai-png
\nA starter notebook how to work with kaggle-tpu can be found here:
\nhttps://www.kaggle.com/philculliton/a-simple-petals-tf-2-2-notebook
", "rawMarkdown": "An other approach would be to make tfrecords out of data and use kaggle-tpu\n\nHow to make tfrecords:\n\nhttps://www.kaggle.com/ryanholbrook/walkthrough-building-a-dataset-of-tfrecords\n\nData converted to png has been already uploaded:\n\nhttps://www.kaggle.com/jonathanbesomi/rsna-miccai-png\n\n\nA starter notebook how to work with kaggle-tpu can be found here:\n\nhttps://www.kaggle.com/philculliton/a-simple-petals-tf-2-2-notebook", "votes": 3, "replies": [ { "id": 1388064, "postDate": "2021-07-14T16:13:45.940Z", "content": "Great references thank you @lucamtb
", "rawMarkdown": "Great references thank you @lucamtb " }, { "id": 1388164, "postDate": "2021-07-14T17:32:46.363Z", "content": "Thanks @lucamtb , That's very helpful!
", "rawMarkdown": "Thanks @lucamtb , That's very helpful!" } ] }, { "id": 1387406, "postDate": "2021-07-14T06:47:33.513Z", "content": "you can work right in Kaggle kernels or pipeline the data into google colab.
\npeople who had high internet speed, more storage, and available GPU in their local system will download this huge data.
Yeah, working directly with Kernels are very logical. Thanks!
", "rawMarkdown": "Yeah, working directly with Kernels are very logical. Thanks!", "votes": 1 } ] }, { "id": 1387547, "postDate": "2021-07-14T09:07:07.583Z", "content": "In case you want to run in local or AWS servers. It's highly recommended to use Kaggle API. Download locally and be a killer on WiFi. The speed on internet servers is very high and should be seamless with Kaggle CLI
", "rawMarkdown": "In case you want to run in local or AWS servers. It's highly recommended to use [Kaggle API](https://github.com/Kaggle/kaggle-api). Download locally and be a killer on WiFi. The speed on internet servers is very high and should be seamless with Kaggle CLI", "votes": 2, "replies": [ { "id": 1388013, "postDate": "2021-07-14T15:33:16.667Z", "content": "Interesting! Thanks for referencing this, I actually got the chance to read up on some of the things that can be accomplished using the API (https://www.kaggle.com/docs/api)…. right now I may stick with the Kaggle Kernels, but I may find myself coming back to the API approach in the future. Thanks for sharing the github link as well!!
\nAlso curious, what has been the most useful aspects of using the API for you, if you currently use it!
", "rawMarkdown": "Interesting! Thanks for referencing this, I actually got the chance to read up on some of the things that can be accomplished using the API (https://www.kaggle.com/docs/api).... right now I may stick with the Kaggle Kernels, but I may find myself coming back to the API approach in the future. Thanks for sharing the github link as well!!\n\nAlso curious, what has been the most useful aspects of using the API for you, if you currently use it!", "votes": 1 }, { "id": 1389333, "postDate": "2021-07-15T16:09:52.490Z", "content": "Glad to help!
\nI use it also to submit submission.csv through API. Sometimes its easy to do via CLI and get scores rather than uploading. Also, I use to get data majority of times
This is a code competition you must run the code in Kaggle kernel and submit entire kernel.
", "rawMarkdown": " This is a code competition you must run the code in Kaggle kernel and submit entire kernel." }, { "id": 1389328, "postDate": "2021-07-15T16:04:22.203Z", "content": "You can train your model on your local machine. But the submission must be done in a kaggle kernel. People here normally use a notebook to train (on kaggle or local) and one notebook to submit.
", "rawMarkdown": "You can train your model on your local machine. But the submission must be done in a kaggle kernel. People here normally use a notebook to train (on kaggle or local) and one notebook to submit. " }, { "id": 1400745, "postDate": "2021-07-26T14:49:42.697Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 1388057, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-07-14T16:07:42.510000", "content": "An other approach would be to make tfrecords out of data and use kaggle-tpu
\nHow to make tfrecords:
\nhttps://www.kaggle.com/ryanholbrook/walkthrough-building-a-dataset-of-tfrecords
\nData converted to png has been already uploaded:
\nhttps://www.kaggle.com/jonathanbesomi/rsna-miccai-png
\nA starter notebook how to work with kaggle-tpu can be found here:
\nhttps://www.kaggle.com/philculliton/a-simple-petals-tf-2-2-notebook
", "votes": 3, "replies": [ { "id": 1388064, "author_name": "tharun_01", "author_url": "", "post_date": "2021-07-14T16:13:45.940000", "content": "Great references thank you @lucamtb
", "votes": 0, "replies": [] }, { "id": 1388164, "author_name": "Berkay Alan", "author_url": "", "post_date": "2021-07-14T17:32:46.363000", "content": "Thanks @lucamtb , That's very helpful!
", "votes": 0, "replies": [] } ] }, { "id": 1387406, "author_name": "tharun_01", "author_url": "", "post_date": "2021-07-14T06:47:33.513000", "content": "you can work right in Kaggle kernels or pipeline the data into google colab.
\npeople who had high internet speed, more storage, and available GPU in their local system will download this huge data.
Yeah, working directly with Kernels are very logical. Thanks!
", "votes": 1, "replies": [] } ] }, { "id": 1387547, "author_name": "Shahebaz Mohammad", "author_url": "", "post_date": "2021-07-14T09:07:07.583000", "content": "In case you want to run in local or AWS servers. It's highly recommended to use Kaggle API. Download locally and be a killer on WiFi. The speed on internet servers is very high and should be seamless with Kaggle CLI
", "votes": 2, "replies": [ { "id": 1388013, "author_name": "ElenaEB", "author_url": "", "post_date": "2021-07-14T15:33:16.667000", "content": "Interesting! Thanks for referencing this, I actually got the chance to read up on some of the things that can be accomplished using the API (https://www.kaggle.com/docs/api)…. right now I may stick with the Kaggle Kernels, but I may find myself coming back to the API approach in the future. Thanks for sharing the github link as well!!
\nAlso curious, what has been the most useful aspects of using the API for you, if you currently use it!
", "votes": 1, "replies": [] }, { "id": 1389333, "author_name": "Shahebaz Mohammad", "author_url": "", "post_date": "2021-07-15T16:09:52.490000", "content": "Glad to help!
\nI use it also to submit submission.csv through API. Sometimes its easy to do via CLI and get scores rather than uploading. Also, I use to get data majority of times
This is a code competition you must run the code in Kaggle kernel and submit entire kernel.
", "votes": 0, "replies": [] }, { "id": 1389328, "author_name": "LucaMTB", "author_url": "", "post_date": "2021-07-15T16:04:22.203000", "content": "You can train your model on your local machine. But the submission must be done in a kaggle kernel. People here normally use a notebook to train (on kaggle or local) and one notebook to submit.
", "votes": 0, "replies": [] }, { "id": 1400745, "author_name": "", "author_url": "", "post_date": "2021-07-26T14:49:42.697000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1387366": "Hi all,\n\nI would like to ask how to download the data. It's so big that bigger than 120 GB. Are yo directly downloading to your local or is there any other solution?\n\nThanks in advance.", "1388057": "An other approach would be to make tfrecords out of data and use kaggle-tpu\n\nHow to make tfrecords:\n\nhttps://www.kaggle.com/ryanholbrook/walkthrough-building-a-dataset-of-tfrecords\n\nData converted to png has been already uploaded:\n\nhttps://www.kaggle.com/jonathanbesomi/rsna-miccai-png\n\n\nA starter notebook how to work with kaggle-tpu can be found here:\n\nhttps://www.kaggle.com/philculliton/a-simple-petals-tf-2-2-notebook", "1387406": "you can work right in Kaggle kernels or pipeline the data into google colab.\npeople who had high internet speed, more storage, and available GPU in their local system will download this huge data.", "1387547": "In case you want to run in local or AWS servers. It's highly recommended to use [Kaggle API](https://github.com/Kaggle/kaggle-api). Download locally and be a killer on WiFi. The speed on internet servers is very high and should be seamless with Kaggle CLI", "1388419": "" } }