{ "id": 269488, "title": "How to download dataset on google colaboratory", "url": "/competitions/landmark-recognition-2021/discussion/269488", "author_name": "motono0223", "post_date": "2021-08-31T22:20:53.831000", "votes": 3, "comment_count": 6, "views": 0, "content": "

**How does everyone download the dataset for goole colaboratory?
\n**

\n

I tried to download dataset on google colaboratory with command below.

\n

[download command]: kaggle competitions download -c landmark-recognition-2021

\n

This command didn't work well.
\nI expected that the zip files of dataset (train.zip, test.zip, …) were downloaded.
\nActually the jpg files had been downloaded and the timeout error occurred.

\n

[result of command]:

\n
\n

Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.12 / client 1.5.4)
\n Downloading 6660a970eb0c00f4.jpg to /input/test
\n 0% 0.00/32.2k [00:00<?, ?B/s]
\n 100% 32.2k/32.2k [00:00<00:00, 50.5MB/s]
\n 429 - Too Many Requests

\n
", "messages": [ { "id": 1498347, "postDate": "2021-08-31T22:20:53.830Z", "content": "

**How does everyone download the dataset for goole colaboratory?
\n**

\n

I tried to download dataset on google colaboratory with command below.

\n

[download command]: kaggle competitions download -c landmark-recognition-2021

\n

This command didn't work well.
\nI expected that the zip files of dataset (train.zip, test.zip, …) were downloaded.
\nActually the jpg files had been downloaded and the timeout error occurred.

\n

[result of command]:

\n
\n

Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.12 / client 1.5.4)
\n Downloading 6660a970eb0c00f4.jpg to /input/test
\n 0% 0.00/32.2k [00:00<?, ?B/s]
\n 100% 32.2k/32.2k [00:00<00:00, 50.5MB/s]
\n 429 - Too Many Requests

\n
", "rawMarkdown": "**How does everyone download the dataset for goole colaboratory?\n**\n\nI tried to download dataset on google colaboratory with command below.\n\n[download command]: kaggle competitions download -c landmark-recognition-2021\n\nThis command didn't work well.\nI expected that the zip files of dataset (train.zip, test.zip, ...) were downloaded.\nActually the jpg files had been downloaded and the timeout error occurred.\n\n[result of command]: \n> Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.12 / client 1.5.4)\n> Downloading 6660a970eb0c00f4.jpg to /input/test\n> 0% 0.00/32.2k [00:00 100% 32.2k/32.2k [00:00<00:00, 50.5MB/s]\n> 429 - Too Many Requests\n", "votes": 3 }, { "id": 1500821, "postDate": "2021-09-02T16:49:36.817Z", "content": "
\n

The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.

\n
\n

Oh, thank you for important information.
\nDo you train the model on the kaggle notebook (not google colab)?
\nI agree that the dataset does not be downloaded if you use the TPU of kaggle notebook.

\n

And I faced another problem that the session wad disconnected after about 20minutes later since my last operation on kaggle notebook, in the case of the notebook took a long time (such as 2hours or more) to train model.
\nDo you know any tips to avoid it?
\n(example: using the script that periodically opens the kaggle notebook url.)

", "rawMarkdown": "> The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.\n\nOh, thank you for important information.\nDo you train the model on the kaggle notebook (not google colab)? \nI agree that the dataset does not be downloaded if you use the TPU of kaggle notebook.\n\nAnd I faced another problem that the session wad disconnected after about 20minutes later since my last operation on kaggle notebook, in the case of the notebook took a long time (such as 2hours or more) to train model.\nDo you know any tips to avoid it?\n(example: using the script that periodically opens the kaggle notebook url.)", "replies": [ { "id": 1501430, "postDate": "2021-09-03T08:03:07.550Z", "content": "

Just save the notebook (click \"Save Version\" at the edit mode), kaggle will run this notebook in the background(up to 9 hours), and the training won't be terminated even if the browser is closed.

\n

This post may help you: https://www.kaggle.com/c/landmark-retrieval-2021/discussion/269420

", "rawMarkdown": "Just save the notebook (click \"Save Version\" at the edit mode), kaggle will run this notebook in the background(up to 9 hours), and the training won't be terminated even if the browser is closed.\n\nThis post may help you: https://www.kaggle.com/c/landmark-retrieval-2021/discussion/269420", "votes": 2 } ] }, { "id": 1500445, "postDate": "2021-09-02T11:39:43.803Z", "content": "

I have also encountered the same problem. And even if you successfully put the data set on the Colab, it is difficult to use a single GPU to complete a training epoch within 12 hours. The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.

", "rawMarkdown": "I have also encountered the same problem. And even if you successfully put the data set on the Colab, it is difficult to use a single GPU to complete a training epoch within 12 hours. The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set." }, { "id": 1500964, "postDate": "2021-09-02T19:27:25.260Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1500963, "postDate": "2021-09-02T19:22:05.200Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1499645, "postDate": "2021-09-01T19:45:47.223Z", "rawMarkdown": "", "votes": 3, "isDeleted": true } ], "comments": [ { "id": 1500821, "author_name": "motono0223", "author_url": "", "post_date": "2021-09-02T16:49:36.817000", "content": "
\n

The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.

\n
\n

Oh, thank you for important information.
\nDo you train the model on the kaggle notebook (not google colab)?
\nI agree that the dataset does not be downloaded if you use the TPU of kaggle notebook.

\n

And I faced another problem that the session wad disconnected after about 20minutes later since my last operation on kaggle notebook, in the case of the notebook took a long time (such as 2hours or more) to train model.
\nDo you know any tips to avoid it?
\n(example: using the script that periodically opens the kaggle notebook url.)

", "votes": 0, "replies": [ { "id": 1501430, "author_name": "Qiao Su", "author_url": "", "post_date": "2021-09-03T08:03:07.550000", "content": "

Just save the notebook (click \"Save Version\" at the edit mode), kaggle will run this notebook in the background(up to 9 hours), and the training won't be terminated even if the browser is closed.

\n

This post may help you: https://www.kaggle.com/c/landmark-retrieval-2021/discussion/269420

", "votes": 2, "replies": [] } ] }, { "id": 1500445, "author_name": "Qiao Su", "author_url": "", "post_date": "2021-09-02T11:39:43.803000", "content": "

I have also encountered the same problem. And even if you successfully put the data set on the Colab, it is difficult to use a single GPU to complete a training epoch within 12 hours. The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.

", "votes": 0, "replies": [] }, { "id": 1500964, "author_name": "", "author_url": "", "post_date": "2021-09-02T19:27:25.260000", "content": "", "votes": 0, "replies": [] }, { "id": 1500963, "author_name": "", "author_url": "", "post_date": "2021-09-02T19:22:05.200000", "content": "", "votes": 0, "replies": [] }, { "id": 1499645, "author_name": "", "author_url": "", "post_date": "2021-09-01T19:45:47.223000", "content": "", "votes": 3, "replies": [] } ], "raw_markdown_by_id": { "1498347": "**How does everyone download the dataset for goole colaboratory?\n**\n\nI tried to download dataset on google colaboratory with command below.\n\n[download command]: kaggle competitions download -c landmark-recognition-2021\n\nThis command didn't work well.\nI expected that the zip files of dataset (train.zip, test.zip, ...) were downloaded.\nActually the jpg files had been downloaded and the timeout error occurred.\n\n[result of command]: \n> Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.12 / client 1.5.4)\n> Downloading 6660a970eb0c00f4.jpg to /input/test\n> 0% 0.00/32.2k [00:00 100% 32.2k/32.2k [00:00<00:00, 50.5MB/s]\n> 429 - Too Many Requests\n", "1500821": "> The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.\n\nOh, thank you for important information.\nDo you train the model on the kaggle notebook (not google colab)? \nI agree that the dataset does not be downloaded if you use the TPU of kaggle notebook.\n\nAnd I faced another problem that the session wad disconnected after about 20minutes later since my last operation on kaggle notebook, in the case of the notebook took a long time (such as 2hours or more) to train model.\nDo you know any tips to avoid it?\n(example: using the script that periodically opens the kaggle notebook url.)", "1500445": "I have also encountered the same problem. And even if you successfully put the data set on the Colab, it is difficult to use a single GPU to complete a training epoch within 12 hours. The only solution I found is to use TPU training because it is faster than GPU, and does not need to download the data set.", "1500964": "", "1500963": "", "1499645": "" } }