Hi, could anyone tell how are they handling the large dataset provided here.\n1) Local execution does not work, since data size>available hard-disk space\n2) Tried Using Google Drive, and mounting on Google Collaboratory , but results in Input/Output error. \nCan anyone provide appropriately sized datasets to work here ?
", "messages": [ { "id": "550931", "postDate": "06/12/2019 06:33:43", "content": "Hi, could anyone tell how are they handling the large dataset provided here.\n1) Local execution does not work, since data size>available hard-disk space\n2) Tried Using Google Drive, and mounting on Google Collaboratory , but results in Input/Output error. \nCan anyone provide appropriately sized datasets to work here ?
", "rawMarkdown": "Hi, could anyone tell how are they handling the large dataset provided here.\n1) Local execution does not work, since data size>available hard-disk space\n2) Tried Using Google Drive, and mounting on Google Collaboratory , but results in Input/Output error. \nCan anyone provide appropriately sized datasets to work here ?", "votes": null }, { "id": "550942", "postDate": "06/12/2019 06:46:38", "content": "Meeting the same problems :(
", "rawMarkdown": "Meeting the same problems :(", "votes": null }, { "id": "551029", "postDate": "06/12/2019 09:05:42", "content": "No need to download entire dataset inorder to participate in the competition. You can probably download any one subset of training data and train your model on that. That would be possible on colab and google drive.
", "rawMarkdown": "No need to download entire dataset inorder to participate in the competition. You can probably download any one subset of training data and train your model on that. That would be possible on colab and google drive.", "votes": null }, { "id": "551160", "postDate": "06/12/2019 12:06:56", "content": "I put up part of the dataset on Kaggle that was resized. You should be able to use that to download into Google Collaboratory if you don't want to use Kaggle.
", "rawMarkdown": "I put up part of the dataset on [Kaggle that was resized](https://www.kaggle.com/c/open-images-2019-object-detection/discussion/94770#latest-550992). You should be able to use that to download into Google Collaboratory if you don't want to use Kaggle.", "votes": null }, { "id": "558875", "postDate": "06/23/2019 04:48:19", "content": "Even a single subset is of 58GB
", "rawMarkdown": "Even a single subset is of 58GB", "votes": null }, { "id": "558930", "postDate": "06/23/2019 07:32:05", "content": "You can use this dataset which is resized to 256*256 and takes only 10 GB of space
\n\nThanks to @mindtrinket
", "rawMarkdown": "You can use [this dataset](https://www.kaggle.com/mindtrinket/google-2019-30k-train/) which is resized to 256*256 and takes only 10 GB of space\n\nThanks to @mindtrinket", "votes": null }, { "id": "559471", "postDate": "06/24/2019 07:00:58", "content": "hey use the reduced data set or download the data set using the wget method.
", "rawMarkdown": "hey use the reduced data set or download the data set using the wget method.", "votes": null } ], "comments": [ { "id": 550942, "author_name": "henryham", "author_url": "", "post_date": "06/12/2019 06:46:38", "content": "Meeting the same problems :(
", "votes": null, "replies": [] }, { "id": 551029, "author_name": "chandanverma", "author_url": "", "post_date": "06/12/2019 09:05:42", "content": "No need to download entire dataset inorder to participate in the competition. You can probably download any one subset of training data and train your model on that. That would be possible on colab and google drive.
", "votes": null, "replies": [ { "id": 558875, "author_name": "vikashpathak", "author_url": "", "post_date": "06/23/2019 04:48:19", "content": "Even a single subset is of 58GB
", "votes": null, "replies": [] }, { "id": 558930, "author_name": "chandanverma", "author_url": "", "post_date": "06/23/2019 07:32:05", "content": "You can use this dataset which is resized to 256*256 and takes only 10 GB of space
\n\nThanks to @mindtrinket
", "votes": null, "replies": [] } ] }, { "id": 551160, "author_name": "mindtrinket", "author_url": "", "post_date": "06/12/2019 12:06:56", "content": "I put up part of the dataset on Kaggle that was resized. You should be able to use that to download into Google Collaboratory if you don't want to use Kaggle.
", "votes": null, "replies": [] }, { "id": 559471, "author_name": "krishnakatyal", "author_url": "", "post_date": "06/24/2019 07:00:58", "content": "hey use the reduced data set or download the data set using the wget method.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "550931": "Hi, could anyone tell how are they handling the large dataset provided here.\n1) Local execution does not work, since data size>available hard-disk space\n2) Tried Using Google Drive, and mounting on Google Collaboratory , but results in Input/Output error. \nCan anyone provide appropriately sized datasets to work here ?", "550942": "Meeting the same problems :(", "551029": "No need to download entire dataset inorder to participate in the competition. You can probably download any one subset of training data and train your model on that. That would be possible on colab and google drive.", "551160": "I put up part of the dataset on [Kaggle that was resized](https://www.kaggle.com/c/open-images-2019-object-detection/discussion/94770#latest-550992). You should be able to use that to download into Google Collaboratory if you don't want to use Kaggle.", "558875": "Even a single subset is of 58GB", "558930": "You can use [this dataset](https://www.kaggle.com/mindtrinket/google-2019-30k-train/) which is resized to 256*256 and takes only 10 GB of space\n\nThanks to @mindtrinket", "559471": "hey use the reduced data set or download the data set using the wget method." }, "source": "meta" }