Does anyone know how can I install the kaggle_datasets module in the Google Colab? I faced this problem trying to run this notebook by @cdeotte in the Google Colab. There is just no such module in the colab environment. I've successfully installed the Kaggle module itself but I still can't use the kaggle_datasets module.
\n\nHere is the import string of code which causes an error:\nfrom kaggle_datasets import KaggleDatasets
You need to run kaggle datasets here on local notebook, print the variable it'll be something like this:
\n'gs://kds-xxxxxxxxxxxxxxxxxxxxxx' x=some numbers
Copy it and paste it on colab where you used KaggleDatasets().get_gcs_path on kaggle. Instead of loader command you have the GCS bucket address as string.
To the best of my knowledge, you cannot install this module outside of Kaggle. So, the workaround is this: run KaggleDatasets on Kaggle to learn the GCS bucket addresses of the datasets of your interest and then use these addresses on Colab. You can store them in some kind of dictionary: GCS['train']='...', GCS['test']='...'. Also, you will need to update these addresses periodically (once in 5-7 days) because they change.
Does anyone know how can I install the kaggle_datasets module in the Google Colab? I faced this problem trying to run this notebook by @cdeotte in the Google Colab. There is just no such module in the colab environment. I've successfully installed the Kaggle module itself but I still can't use the kaggle_datasets module.
\n\nHere is the import string of code which causes an error:\nfrom kaggle_datasets import KaggleDatasets
Right, there is no official Kaggle support for this. Despite this, I have developed a solution that works for both Kaggle and Colab - described here: here. (required kaggle API key connected with your account)
", "rawMarkdown": "Right, there is no official Kaggle support for this. Despite this, I have developed a solution that works for both Kaggle and Colab - described here: [here](https://www.kaggle.com/wrrosa/gcs-bucket-addresses-utility-package). (required kaggle API key connected with your account)" } ], "comments": [ { "id": 963170, "author_name": "Ertuğrul Demir", "author_url": "", "post_date": "2020-08-08T18:11:27.377000", "content": "You need to run kaggle datasets here on local notebook, print the variable it'll be something like this:
\n'gs://kds-xxxxxxxxxxxxxxxxxxxxxx' x=some numbers
Copy it and paste it on colab where you used KaggleDatasets().get_gcs_path on kaggle. Instead of loader command you have the GCS bucket address as string.
To the best of my knowledge, you cannot install this module outside of Kaggle. So, the workaround is this: run KaggleDatasets on Kaggle to learn the GCS bucket addresses of the datasets of your interest and then use these addresses on Colab. You can store them in some kind of dictionary: GCS['train']='...', GCS['test']='...'. Also, you will need to update these addresses periodically (once in 5-7 days) because they change.
Right, there is no official Kaggle support for this. Despite this, I have developed a solution that works for both Kaggle and Colab - described here: here. (required kaggle API key connected with your account)
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "963170": "You need to run kaggle datasets here on local notebook, print the variable it'll be something like this:\n\n`'gs://kds-xxxxxxxxxxxxxxxxxxxxxx'` x=some numbers\n\nCopy it and paste it on colab where you used `KaggleDatasets().get_gcs_path` on kaggle. Instead of loader command you have the GCS bucket address as string.", "963154": "To the best of my knowledge, you cannot install this module outside of Kaggle. So, the workaround is this: run `KaggleDatasets` on Kaggle to learn the GCS bucket addresses of the datasets of your interest and then use these addresses on Colab. You can store them in some kind of dictionary: `GCS['train']='...'`, `GCS['test']='...'`. Also, you will need to update these addresses periodically (once in 5-7 days) because they change.", "963054": "Does anyone know how can I install the kaggle\\_datasets module in the Google Colab? I faced this problem trying to run [this notebook](https://www.kaggle.com/cdeotte/triple-stratified-kfold-with-tfrecords) by @cdeotte in the Google Colab. There is just no such module in the colab environment. I've successfully installed the Kaggle module itself but I still can't use the kaggle_datasets module.\n\nHere is the import string of code which causes an error:\n`from kaggle_datasets import KaggleDatasets`\n", "963242": "Right, there is no official Kaggle support for this. Despite this, I have developed a solution that works for both Kaggle and Colab - described here: [here](https://www.kaggle.com/wrrosa/gcs-bucket-addresses-utility-package). (required kaggle API key connected with your account)" } }