Downloading from kaggle.com is very slow. So, I try API but only download part of this dataset.
\nCan somebody help?
This code will allow you to quickly download all the data directly from Kaggle to your Colab notebook. Please run each block of code in a separate cell in the same notebook where you will be training your model. This method uses fuze-zip to save space. It will allow you to access the data without unzipping it.
\n# Uninstall and reinstall kaggle\n!pip uninstall -y kaggle\n!pip install kaggle\n# Set environment variables for using the Kaggle API.\n\n# Go to Kaggle.\n# Click on account.\n# Click the 'Create New API Token' button.\n# A .json file will download.\n# Open the file with a text editor and\n# copy and paste your username and key from that file below.\n\nimport os\n\nos.environ[\"KAGGLE_USERNAME\"] = 'your_kaggle_username'\nos.environ[\"KAGGLE_KEY\"] = 'your_kaggle_api_key'\n# Download the data from the kaggle competition\n\nraw_data_dir = \"input/raw\"\n\n!kaggle competitions download -c cassava-leaf-disease-classification -p {raw_data_dir}\n# Check that the zip file was downloaded\n!ls {raw_data_dir}\n# Fuze-zip allows us to access zipped folders without\n# first unzipping them. This is helpful when working\n# with large files.\n\n!apt-get install -y fuse-zip\n# Apply fuze-zip to the data\n\ninput_dir = \"/tmp/kaggle-data\"\n!mkdir {input_dir}\n!fuse-zip input/raw/cassava-leaf-disease-classification.zip {input_dir}\n# Check that everything is working.\n!ls {input_dir}\n# Use this path to access the downloaded folders\n\nbase_path = '/tmp/kaggle-data/'\n\nos.listdir(base_path)\nThanks bro. This finally really helped!!!!
\nThanks a lot
Thank you, I am a novice, please tell me how can I view my 'your_kaggle_api_key' .
", "rawMarkdown": "Thank you, I am a novice, please tell me how can I view my 'your_kaggle_api_key' ." }, { "id": 1092860, "postDate": "2020-11-27T08:22:08.627Z", "content": "@crawler3, I hope this will help you. API credentials
", "rawMarkdown": "@crawler3, I hope this will help you. [API credentials](https://github.com/Kaggle/kaggle-api#api-credentials)", "replies": [ { "id": 1092886, "postDate": "2020-11-27T08:39:26.397Z", "content": "Thank you, it helped me a lot。
", "rawMarkdown": "Thank you, it helped me a lot。", "votes": 1 } ] }, { "id": 1109724, "postDate": "2020-12-12T02:17:54.943Z", "content": "Thank you very much
", "rawMarkdown": "Thank you very much", "votes": 1 } ] }, { "id": 1085020, "postDate": "2020-11-20T15:52:58.563Z", "content": "Downloading from kaggle.com is very slow. So, I try API but only download part of this dataset.
\nCan somebody help?
great !!!!!!👍
", "rawMarkdown": "great !!!!!!👍", "votes": 1 }, { "id": 1120104, "postDate": "2020-12-20T15:16:33.293Z", "content": "Hi Marsh, you know what?
\nYou are awesome!
Hi,
\nI am also facing some issues with the kaggle api and I have opened a different discussion thread. Can you look into that and share whether you are having the same issue or not?
\nThanks.
[Issue] Downloading data Using kaggle-api\n
", "rawMarkdown": "Hi,\nI am also facing some issues with the kaggle api and I have opened a different discussion thread. Can you look into that and share whether you are having the same issue or not?\nThanks.\n\n[[Issue] Downloading data Using kaggle-api\n](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198594)" }, { "id": 1085588, "postDate": "2020-11-21T02:21:20.477Z", "content": "You can download it to your Drive, then mount Drive everytime you use it.
", "rawMarkdown": "You can download it to your Drive, then mount Drive everytime you use it." }, { "id": 1087919, "postDate": "2020-11-23T06:53:52.217Z", "rawMarkdown": "", "votes": -1, "isDeleted": true } ], "comments": [ { "id": 1085623, "author_name": "vbookshelf", "author_url": "", "post_date": "2020-11-21T03:11:53.657000", "content": "This code will allow you to quickly download all the data directly from Kaggle to your Colab notebook. Please run each block of code in a separate cell in the same notebook where you will be training your model. This method uses fuze-zip to save space. It will allow you to access the data without unzipping it.
\n# Uninstall and reinstall kaggle\n!pip uninstall -y kaggle\n!pip install kaggle\n# Set environment variables for using the Kaggle API.\n\n# Go to Kaggle.\n# Click on account.\n# Click the 'Create New API Token' button.\n# A .json file will download.\n# Open the file with a text editor and\n# copy and paste your username and key from that file below.\n\nimport os\n\nos.environ[\"KAGGLE_USERNAME\"] = 'your_kaggle_username'\nos.environ[\"KAGGLE_KEY\"] = 'your_kaggle_api_key'\n# Download the data from the kaggle competition\n\nraw_data_dir = \"input/raw\"\n\n!kaggle competitions download -c cassava-leaf-disease-classification -p {raw_data_dir}\n# Check that the zip file was downloaded\n!ls {raw_data_dir}\n# Fuze-zip allows us to access zipped folders without\n# first unzipping them. This is helpful when working\n# with large files.\n\n!apt-get install -y fuse-zip\n# Apply fuze-zip to the data\n\ninput_dir = \"/tmp/kaggle-data\"\n!mkdir {input_dir}\n!fuse-zip input/raw/cassava-leaf-disease-classification.zip {input_dir}\n# Check that everything is working.\n!ls {input_dir}\n# Use this path to access the downloaded folders\n\nbase_path = '/tmp/kaggle-data/'\n\nos.listdir(base_path)\nThanks bro. This finally really helped!!!!
\nThanks a lot
Thank you, I am a novice, please tell me how can I view my 'your_kaggle_api_key' .
", "votes": 0, "replies": [] }, { "id": 1092860, "author_name": "jabertuhin", "author_url": "", "post_date": "2020-11-27T08:22:08.627000", "content": "@crawler3, I hope this will help you. API credentials
", "votes": 0, "replies": [ { "id": 1092886, "author_name": "Crawler", "author_url": "", "post_date": "2020-11-27T08:39:26.397000", "content": "Thank you, it helped me a lot。
", "votes": 1, "replies": [] } ] }, { "id": 1109724, "author_name": "Benja Jorquera", "author_url": "", "post_date": "2020-12-12T02:17:54.943000", "content": "Thank you very much
", "votes": 1, "replies": [] } ] }, { "id": 1120171, "author_name": "dnalexen", "author_url": "", "post_date": "2020-12-20T16:09:07.947000", "content": "great !!!!!!👍
", "votes": 1, "replies": [] }, { "id": 1120104, "author_name": "Freidoun", "author_url": "", "post_date": "2020-12-20T15:16:33.293000", "content": "Hi Marsh, you know what?
\nYou are awesome!
Hi,
\nI am also facing some issues with the kaggle api and I have opened a different discussion thread. Can you look into that and share whether you are having the same issue or not?
\nThanks.
[Issue] Downloading data Using kaggle-api\n
", "votes": 0, "replies": [] }, { "id": 1085588, "author_name": "Long Luu", "author_url": "", "post_date": "2020-11-21T02:21:20.477000", "content": "You can download it to your Drive, then mount Drive everytime you use it.
", "votes": 0, "replies": [] }, { "id": 1087919, "author_name": "", "author_url": "", "post_date": "2020-11-23T06:53:52.217000", "content": "", "votes": -1, "replies": [] } ], "raw_markdown_by_id": { "1085623": "This code will allow you to quickly download all the data directly from Kaggle to your Colab notebook. Please run each block of code in a separate cell in the same notebook where you will be training your model. This method uses fuze-zip to save space. It will allow you to access the data without unzipping it.\n\n```\n# Uninstall and reinstall kaggle\n!pip uninstall -y kaggle\n!pip install kaggle\n```\n\n\n```\n# Set environment variables for using the Kaggle API.\n\n# Go to Kaggle.\n# Click on account.\n# Click the 'Create New API Token' button.\n# A .json file will download.\n# Open the file with a text editor and\n# copy and paste your username and key from that file below.\n\nimport os\n\nos.environ[\"KAGGLE_USERNAME\"] = 'your_kaggle_username'\nos.environ[\"KAGGLE_KEY\"] = 'your_kaggle_api_key'\n```\n\n```\n# Download the data from the kaggle competition\n\nraw_data_dir = \"input/raw\"\n\n!kaggle competitions download -c cassava-leaf-disease-classification -p {raw_data_dir}\n```\n\n```\n# Check that the zip file was downloaded\n!ls {raw_data_dir}\n```\n\n\n```\n# Fuze-zip allows us to access zipped folders without\n# first unzipping them. This is helpful when working\n# with large files.\n\n!apt-get install -y fuse-zip\n```\n\n```\n# Apply fuze-zip to the data\n\ninput_dir = \"/tmp/kaggle-data\"\n!mkdir {input_dir}\n!fuse-zip input/raw/cassava-leaf-disease-classification.zip {input_dir}\n```\n\n```\n# Check that everything is working.\n!ls {input_dir}\n```\n\n```\n# Use this path to access the downloaded folders\n\nbase_path = '/tmp/kaggle-data/'\n\nos.listdir(base_path)\n```", "1085020": "Downloading from kaggle.com is very slow. So, I try API but only download part of this dataset.\nCan somebody help?", "1120171": "great !!!!!!👍", "1120104": "Hi Marsh, you know what?\nYou are awesome!", "1086726": "Hi,\nI am also facing some issues with the kaggle api and I have opened a different discussion thread. Can you look into that and share whether you are having the same issue or not?\nThanks.\n\n[[Issue] Downloading data Using kaggle-api\n](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198594)", "1085588": "You can download it to your Drive, then mount Drive everytime you use it.", "1087919": "" } }