I would like to download just the new data with Kaggle command line tool. I do not want to download the additional 70GB of old data. Does anyone know how to do this?
\nTo download a single file, we type
\nkaggle competitions download -f [FILE_NAME] -c [competition]\nHow do we download a folder of images?
", "messages": [ { "id": 1469361, "postDate": "2021-08-12T21:48:08.580Z", "content": "I would like to download just the new data with Kaggle command line tool. I do not want to download the additional 70GB of old data. Does anyone know how to do this?
\nTo download a single file, we type
\nkaggle competitions download -f [FILE_NAME] -c [competition]\nHow do we download a folder of images?
", "rawMarkdown": "I would like to download just the new data with Kaggle command line tool. I do not want to download the additional 70GB of old data. Does anyone know how to do this?\n\nTo download a single file, we type\n\n kaggle competitions download -f [FILE_NAME] -c [competition]\n\nHow do we download a folder of images?", "votes": 4 }, { "id": 1479727, "postDate": "2021-08-18T15:54:30.120Z", "content": "It's a late response.
\nI use wget [url].
\nTo get the url of a folder, just try to download the folder with any Browser(like Chrome), and you can get the url in \"download\" by right clicking it.
I downloaded train folder from competitions data page, then I uploaded the zip file to the server.
\nYou can select train folder, then click the download button, you can download only train folder.
Selecting \"train\" and \"test\" and downloading as zip was also the way i did it.
\nNow, if you are working with remote machines, that reset every time you run them up, you probably want an automatic download script for the data. Selecting a subset would save a lot of bandwidth, that is not always needed. Adding this functionality to the kaggle command line tool would be greatly appreciated. Hosting the already available data on an additional server doesn't really sound like an optimal solution.
Thanks. Kaggle should allow wildcards in their CLI like below (and package the files into single zip download):
\nkaggle competitions download -f train/*/* -c seti-breakthrough-listen\nin the same way that zip allows wildcards to avoid unzipping old data (if we are required to download both)
\nunzip seti-breakthrough-listen.zip -x old_leaky_data/*/*/*\nGreat concept!
\nIf you ask for such feature in product feedback, count me in for supporting the proposal.
I posted a feature request here. Everyone who thinks this would be helpful should upvote. Thanks
\nhttps://www.kaggle.com/product-feedback/264838
Upvoted @cdeotte !
", "rawMarkdown": "Upvoted @cdeotte !", "votes": 1 } ] }, { "id": 1469377, "postDate": "2021-08-12T22:18:26.770Z", "content": "What about this solution ?
", "rawMarkdown": "What about [this solution](https://www.kaggle.com/c/seti-breakthrough-listen/discussion/253912) ?", "votes": 2 } ], "comments": [ { "id": 1479727, "author_name": "Last Scene", "author_url": "", "post_date": "2021-08-18T15:54:30.120000", "content": "It's a late response.
\nI use wget [url].
\nTo get the url of a folder, just try to download the folder with any Browser(like Chrome), and you can get the url in \"download\" by right clicking it.
I downloaded train folder from competitions data page, then I uploaded the zip file to the server.
\nYou can select train folder, then click the download button, you can download only train folder.
Selecting \"train\" and \"test\" and downloading as zip was also the way i did it.
\nNow, if you are working with remote machines, that reset every time you run them up, you probably want an automatic download script for the data. Selecting a subset would save a lot of bandwidth, that is not always needed. Adding this functionality to the kaggle command line tool would be greatly appreciated. Hosting the already available data on an additional server doesn't really sound like an optimal solution.
Thanks. Kaggle should allow wildcards in their CLI like below (and package the files into single zip download):
\nkaggle competitions download -f train/*/* -c seti-breakthrough-listen\nin the same way that zip allows wildcards to avoid unzipping old data (if we are required to download both)
\nunzip seti-breakthrough-listen.zip -x old_leaky_data/*/*/*\nGreat concept!
\nIf you ask for such feature in product feedback, count me in for supporting the proposal.
I posted a feature request here. Everyone who thinks this would be helpful should upvote. Thanks
\nhttps://www.kaggle.com/product-feedback/264838
Upvoted @cdeotte !
", "votes": 1, "replies": [] } ] }, { "id": 1469377, "author_name": "kkiller", "author_url": "", "post_date": "2021-08-12T22:18:26.770000", "content": "What about this solution ?
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "1469361": "I would like to download just the new data with Kaggle command line tool. I do not want to download the additional 70GB of old data. Does anyone know how to do this?\n\nTo download a single file, we type\n\n kaggle competitions download -f [FILE_NAME] -c [competition]\n\nHow do we download a folder of images?", "1479727": "It's a late response.\nI use wget [url].\nTo get the url of a folder, just try to download the folder with any Browser(like Chrome), and you can get the url in \"download\" by right clicking it.", "1470054": "I downloaded train folder from competitions data page, then I uploaded the zip file to the server.\nYou can select train folder, then click the download button, you can download only train folder.", "1469377": "What about [this solution](https://www.kaggle.com/c/seti-breakthrough-listen/discussion/253912) ?" } }