As the title says, I wonder how do you download the train.zip and validate.zip data.\nI failed with wget, and that may have relation with china's gfw. What's more, if I just click the link, it may cost me a week to finish downloading the data. Of course, I am now in china.
", "messages": [ { "id": "101710", "postDate": "12/16/2015 16:23:37", "content": "As the title says, I wonder how do you download the train.zip and validate.zip data.\nI failed with wget, and that may have relation with china's gfw. What's more, if I just click the link, it may cost me a week to finish downloading the data. Of course, I am now in china.
", "rawMarkdown": "As the title says, I wonder how do you download the train.zip and validate.zip data.\r\nI failed with wget, and that may have relation with china's gfw. What's more, if I just click the link, it may cost me a week to finish downloading the data. Of course, I am now in china.", "votes": null }, { "id": "101731", "postDate": "12/16/2015 18:55:36", "content": "Hi, @dotcoming!
\n\nI downloaded with Google Chrome and "Free Download Manager" (open source), plus the Chrome plugin for FDM.
\n\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).
\n\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.
\n\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page').
\n\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.
\n\nFDM is able to resume the download if something goes wrong.
\n\nHope this help!
\n\nPS: sorry if I made mistakes with the English language! :-)
", "rawMarkdown": "Hi, @dotcoming!\r\n\r\nI downloaded with Google Chrome and \"Free Download Manager\" (open source), plus the Chrome plugin for FDM.\r\n\r\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).\r\n\r\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.\r\n\r\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page'). \r\n\r\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.\r\n\r\nFDM is able to resume the download if something goes wrong.\r\n\r\nHope this help!\r\n\r\nPS: sorry if I made mistakes with the English language! :-)", "votes": null }, { "id": "101752", "postDate": "12/16/2015 21:14:02", "content": "Hi,
\n\nDatasets download with Firefox basic download withouy any plugin. It worked well. \nHope it will help.
", "rawMarkdown": "Hi,\r\n\r\nDatasets download with Firefox basic download withouy any plugin. It worked well. \r\nHope it will help.", "votes": null }, { "id": "101761", "postDate": "12/16/2015 22:12:17", "content": "Make sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.
\n\n\n\n\nMax. file size 4,294,967,295 bytes (4 GB - 1)
\n
https://en.wikipedia.org/wiki/File_Allocation_Table
\n\n\n\n\nMax. file size 16 EiB
\n
https://en.wikipedia.org/wiki/NTFS
\n\nEdit:
\n\nWhy the downvote? I had exactly this problem.
", "rawMarkdown": "Make sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.\r\n\r\n> Max. file size \t4,294,967,295 bytes (4 GB - 1)\r\n\r\nhttps://en.wikipedia.org/wiki/File_Allocation_Table\r\n\r\n> Max. file size \t16 EiB\r\n\r\nhttps://en.wikipedia.org/wiki/NTFS\r\n\r\nEdit:\r\n\r\nWhy the downvote? I had exactly this problem.", "votes": null }, { "id": "101773", "postDate": "12/17/2015 01:41:39", "content": "[quote=Alexandre Gomiero;101731]
\n\nHi, @dotcoming!
\n\nI downloaded with Google Chrome and "Free Download Manager" (open source), plus the Chrome plugin for FDM.
\n\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).
\n\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.
\n\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page').
\n\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.
\n\nFDM is able to resume the download if something goes wrong.
\n\nHope this help!
\n\nPS: sorry if I made mistakes with the English language! :-)
\n\n[/quote]\nAlexandre Gomiero,
\n\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.
\n\nThanks again. :)
", "rawMarkdown": "[quote=Alexandre Gomiero;101731]\r\n\r\nHi, @dotcoming!\r\n\r\nI downloaded with Google Chrome and \"Free Download Manager\" (open source), plus the Chrome plugin for FDM.\r\n\r\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).\r\n\r\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.\r\n\r\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page'). \r\n\r\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.\r\n\r\nFDM is able to resume the download if something goes wrong.\r\n\r\nHope this help!\r\n\r\nPS: sorry if I made mistakes with the English language! :-)\r\n\r\n[/quote]\r\nAlexandre Gomiero,\r\n \r\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.\r\n\r\nThanks again. :)", "votes": null }, { "id": "101776", "postDate": "12/17/2015 01:44:30", "content": "[quote=Xavier Van Ausloos;101752]
\n\nHi,
\n\nDatasets download with Firefox basic download withouy any plugin. It worked well. \nHope it will help.
\n\n[/quote]\nThanks for your advice really.
", "rawMarkdown": "[quote=Xavier Van Ausloos;101752]\r\n\r\nHi,\r\n\r\nDatasets download with Firefox basic download withouy any plugin. It worked well. \r\nHope it will help. \r\n\r\n[/quote]\r\nThanks for your advice really.", "votes": null }, { "id": "101777", "postDate": "12/17/2015 01:47:50", "content": "[quote=h-wit;101761]
\n\nMake sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.
\n\n\n\n\nMax. file size 4,294,967,295 bytes (4 GB - 1)
\n
https://en.wikipedia.org/wiki/File_Allocation_Table
\n\n\n\n\nMax. file size 16 EiB
\n
https://en.wikipedia.org/wiki/NTFS
\n\nEdit:
\n\nWhy the downvote? I had exactly this problem.
\n\n[/quote]\nYes, you r right, that may be a problem. But my actual barrier is the slow speed network, now I solve it by using a software.
\n\nBy the way, thanks for your learning material "fat32 vs ntfs".
", "rawMarkdown": "[quote=h-wit;101761]\r\n\r\nMake sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.\r\n\r\n> Max. file size \t4,294,967,295 bytes (4 GB - 1)\r\n\r\nhttps://en.wikipedia.org/wiki/File_Allocation_Table\r\n\r\n> Max. file size \t16 EiB\r\n\r\nhttps://en.wikipedia.org/wiki/NTFS\r\n\r\nEdit:\r\n\r\nWhy the downvote? I had exactly this problem.\r\n\r\n[/quote]\r\nYes, you r right, that may be a problem. But my actual barrier is the slow speed network, now I solve it by using a software. \r\n\r\nBy the way, thanks for your learning material \"fat32 vs ntfs\".", "votes": null }, { "id": "102086", "postDate": "12/19/2015 09:58:26", "content": "[quote=dotcoming;101773]
\n\n[quote=Alexandre Gomiero;101731]
\n\nHi, @dotcoming!
\n\nI downloaded with Google Chrome and "Free Download Manager" (open source), plus the Chrome plugin for FDM.
\n\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).
\n\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.
\n\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page').
\n\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.
\n\nFDM is able to resume the download if something goes wrong.
\n\nHope this help!
\n\nPS: sorry if I made mistakes with the English language! :-)
\n\n[/quote]\nAlexandre Gomiero,
\n\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.
\n\nThanks again. :)
\n\n[/quote]
\n\nIt seems no help for me ，Can you share it via Baidu Yun to me， If the competition allow it
", "rawMarkdown": "[quote=dotcoming;101773]\r\n\r\n[quote=Alexandre Gomiero;101731]\r\n\r\nHi, @dotcoming!\r\n\r\nI downloaded with Google Chrome and \"Free Download Manager\" (open source), plus the Chrome plugin for FDM.\r\n\r\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).\r\n\r\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.\r\n\r\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page'). \r\n\r\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.\r\n\r\nFDM is able to resume the download if something goes wrong.\r\n\r\nHope this help!\r\n\r\nPS: sorry if I made mistakes with the English language! :-)\r\n\r\n[/quote]\r\nAlexandre Gomiero,\r\n \r\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.\r\n\r\nThanks again. :)\r\n\r\n[/quote]\r\n\r\nIt seems no help for me ，Can you share it via Baidu Yun to me， If the competition allow it", "votes": null }, { "id": "104564", "postDate": "01/13/2016 22:15:43", "content": "The option with Free Download Manager did not work for me. I am on Mac OS X El Capitan. I could not figure out how to change query size.
\n\nI found an extension for Firefox, called DownThemAll! This solved the problem, i was able to download data with the good speed.
\n\nHope this helps.
", "rawMarkdown": "The option with Free Download Manager did not work for me. I am on Mac OS X El Capitan. I could not figure out how to change query size.\r\n\r\nI found an extension for Firefox, called DownThemAll! This solved the problem, i was able to download data with the good speed.\r\n\r\nHope this helps.", "votes": null }, { "id": "104567", "postDate": "01/13/2016 22:40:47", "content": "I used DownThemAll too.
", "rawMarkdown": "I used DownThemAll too.", "votes": null }, { "id": "104576", "postDate": "01/14/2016 01:08:49", "content": "Can Kaggle provide some official S3 location for these data, since many people will use AWS GPU instance anyway?
\n\nBy the way, one can export cache from the browser, upload to AWS, and download from AWS using wget with this cache. It can reach the max speed. Details are at https://www.kaggle.com/forums/f/15/kaggle-forum/t/6604/downloading-data-via-command-line Plus some nice guy's github gist https://gist.github.com/rgbkrk/7340130
", "rawMarkdown": "Can Kaggle provide some official S3 location for these data, since many people will use AWS GPU instance anyway?\r\n\r\nBy the way, one can export cache from the browser, upload to AWS, and download from AWS using wget with this cache. It can reach the max speed. Details are at https://www.kaggle.com/forums/f/15/kaggle-forum/t/6604/downloading-data-via-command-line Plus some nice guy's github gist https://gist.github.com/rgbkrk/7340130", "votes": null }, { "id": "104611", "postDate": "01/14/2016 11:50:35", "content": "I am not able to find the DICOM Image Data set. Can anybody let me know from where we will download the DICOM files
", "rawMarkdown": "I am not able to find the DICOM Image Data set. Can anybody let me know from where we will download the DICOM files", "votes": null }, { "id": "104613", "postDate": "01/14/2016 13:29:51", "content": "The DICOM data is in train.zip and in validate.zip (download them here: https://www.kaggle.com/c/second-annual-data-science-bowl/data). \nUnzip them and you get all the folders with all the DICOM images.
", "rawMarkdown": "The DICOM data is in train.zip and in validate.zip (download them here: https://www.kaggle.com/c/second-annual-data-science-bowl/data). \r\nUnzip them and you get all the folders with all the DICOM images.", "votes": null }, { "id": "106322", "postDate": "01/29/2016 23:44:40", "content": "I am in Australia and it has been a struggle trying to download files of this size (!!!). Network timeouts are the main bug bear. Perhaps files should be split up into 2-3 GB size zip files. Even with a mac download manager (this took 2 attempts as the first product was unstable), I got zip file corruption errors, which has left me with a useless 13GB file on my HD.
\n\nI will run a windows 7 virtual machine and try Free DownLoad Manager - maybe its the Mac download managers.
", "rawMarkdown": "I am in Australia and it has been a struggle trying to download files of this size (!!!). Network timeouts are the main bug bear. Perhaps files should be split up into 2-3 GB size zip files. Even with a mac download manager (this took 2 attempts as the first product was unstable), I got zip file corruption errors, which has left me with a useless 13GB file on my HD.\r\n\r\nI will run a windows 7 virtual machine and try Free DownLoad Manager - maybe its the Mac download managers.", "votes": null }, { "id": "106335", "postDate": "01/30/2016 08:50:40", "content": "On Ubuntu I used wget with the auto resume option.
\n\nTo handle the authentication you need to login via your browser and then export your cookies, like here-> https://www.kaggle.com/c/belkin-energy-disaggregation-competition/forums/t/5118/downloading-data-via-wget/96790
", "rawMarkdown": "On Ubuntu I used wget with the auto resume option. \r\n\r\nTo handle the authentication you need to login via your browser and then export your cookies, like here-> https://www.kaggle.com/c/belkin-energy-disaggregation-competition/forums/t/5118/downloading-data-via-wget/96790", "votes": null }, { "id": "108806", "postDate": "02/20/2016 03:31:38", "content": "@Peter thank you for the tip. It worked perfectly. The one thing that tripped me up was that I needed to add sudo to my wget call. Otherwise, I would get a file not found error. Thanks again!
", "rawMarkdown": "Peter thank you for the tip. It worked perfectly. The one thing that tripped me up was that I needed to add sudo to my wget call. Otherwise, I would get a file not found error. Thanks again!", "votes": null } ], "comments": [ { "id": 101731, "author_name": "gomiero", "author_url": "", "post_date": "12/16/2015 18:55:36", "content": "Hi, @dotcoming!
\n\nI downloaded with Google Chrome and "Free Download Manager" (open source), plus the Chrome plugin for FDM.
\n\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).
\n\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.
\n\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page').
\n\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.
\n\nFDM is able to resume the download if something goes wrong.
\n\nHope this help!
\n\nPS: sorry if I made mistakes with the English language! :-)
", "votes": null, "replies": [] }, { "id": 101752, "author_name": "vanausloos", "author_url": "", "post_date": "12/16/2015 21:14:02", "content": "Hi,
\n\nDatasets download with Firefox basic download withouy any plugin. It worked well. \nHope it will help.
", "votes": null, "replies": [] }, { "id": 101761, "author_name": "halfwit", "author_url": "", "post_date": "12/16/2015 22:12:17", "content": "Make sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.
\n\n\n\n\nMax. file size 4,294,967,295 bytes (4 GB - 1)
\n
https://en.wikipedia.org/wiki/File_Allocation_Table
\n\n\n\n\nMax. file size 16 EiB
\n
https://en.wikipedia.org/wiki/NTFS
\n\nEdit:
\n\nWhy the downvote? I had exactly this problem.
", "votes": null, "replies": [] }, { "id": 101773, "author_name": "dotcoming", "author_url": "", "post_date": "12/17/2015 01:41:39", "content": "[quote=Alexandre Gomiero;101731]
\n\nHi, @dotcoming!
\n\nI downloaded with Google Chrome and "Free Download Manager" (open source), plus the Chrome plugin for FDM.
\n\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).
\n\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.
\n\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page').
\n\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.
\n\nFDM is able to resume the download if something goes wrong.
\n\nHope this help!
\n\nPS: sorry if I made mistakes with the English language! :-)
\n\n[/quote]\nAlexandre Gomiero,
\n\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.
\n\nThanks again. :)
", "votes": null, "replies": [] }, { "id": 101776, "author_name": "dotcoming", "author_url": "", "post_date": "12/17/2015 01:44:30", "content": "[quote=Xavier Van Ausloos;101752]
\n\nHi,
\n\nDatasets download with Firefox basic download withouy any plugin. It worked well. \nHope it will help.
\n\n[/quote]\nThanks for your advice really.
", "votes": null, "replies": [] }, { "id": 101777, "author_name": "dotcoming", "author_url": "", "post_date": "12/17/2015 01:47:50", "content": "[quote=h-wit;101761]
\n\nMake sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.
\n\n\n\n\nMax. file size 4,294,967,295 bytes (4 GB - 1)
\n
https://en.wikipedia.org/wiki/File_Allocation_Table
\n\n\n\n\nMax. file size 16 EiB
\n
https://en.wikipedia.org/wiki/NTFS
\n\nEdit:
\n\nWhy the downvote? I had exactly this problem.
\n\n[/quote]\nYes, you r right, that may be a problem. But my actual barrier is the slow speed network, now I solve it by using a software.
\n\nBy the way, thanks for your learning material "fat32 vs ntfs".
", "votes": null, "replies": [] }, { "id": 102086, "author_name": "meanku", "author_url": "", "post_date": "12/19/2015 09:58:26", "content": "[quote=dotcoming;101773]
\n\n[quote=Alexandre Gomiero;101731]
\n\nHi, @dotcoming!
\n\nI downloaded with Google Chrome and "Free Download Manager" (open source), plus the Chrome plugin for FDM.
\n\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).
\n\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.
\n\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page').
\n\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.
\n\nFDM is able to resume the download if something goes wrong.
\n\nHope this help!
\n\nPS: sorry if I made mistakes with the English language! :-)
\n\n[/quote]\nAlexandre Gomiero,
\n\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.
\n\nThanks again. :)
\n\n[/quote]
\n\nIt seems no help for me ，Can you share it via Baidu Yun to me， If the competition allow it
", "votes": null, "replies": [] }, { "id": 104564, "author_name": "", "author_url": "", "post_date": "01/13/2016 22:15:43", "content": "The option with Free Download Manager did not work for me. I am on Mac OS X El Capitan. I could not figure out how to change query size.
\n\nI found an extension for Firefox, called DownThemAll! This solved the problem, i was able to download data with the good speed.
\n\nHope this helps.
", "votes": null, "replies": [] }, { "id": 104567, "author_name": "elenacuoco", "author_url": "", "post_date": "01/13/2016 22:40:47", "content": "I used DownThemAll too.
", "votes": null, "replies": [] }, { "id": 104576, "author_name": "phunter", "author_url": "", "post_date": "01/14/2016 01:08:49", "content": "Can Kaggle provide some official S3 location for these data, since many people will use AWS GPU instance anyway?
\n\nBy the way, one can export cache from the browser, upload to AWS, and download from AWS using wget with this cache. It can reach the max speed. Details are at https://www.kaggle.com/forums/f/15/kaggle-forum/t/6604/downloading-data-via-command-line Plus some nice guy's github gist https://gist.github.com/rgbkrk/7340130
", "votes": null, "replies": [] }, { "id": 104611, "author_name": "saravananrmca", "author_url": "", "post_date": "01/14/2016 11:50:35", "content": "I am not able to find the DICOM Image Data set. Can anybody let me know from where we will download the DICOM files
", "votes": null, "replies": [] }, { "id": 104613, "author_name": "icedragon", "author_url": "", "post_date": "01/14/2016 13:29:51", "content": "The DICOM data is in train.zip and in validate.zip (download them here: https://www.kaggle.com/c/second-annual-data-science-bowl/data). \nUnzip them and you get all the folders with all the DICOM images.
", "votes": null, "replies": [] }, { "id": 106322, "author_name": "heisenbug", "author_url": "", "post_date": "01/29/2016 23:44:40", "content": "I am in Australia and it has been a struggle trying to download files of this size (!!!). Network timeouts are the main bug bear. Perhaps files should be split up into 2-3 GB size zip files. Even with a mac download manager (this took 2 attempts as the first product was unstable), I got zip file corruption errors, which has left me with a useless 13GB file on my HD.
\n\nI will run a windows 7 virtual machine and try Free DownLoad Manager - maybe its the Mac download managers.
", "votes": null, "replies": [] }, { "id": 106335, "author_name": "ptd006", "author_url": "", "post_date": "01/30/2016 08:50:40", "content": "On Ubuntu I used wget with the auto resume option.
\n\nTo handle the authentication you need to login via your browser and then export your cookies, like here-> https://www.kaggle.com/c/belkin-energy-disaggregation-competition/forums/t/5118/downloading-data-via-wget/96790
", "votes": null, "replies": [] }, { "id": 108806, "author_name": "joncarlisle", "author_url": "", "post_date": "02/20/2016 03:31:38", "content": "@Peter thank you for the tip. It worked perfectly. The one thing that tripped me up was that I needed to add sudo to my wget call. Otherwise, I would get a file not found error. Thanks again!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "101710": "As the title says, I wonder how do you download the train.zip and validate.zip data.\r\nI failed with wget, and that may have relation with china's gfw. What's more, if I just click the link, it may cost me a week to finish downloading the data. Of course, I am now in china.", "101731": "Hi, @dotcoming!\r\n\r\nI downloaded with Google Chrome and \"Free Download Manager\" (open source), plus the Chrome plugin for FDM.\r\n\r\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).\r\n\r\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.\r\n\r\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page'). \r\n\r\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.\r\n\r\nFDM is able to resume the download if something goes wrong.\r\n\r\nHope this help!\r\n\r\nPS: sorry if I made mistakes with the English language! :-)", "101752": "Hi,\r\n\r\nDatasets download with Firefox basic download withouy any plugin. It worked well. \r\nHope it will help.", "101761": "Make sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.\r\n\r\n> Max. file size \t4,294,967,295 bytes (4 GB - 1)\r\n\r\nhttps://en.wikipedia.org/wiki/File_Allocation_Table\r\n\r\n> Max. file size \t16 EiB\r\n\r\nhttps://en.wikipedia.org/wiki/NTFS\r\n\r\nEdit:\r\n\r\nWhy the downvote? I had exactly this problem.", "101773": "[quote=Alexandre Gomiero;101731]\r\n\r\nHi, @dotcoming!\r\n\r\nI downloaded with Google Chrome and \"Free Download Manager\" (open source), plus the Chrome plugin for FDM.\r\n\r\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).\r\n\r\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.\r\n\r\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page'). \r\n\r\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.\r\n\r\nFDM is able to resume the download if something goes wrong.\r\n\r\nHope this help!\r\n\r\nPS: sorry if I made mistakes with the English language! :-)\r\n\r\n[/quote]\r\nAlexandre Gomiero,\r\n \r\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.\r\n\r\nThanks again. :)", "101776": "[quote=Xavier Van Ausloos;101752]\r\n\r\nHi,\r\n\r\nDatasets download with Firefox basic download withouy any plugin. It worked well. \r\nHope it will help. \r\n\r\n[/quote]\r\nThanks for your advice really.", "101777": "[quote=h-wit;101761]\r\n\r\nMake sure you don't try to download to a FAT32 drive, use NTFS instead. I think this may be a major cause of download problems, they should warn on the data page or split the data into files <4 GB.\r\n\r\n> Max. file size \t4,294,967,295 bytes (4 GB - 1)\r\n\r\nhttps://en.wikipedia.org/wiki/File_Allocation_Table\r\n\r\n> Max. file size \t16 EiB\r\n\r\nhttps://en.wikipedia.org/wiki/NTFS\r\n\r\nEdit:\r\n\r\nWhy the downvote? I had exactly this problem.\r\n\r\n[/quote]\r\nYes, you r right, that may be a problem. But my actual barrier is the slow speed network, now I solve it by using a software. \r\n\r\nBy the way, thanks for your learning material \"fat32 vs ntfs\".", "102086": "[quote=dotcoming;101773]\r\n\r\n[quote=Alexandre Gomiero;101731]\r\n\r\nHi, @dotcoming!\r\n\r\nI downloaded with Google Chrome and \"Free Download Manager\" (open source), plus the Chrome plugin for FDM.\r\n\r\nAfter install the FDM, it has an option to integrate with Chrome to monitor downloads (it's disabled in the default installation and I turn it on).\r\n\r\nIt shows a blue box down right the browser that you can drag'n drop the download link, and a FDM dialog appears, with options to download.\r\n\r\nBefore click on 'Ok' button, I clicked on 'Query Size' button to ensure that the right file is to be downloaded (not the '14kb login page'). \r\n\r\nThe download was splitted into 4 threads and I've been able to get about 7 to 10Mb/s of download speed.\r\n\r\nFDM is able to resume the download if something goes wrong.\r\n\r\nHope this help!\r\n\r\nPS: sorry if I made mistakes with the English language! :-)\r\n\r\n[/quote]\r\nAlexandre Gomiero,\r\n \r\nThanks for ur impressive advice. It really help a lot, I copied the download link from the website to a software frequently used in china, and the speed booms up.\r\n\r\nThanks again. :)\r\n\r\n[/quote]\r\n\r\nIt seems no help for me ，Can you share it via Baidu Yun to me， If the competition allow it", "104564": "The option with Free Download Manager did not work for me. I am on Mac OS X El Capitan. I could not figure out how to change query size.\r\n\r\nI found an extension for Firefox, called DownThemAll! This solved the problem, i was able to download data with the good speed.\r\n\r\nHope this helps.", "104567": "I used DownThemAll too.", "104576": "Can Kaggle provide some official S3 location for these data, since many people will use AWS GPU instance anyway?\r\n\r\nBy the way, one can export cache from the browser, upload to AWS, and download from AWS using wget with this cache. It can reach the max speed. Details are at https://www.kaggle.com/forums/f/15/kaggle-forum/t/6604/downloading-data-via-command-line Plus some nice guy's github gist https://gist.github.com/rgbkrk/7340130", "104611": "I am not able to find the DICOM Image Data set. Can anybody let me know from where we will download the DICOM files", "104613": "The DICOM data is in train.zip and in validate.zip (download them here: https://www.kaggle.com/c/second-annual-data-science-bowl/data). \r\nUnzip them and you get all the folders with all the DICOM images.", "106322": "I am in Australia and it has been a struggle trying to download files of this size (!!!). Network timeouts are the main bug bear. Perhaps files should be split up into 2-3 GB size zip files. Even with a mac download manager (this took 2 attempts as the first product was unstable), I got zip file corruption errors, which has left me with a useless 13GB file on my HD.\r\n\r\nI will run a windows 7 virtual machine and try Free DownLoad Manager - maybe its the Mac download managers.", "106335": "On Ubuntu I used wget with the auto resume option. \r\n\r\nTo handle the authentication you need to login via your browser and then export your cookies, like here-> https://www.kaggle.com/c/belkin-energy-disaggregation-competition/forums/t/5118/downloading-data-via-wget/96790", "108806": "Peter thank you for the tip. It worked perfectly. The one thing that tripped me up was that I needed to add sudo to my wget call. Otherwise, I would get a file not found error. Thanks again!" }, "source": "meta" }