{ "id": 3231, "title": "Downloading Data", "url": "/competitions/flight/discussion/3231", "author_name": "", "post_date": "2012-12-01T21:15:25.877Z", "votes": null, "comment_count": 13, "views": 9802, "content": "

This is the 4th time that downloading the data files failed.

\r\n

Can anyone suggest some ways to download the data file? I am using Ubuntu.

\r\n

Or can the admin split the datafile into 2 and upload?

", "messages": [ { "id": "17326", "postDate": "12/01/2012 21:15:25", "content": "

This is the 4th time that downloading the data files failed.

\r\n

Can anyone suggest some ways to download the data file? I am using Ubuntu.

\r\n

Or can the admin split the datafile into 2 and upload?

", "rawMarkdown": "", "votes": null }, { "id": "17331", "postDate": "12/02/2012 03:29:02", "content": "

Just so you know (not much help I realise!!), I am on Ubuntu, downloaded it fine using Chromium 1st time, so it is possible (I used the smaller (1.1GB file!!). A suggestion though - maybe try a download tool that supports restarts - uGet looks like it might\r\n do the job (I haven't tried it - just reading the features list), there may be others... so you can pick up from where you left off.

\r\n

Otherwise, maybe Kaggle admin can make a bittorrent file available which can definitely be downloaded in bits with stops and starts...

", "rawMarkdown": "", "votes": null }, { "id": "17374", "postDate": "12/04/2012 05:03:41", "content": "

I'm also facing the same problem with downloading the big data file. If admin can make a torrent file then problem will be solved.

", "rawMarkdown": "", "votes": null }, { "id": "17379", "postDate": "12/04/2012 13:44:22", "content": "

I am using idm to download files to computer, and it is my 4th time .

", "rawMarkdown": "", "votes": null }, { "id": "17380", "postDate": "12/04/2012 14:20:46", "content": "

I am also facing problem while downloading the data. Is there a different way to download the same?

", "rawMarkdown": "", "votes": null }, { "id": "17381", "postDate": "12/04/2012 14:39:43", "content": "

The website is probably timing out for the login session, which causes the download to fail.

\r\n

I have used Download Accelerator Plus for download and it worked fine. I have tried it on windows only.

\r\n

Here is the URL for DAP... http://www.speedbit.com/

\r\n

All the best.

", "rawMarkdown": "", "votes": null }, { "id": "17384", "postDate": "12/04/2012 19:13:23", "content": "

I don't work for Amazon, but if you have one of their cloud servers, they are able to download the whole thing in < 5 minutes. They get about 2-3MB/s. Even the free tier gets that kind of speed, but you won't have much processing power at that level.

", "rawMarkdown": "", "votes": null }, { "id": "17417", "postDate": "12/05/2012 18:58:54", "content": "

Sorry about this. We can't create a torrent, as we need to make sure that only people who have accepted the rules can download the data.

\r\n

For future data releases (there will be a data correction soon), we will split the data into smaller files, which should solve the problem.

", "rawMarkdown": "", "votes": null }, { "id": "17445", "postDate": "12/06/2012 03:53:23", "content": "

Thanks David. Although this time I successfully downloaded it after numerous unsuccessful attempts, had the data files been split, it would have been easier.

", "rawMarkdown": "", "votes": null }, { "id": "17455", "postDate": "12/06/2012 15:01:10", "content": "

I had a problem with the InitialTrainingSet.zip file, but I was able to resolve it and thought I'd share to try to save someone else a headache.

\r\n

I'm using a Mac, and was able to download the InitialTrainingSet.zip file in Firefox just fine, but when I tried to uncompress the data in Finder, the Archive Utility gave an error message as as follows.

\r\n

Unable to expand "InitialTrainingSet.zip" into "GE data".

\r\n

(Error 1 - Operation not permitted.)

\r\n

The workaround was to open a shell (in terminal) and run the command "unzip InitialTrainingSet.zip".  Then the data uncompressed just fine.  :-)

\r\n

 

\r\n

Sorry about this. We can't create a torrent, as we need to make sure that only people who have accepted the rules can download the data.

\r\n

So put 98% of the data in a torrent, and then hold back the remaining 2% to be downloaded only through the 'I accept the rules" link.

\r\n

Splitting the data into seperate downloads would be OK too, some people might actually prefer that because they don't know how to use BitTorrent, but I'd prefer a torrent.

\r\n

I too had to make multiple attempts before I was able to successfully download the data, and I'd bet that the problem will be worse for later data sets as there might be more people trying to access the data at once.

\r\n

Danny

", "rawMarkdown": "", "votes": null }, { "id": "19004", "postDate": "01/05/2013 19:51:18", "content": "

I agree with @dannymulligan you can release the files as a torrent. Have 1 digest file required to decompress the files received through the registration wall. The rest 99.9% of the data as a torrent. The size of the files are painful otherwise.

", "rawMarkdown": "", "votes": null }, { "id": "19064", "postDate": "01/07/2013 14:20:24", "content": "

The admins seriously need to consider this problem instead of hoping that it'll go away itself. I've been waiting for a reply here for over 3 weeks now. I understand you don't want to release the data to those who haven't accepted the rules, but the least\r\n you can do is increase the login timeout duration so that at a 5 Mbps connection, I don't get logged out after downloading 700 MB every time, for 4 times in a row.

", "rawMarkdown": "", "votes": null }, { "id": "19069", "postDate": "01/07/2013 17:53:09", "content": "

Hey guys,

\r\n

We hear you and will work on developing a more robust way to deliver large files to all corners of the globe. In the meanwhile, if you're still having issues, please fill this out and we'll get the data to you:

\r\n

https://www.kaggle.com/request-data

\r\n

Best,
\r\nWill 

", "rawMarkdown": "", "votes": null }, { "id": "19087", "postDate": "01/08/2013 10:16:39", "content": "

Thanks so much Will! :)

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 17331, "author_name": "marka25933", "author_url": "", "post_date": "12/02/2012 03:29:02", "content": "

Just so you know (not much help I realise!!), I am on Ubuntu, downloaded it fine using Chromium 1st time, so it is possible (I used the smaller (1.1GB file!!). A suggestion though - maybe try a download tool that supports restarts - uGet looks like it might\r\n do the job (I haven't tried it - just reading the features list), there may be others... so you can pick up from where you left off.

\r\n

Otherwise, maybe Kaggle admin can make a bittorrent file available which can definitely be downloaded in bits with stops and starts...

", "votes": null, "replies": [] }, { "id": 17374, "author_name": "ankitksharma", "author_url": "", "post_date": "12/04/2012 05:03:41", "content": "

I'm also facing the same problem with downloading the big data file. If admin can make a torrent file then problem will be solved.

", "votes": null, "replies": [] }, { "id": 17379, "author_name": "mohitkhanna0", "author_url": "", "post_date": "12/04/2012 13:44:22", "content": "

I am using idm to download files to computer, and it is my 4th time .

", "votes": null, "replies": [] }, { "id": 17380, "author_name": "sircausticmail", "author_url": "", "post_date": "12/04/2012 14:20:46", "content": "

I am also facing problem while downloading the data. Is there a different way to download the same?

", "votes": null, "replies": [] }, { "id": 17381, "author_name": "nrupal", "author_url": "", "post_date": "12/04/2012 14:39:43", "content": "

The website is probably timing out for the login session, which causes the download to fail.

\r\n

I have used Download Accelerator Plus for download and it worked fine. I have tried it on windows only.

\r\n

Here is the URL for DAP... http://www.speedbit.com/

\r\n

All the best.

", "votes": null, "replies": [] }, { "id": 17384, "author_name": "mcstar", "author_url": "", "post_date": "12/04/2012 19:13:23", "content": "

I don't work for Amazon, but if you have one of their cloud servers, they are able to download the whole thing in < 5 minutes. They get about 2-3MB/s. Even the free tier gets that kind of speed, but you won't have much processing power at that level.

", "votes": null, "replies": [] }, { "id": 17417, "author_name": "dchudz", "author_url": "", "post_date": "12/05/2012 18:58:54", "content": "

Sorry about this. We can't create a torrent, as we need to make sure that only people who have accepted the rules can download the data.

\r\n

For future data releases (there will be a data correction soon), we will split the data into smaller files, which should solve the problem.

", "votes": null, "replies": [] }, { "id": 17445, "author_name": "mahendrakariya", "author_url": "", "post_date": "12/06/2012 03:53:23", "content": "

Thanks David. Although this time I successfully downloaded it after numerous unsuccessful attempts, had the data files been split, it would have been easier.

", "votes": null, "replies": [] }, { "id": 17455, "author_name": "dannymulligan", "author_url": "", "post_date": "12/06/2012 15:01:10", "content": "

I had a problem with the InitialTrainingSet.zip file, but I was able to resolve it and thought I'd share to try to save someone else a headache.

\r\n

I'm using a Mac, and was able to download the InitialTrainingSet.zip file in Firefox just fine, but when I tried to uncompress the data in Finder, the Archive Utility gave an error message as as follows.

\r\n

Unable to expand "InitialTrainingSet.zip" into "GE data".

\r\n

(Error 1 - Operation not permitted.)

\r\n

The workaround was to open a shell (in terminal) and run the command "unzip InitialTrainingSet.zip".  Then the data uncompressed just fine.  :-)

\r\n

 

\r\n

Sorry about this. We can't create a torrent, as we need to make sure that only people who have accepted the rules can download the data.

\r\n

So put 98% of the data in a torrent, and then hold back the remaining 2% to be downloaded only through the 'I accept the rules" link.

\r\n

Splitting the data into seperate downloads would be OK too, some people might actually prefer that because they don't know how to use BitTorrent, but I'd prefer a torrent.

\r\n

I too had to make multiple attempts before I was able to successfully download the data, and I'd bet that the problem will be worse for later data sets as there might be more people trying to access the data at once.

\r\n

Danny

", "votes": null, "replies": [] }, { "id": 19004, "author_name": "firestrand", "author_url": "", "post_date": "01/05/2013 19:51:18", "content": "

I agree with @dannymulligan you can release the files as a torrent. Have 1 digest file required to decompress the files received through the registration wall. The rest 99.9% of the data as a torrent. The size of the files are painful otherwise.

", "votes": null, "replies": [] }, { "id": 19064, "author_name": "anant90", "author_url": "", "post_date": "01/07/2013 14:20:24", "content": "

The admins seriously need to consider this problem instead of hoping that it'll go away itself. I've been waiting for a reply here for over 3 weeks now. I understand you don't want to release the data to those who haven't accepted the rules, but the least\r\n you can do is increase the login timeout duration so that at a 5 Mbps connection, I don't get logged out after downloading 700 MB every time, for 4 times in a row.

", "votes": null, "replies": [] }, { "id": 19069, "author_name": "wcukierski", "author_url": "", "post_date": "01/07/2013 17:53:09", "content": "

Hey guys,

\r\n

We hear you and will work on developing a more robust way to deliver large files to all corners of the globe. In the meanwhile, if you're still having issues, please fill this out and we'll get the data to you:

\r\n

https://www.kaggle.com/request-data

\r\n

Best,
\r\nWill 

", "votes": null, "replies": [] }, { "id": 19087, "author_name": "anant90", "author_url": "", "post_date": "01/08/2013 10:16:39", "content": "

Thanks so much Will! :)

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "17326": "", "17331": "", "17374": "", "17379": "", "17380": "", "17381": "", "17384": "", "17417": "", "17445": "", "17455": "", "19004": "", "19064": "", "19069": "", "19087": "" }, "source": "meta" }