{
  "id": 39490,
  "title": "Very slow downloading (train.bson)",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/39490",
  "author_name": "",
  "post_date": "2017-09-15T03:35:18.460124800Z",
  "votes": 4,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I'm trying to download train.bson, but download is very slow(about 50 kb/s), <br>\ntest.bson is quick, about 5-10 mb/s.  </p>\n\n<p>Download of test.bson is from <a href=\"https://storage.googleapis.com/\">https://storage.googleapis.com</a>, <br>\ntrain.bson is from <a href=\"https://kaggle2.blob.core.windows.net/\">https://kaggle2.blob.core.windows.net</a>.  </p>\n\n<p>I (we?) want to provide train.bson on <a href=\"https://storage.googleapis.com/\">https://storage.googleapis.com</a>.</p>\n\n<p>Best regards,</p>",
  "messages": [
    {
      "id": "221387",
      "postDate": "09/15/2017 03:35:18",
      "content": "<p>I'm trying to download train.bson, but download is very slow(about 50 kb/s), <br>\ntest.bson is quick, about 5-10 mb/s.  </p>\n\n<p>Download of test.bson is from <a href=\"https://storage.googleapis.com/\">https://storage.googleapis.com</a>, <br>\ntrain.bson is from <a href=\"https://kaggle2.blob.core.windows.net/\">https://kaggle2.blob.core.windows.net</a>.  </p>\n\n<p>I (we?) want to provide train.bson on <a href=\"https://storage.googleapis.com/\">https://storage.googleapis.com</a>.</p>\n\n<p>Best regards,</p>",
      "rawMarkdown": "I'm trying to download train.bson, but download is very slow(about 50 kb/s),  \ntest.bson is quick, about 5-10 mb/s.  \n\nDownload of test.bson is from https://storage.googleapis.com,  \ntrain.bson is from https://kaggle2.blob.core.windows.net.  \n\nI (we?) want to provide train.bson on https://storage.googleapis.com.\n\nBest regards,",
      "votes": null
    },
    {
      "id": "221438",
      "postDate": "09/15/2017 09:23:55",
      "content": "<p>Similar story here ... train.bson loads at 2Mo/s while test.bson loads at a confortable 50Mo/s ? Any options to make the download faster for train.bson ?</p>",
      "rawMarkdown": "Similar story here ... train.bson loads at 2Mo/s while test.bson loads at a confortable 50Mo/s ? Any options to make the download faster for train.bson ?",
      "votes": null
    },
    {
      "id": "221475",
      "postDate": "09/15/2017 12:06:51",
      "content": "<p>It seems the file \"train.bson\" is ordered by the field \"product_id\". It would be nice to split whole file into partitions as separate files. Definitely, it could help to download training data easlier.</p>",
      "rawMarkdown": "It seems the file \"train.bson\" is ordered by the field \"product_id\". It would be nice to split whole file into partitions as separate files. Definitely, it could help to download training data easlier.",
      "votes": null
    },
    {
      "id": "221505",
      "postDate": "09/15/2017 13:56:27",
      "content": "<p>Yup, I guess it's the same for everyone. Probably limit  of the bandwitch used so everyone can download it properly.</p>",
      "rawMarkdown": "Yup, I guess it's the same for everyone. Probably limit  of the bandwitch used so everyone can download it properly.",
      "votes": null
    },
    {
      "id": "221539",
      "postDate": "09/15/2017 15:40:59",
      "content": "<p>Thanks for the feedback. We're in the process of moving to a more performant file sharing solution (from Azure to Google Cloud Storage). The train.bson didn't get copied to GCS correctly. That has been fixed, and you should see download speeds markedly increase for that file.</p>",
      "rawMarkdown": "Thanks for the feedback. We're in the process of moving to a more performant file sharing solution (from Azure to Google Cloud Storage). The train.bson didn't get copied to GCS correctly. That has been fixed, and you should see download speeds markedly increase for that file.",
      "votes": null
    },
    {
      "id": "221550",
      "postDate": "09/15/2017 16:04:21",
      "content": "<p>Thank you very much, inversion!! <br>\nDownload speed is 300 Kbytes/s -&gt; 10 Mbytes/s under my environment:) <br>\nI'll challenge and enjoy this competition tomorrow morning(1am in Japan...)!!</p>",
      "rawMarkdown": "Thank you very much, inversion!!  \nDownload speed is 300 Kbytes/s -&gt; 10 Mbytes/s under my environment:)  \nI'll challenge and enjoy this competition tomorrow morning(1am in Japan...)!!",
      "votes": null
    },
    {
      "id": "221608",
      "postDate": "09/15/2017 20:07:01",
      "content": "<p>great , train is downloading now also at 400Mbps !</p>",
      "rawMarkdown": "great , train is downloading now also at 400Mbps !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 221438,
      "author_name": "intelpen",
      "author_url": "",
      "post_date": "09/15/2017 09:23:55",
      "content": "<p>Similar story here ... train.bson loads at 2Mo/s while test.bson loads at a confortable 50Mo/s ? Any options to make the download faster for train.bson ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 221475,
      "author_name": "victorir",
      "author_url": "",
      "post_date": "09/15/2017 12:06:51",
      "content": "<p>It seems the file \"train.bson\" is ordered by the field \"product_id\". It would be nice to split whole file into partitions as separate files. Definitely, it could help to download training data easlier.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 221505,
      "author_name": "rdoume",
      "author_url": "",
      "post_date": "09/15/2017 13:56:27",
      "content": "<p>Yup, I guess it's the same for everyone. Probably limit  of the bandwitch used so everyone can download it properly.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 221539,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "09/15/2017 15:40:59",
      "content": "<p>Thanks for the feedback. We're in the process of moving to a more performant file sharing solution (from Azure to Google Cloud Storage). The train.bson didn't get copied to GCS correctly. That has been fixed, and you should see download speeds markedly increase for that file.</p>",
      "votes": null,
      "replies": [
        {
          "id": 221550,
          "author_name": "kambarakun",
          "author_url": "",
          "post_date": "09/15/2017 16:04:21",
          "content": "<p>Thank you very much, inversion!! <br>\nDownload speed is 300 Kbytes/s -&gt; 10 Mbytes/s under my environment:) <br>\nI'll challenge and enjoy this competition tomorrow morning(1am in Japan...)!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 221608,
      "author_name": "intelpen",
      "author_url": "",
      "post_date": "09/15/2017 20:07:01",
      "content": "<p>great , train is downloading now also at 400Mbps !</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "221387": "I'm trying to download train.bson, but download is very slow(about 50 kb/s),  \ntest.bson is quick, about 5-10 mb/s.  \n\nDownload of test.bson is from https://storage.googleapis.com,  \ntrain.bson is from https://kaggle2.blob.core.windows.net.  \n\nI (we?) want to provide train.bson on https://storage.googleapis.com.\n\nBest regards,",
    "221438": "Similar story here ... train.bson loads at 2Mo/s while test.bson loads at a confortable 50Mo/s ? Any options to make the download faster for train.bson ?",
    "221475": "It seems the file \"train.bson\" is ordered by the field \"product_id\". It would be nice to split whole file into partitions as separate files. Definitely, it could help to download training data easlier.",
    "221505": "Yup, I guess it's the same for everyone. Probably limit  of the bandwitch used so everyone can download it properly.",
    "221539": "Thanks for the feedback. We're in the process of moving to a more performant file sharing solution (from Azure to Google Cloud Storage). The train.bson didn't get copied to GCS correctly. That has been fixed, and you should see download speeds markedly increase for that file.",
    "221550": "Thank you very much, inversion!!  \nDownload speed is 300 Kbytes/s -&gt; 10 Mbytes/s under my environment:)  \nI'll challenge and enjoy this competition tomorrow morning(1am in Japan...)!!",
    "221608": "great , train is downloading now also at 400Mbps !"
  },
  "source": "meta"
}