{
  "id": 177351,
  "title": "Newbie question: Can I read gs://xxx dataset directly on Colab?",
  "url": "/competitions/birdsong-recognition/discussion/177351",
  "author_name": "",
  "post_date": "2020-08-25T15:54:26.365274Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hi I would like to train the model in Colab <strong>without uploading the dataset in my own Google drive / GCP</strong> (too bulky)<br>\nI can read csv file directly from gs:// link but the trainloader cannot read the wav file directly.<br>\nMay I know how to read the wav file through gs link in this comp?</p>\n<p>I hv googled for few hrs but most are teaching me to upload the file into Gdrive /GCP then mount in colab, which is not what I want</p>\n<p>Thanks so much🙏🙏🙏</p>\n<pre><code>[['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC137570.wav',\n  'aldfly'],\n ['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC138639.wav',\n  'aldfly'],...etc\n</code></pre>\n<p>Here is the error code</p>\n<pre><code>RuntimeError: Error opening 'gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/bewwre/XC217912.wav': System error.\n</code></pre>",
  "messages": [
    {
      "id": "985280",
      "postDate": "08/25/2020 15:54:26",
      "content": "<p>Hi I would like to train the model in Colab <strong>without uploading the dataset in my own Google drive / GCP</strong> (too bulky)<br>\nI can read csv file directly from gs:// link but the trainloader cannot read the wav file directly.<br>\nMay I know how to read the wav file through gs link in this comp?</p>\n<p>I hv googled for few hrs but most are teaching me to upload the file into Gdrive /GCP then mount in colab, which is not what I want</p>\n<p>Thanks so much🙏🙏🙏</p>\n<pre><code>[['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC137570.wav',\n  'aldfly'],\n ['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC138639.wav',\n  'aldfly'],...etc\n</code></pre>\n<p>Here is the error code</p>\n<pre><code>RuntimeError: Error opening 'gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/bewwre/XC217912.wav': System error.\n</code></pre>",
      "rawMarkdown": "Hi I would like to train the model in Colab **without uploading the dataset in my own Google drive / GCP** (too bulky)\nI can read csv file directly from gs:// link but the trainloader cannot read the wav file directly.\nMay I know how to read the wav file through gs link in this comp?\n\nI hv googled for few hrs but most are teaching me to upload the file into Gdrive /GCP then mount in colab, which is not what I want\n\nThanks so much🙏🙏🙏\n\n```\n[['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC137570.wav',\n  'aldfly'],\n ['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC138639.wav',\n  'aldfly'],...etc\n```\n\nHere is the error code\n```\nRuntimeError: Error opening 'gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/bewwre/XC217912.wav': System error.\n```",
      "votes": null
    },
    {
      "id": "986291",
      "postDate": "08/26/2020 10:52:13",
      "content": "<p>it's use diff method to read file</p>",
      "rawMarkdown": "it's use diff method to read file",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 986291,
      "author_name": "doanquanvietnamca",
      "author_url": "",
      "post_date": "08/26/2020 10:52:13",
      "content": "<p>it's use diff method to read file</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "985280": "Hi I would like to train the model in Colab **without uploading the dataset in my own Google drive / GCP** (too bulky)\nI can read csv file directly from gs:// link but the trainloader cannot read the wav file directly.\nMay I know how to read the wav file through gs link in this comp?\n\nI hv googled for few hrs but most are teaching me to upload the file into Gdrive /GCP then mount in colab, which is not what I want\n\nThanks so much🙏🙏🙏\n\n```\n[['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC137570.wav',\n  'aldfly'],\n ['gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/aldfly/XC138639.wav',\n  'aldfly'],...etc\n```\n\nHere is the error code\n```\nRuntimeError: Error opening 'gs://kds-415a60f0bab644c1dede2edf10f91cfeaad783c1b318ff35aa8a6057/bewwre/XC217912.wav': System error.\n```",
    "986291": "it's use diff method to read file"
  },
  "source": "meta"
}