{
  "id": 103724,
  "title": "How to use GCP bucket?",
  "url": "/competitions/open-images-2019-object-detection/discussion/103724",
  "author_name": "",
  "post_date": "2019-08-11T10:07:22.908005200Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Can anyone please guide me on how to use data that is stored in GCP bucket for training purpose?</p>",
  "messages": [
    {
      "id": "596794",
      "postDate": "08/11/2019 10:07:22",
      "content": "<p>Can anyone please guide me on how to use data that is stored in GCP bucket for training purpose?</p>",
      "rawMarkdown": "Can anyone please guide me on how to use data that is stored in GCP bucket for training purpose?",
      "votes": null
    },
    {
      "id": "596832",
      "postDate": "08/11/2019 11:31:04",
      "content": "<p>Copy to a local disk in VM or your local machine, i'm afraid - while libs like pandas can read from gs directly, its not a workable option for image-based tasks.</p>",
      "rawMarkdown": "Copy to a local disk in VM or your local machine, i'm afraid - while libs like pandas can read from gs directly, its not a workable option for image-based tasks.",
      "votes": null
    },
    {
      "id": "597068",
      "postDate": "08/11/2019 18:51:06",
      "content": "<p>Hi! <a href=\"/sudhiriitb\">@sudhiriitb</a> If you have a VM running on GCP and you use custom estimators you can work with the tensorflow datasets to generate a batch of images efficiently from GCS for training purposes\n<a href=\"https://www.tensorflow.org/datasets\">https://www.tensorflow.org/datasets</a>\n<a href=\"https://www.tensorflow.org/guide/performance/datasets\">https://www.tensorflow.org/guide/performance/datasets</a>\nKeep the VM and GCS in the same region for impresive  transfer speed</p>",
      "rawMarkdown": "Hi! @sudhiriitb If you have a VM running on GCP and you use custom estimators you can work with the tensorflow datasets to generate a batch of images efficiently from GCS for training purposes\nhttps://www.tensorflow.org/datasets\nhttps://www.tensorflow.org/guide/performance/datasets\nKeep the VM and GCS in the same region for impresive  transfer speed",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 596832,
      "author_name": "konradb",
      "author_url": "",
      "post_date": "08/11/2019 11:31:04",
      "content": "<p>Copy to a local disk in VM or your local machine, i'm afraid - while libs like pandas can read from gs directly, its not a workable option for image-based tasks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 597068,
      "author_name": "cv13j0",
      "author_url": "",
      "post_date": "08/11/2019 18:51:06",
      "content": "<p>Hi! <a href=\"/sudhiriitb\">@sudhiriitb</a> If you have a VM running on GCP and you use custom estimators you can work with the tensorflow datasets to generate a batch of images efficiently from GCS for training purposes\n<a href=\"https://www.tensorflow.org/datasets\">https://www.tensorflow.org/datasets</a>\n<a href=\"https://www.tensorflow.org/guide/performance/datasets\">https://www.tensorflow.org/guide/performance/datasets</a>\nKeep the VM and GCS in the same region for impresive  transfer speed</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "596794": "Can anyone please guide me on how to use data that is stored in GCP bucket for training purpose?",
    "596832": "Copy to a local disk in VM or your local machine, i'm afraid - while libs like pandas can read from gs directly, its not a workable option for image-based tasks.",
    "597068": "Hi! @sudhiriitb If you have a VM running on GCP and you use custom estimators you can work with the tensorflow datasets to generate a batch of images efficiently from GCS for training purposes\nhttps://www.tensorflow.org/datasets\nhttps://www.tensorflow.org/guide/performance/datasets\nKeep the VM and GCS in the same region for impresive  transfer speed"
  },
  "source": "meta"
}