{
  "id": 229874,
  "title": "Dataset Storage when Using PyTorch on TPU",
  "url": "/competitions/bms-molecular-translation/discussion/229874",
  "author_name": "",
  "post_date": "2021-04-01T06:55:22.586161100Z",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>When using TF/Keras with TPU, it's required to store data in GCS bucket and preferrably as tfrecord to increase training speed. I was wondering if there's any similar requirement or tricks when using PyTorch on TPU.</p>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "1259122",
      "postDate": "04/01/2021 06:55:22",
      "content": "<p>When using TF/Keras with TPU, it's required to store data in GCS bucket and preferrably as tfrecord to increase training speed. I was wondering if there's any similar requirement or tricks when using PyTorch on TPU.</p>\n<p>Thanks!</p>",
      "rawMarkdown": "When using TF/Keras with TPU, it's required to store data in GCS bucket and preferrably as tfrecord to increase training speed. I was wondering if there's any similar requirement or tricks when using PyTorch on TPU.\n\nThanks!",
      "votes": null
    },
    {
      "id": "1259423",
      "postDate": "04/01/2021 12:00:02",
      "content": "<p>With pytorch you can use the data on your local disk.</p>",
      "rawMarkdown": "With pytorch you can use the data on your local disk.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1259423,
      "author_name": "tonyxu",
      "author_url": "",
      "post_date": "04/01/2021 12:00:02",
      "content": "<p>With pytorch you can use the data on your local disk.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1259122": "When using TF/Keras with TPU, it's required to store data in GCS bucket and preferrably as tfrecord to increase training speed. I was wondering if there's any similar requirement or tricks when using PyTorch on TPU.\n\nThanks!",
    "1259423": "With pytorch you can use the data on your local disk."
  },
  "source": "meta"
}