{
  "id": 129855,
  "title": "How to run Text Models on TPUs?",
  "url": "/competitions/flower-classification-with-tpus/discussion/129855",
  "author_name": "",
  "post_date": "2020-02-11T02:40:40.664519400Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Is there any way to create tf records of tokenized &amp; integer encoded text sequences, such that we can then input them into a language model (e.g. BERT) that is loaded onto the TPU?</p>\n\n<p><a href=\"/mgornergoogle\">@mgornergoogle</a> </p>",
  "messages": [
    {
      "id": "741958",
      "postDate": "02/11/2020 02:40:40",
      "content": "<p>Is there any way to create tf records of tokenized &amp; integer encoded text sequences, such that we can then input them into a language model (e.g. BERT) that is loaded onto the TPU?</p>\n\n<p><a href=\"/mgornergoogle\">@mgornergoogle</a> </p>",
      "rawMarkdown": "Is there any way to create tf records of tokenized &amp; integer encoded text sequences, such that we can then input them into a language model (e.g. BERT) that is loaded onto the TPU?\n\n@mgornergoogle",
      "votes": null
    },
    {
      "id": "742224",
      "postDate": "02/11/2020 06:59:06",
      "content": "<p>I am not sure about storing a text dataset in tfrecords but there is a tutorial over <a href=\"https://cloud.google.com/tpu/docs/tutorials/bert\">here</a> for fine-tuning BERT on TPUs.</p>\n\n<p>There is also a tutorial <a href=\"https://www.youtube.com/watch?v=B_P0ZIXspOU&amp;t=1s\">here</a> for doing the same with PyTorch.</p>\n\n<p>Hope this helps.</p>",
      "rawMarkdown": "I am not sure about storing a text dataset in tfrecords but there is a tutorial over [here](https://cloud.google.com/tpu/docs/tutorials/bert) for fine-tuning BERT on TPUs.\n\nThere is also a tutorial [here](https://www.youtube.com/watch?v=B_P0ZIXspOU&amp;t=1s) for doing the same with PyTorch.\n\nHope this helps.",
      "votes": null
    },
    {
      "id": "742881",
      "postDate": "02/11/2020 16:10:53",
      "content": "<p>THanks for sharing!</p>",
      "rawMarkdown": "THanks for sharing!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 742224,
      "author_name": "tanlikesmath",
      "author_url": "",
      "post_date": "02/11/2020 06:59:06",
      "content": "<p>I am not sure about storing a text dataset in tfrecords but there is a tutorial over <a href=\"https://cloud.google.com/tpu/docs/tutorials/bert\">here</a> for fine-tuning BERT on TPUs.</p>\n\n<p>There is also a tutorial <a href=\"https://www.youtube.com/watch?v=B_P0ZIXspOU&amp;t=1s\">here</a> for doing the same with PyTorch.</p>\n\n<p>Hope this helps.</p>",
      "votes": null,
      "replies": [
        {
          "id": 742881,
          "author_name": "xhlulu",
          "author_url": "",
          "post_date": "02/11/2020 16:10:53",
          "content": "<p>THanks for sharing!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "741958": "Is there any way to create tf records of tokenized &amp; integer encoded text sequences, such that we can then input them into a language model (e.g. BERT) that is loaded onto the TPU?\n\n@mgornergoogle",
    "742224": "I am not sure about storing a text dataset in tfrecords but there is a tutorial over [here](https://cloud.google.com/tpu/docs/tutorials/bert) for fine-tuning BERT on TPUs.\n\nThere is also a tutorial [here](https://www.youtube.com/watch?v=B_P0ZIXspOU&amp;t=1s) for doing the same with PyTorch.\n\nHope this helps.",
    "742881": "THanks for sharing!"
  },
  "source": "meta"
}