Yesterday, I got access to run 5 regular cloud TPU's and 10 preermptible cloud TPU's for free for the next 60 days.
\n\nI am currently working on a slightly modified version of @mmmarchetti 's code. Not sure of steps to be able to do the fine-tuning on TPU's. The code is based on TF2.0 so not sure about it's compatibility with TPU's either. Should I switch to TF1.0? I guess TF2.1 works with TPU but it's still a bit rough around the edges. I'm new to all this and wouldn't like to spend most of my time debugging.
\n\n@guozhiyu0914 @dynamicwebpaige @seesee @xincuimath
", "messages": [ { "id": "699336", "postDate": "12/20/2019 10:21:32", "content": "Yesterday, I got access to run 5 regular cloud TPU's and 10 preermptible cloud TPU's for free for the next 60 days.
\n\nI am currently working on a slightly modified version of @mmmarchetti 's code. Not sure of steps to be able to do the fine-tuning on TPU's. The code is based on TF2.0 so not sure about it's compatibility with TPU's either. Should I switch to TF1.0? I guess TF2.1 works with TPU but it's still a bit rough around the edges. I'm new to all this and wouldn't like to spend most of my time debugging.
\n\n@guozhiyu0914 @dynamicwebpaige @seesee @xincuimath
", "rawMarkdown": "Yesterday, I got access to run 5 regular cloud TPU's and 10 preermptible cloud TPU's for free for the next 60 days. \n\nI am currently working on a slightly modified version of @mmmarchetti 's [code](https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers). Not sure of steps to be able to do the fine-tuning on TPU's. The code is based on TF2.0 so not sure about it's compatibility with TPU's either. Should I switch to TF1.0? I guess TF2.1 works with TPU but it's still a bit rough around the edges. I'm new to all this and wouldn't like to spend most of my time debugging.\n\n@guozhiyu0914 @dynamicwebpaige @seesee @xincuimath", "votes": null }, { "id": "699638", "postDate": "12/20/2019 17:30:03", "content": "Hi, Rohit! Thank you for the question.
\n\nIf you're just getting started with TPUs and TensorFlow, I strongly suggest taking a look at TF 2.1 rather than TPUEstimator in TF 1.x. Details can be found in this installation guide from the Kaggle and Google Cloud teams. We have also released an example of BERT fine-tuning with Cloud TPUs, using TensorFlow.
The API docs for tf.distribute.experimental.TPUStrategy might also be useful, as well as the discussion on our TPU post. Let us know if you have any questions!
Thanks a ton @dynamicwebpaige , you've always been very helpful.
", "rawMarkdown": "Thanks a ton @dynamicwebpaige , you've always been very helpful.", "votes": null } ], "comments": [ { "id": 699638, "author_name": "dynamicwebpaige", "author_url": "", "post_date": "12/20/2019 17:30:03", "content": "Hi, Rohit! Thank you for the question.
\n\nIf you're just getting started with TPUs and TensorFlow, I strongly suggest taking a look at TF 2.1 rather than TPUEstimator in TF 1.x. Details can be found in this installation guide from the Kaggle and Google Cloud teams. We have also released an example of BERT fine-tuning with Cloud TPUs, using TensorFlow.
The API docs for tf.distribute.experimental.TPUStrategy might also be useful, as well as the discussion on our TPU post. Let us know if you have any questions!
Thanks a ton @dynamicwebpaige , you've always been very helpful.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "699336": "Yesterday, I got access to run 5 regular cloud TPU's and 10 preermptible cloud TPU's for free for the next 60 days. \n\nI am currently working on a slightly modified version of @mmmarchetti 's [code](https://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers). Not sure of steps to be able to do the fine-tuning on TPU's. The code is based on TF2.0 so not sure about it's compatibility with TPU's either. Should I switch to TF1.0? I guess TF2.1 works with TPU but it's still a bit rough around the edges. I'm new to all this and wouldn't like to spend most of my time debugging.\n\n@guozhiyu0914 @dynamicwebpaige @seesee @xincuimath", "699638": "Hi, Rohit! Thank you for the question.\n\nIf you're just getting started with TPUs and TensorFlow, I strongly suggest taking a look at TF 2.1 rather than `TPUEstimator` in TF 1.x. Details can be found in [this installation guide](https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/fast-and-lean-data-science/README-TF2.1.md) from the Kaggle and Google Cloud teams. We have also released an example of [BERT fine-tuning with Cloud TPUs](https://github.com/tensorflow/models/blob/master/official/nlp/bert/bert_cloud_tpu.md), using TensorFlow. \n\nThe [API docs for `tf.distribute.experimental.TPUStrategy`](https://www.tensorflow.org/api_docs/python/tf/distribute/experimental/TPUStrategy?hl=en&version=stable) might also be useful, as well as the discussion on our [TPU post](https://www.kaggle.com/c/tensorflow2-question-answering/discussion/121572). Let us know if you have any questions!", "699820": "Thanks a ton @dynamicwebpaige , you've always been very helpful." }, "source": "meta" }