TFRecords, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with tf.data this way the transformation will be added to TF graph, here are some examples:CutMix and MixUp techniques to increase models generalization.Also, check out the two previous competitions where TPUs were heavily used Flower Classification with TPUs and Jigsaw Multilingual Toxic Comment Classification. If you have any more nice resources, post here and let us know!
", "messages": [ { "id": 892528, "postDate": "2020-06-19T01:38:54.787Z", "content": "TFRecords, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with tf.data this way the transformation will be added to TF graph, here are some examples:CutMix and MixUp techniques to increase models generalization.Also, check out the two previous competitions where TPUs were heavily used Flower Classification with TPUs and Jigsaw Multilingual Toxic Comment Classification. If you have any more nice resources, post here and let us know!
", "rawMarkdown": "### Guides\n- [Tensorflow quick guide on using TPU](https://www.tensorflow.org/guide/tpu#improving_performance_by_multiple_steps_within_tffunction)\n- [Kaggle TPU Youtube playlist](https://www.youtube.com/playlist?list=PLqFaTIg4myu-1c3ygYzakW8-hNzQG59-5)\n- [Tips for improving TPU usage](https://cloud.google.com/tpu/docs/troubleshooting#memory-usage)\n- Training optimizations [disscused here](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/135443) by Martin\n\n### Documentations\n- [Kaggle TPU documentation](https://www.kaggle.com/docs/tpu)\n- [Cloud TPU documentation](https://cloud.google.com/tpu/docs/)\n- In case you wanna use [Mixed precision](https://www.tensorflow.org/guide/mixed_precision)\n- In case you wanna use [XLA](https://www.tensorflow.org/xla)\n\n### A couple of starting kernels from previous competitions\n- [Custom Training Loop with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu/notebook)\n- [Getting started with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu/notebook)\n- Check out many more from the [previous competition](https://www.kaggle.com/c/flower-classification-with-tpus/notebooks)\n\n### Some interesting examples from the community\n- **Data augmentation and processing using `TFRecords`**, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with `tf.data` this way the transformation will be added to `TF graph`, here are some examples:\n - [Rotation Augmentation GPU/TPU - [0.96+]](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96), many custom data transformations with a bunch of explanations on why it is more computationally efficient.\n - [Make Chris Deotte's data augmentation faster](https://www.kaggle.com/yihdarshieh/make-chris-deotte-s-data-augmentation-faster), making these transformations even faster by running them inside the TPU.\n - [CutMix and MixUp on GPU/TPU](https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu), Implementation of `CutMix and MixUp` techniques to increase models generalization.\n - [Flower with TPUs - Advanced augmentations](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-advanced-augmentations/notebook), one idea to give more control of what augmentation you doing on your data batches.\n\nAlso, check out the two previous competitions where TPUs were heavily used [Flower Classification with TPUs](https://www.kaggle.com/c/flower-classification-with-tpus) and [Jigsaw Multilingual Toxic Comment Classification](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification). If you have any more nice resources, post here and let us know!", "votes": 13 }, { "id": 1119148, "postDate": "2020-12-19T18:42:49.350Z", "content": "Nice Resources! I'm looking into them.
", "rawMarkdown": "Nice Resources! I'm looking into them.", "votes": 1 }, { "id": 895001, "postDate": "2020-06-21T02:52:54.017Z", "content": "Nice. You should add a section about data augmentation of TFRecords.
", "rawMarkdown": "Nice. You should add a section about data augmentation of TFRecords.", "votes": 1, "replies": [ { "id": 895626, "postDate": "2020-06-21T14:03:06.740Z", "content": "Good Idea @cdeotte !
", "rawMarkdown": "Good Idea @cdeotte !" } ] }, { "id": 3208576, "postDate": "2025-05-24T11:33:03.300Z", "content": "I am new to TPU. I think this information will be useful. Thanks.
", "rawMarkdown": "I am new to TPU. I think this information will be useful. Thanks." }, { "id": 2667573, "postDate": "2024-02-25T08:01:57.523Z", "content": "RuntimeError: Mixing different tf.distribute.Strategy objects: is not
\nHow to solve it ?
https://medium.com/@Vishal_v_cool/how-to-use-tpu-v3-8-on-kaggle-with-tensorflow-2-18-0-e40d590f64b9
", "rawMarkdown": "https://medium.com/@Vishal_v_cool/how-to-use-tpu-v3-8-on-kaggle-with-tensorflow-2-18-0-e40d590f64b9" } ] }, { "id": 895628, "postDate": "2020-06-21T14:06:03.630Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1119148, "author_name": "Xabier Lahuerta Vázquez", "author_url": "", "post_date": "2020-12-19T18:42:49.350000", "content": "Nice Resources! I'm looking into them.
", "votes": 1, "replies": [] }, { "id": 895001, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-21T02:52:54.017000", "content": "Nice. You should add a section about data augmentation of TFRecords.
", "votes": 1, "replies": [ { "id": 895626, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-06-21T14:03:06.740000", "content": "Good Idea @cdeotte !
", "votes": 0, "replies": [] } ] }, { "id": 3208576, "author_name": "kukazuo", "author_url": "", "post_date": "2025-05-24T11:33:03.300000", "content": "I am new to TPU. I think this information will be useful. Thanks.
", "votes": 0, "replies": [] }, { "id": 2667573, "author_name": "Edison", "author_url": "", "post_date": "2024-02-25T08:01:57.523000", "content": "RuntimeError: Mixing different tf.distribute.Strategy objects: is not
\nHow to solve it ?
https://medium.com/@Vishal_v_cool/how-to-use-tpu-v3-8-on-kaggle-with-tensorflow-2-18-0-e40d590f64b9
", "votes": 0, "replies": [] } ] }, { "id": 895628, "author_name": "", "author_url": "", "post_date": "2020-06-21T14:06:03.630000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "892528": "### Guides\n- [Tensorflow quick guide on using TPU](https://www.tensorflow.org/guide/tpu#improving_performance_by_multiple_steps_within_tffunction)\n- [Kaggle TPU Youtube playlist](https://www.youtube.com/playlist?list=PLqFaTIg4myu-1c3ygYzakW8-hNzQG59-5)\n- [Tips for improving TPU usage](https://cloud.google.com/tpu/docs/troubleshooting#memory-usage)\n- Training optimizations [disscused here](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/135443) by Martin\n\n### Documentations\n- [Kaggle TPU documentation](https://www.kaggle.com/docs/tpu)\n- [Cloud TPU documentation](https://cloud.google.com/tpu/docs/)\n- In case you wanna use [Mixed precision](https://www.tensorflow.org/guide/mixed_precision)\n- In case you wanna use [XLA](https://www.tensorflow.org/xla)\n\n### A couple of starting kernels from previous competitions\n- [Custom Training Loop with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu/notebook)\n- [Getting started with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu/notebook)\n- Check out many more from the [previous competition](https://www.kaggle.com/c/flower-classification-with-tpus/notebooks)\n\n### Some interesting examples from the community\n- **Data augmentation and processing using `TFRecords`**, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with `tf.data` this way the transformation will be added to `TF graph`, here are some examples:\n - [Rotation Augmentation GPU/TPU - [0.96+]](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96), many custom data transformations with a bunch of explanations on why it is more computationally efficient.\n - [Make Chris Deotte's data augmentation faster](https://www.kaggle.com/yihdarshieh/make-chris-deotte-s-data-augmentation-faster), making these transformations even faster by running them inside the TPU.\n - [CutMix and MixUp on GPU/TPU](https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu), Implementation of `CutMix and MixUp` techniques to increase models generalization.\n - [Flower with TPUs - Advanced augmentations](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-advanced-augmentations/notebook), one idea to give more control of what augmentation you doing on your data batches.\n\nAlso, check out the two previous competitions where TPUs were heavily used [Flower Classification with TPUs](https://www.kaggle.com/c/flower-classification-with-tpus) and [Jigsaw Multilingual Toxic Comment Classification](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification). If you have any more nice resources, post here and let us know!", "1119148": "Nice Resources! I'm looking into them.", "895001": "Nice. You should add a section about data augmentation of TFRecords.", "3208576": "I am new to TPU. I think this information will be useful. Thanks.", "2667573": "RuntimeError: Mixing different tf.distribute.Strategy objects: