{ "id": 332442, "title": "[TPU Flower Classification] Summary of Top discussion posts", "url": "/competitions/tpu-getting-started/discussion/332442", "author_name": "The Devastator", "post_date": "2022-06-21T16:57:10.747000", "votes": 7, "comment_count": 0, "views": null, "content": "

Flower Classification on TPU

\n

Summary of Top discussion posts

\n
\n

Introduction Notebook
\nby kurianbenoy

\n\n

Research Papers: Deep Learning for Flower Classification
\nby satoshidatamoto

\n\n

TFRecords
\nby davidedwards1

\n\n

Efficient Net has low sparse categorical accuracy and validation accuracy
\nby yuzheni

\n\n
EPOCHS = 12\nfrom tensorflow.keras.layers.experimental import preprocessing\nwith strategy.scope():\n    pretrained_model = tf.keras.applications.EfficientNetB4(\n        weights=\"imagenet\",\n        include_top=False ,\n        input_shape=[*IMAGE_SIZE, 3], pooling=\"max\"\n    )\n    pretrained_model.trainable = False\n    model = tf.keras.Sequential([\n        # To a base pretrained on ImageNet to extract features from images...\n        #tf.keras.layers.experimental.preprocessing.RandomFlip('horizontal'),\n        pretrained_model,\n        tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n    ])\n
\n\n

Starter Notebooks To Keep You Entertained!
\nby satoshidatamoto

\n\n

Model fitting very slow using TPU
\nby yanngarcia

\n\n

Some TPU beginner tips
\nby dimitreoliveira

\n\n

TPU only Tensorflow? or we can use PyTorch?
\nby paantya

\n\n

Detailed guide to custom training with TPUs
\nby yihdarshieh

\n\n

To begin with TPUs
\nby aryansakhala

\n\n

BiT: Transfer Learning in tensorflow
\nby ibrahimsherify

\n\n

Live (+recorded) deep dive on: Computer Vision - Petals to the Metal🌻🌸🌹
\nby georgezoto

\n\n

Cool TPU beginner resources
\nby dimitreoliveira

\n\n

Guides

\n\n

Documentations

\n\n

A couple of starting kernels from previous competitions

\n\n

Some interesting examples from the community

\n

Data augmentation and processing using TFRecords, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with tf.data this way the transformation will be added to TF graph, here are some examples:

\n\n

Also, check out the two previous competitions where TPUs were heavily used Flower Classification with TPUs and Jigsaw Multilingual Toxic Comment Classification. If you have any more nice resources, post here and let us know!

\n

Need good resources for understanding TPU
\nby pandekp

\n\n

TPU Basics

\n\n

Great TPU Notebooks built w/ Tensorflow

\n\n

Data Augmentation w/ TPU's

\n\n

What steps can improve Confusion Matrix sharpness?
\nby mpwolke

\n\n

TFRecords to Pandas.core.DataFrame
\nby divyosmi2009

\n", "messages": [ { "id": 1828404, "postDate": "2022-06-21T16:57:10.747Z", "content": "

Flower Classification on TPU

\n

Summary of Top discussion posts

\n
\n

Introduction Notebook
\nby kurianbenoy

\n\n

Research Papers: Deep Learning for Flower Classification
\nby satoshidatamoto

\n\n

TFRecords
\nby davidedwards1

\n\n

Efficient Net has low sparse categorical accuracy and validation accuracy
\nby yuzheni

\n\n
EPOCHS = 12\nfrom tensorflow.keras.layers.experimental import preprocessing\nwith strategy.scope():\n    pretrained_model = tf.keras.applications.EfficientNetB4(\n        weights=\"imagenet\",\n        include_top=False ,\n        input_shape=[*IMAGE_SIZE, 3], pooling=\"max\"\n    )\n    pretrained_model.trainable = False\n    model = tf.keras.Sequential([\n        # To a base pretrained on ImageNet to extract features from images...\n        #tf.keras.layers.experimental.preprocessing.RandomFlip('horizontal'),\n        pretrained_model,\n        tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n    ])\n
\n\n

Starter Notebooks To Keep You Entertained!
\nby satoshidatamoto

\n\n

Model fitting very slow using TPU
\nby yanngarcia

\n\n

Some TPU beginner tips
\nby dimitreoliveira

\n\n

TPU only Tensorflow? or we can use PyTorch?
\nby paantya

\n\n

Detailed guide to custom training with TPUs
\nby yihdarshieh

\n\n

To begin with TPUs
\nby aryansakhala

\n\n

BiT: Transfer Learning in tensorflow
\nby ibrahimsherify

\n\n

Live (+recorded) deep dive on: Computer Vision - Petals to the Metal🌻🌸🌹
\nby georgezoto

\n\n

Cool TPU beginner resources
\nby dimitreoliveira

\n\n

Guides

\n\n

Documentations

\n\n

A couple of starting kernels from previous competitions

\n\n

Some interesting examples from the community

\n

Data augmentation and processing using TFRecords, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with tf.data this way the transformation will be added to TF graph, here are some examples:

\n\n

Also, check out the two previous competitions where TPUs were heavily used Flower Classification with TPUs and Jigsaw Multilingual Toxic Comment Classification. If you have any more nice resources, post here and let us know!

\n

Need good resources for understanding TPU
\nby pandekp

\n\n

TPU Basics

\n\n

Great TPU Notebooks built w/ Tensorflow

\n\n

Data Augmentation w/ TPU's

\n\n

What steps can improve Confusion Matrix sharpness?
\nby mpwolke

\n\n

TFRecords to Pandas.core.DataFrame
\nby divyosmi2009

\n", "rawMarkdown": "### Flower Classification on TPU\n#### Summary of Top discussion posts\n_____\n\n[Introduction Notebook](https://www.kaggle.com/competitions/tpu-getting-started/discussion/160793) \nby [kurianbenoy](kurianbenoy)\n\n- Sharing an [introduction notebook](https://www.kaggle.com/kurianbenoy/introduction-kernel-for-petal2metal-flowers) for getting started in the competition\n- **Tips to improve the score:**\n- Use augmentations ([discussions](https://www.kaggle.com/c/flower-classification-with-tpus/discussion))\n- [Techniques to improve scores](https://www.kaggle.com/romanweilguny/tpu-flowers-first-love)\n- Check the best score notebooks in Flower classification comp(there are SOTA models out there)\n- Use Big Transfer architecture: [code](https://github.com/google-research/big_transfer)\n\n[Research Papers: Deep Learning for Flower Classification](https://www.kaggle.com/competitions/tpu-getting-started/discussion/327120)\nby [satoshidatamoto](satoshidatamoto)\n\n- Sharing a list of deep learning papers for flower classification\n\n[TFRecords](https://www.kaggle.com/competitions/tpu-getting-started/discussion/166503)\nby [davidedwards1](davidedwards1)\n\n- A discussion on TFRecords, [starter tutorial here](https://www.kaggle.com/ryanholbrook/tfrecords-basics)\n\n[Efficient Net has low sparse categorical accuracy and validation accuracy](https://www.kaggle.com/competitions/tpu-getting-started/discussion/199759)\nby [yuzheni](yuzheni)\n\n- The author askes about the following code, reporting it to have low performance:\n\n```python\nEPOCHS = 12\nfrom tensorflow.keras.layers.experimental import preprocessing\nwith strategy.scope():\n pretrained_model = tf.keras.applications.EfficientNetB4(\n weights=\"imagenet\",\n include_top=False ,\n input_shape=[*IMAGE_SIZE, 3], pooling=\"max\"\n )\n pretrained_model.trainable = False\n model = tf.keras.Sequential([\n # To a base pretrained on ImageNet to extract features from images...\n #tf.keras.layers.experimental.preprocessing.RandomFlip('horizontal'),\n pretrained_model,\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\n```\n\n- **From the comments:** `pretrained_model.trainable = True`\n\n[Starter Notebooks To Keep You Entertained!](https://www.kaggle.com/competitions/tpu-getting-started/discussion/325287) \nby [satoshidatamoto](satoshidatamoto)\n\n- Listing starter notebooks for this competition.\n\n[Model fitting very slow using TPU](https://www.kaggle.com/competitions/tpu-getting-started/discussion/218062) \nby [yanngarcia](yanngarcia)\n\n- Asking for help debugging a model that is taking a long time to fit using TPU.\n- From the comments: The problem was that the author used augmentations for both training and validation and it was slow.\n\n[Some TPU beginner tips](https://www.kaggle.com/competitions/tpu-getting-started/discussion/159854) \nby [dimitreoliveira](dimitreoliveira)\n\n- **The author shares some TPU beginner tips for this competition, From the post**\n- With TPU your labels need to be of the same type (in case you are using more than 1 data source).\n- Beware of memory issues, I got many of them and some are not easy to spot just looking at error messages.\n- Keep your `batch size` large to take more advantage of the TPU resources, but keep in mind that you may need to adjust your `learning rate`.\n- If you are using `Tensorflow` try to use `tf.data` to load and manipulate your data, it will make things faster!\n- Try to make any data manipulation inside the TPU whenever you can, or at least accelerate your CPU data processing, for example, if you are doing data augmentation you can `flip` images with `tf.data` API with `tf.image.random_flip_left_right(image)`.\n- If you are training more than one model on the same kernel make sure to use `tf.tpu.experimental.initialize_tpu_system(tpu)` to clear TPU memory before loading the next model [here is an example](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-k-fold-optimized-training-loop/notebook#Optimized-training-loop).\n- Make sure you end your kernel after committing or finish editing it, if you don't it will run some minutes after automatic shut down (I think 15 minutes).\n- Watch your TPU idle time to make sure you are using your TPU as much as you should, as [pointed here](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/133447) by Martin.\n\n[TPU only Tensorflow? or we can use PyTorch?](https://www.kaggle.com/competitions/tpu-getting-started/discussion/160958) \nby [paantya](paantya)\n\n- Asking if PyTorch is supported for training in TPU.\n- From the comments:\n- From the comments: Yes, PyTorch XLA supports TPU ([example](https://www.kaggle.com/dhananjay3/fast-pytorch-xla-for-tpu-with-multiprocessing)).\n\n[Detailed guide to custom training with TPUs](https://www.kaggle.com/competitions/tpu-getting-started/discussion/172593)\nby [yihdarshieh](yihdarshieh)\n\n- Sharing a [public notebook](https://www.kaggle.com/yihdarshieh/detailed-guide-to-custom-training-with-tpus) about custom training with TPUs.\n- It combines several of my previous notebooks, including\n- custom training\n- gradient accumulation\n- oversampling\n- perspective transformation (as data augmentation)\n\n[To begin with TPUs](https://www.kaggle.com/competitions/tpu-getting-started/discussion/269317) \nby [aryansakhala](aryansakhala)\n\n- Sharing a [medium blog](https://medium.com/analytics-vidhya/tpu-training-made-easy-with-colab-3b73b920878f) as a guide to begin with TPUs.\n\n[ BiT: Transfer Learning in tensorflow](https://www.kaggle.com/competitions/tpu-getting-started/discussion/213329) \nby [ibrahimsherify](ibrahimsherify)\n\n- Sharing a [notebook](https://www.kaggle.com/ibrahimsherify/bit-transfer-learning-in-tf) and [dataset](https://www.kaggle.com/ibrahimsherify/bit-tf-hub-models) about using BigTransfer (BiT) pretrained model for transfer learning.\n- Uploaded TF hub model as a [Kaggle dataset](https://www.kaggle.com/ibrahimsherify/bit-tf-hub-models)\n\n[Live (+recorded) deep dive on: Computer Vision - Petals to the Metal🌻🌸🌹](https://www.kaggle.com/competitions/tpu-getting-started/discussion/209865) \nby [georgezoto](georgezoto)\n\n- Learning from each other\n- Sharing a [public notebook](https://www.kaggle.com/georgezoto/computer-vision-petals-to-the-metal) and a [youtube plalist](https://www.youtube.com/playlist?list=PL2EBNWEnwV2ciJiH6TmL8v_o42rb65RrN) of a kaggle course.\n\n[Cool TPU beginner resources](https://www.kaggle.com/competitions/tpu-getting-started/discussion/159853) \nby [dimitreoliveira](dimitreoliveira)\n\n- Sharing many TPU beginner resources\n\n### Guides\n- [Tensorflow quick guide on using TPU](https://www.tensorflow.org/guide/tpu#improving_performance_by_multiple_steps_within_tffunction)\n- [Kaggle TPU Youtube playlist](https://www.youtube.com/playlist?list=PLqFaTIg4myu-1c3ygYzakW8-hNzQG59-5)\n- [Tips for improving TPU usage](https://cloud.google.com/tpu/docs/troubleshooting#memory-usage)\n- Training optimizations [disscused here](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/135443) by Martin\n\n### Documentations\n- [Kaggle TPU documentation](https://www.kaggle.com/docs/tpu)\n- [Cloud TPU documentation](https://cloud.google.com/tpu/docs/)\n- In case you wanna use [Mixed precision](https://www.tensorflow.org/guide/mixed_precision)\n- In case you wanna use [XLA](https://www.tensorflow.org/xla)\n\n#### A couple of starting kernels from previous competitions\n\n- [Custom Training Loop with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu/notebook)\n- [Getting started with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu/notebook)\n- Check out many more from the [previous competition](https://www.kaggle.com/c/flower-classification-with-tpus/notebooks)\n\n### Some interesting examples from the community\n**Data augmentation and processing using TFRecords**, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with tf.data this way the transformation will be added to TF graph, here are some examples:\n- [Rotation Augmentation GPU/TPU - [0.96+]](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96), many custom data transformations with a bunch of explanations on why it is more computationally efficient.\n- [Make Chris Deotte's data augmentation faster](https://www.kaggle.com/yihdarshieh/make-chris-deotte-s-data-augmentation-faster), making these transformations even faster by running them inside the TPU.\n- [CutMix and MixUp on GPU/TPU](https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu), Implementation of CutMix and MixUp techniques to increase models generalization.\n- [Flower with TPUs - Advanced augmentations](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-advanced-augmentations/notebook), one idea to give more control of what augmentation you doing on your data batches.\n\nAlso, check out the two previous competitions where TPUs were heavily used [Flower Classification with TPUs](https://www.kaggle.com/c/flower-classification-with-tpus) and [Jigsaw Multilingual Toxic Comment Classification](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification). If you have any more nice resources, post here and let us know!\n\n[Need good resources for understanding TPU](https://www.kaggle.com/competitions/tpu-getting-started/discussion/222684) \nby [pandekp](pandekp)\n\n- Asking for good resources for understanding TPU.\n- From the comments:\n\n**TPU Basics**\n\n- [Getting Started with TPU's](https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease)\n- [TPU Docs](https://www.kaggle.com/docs/tpu)\n- [TFRecords Basics](https://www.kaggle.com/ryanholbrook/tfrecords-basics)\n\n**Great TPU Notebooks built w/ Tensorflow**\n\n- [dimitreoliveira's notebook](https://www.kaggle.com/dimitreoliveira/flower-classification-with-tpus-eda-and-baseline)\n- [Xhlulu's flower competition notebook](https://www.kaggle.com/xhlulu/flowers-tpu-concise-efficientnet-b7)\n\n**Data Augmentation w/ TPU's**\n- [TF Data Augmentation Docs](https://www.kaggle.com/competitions/tpu-getting-started/discussion/tensorflow.org/tutorials/images/data_augmentation)\n- [TF Image Docs](https://www.tensorflow.org/api_docs/python/tf/image)\n\n[What steps can improve Confusion Matrix sharpness?](https://www.kaggle.com/competitions/tpu-getting-started/discussion/160533)\nby [mpwolke](mpwolke)\n\n- Asking what steps can be taken to improve the model on rare edge cases.\n- From the comments:\n- In the [Accelerator Power Hour for data science professionals with Kaggle Grandmasters](https://www.youtube.com/watch?v=DEuvGh4ZwaY), shared a very useful tip about how to increase accuracy for rare classes in post-processing\n\n[TFRecords to Pandas.core.DataFrame](https://www.kaggle.com/competitions/tpu-getting-started/discussion/197988) \nby [divyosmi2009](divyosmi2009)\n\n- Asking how to convert the Prefetch data set into pandas data frame to do eda.\n- From the comments:\n- `pandas_tfrecords.tfrecords_to_pandas(file_paths, schema=None, compression_type='auto', cast=True)\n", "votes": 7 } ], "comments": [], "raw_markdown_by_id": { "1828404": "### Flower Classification on TPU\n#### Summary of Top discussion posts\n_____\n\n[Introduction Notebook](https://www.kaggle.com/competitions/tpu-getting-started/discussion/160793) \nby [kurianbenoy](kurianbenoy)\n\n- Sharing an [introduction notebook](https://www.kaggle.com/kurianbenoy/introduction-kernel-for-petal2metal-flowers) for getting started in the competition\n- **Tips to improve the score:**\n- Use augmentations ([discussions](https://www.kaggle.com/c/flower-classification-with-tpus/discussion))\n- [Techniques to improve scores](https://www.kaggle.com/romanweilguny/tpu-flowers-first-love)\n- Check the best score notebooks in Flower classification comp(there are SOTA models out there)\n- Use Big Transfer architecture: [code](https://github.com/google-research/big_transfer)\n\n[Research Papers: Deep Learning for Flower Classification](https://www.kaggle.com/competitions/tpu-getting-started/discussion/327120)\nby [satoshidatamoto](satoshidatamoto)\n\n- Sharing a list of deep learning papers for flower classification\n\n[TFRecords](https://www.kaggle.com/competitions/tpu-getting-started/discussion/166503)\nby [davidedwards1](davidedwards1)\n\n- A discussion on TFRecords, [starter tutorial here](https://www.kaggle.com/ryanholbrook/tfrecords-basics)\n\n[Efficient Net has low sparse categorical accuracy and validation accuracy](https://www.kaggle.com/competitions/tpu-getting-started/discussion/199759)\nby [yuzheni](yuzheni)\n\n- The author askes about the following code, reporting it to have low performance:\n\n```python\nEPOCHS = 12\nfrom tensorflow.keras.layers.experimental import preprocessing\nwith strategy.scope():\n pretrained_model = tf.keras.applications.EfficientNetB4(\n weights=\"imagenet\",\n include_top=False ,\n input_shape=[*IMAGE_SIZE, 3], pooling=\"max\"\n )\n pretrained_model.trainable = False\n model = tf.keras.Sequential([\n # To a base pretrained on ImageNet to extract features from images...\n #tf.keras.layers.experimental.preprocessing.RandomFlip('horizontal'),\n pretrained_model,\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\n```\n\n- **From the comments:** `pretrained_model.trainable = True`\n\n[Starter Notebooks To Keep You Entertained!](https://www.kaggle.com/competitions/tpu-getting-started/discussion/325287) \nby [satoshidatamoto](satoshidatamoto)\n\n- Listing starter notebooks for this competition.\n\n[Model fitting very slow using TPU](https://www.kaggle.com/competitions/tpu-getting-started/discussion/218062) \nby [yanngarcia](yanngarcia)\n\n- Asking for help debugging a model that is taking a long time to fit using TPU.\n- From the comments: The problem was that the author used augmentations for both training and validation and it was slow.\n\n[Some TPU beginner tips](https://www.kaggle.com/competitions/tpu-getting-started/discussion/159854) \nby [dimitreoliveira](dimitreoliveira)\n\n- **The author shares some TPU beginner tips for this competition, From the post**\n- With TPU your labels need to be of the same type (in case you are using more than 1 data source).\n- Beware of memory issues, I got many of them and some are not easy to spot just looking at error messages.\n- Keep your `batch size` large to take more advantage of the TPU resources, but keep in mind that you may need to adjust your `learning rate`.\n- If you are using `Tensorflow` try to use `tf.data` to load and manipulate your data, it will make things faster!\n- Try to make any data manipulation inside the TPU whenever you can, or at least accelerate your CPU data processing, for example, if you are doing data augmentation you can `flip` images with `tf.data` API with `tf.image.random_flip_left_right(image)`.\n- If you are training more than one model on the same kernel make sure to use `tf.tpu.experimental.initialize_tpu_system(tpu)` to clear TPU memory before loading the next model [here is an example](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-k-fold-optimized-training-loop/notebook#Optimized-training-loop).\n- Make sure you end your kernel after committing or finish editing it, if you don't it will run some minutes after automatic shut down (I think 15 minutes).\n- Watch your TPU idle time to make sure you are using your TPU as much as you should, as [pointed here](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/133447) by Martin.\n\n[TPU only Tensorflow? or we can use PyTorch?](https://www.kaggle.com/competitions/tpu-getting-started/discussion/160958) \nby [paantya](paantya)\n\n- Asking if PyTorch is supported for training in TPU.\n- From the comments:\n- From the comments: Yes, PyTorch XLA supports TPU ([example](https://www.kaggle.com/dhananjay3/fast-pytorch-xla-for-tpu-with-multiprocessing)).\n\n[Detailed guide to custom training with TPUs](https://www.kaggle.com/competitions/tpu-getting-started/discussion/172593)\nby [yihdarshieh](yihdarshieh)\n\n- Sharing a [public notebook](https://www.kaggle.com/yihdarshieh/detailed-guide-to-custom-training-with-tpus) about custom training with TPUs.\n- It combines several of my previous notebooks, including\n- custom training\n- gradient accumulation\n- oversampling\n- perspective transformation (as data augmentation)\n\n[To begin with TPUs](https://www.kaggle.com/competitions/tpu-getting-started/discussion/269317) \nby [aryansakhala](aryansakhala)\n\n- Sharing a [medium blog](https://medium.com/analytics-vidhya/tpu-training-made-easy-with-colab-3b73b920878f) as a guide to begin with TPUs.\n\n[ BiT: Transfer Learning in tensorflow](https://www.kaggle.com/competitions/tpu-getting-started/discussion/213329) \nby [ibrahimsherify](ibrahimsherify)\n\n- Sharing a [notebook](https://www.kaggle.com/ibrahimsherify/bit-transfer-learning-in-tf) and [dataset](https://www.kaggle.com/ibrahimsherify/bit-tf-hub-models) about using BigTransfer (BiT) pretrained model for transfer learning.\n- Uploaded TF hub model as a [Kaggle dataset](https://www.kaggle.com/ibrahimsherify/bit-tf-hub-models)\n\n[Live (+recorded) deep dive on: Computer Vision - Petals to the Metal🌻🌸🌹](https://www.kaggle.com/competitions/tpu-getting-started/discussion/209865) \nby [georgezoto](georgezoto)\n\n- Learning from each other\n- Sharing a [public notebook](https://www.kaggle.com/georgezoto/computer-vision-petals-to-the-metal) and a [youtube plalist](https://www.youtube.com/playlist?list=PL2EBNWEnwV2ciJiH6TmL8v_o42rb65RrN) of a kaggle course.\n\n[Cool TPU beginner resources](https://www.kaggle.com/competitions/tpu-getting-started/discussion/159853) \nby [dimitreoliveira](dimitreoliveira)\n\n- Sharing many TPU beginner resources\n\n### Guides\n- [Tensorflow quick guide on using TPU](https://www.tensorflow.org/guide/tpu#improving_performance_by_multiple_steps_within_tffunction)\n- [Kaggle TPU Youtube playlist](https://www.youtube.com/playlist?list=PLqFaTIg4myu-1c3ygYzakW8-hNzQG59-5)\n- [Tips for improving TPU usage](https://cloud.google.com/tpu/docs/troubleshooting#memory-usage)\n- Training optimizations [disscused here](https://www.kaggle.com/c/flower-classification-with-tpus/discussion/135443) by Martin\n\n### Documentations\n- [Kaggle TPU documentation](https://www.kaggle.com/docs/tpu)\n- [Cloud TPU documentation](https://cloud.google.com/tpu/docs/)\n- In case you wanna use [Mixed precision](https://www.tensorflow.org/guide/mixed_precision)\n- In case you wanna use [XLA](https://www.tensorflow.org/xla)\n\n#### A couple of starting kernels from previous competitions\n\n- [Custom Training Loop with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu/notebook)\n- [Getting started with 100+ flowers on TPU](https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu/notebook)\n- Check out many more from the [previous competition](https://www.kaggle.com/c/flower-classification-with-tpus/notebooks)\n\n### Some interesting examples from the community\n**Data augmentation and processing using TFRecords**, as mentioned on this other topic, to take full advantage of TPU you should do every possible computation inside the TPU, or accelerate your CPU data processing, including data augmentation, this can be achieved doing the transformation with tf.data this way the transformation will be added to TF graph, here are some examples:\n- [Rotation Augmentation GPU/TPU - [0.96+]](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96), many custom data transformations with a bunch of explanations on why it is more computationally efficient.\n- [Make Chris Deotte's data augmentation faster](https://www.kaggle.com/yihdarshieh/make-chris-deotte-s-data-augmentation-faster), making these transformations even faster by running them inside the TPU.\n- [CutMix and MixUp on GPU/TPU](https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu), Implementation of CutMix and MixUp techniques to increase models generalization.\n- [Flower with TPUs - Advanced augmentations](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-advanced-augmentations/notebook), one idea to give more control of what augmentation you doing on your data batches.\n\nAlso, check out the two previous competitions where TPUs were heavily used [Flower Classification with TPUs](https://www.kaggle.com/c/flower-classification-with-tpus) and [Jigsaw Multilingual Toxic Comment Classification](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification). If you have any more nice resources, post here and let us know!\n\n[Need good resources for understanding TPU](https://www.kaggle.com/competitions/tpu-getting-started/discussion/222684) \nby [pandekp](pandekp)\n\n- Asking for good resources for understanding TPU.\n- From the comments:\n\n**TPU Basics**\n\n- [Getting Started with TPU's](https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease)\n- [TPU Docs](https://www.kaggle.com/docs/tpu)\n- [TFRecords Basics](https://www.kaggle.com/ryanholbrook/tfrecords-basics)\n\n**Great TPU Notebooks built w/ Tensorflow**\n\n- [dimitreoliveira's notebook](https://www.kaggle.com/dimitreoliveira/flower-classification-with-tpus-eda-and-baseline)\n- [Xhlulu's flower competition notebook](https://www.kaggle.com/xhlulu/flowers-tpu-concise-efficientnet-b7)\n\n**Data Augmentation w/ TPU's**\n- [TF Data Augmentation Docs](https://www.kaggle.com/competitions/tpu-getting-started/discussion/tensorflow.org/tutorials/images/data_augmentation)\n- [TF Image Docs](https://www.tensorflow.org/api_docs/python/tf/image)\n\n[What steps can improve Confusion Matrix sharpness?](https://www.kaggle.com/competitions/tpu-getting-started/discussion/160533)\nby [mpwolke](mpwolke)\n\n- Asking what steps can be taken to improve the model on rare edge cases.\n- From the comments:\n- In the [Accelerator Power Hour for data science professionals with Kaggle Grandmasters](https://www.youtube.com/watch?v=DEuvGh4ZwaY), shared a very useful tip about how to increase accuracy for rare classes in post-processing\n\n[TFRecords to Pandas.core.DataFrame](https://www.kaggle.com/competitions/tpu-getting-started/discussion/197988) \nby [divyosmi2009](divyosmi2009)\n\n- Asking how to convert the Prefetch data set into pandas data frame to do eda.\n- From the comments:\n- `pandas_tfrecords.tfrecords_to_pandas(file_paths, schema=None, compression_type='auto', cast=True)\n" } }