Hey everyone, as the competition coming to an end I'm pretty sure a lot of us are trying to use TPUs efficiently. I'm currently struggling with this specially with augmentations most of the time TPU is just waiting for data to load, what's the solution ? and also what are the tips and tricks to make notebooks run faster ?
\nThanks and good luck !
Hey everyone, as the competition coming to an end I'm pretty sure a lot of us are trying to use TPUs efficiently. I'm currently struggling with this specially with augmentations most of the time TPU is just waiting for data to load, what's the solution ? and also what are the tips and tricks to make notebooks run faster ?
\nThanks and good luck !
If you want to speed up time by 20-30%. Perhaps no mixed precision is a bit more accurate, however.
\n# SET MIXED PRECISION\nMIXED_PRECISION = True\nXLA_ACCELERATE = True\n\nif MIXED_PRECISION:\n from tensorflow.keras.mixed_precision import experimental as mixed_precision\n if DEVICE == 'TPU': policy = tf.keras.mixed_precision.experimental.Policy('mixed_bfloat16')\n else: policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')\n mixed_precision.set_policy(policy)\n print('Mixed precision enabled')\n\nif XLA_ACCELERATE:\n tf.config.optimizer.set_jit(True)\n print('Accelerated Linear Algebra enabled')\nand you have to change the output layer to float32
\nx = tf.keras.layers.Dense(1,activation='sigmoid',dtype='float32')(x)\nHey ! I am really not an expert on TPUs (and in Machine Learning in general) but I've made a notebook about how to use TPU while doing data augmentation on the MNIST digit recognizer competition.
\nHere is the link https://www.kaggle.com/foucardm/data-augmentation-on-tpu-for-mnist-classification/ (I am not even sure this how to share a notebook).
Hope this will be useful.
", "rawMarkdown": "Hey ! I am really not an expert on TPUs (and in Machine Learning in general) but I've made a notebook about how to use TPU while doing data augmentation on the MNIST digit recognizer competition.\nHere is the link https://www.kaggle.com/foucardm/data-augmentation-on-tpu-for-mnist-classification/ (I am not even sure this how to share a notebook).\n\nHope this will be useful.", "votes": 1, "replies": [ { "id": 969658, "postDate": "2020-08-13T20:42:15.380Z", "content": "Hey thanks thanks for sharing btw this is how you link something : while wrting a comment there's an insert link button use it to put the link into a word that you can click on it (click in this example)
", "rawMarkdown": "Hey thanks thanks for sharing btw this is how you link something : while wrting a comment there's an insert link button use it to put the link into a word that you can [click ](https://www.kaggle.com/foucardm/data-augmentation-on-tpu-for-mnist-classification/)on it (click in this example)", "votes": 1 } ] }, { "id": 968298, "postDate": "2020-08-12T21:05:17.057Z", "content": "Bookmarked. I have the same questions being very new to TPU.
\nI am also using TPU on Google Colab. I would love to hear tips/tricks to streamline notebooks on Kaggle and Colab.
\nThanks!
", "rawMarkdown": "Bookmarked. I have the same questions being very new to TPU.\n\nI am also using TPU on Google Colab. I would love to hear tips/tricks to streamline notebooks on Kaggle and Colab. \n\nThanks!", "votes": 1 } ], "comments": [ { "id": 969540, "author_name": "Tim Yee", "author_url": "", "post_date": "2020-08-13T18:56:46.833000", "content": "If you want to speed up time by 20-30%. Perhaps no mixed precision is a bit more accurate, however.
\n# SET MIXED PRECISION\nMIXED_PRECISION = True\nXLA_ACCELERATE = True\n\nif MIXED_PRECISION:\n from tensorflow.keras.mixed_precision import experimental as mixed_precision\n if DEVICE == 'TPU': policy = tf.keras.mixed_precision.experimental.Policy('mixed_bfloat16')\n else: policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')\n mixed_precision.set_policy(policy)\n print('Mixed precision enabled')\n\nif XLA_ACCELERATE:\n tf.config.optimizer.set_jit(True)\n print('Accelerated Linear Algebra enabled')\nand you have to change the output layer to float32
\nx = tf.keras.layers.Dense(1,activation='sigmoid',dtype='float32')(x)\nHey ! I am really not an expert on TPUs (and in Machine Learning in general) but I've made a notebook about how to use TPU while doing data augmentation on the MNIST digit recognizer competition.
\nHere is the link https://www.kaggle.com/foucardm/data-augmentation-on-tpu-for-mnist-classification/ (I am not even sure this how to share a notebook).
Hope this will be useful.
", "votes": 1, "replies": [ { "id": 969658, "author_name": "Aziz_Belaweid", "author_url": "", "post_date": "2020-08-13T20:42:15.380000", "content": "Hey thanks thanks for sharing btw this is how you link something : while wrting a comment there's an insert link button use it to put the link into a word that you can click on it (click in this example)
", "votes": 1, "replies": [] } ] }, { "id": 968298, "author_name": "Leo Dzung", "author_url": "", "post_date": "2020-08-12T21:05:17.057000", "content": "Bookmarked. I have the same questions being very new to TPU.
\nI am also using TPU on Google Colab. I would love to hear tips/tricks to streamline notebooks on Kaggle and Colab.
\nThanks!
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "967033": "Hey everyone, as the competition coming to an end I'm pretty sure a lot of us are trying to use TPUs efficiently. I'm currently struggling with this specially with augmentations most of the time TPU is just waiting for data to load, what's the solution ? and also what are the tips and tricks to make notebooks run faster ? \nThanks and good luck !", "969540": "If you want to speed up time by 20-30%. Perhaps no mixed precision is a bit more accurate, however.\n\n```\n# SET MIXED PRECISION\nMIXED_PRECISION = True\nXLA_ACCELERATE = True\n\nif MIXED_PRECISION:\n from tensorflow.keras.mixed_precision import experimental as mixed_precision\n if DEVICE == 'TPU': policy = tf.keras.mixed_precision.experimental.Policy('mixed_bfloat16')\n else: policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')\n mixed_precision.set_policy(policy)\n print('Mixed precision enabled')\n\nif XLA_ACCELERATE:\n tf.config.optimizer.set_jit(True)\n print('Accelerated Linear Algebra enabled')\n```\nand you have to change the output layer to float32\n\n```\nx = tf.keras.layers.Dense(1,activation='sigmoid',dtype='float32')(x)\n```", "969504": "Hey ! I am really not an expert on TPUs (and in Machine Learning in general) but I've made a notebook about how to use TPU while doing data augmentation on the MNIST digit recognizer competition.\nHere is the link https://www.kaggle.com/foucardm/data-augmentation-on-tpu-for-mnist-classification/ (I am not even sure this how to share a notebook).\n\nHope this will be useful.", "968298": "Bookmarked. I have the same questions being very new to TPU.\n\nI am also using TPU on Google Colab. I would love to hear tips/tricks to streamline notebooks on Kaggle and Colab. \n\nThanks!" } }