Hi everyone, the standard augmentation on TPU showed by the starter kernel is not very close to what we are used to doing like the albumentations lib, it may be hard to group and combine transformations and decide how and when to use each of them, so I've created a kernel to show one possible solution to that, and give us more freedom to augment images and still keep TPU efficiency.
Here is the kernel link, basically it uses probabilities to group and combine transformations as we do with imgaug and albumentations, there is still a lot of room to improve so feedback is welcomed.
\n", "messages": [ { "id": "759956", "postDate": "02/29/2020 16:49:29", "content": "ps: I've also used @cdeotte 's custom transformations but broken down to single functions.
\n
Hi everyone, the standard augmentation on TPU showed by the starter kernel is not very close to what we are used to doing like the albumentations lib, it may be hard to group and combine transformations and decide how and when to use each of them, so I've created a kernel to show one possible solution to that, and give us more freedom to augment images and still keep TPU efficiency.
Here is the kernel link, basically it uses probabilities to group and combine transformations as we do with imgaug and albumentations, there is still a lot of room to improve so feedback is welcomed.
\n", "rawMarkdown": "Hi everyone, the standard augmentation on TPU showed by the starter kernel is not very close to what we are used to doing like the `albumentations` lib, it may be hard to group and combine transformations and decide how and when to use each of them, so I've created a kernel to show one possible solution to that, and give us more freedom to augment images and still keep TPU efficiency.\n\nHere is the [kernel link](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-advanced-augmentation/), basically it uses probabilities to group and combine transformations as we do with `imgaug` and `albumentations`, there is still a lot of room to improve so feedback is welcomed.\n\n> ps: I've also used @cdeotte 's custom transformations but broken down to single functions.", "votes": null }, { "id": "760330", "postDate": "03/01/2020 06:13:35", "content": "ps: I've also used @cdeotte 's custom transformations but broken down to single functions.
\n
Nice way of applying augmentations.
", "rawMarkdown": "Nice way of applying augmentations.", "votes": null }, { "id": "761721", "postDate": "03/02/2020 21:44:14", "content": "Fantastic. Thank you for the contribution.
", "rawMarkdown": "Fantastic. Thank you for the contribution.", "votes": null }, { "id": "761739", "postDate": "03/02/2020 22:10:47", "content": "hey @mgornergoogle I have a question would be more efficient to wrap the augmentation function with @tf function? like this:
```\n@tf function\ndef data_augment(image, label):\n p_crop = tf.random.uniform([1], minval=0, maxval=1, dtype='float32', seed=seed)
\n\n## Pixel-level transforms\nif p_pixel >= .4: # pixel transformations\n if p_pixel >= .85:\n image = tf.image.random_saturation(image, lower=0, upper=2, seed=seed)\n elif p_pixel >= .65:\n image = tf.image.random_contrast(image, lower=.8, upper=2, seed=seed)\n elif p_pixel >= .5:\n image = tf.image.random_brightness(image, max_delta=.2, seed=seed)\n else:\n image = tf.image.adjust_gamma(image, gamma=.6)\n\nreturn image, label\n```
", "rawMarkdown": "hey @mgornergoogle I have a question would be more efficient to wrap the augmentation function with `@tf function`? like this:\n\n```\n@tf function\ndef data_augment(image, label):\n p_crop = tf.random.uniform([1], minval=0, maxval=1, dtype='float32', seed=seed)\n \n ## Pixel-level transforms\n if p_pixel >= .4: # pixel transformations\n if p_pixel >= .85:\n image = tf.image.random_saturation(image, lower=0, upper=2, seed=seed)\n elif p_pixel >= .65:\n image = tf.image.random_contrast(image, lower=.8, upper=2, seed=seed)\n elif p_pixel >= .5:\n image = tf.image.random_brightness(image, max_delta=.2, seed=seed)\n else:\n image = tf.image.adjust_gamma(image, gamma=.6)\n\n return image, label\n```", "votes": null }, { "id": "761760", "postDate": "03/02/2020 22:48:57", "content": "As far as I know this should be automatic for code inside of tf.data.Datset but I'm not 100% on this. I'll try and get an authoritative answer.
", "rawMarkdown": "As far as I know this should be automatic for code inside of tf.data.Datset but I'm not 100% on this. I'll try and get an authoritative answer.", "votes": null } ], "comments": [ { "id": 760330, "author_name": "gskdhiman", "author_url": "", "post_date": "03/01/2020 06:13:35", "content": "Nice way of applying augmentations.
", "votes": null, "replies": [] }, { "id": 761721, "author_name": "mgorner", "author_url": "", "post_date": "03/02/2020 21:44:14", "content": "Fantastic. Thank you for the contribution.
", "votes": null, "replies": [ { "id": 761739, "author_name": "dimitreoliveira", "author_url": "", "post_date": "03/02/2020 22:10:47", "content": "hey @mgornergoogle I have a question would be more efficient to wrap the augmentation function with @tf function? like this:
```\n@tf function\ndef data_augment(image, label):\n p_crop = tf.random.uniform([1], minval=0, maxval=1, dtype='float32', seed=seed)
\n\n## Pixel-level transforms\nif p_pixel >= .4: # pixel transformations\n if p_pixel >= .85:\n image = tf.image.random_saturation(image, lower=0, upper=2, seed=seed)\n elif p_pixel >= .65:\n image = tf.image.random_contrast(image, lower=.8, upper=2, seed=seed)\n elif p_pixel >= .5:\n image = tf.image.random_brightness(image, max_delta=.2, seed=seed)\n else:\n image = tf.image.adjust_gamma(image, gamma=.6)\n\nreturn image, label\n```
", "votes": null, "replies": [] }, { "id": 761760, "author_name": "mgorner", "author_url": "", "post_date": "03/02/2020 22:48:57", "content": "As far as I know this should be automatic for code inside of tf.data.Datset but I'm not 100% on this. I'll try and get an authoritative answer.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "759956": "Hi everyone, the standard augmentation on TPU showed by the starter kernel is not very close to what we are used to doing like the `albumentations` lib, it may be hard to group and combine transformations and decide how and when to use each of them, so I've created a kernel to show one possible solution to that, and give us more freedom to augment images and still keep TPU efficiency.\n\nHere is the [kernel link](https://www.kaggle.com/dimitreoliveira/flower-with-tpus-advanced-augmentation/), basically it uses probabilities to group and combine transformations as we do with `imgaug` and `albumentations`, there is still a lot of room to improve so feedback is welcomed.\n\n> ps: I've also used @cdeotte 's custom transformations but broken down to single functions.", "760330": "Nice way of applying augmentations.", "761721": "Fantastic. Thank you for the contribution.", "761739": "hey @mgornergoogle I have a question would be more efficient to wrap the augmentation function with `@tf function`? like this:\n\n```\n@tf function\ndef data_augment(image, label):\n p_crop = tf.random.uniform([1], minval=0, maxval=1, dtype='float32', seed=seed)\n \n ## Pixel-level transforms\n if p_pixel >= .4: # pixel transformations\n if p_pixel >= .85:\n image = tf.image.random_saturation(image, lower=0, upper=2, seed=seed)\n elif p_pixel >= .65:\n image = tf.image.random_contrast(image, lower=.8, upper=2, seed=seed)\n elif p_pixel >= .5:\n image = tf.image.random_brightness(image, max_delta=.2, seed=seed)\n else:\n image = tf.image.adjust_gamma(image, gamma=.6)\n\n return image, label\n```", "761760": "As far as I know this should be automatic for code inside of tf.data.Datset but I'm not 100% on this. I'll try and get an authoritative answer." }, "source": "meta" }