{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nfrom fastai.vision.all import *\nimport albumentations as A # the albumentations library has the transformations we will be using","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\nThe goal of this notebook is to succinctly demonstrate the general process of transforming images via the `Albumentations` library for use in `fastai` `DataBlock`s.\n\n## Sources\n* [Albumentations Library](https://github.com/albumentations-team/albumentations)\n* [Tutorial: Custom Transforms | Fastai](https://docs.fast.ai/tutorial.albumentations.html)\n* [Transform class documentation](https://fastcore.fast.ai/transform#Transform)\n\n## Minimum Working Code Template\nScroll to the bottom if all you're interested in is a minimal working code template for creating a transformation that can be passed to a `fastai` `DataBlock`.\n\n## Global options\n* TEST = True to use only a small subset of images to save time/resources."},{"metadata":{"trusted":true},"cell_type":"code","source":"TEST = True\n\ndef set_seeds():\n    random.seed(42)\n    np.random.seed(12345)\n    torch.manual_seed(1234)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Setup\n\nNothing new here -- the same process as used previously to set up the data for the Cassava competition. We will use only a small subset of the images for testing purposes."},{"metadata":{"trusted":true},"cell_type":"code","source":"path = Path('../input/cassava-leaf-disease-classification')\ntrain_df = pd.read_csv(path/'train.csv')\ntrain_df['image_id'] = train_df['image_id'].apply(lambda x: f'train_images/{x}')\n\nif TEST: train_df = train_df[0:100] # use only 100 training examples if TEST is True\n    \ntrain_df.head(), train_df.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Making Labels More Interpretable"},{"metadata":{"trusted":true},"cell_type":"code","source":"idx2lbl = {0:\"Cassava Bacterial Blight (CBB)\",\n          1:\"Cassava Brown Streak Disease (CBSD)\",\n          2:\"Cassava Green Mottle (CGM)\",\n          3:\"Cassava Mosaic Disease (CMD)\",\n          4:\"Healthy\"}\n\ntrain_df['label'].replace(idx2lbl, inplace=True)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Preparing our Image Transformation(s)\nWe will be using the `albumentations` library, which provides many different image transformation options. Our goal, then, is to make the transformations from that library usable within the `fastai` `DataBlock` API.\n\nFirst, we will look at a single example to make sure we can correctly implement the transformation of interest. Here is the base image we'll be transforming."},{"metadata":{"trusted":true},"cell_type":"code","source":"img = PILImage.create(path/train_df['image_id'][49])\nimg = img.resize((224,224))\nimg","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Transformations as Simple Functions\nWe will begin by defining some simple functions for transforming the images and visualizing the transformations. At this phase, we won't worry about making them work with the fastai `DataBlock`s.\n\nWe start by defining a generic function that should work for any of the `albumentations` transforms. This package handles the necessary transformations between data types. We have `PILImage` images while the package works on `numpy` images, so we need to convert between types."},{"metadata":{"trusted":true},"cell_type":"code","source":"def aug_tfm(img): \n    np_img = np.array(img) # converts image to numpy array\n    aug_img = aug(image=np_img)['image'] # applies transformation (defined outside of function)\n    return PILImage.create(aug_img) #returns and visualizes PILImage","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = A.ToGray(p=1)\naug_tfm(img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Dropout"},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = A.CoarseDropout(p=1, min_holes = 40, max_holes=50)\naug_tfm(img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Fog"},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = A.RandomFog(p=1)\naug_tfm(img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Compositions of Transformations\nMultiple transformations can be combined in a single pipeline."},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = A.Compose([\n    A.ToGray(p=1),\n    A.RandomFog(p=1),\n    A.CoarseDropout(p=1, min_holes = 40, max_holes=50),\n])\naug_tfm(img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Making these Transformations Work with Fastai\n\nWe will now make these transformations work with the fastai `DataBlock` API. We will demonstrate using the `CoarseDropout` transformation defined above, as it provides a highly-visible transformation, making it immediately obvious whether the transformation was successfully applied.\n\n## \"Baseline\" `DataBlock`\nFirst we show our datablock without any transformations applied."},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_x(row): return path/row['image_id']\ndef get_y(row): return row['label']\n\nset_seeds()\ndb = DataBlock(blocks = (ImageBlock, CategoryBlock),\n                 get_x = get_x,\n                 get_y = get_y,\n                 splitter = RandomSplitter(valid_pct=0.2),\n                 item_tfms = [Resize(224)],\n                 batch_tfms = [*aug_transforms(), Normalize.from_stats(*imagenet_stats)])\n\nbs=10 if TEST else 64\ndls = db.dataloaders(train_df, bs=bs)\ndls.show_batch(max_n = 3, figsize=((12,12)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Transformations in the `DataBlock`\n\nNext, we apply our transformations as `item_tfms`. To do this, we need to package our transforms into a class that provides a few extra details to the `DataBlock`.\n- `split_idx`: `0` is for training set; `1` is for validation set; `none` is for both.\n- `order` tells when to run relative to the other transforms. So `order=2` in the example below says to run the transform after the inital resize.\n\nAs with the function we defined above, the class we defined below is very modular. We can try out different definitions of `aug` with the `MyTransform` class."},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = A.CoarseDropout(p=1, min_holes = 40, max_holes=50)\n\nclass MyTransform(Transform):\n    split_idx=None #runs on training and valid\n    order = 2 # runs after initial resize\n    def __init__(self, aug): self.aug = aug\n    def encodes(self, img: PILImage):\n        aug_img = self.aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)\nset_seeds()\ndb = DataBlock(blocks = (ImageBlock, CategoryBlock),\n                 get_x = get_x,\n                 get_y = get_y,\n                 splitter = RandomSplitter(valid_pct=0.2),\n                 item_tfms = [Resize(224), MyTransform(aug)],\n                 batch_tfms = [*aug_transforms(), Normalize.from_stats(*imagenet_stats)])\n\nbs=10 if TEST else 64\ndls = db.dataloaders(train_df, bs=bs)\ndls.show_batch(max_n = 3, figsize=((12,12)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Because we specified `idx=None`, this transformation was applied to the validation set as well."},{"metadata":{"trusted":true},"cell_type":"code","source":"set_seeds()\ndls.valid.show_batch(figsize=((12,12)), max_n = 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below, we demonstrate that changing the `split_idx` argument to `0` ensures the transformation is *not* applied to the validation set."},{"metadata":{"trusted":true},"cell_type":"code","source":"class MyTransform(Transform):\n    split_idx=0 #runs on training and valid\n    order = 2 # runs after initial resize\n    def __init__(self, aug): self.aug = aug\n    def encodes(self, img: PILImage):\n        aug_img = self.aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)\nset_seeds()   \ndb = DataBlock(blocks = (ImageBlock, CategoryBlock),\n                 get_x = get_x,\n                 get_y = get_y,\n                 splitter = RandomSplitter(valid_pct=0.2),\n                 item_tfms = [Resize(224), MyTransform(aug)],\n                 batch_tfms = [*aug_transforms(), Normalize.from_stats(*imagenet_stats)])\n\nbs=10 if TEST else 64\ndls = db.dataloaders(train_df, bs=bs)\ndls.valid.show_batch(max_n = 3, figsize=((12,12)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## A note note on `batch_tfms`\n\nI tried to apply this with `batch_tfms` with no real expectation of it working. The class defined above is clearly defined to work on a single image, not on a batch, so unless there's some magic happening in the background, I wouldn't expect it to work.\n\nThere is an interesting discussion [here](https://forums.fast.ai/t/is-it-possible-to-combine-aug-transforms-with-some-transforms-of-albumentations-for-segmentation/66666/13) on the topic for anyone interested, but for our purposes, sticking with `item_tfms` is sufficient.\n\n# Minimal Working Code Template"},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = A.CoarseDropout(p=1, min_holes = 40, max_holes=50) # or whatever transform from albumentations you want to use\n\nclass MyTransform(Transform):\n    split_idx=None #runs on training and valid (0 for train, 1 for valid)\n    order = 2 # runs after initial resize\n    def __init__(self, aug): self.aug = aug\n    def encodes(self, img: PILImage):\n        aug_img = self.aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)\n\ndb = DataBlock(blocks = (ImageBlock, CategoryBlock),\n                 get_x = get_x,\n                 get_y = get_y,\n                 splitter = RandomSplitter(valid_pct=0.2),\n                 item_tfms = [Resize(224), MyTransform(aug)], # put the defined class here.\n                 batch_tfms = [*aug_transforms(), Normalize.from_stats(*imagenet_stats)])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}