{"cells":[{"metadata":{},"cell_type":"markdown","source":"<img src='https://albumentations.ai/docs/images/logo.png' width='160'>\n\n\n<h1><center>[Tutorial] Albumentations with masks</center><h1>\n    \n# <a id='1'>Let's start augmentation with masks🔥 </a>"},{"metadata":{},"cell_type":"markdown","source":"- Version `13` : Add `CoarseDropout with Masks` and more examples\n- Version `8` : Add basic guideline and simple examples"},{"metadata":{},"cell_type":"markdown","source":"## If this kernel is useful, <font color='orange'>please upvote</font>!"},{"metadata":{},"cell_type":"markdown","source":"# What is Albumentations\n\n`Albumentations` is a Python library for fast and flexible image augmentations. \n\nAlbumentations efficiently implements a rich variety of image transform operations that are optimized for performance, and does so while providing a concise, yet powerful image augmentation interface for different computer vision tasks, including object classification, segmentation, and detection.\n\n<img src='https://camo.githubusercontent.com/3bb6e4bb500d96ad7bb4e4047af22a63ddf3242a894adf55ebffd3e184e4d113/68747470733a2f2f686162726173746f726167652e6f72672f776562742f62642f6e652f72762f62646e6572763563746b75646d73617a6e687734637273646669772e6a706567' width='640'>\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"## New feature - releases 0.5.1 \n\n`A.ToTensorV2` now supports an additional argument transpose_mask (False by default).\n\nIf the argument is set to True and an input mask has 3 dimensions, A.ToTensorV2 will transpose dimensions of a mask tensor in addition to transposing dimensions of an image tensor.\n\n"},{"metadata":{},"cell_type":"markdown","source":"# Installation"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-output":false},"cell_type":"code","source":"!pip install -U git+https://github.com/albumentations-team/albumentations > /dev/null 2>&1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport cv2\nimport numpy as np\n\nimport torch\nfrom torch.utils.data import TensorDataset, DataLoader, Dataset\nimport albumentations as A\nfrom albumentations.pytorch import ToTensorV2\n\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Dataset"},{"metadata":{},"cell_type":"markdown","source":"https://www.kaggle.com/orkatz2/hubmap-res34unet-baseline-train"},{"metadata":{"trusted":true},"cell_type":"code","source":"images_path = '../input/hubmap-256x256/train'\nmasks_path = '../input/hubmap-256x256/masks'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class HuBMAPDataset(Dataset):\n    def __init__(self, ids, transforms=None, preprocessing=None):\n        self.ids = ids\n        self.transforms = transforms\n        self.preprocessing = preprocessing\n    def __getitem__(self, idx):\n        name = self.ids[idx]\n        img = cv2.imread(f\"{images_path}/{name}\")\n        mask = cv2.imread(f\"{masks_path}/{name}\")[:,:,0:1]\n        if self.transforms:\n            augmented = self.transforms(image=img, mask=mask)\n            img = augmented['image']\n            mask = augmented['mask']\n        '''\n        # Now, we are use new feature in albumentations\n        if self.preprocessing:\n            preprocessed = self.preprocessing(image=img, mask=mask)\n            img = preprocessed['image']\n            mask = preprocessed['mask']\n        '''\n        return img, mask\n\n    def __len__(self):\n        return len(self.ids)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Albumentations with masks"},{"metadata":{},"cell_type":"markdown","source":"Use `transpose_mask`=`True`."},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_train_augmentation(size=1024):\n    return A.Compose([\n        A.HorizontalFlip(),\n        A.VerticalFlip(),\n        A.ShiftScaleRotate(),\n        A.CoarseDropout(max_holes=8, max_height=20, max_width=20, mask_fill_value=0, always_apply=True), # For visualization, set always_apply=True.\n        A.Resize(size,size, always_apply=True),\n        A.Normalize(\n            mean=(0.485, 0.456, 0.406),\n            std=(0.229, 0.224, 0.225)\n        ),\n        ## Check transpose_mask=True\n        ToTensorV2(transpose_mask=True)\n    ])\n\ndef get_valid_augmentation(size=1024):\n    return A.Compose([\n        A.Resize(size,size, always_apply=True),\n        A.Normalize(\n            mean=(0.485, 0.456, 0.406),\n            std=(0.229, 0.224, 0.225)\n        ),\n        ## Check transpose_mask=True\n        ToTensorV2(transpose_mask=True)\n    ])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = os.listdir(images_path)\ntrain_lsit = list(set([row.split(\"_\")[0] for row in data]))\ntrain_idx = [row for row in data if row.split(\"_\")[0] in train_lsit[:-2]]\nvalid_idx = [row for row in data if row.split(\"_\")[0] not in train_lsit[:-2]]\nlen(train_idx),len(valid_idx)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_aug = get_train_augmentation(size=1024)\ntrain_dataset = HuBMAPDataset(\n    ids=train_idx,\n    transforms=train_aug)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Applying augmentation with mask"},{"metadata":{},"cell_type":"markdown","source":"## First\nFocus on that image and compare second one.\n\nWe can see also `CoarseDropout` results between image and mask."},{"metadata":{"trusted":true},"cell_type":"code","source":"# I pick train_dataset[11] for mask\nimg, mask = train_dataset[11]\nimg = img.permute(1,2,0).detach().numpy()\nmask = mask.permute(1,2,0).detach().numpy()\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(img)\nax[1].imshow(mask[:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Second"},{"metadata":{"trusted":true},"cell_type":"code","source":"# I pick train_dataset[11] for mask\nimg, mask = train_dataset[11]\nimg = img.permute(1,2,0).detach().numpy()\nmask = mask.permute(1,2,0).detach().numpy()\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(img)\nax[1].imshow(mask[:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Different augmentations **are applied to** `image` and `mask`!"},{"metadata":{},"cell_type":"markdown","source":"# Multiple View"},{"metadata":{"trusted":true},"cell_type":"code","source":"def collate_fn(batch):\n    return tuple(zip(*batch))\n\ntrain_loader = DataLoader(\n    train_dataset, \n    batch_size=16,\n    collate_fn=collate_fn\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"n_rows=4\nn_cols=4\n\ntrain_loader = DataLoader(train_dataset, batch_size=16, collate_fn=collate_fn)\n\nimages, masks = next(iter(train_loader))\n\n# plot some augmentations!\nfig, ax = plt.subplots(figsize=(20, 20),  nrows=n_rows, ncols=n_cols)\nfor i in range (n_rows*n_cols):     \n    image = images[i].permute(1,2,0).detach().numpy()\n    mask = masks[i].permute(1,2,0).detach().numpy()\n\n    ax[i // n_rows][i % n_cols].imshow(image)\n    ax[i // n_rows][i % n_cols].imshow(mask[:,:,0], cmap=\"hot\", alpha=0.7)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Let's start more Albumentation with masks"},{"metadata":{},"cell_type":"markdown","source":"https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter"},{"metadata":{},"cell_type":"markdown","source":"## Base Image and mask\n\nI used `train_dataset[11]` for visualization."},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset_index = 11","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"valid_aug = get_valid_augmentation(size=1024)\ntrain_dataset = HuBMAPDataset(\n    ids=train_idx,\n    transforms=valid_aug)\n\nimg, mask = train_dataset[dataset_index]\n\nimg = img.permute(1,2,0).detach().numpy()\nmask = mask.permute(1,2,0).detach().numpy()\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(img)\nax[1].imshow(mask[:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## HorizontalFlip"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"transformed_mask = A.HorizontalFlip(always_apply=True)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## VerticalFlip"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"transformed_mask = A.VerticalFlip(always_apply=True)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## RandomRotate90"},{"metadata":{"trusted":true},"cell_type":"code","source":"transformed_mask = A.RandomRotate90(always_apply=True)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## ShiftScaleRotate"},{"metadata":{"trusted":true},"cell_type":"code","source":"transformed_mask = A.ShiftScaleRotate(shift_limit=0.25, \n                                      scale_limit=0.25, \n                                      rotate_limit=15, \n                                      border_mode=cv2.BORDER_REFLECT,\n                                      always_apply=True)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## OpticalDistortion"},{"metadata":{},"cell_type":"markdown","source":"- Parameters such as `distort_limit`, `shift_limit` are exaggerated for visualization.\n- Be careful of use."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"transformed_mask = A.OpticalDistortion(always_apply=True, \n                                       distort_limit=0.85, \n                                       shift_limit=0.85, \n                                       mask_value=0)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## GridDistortion"},{"metadata":{},"cell_type":"markdown","source":"- Parameters such as `distort_limit` are exaggerated for visualization.\n- Be careful of use."},{"metadata":{"trusted":true},"cell_type":"code","source":"transformed_mask = A.GridDistortion(always_apply=True, \n                                    distort_limit=0.85, \n                                    mask_value=0)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## ElasticTransform"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"transformed_mask = A.ElasticTransform(alpha=120, \n                                      sigma=120 * 0.05, \n                                      alpha_affine=120 * 0.03, \n                                      always_apply=True,\n                                      mask_value=0)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## CoarseDropout"},{"metadata":{"trusted":true},"cell_type":"code","source":"transformed_mask = A.CoarseDropout(max_holes=8, \n                                   max_height=50, \n                                   max_width=50, \n                                   mask_fill_value=0, \n                                   always_apply=True)(image=mask)\n\nfig, ax = plt.subplots(figsize=(16, 8),  nrows=1, ncols=2)\nax[0].imshow(mask[:,:,0])\nax[1].imshow(transformed_mask['image'][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## If this kernel is useful, <font color='orange'>please upvote</font>!\n- See you next time!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}