{"cells":[{"metadata":{"_uuid":"a97539877dab3f0fdaf09979568d6b866fc62f0d"},"cell_type":"markdown","source":"In this kernel we will train a segmention model using 450 fluke masks provided by [Dene](http://https://www.kaggle.com/dene33). I was a little bit skeptical about segmenting flukes mainly because you loose the **sea** information. If there was  some correlation between the sea waves and the classes of whales then segmentation would rule this out. However, I gave it a try (don't tested it yet in my models). \n\nI will use models from the great repo [segmentation_models](http://https://github.com/qubvel/segmentation_models) (thank you [Pavel](https://www.kaggle.com/pavel92) for the great work). The whole task  is a straightforward single class segmentation and hopefully 450 fluke  masks would be enought  to generalize to the whole train and test set."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nseed = 66\nnp.random.seed(seed)\nimport cv2\nimport json\nimport glob\nimport os\nfrom tqdm import *\nfrom shutil  import copyfile, rmtree \n\n\nfrom matplotlib import pyplot as plt\n\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.models import Sequential, Model\nfrom keras.layers import Dropout, Flatten, Dense, SpatialDropout2D, Input\nfrom keras import applications\nfrom keras.preprocessing import image\nfrom keras.preprocessing.image import load_img\n\nfrom keras import backend as K\nprint (K.image_dim_ordering())\nprint (K.image_data_format())\nfrom tqdm import tqdm_notebook, tqdm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d82d4c53afeca4e680c1e3c1accec17dc8bdd228"},"cell_type":"code","source":"TRAIN_PATH = '../input/humpback-whale-identification/train/'\nTEST_PATH = '../input/humpback-whale-identification/test/'\nMASK_PATH = '../input/whales-masks-500/masks/'","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e82cbf90abfeba47046248c369fa9402a959d873"},"cell_type":"markdown","source":"Let us parse the 450  intial masks (courtesy of **Dene**)"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"mask_files = os.listdir(MASK_PATH)\nmask_files = [m for m in mask_files if 'mask' in m]\nX = []\nM = []\nfor mask_file in mask_files:\n    img_file = mask_file.split('_')[0] + '.jpg'\n    img = cv2.imread(TRAIN_PATH + img_file)\n    mask = cv2.imread(MASK_PATH + mask_file, 0)\n    mask[mask>108]= 255\n    mask[mask<=108]= 0\n    X.append(img)\n    M.append(mask)\n  \nX = np.array(X)\nM = np.array(M)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b66742abbaf842cf0ea8a979e1e2f6ba071ba200"},"cell_type":"markdown","source":"Now let's install: [segmentation_models](http://https://github.com/qubvel/segmentation_models)"},{"metadata":{"trusted":true,"_uuid":"7f582dbc715ca1582d215ada6ca38c0e5173b968"},"cell_type":"code","source":"!pip install segmentation_models","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"02ee64cdee97255fcccf4b2d116452dafaefbd3a"},"cell_type":"markdown","source":"Another great package for fast image augmentation is  [albumentations](http://https://github.com/albu/albumentations)."},{"metadata":{"trusted":true,"_uuid":"db532cd80ceca295299238df14029bd911a2085b"},"cell_type":"code","source":"!pip install albumentations","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6243466355f3e3bf929ddb12bf8f7657c5f60766"},"cell_type":"code","source":"from albumentations import (\n    HorizontalFlip, VerticalFlip, IAAPerspective, ShiftScaleRotate, CLAHE, RandomRotate90,\n    Transpose, ShiftScaleRotate, Blur, OpticalDistortion, GridDistortion, HueSaturationValue,\n    IAAAdditiveGaussianNoise, GaussNoise, MotionBlur, MedianBlur, IAAPiecewiseAffine,\n    IAASharpen, IAAEmboss, RandomContrast, RandomBrightness, Flip, OneOf, Compose, RandomGamma, Rotate,IAAAffine\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"73288adfd04bca5d071d4a9e72a596f727584630"},"cell_type":"code","source":"aug_null = Compose([])\naug = Compose([ \n    Blur(p=0.5, blur_limit=2),\n    IAAAffine(p=0.5, shear=5),\n    HorizontalFlip(p=0.5),              \n    #VerticalFlip(p=0.5),              \n    Rotate(limit=5, p=0.3),\n    #CLAHE(p=0.3),\n    RandomContrast(p=0.2, limit=0.1),\n    RandomBrightness(p=0.2, limit=0.1),\n    #RandomGamma(p=0.2, gamma_limit=(90, 110))\n])\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cda73af030aaa86b785e85dfeb7962a1692443a6"},"cell_type":"markdown","source":"Extending keras.utils.Sequence structure can be really handy. It provides multithreaded access and ensures each input image will be considered exactly one time per epoch."},{"metadata":{"trusted":true,"_uuid":"575b6507f9ea4822a4cea4632087dc5f52769594"},"cell_type":"code","source":"import keras\nfrom segmentation_models.backbones.classification_models.classification_models.resnet import preprocess_input\n\nclass DataGenerator(keras.utils.Sequence):\n    'Generates data for Keras'\n    def __init__(self, X, M, batch_size=32,\n                 dim=(299,299),  shuffle=True, \n                 preprocess_input=preprocess_input, \n                 aug=aug_null, min_mask=2 ):\n        'Initialization'\n        self.X = X\n        self.M = M\n        self.batch_size = batch_size\n        self.n_classes = 1\n        self.shuffle = shuffle\n        self.preprocess_input = preprocess_input\n        self.aug = aug\n        self.on_epoch_end()\n        self.dim = dim\n\n    def __len__(self):\n        'Denotes the number of batches per epoch'\n        return int(np.floor((len(self.X) / self.batch_size) / 1) )\n\n    def __getitem__(self, index):\n        'Generate one batch of data'\n        # Generate indexes of the batch\n        \n        end_index = min((index+1)*self.batch_size, len(self.indexes))\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n\n\n        # Generate data\n        X, Y = self.__data_generation(indexes)\n\n        return X, Y\n\n    def on_epoch_end(self):\n        'Updates indexes after each epoch'\n        self.indexes = np.arange(len(self.X))\n        if self.shuffle == True:\n            np.random.shuffle(self.indexes)\n\n    def __data_generation(self, indexes):\n        'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)\n        \n        batch_size = len(indexes)\n        \n        # Initialization\n        XX = np.empty((batch_size, self.dim[0], self.dim[1], 3), dtype='float32')\n        YY = np.empty((batch_size, self.dim[0], self.dim[1], 1), dtype='float32')\n\n        # Generate data\n        for i, ID in enumerate(indexes):\n            # Store sample\n            img = self.X[ID]\n            if img.shape[0] != self.dim[0]:\n                img = cv2.resize(img, self.dim, cv2.INTER_CUBIC)\n            mask = self.M[ID]\n            if mask.shape[0] != self.dim[0]:\n                mask = cv2.resize(mask, self.dim, cv2.INTER_AREA)\n            \n            # Store class\n            augmented = self.aug(image=img, mask=mask)\n            aug_img = augmented['image']\n            aug_mask = augmented['mask']\n            aug_mask = np.expand_dims(aug_mask, axis=-1)\n            aug_mask = aug_mask/255\n            \n            assert (np.max(aug_mask)<= 1.0 and  np.min(aug_mask) >= 0)\n            aug_mask[aug_mask>0.5] = 1\n            aug_mask[aug_mask<0.5] = 0\n            \n            YY[i,] = aug_mask.astype('float32')\n            XX[i,] = aug_img.astype('float32')\n    \n       \n        XX = self.preprocess_input(XX)\n\n        return XX, YY","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"467e5522981763629b6b9a1f05de678abbfd2d55"},"cell_type":"markdown","source":"We define the default preprocessing for resnet architectures and create train and validation generators (`keras.utils.Sequence`) "},{"metadata":{"trusted":true,"_uuid":"02db0df46f33d80709c78cc795973478d69a885f"},"cell_type":"code","source":"from segmentation_models.backbones import get_preprocessing\npreprocess_input = get_preprocessing('resnet34')\n\ntraining_generator = DataGenerator(X[:400], M[:400], batch_size=16,  dim=(384, 384), aug=aug, \n                                   preprocess_input=preprocess_input)\nvalid_genarator = DataGenerator(X[400:], M[400:], batch_size=16, dim=(384, 384), aug=aug_null, \n                                preprocess_input=preprocess_input, shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7690af28b5435a10c6bec5cd1fd504a4781ece3a"},"cell_type":"code","source":"x, y= training_generator[7]\nnp.max(x), x.shape, y.shape, np.max(y), np.unique(y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c1514a48e9868e996cd52cd5e7eaac2612a2e122"},"cell_type":"code","source":"plt.imshow(y[9, ..., 0])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1dcc0353723f15a1541ec9d8617dfa449b1189b2"},"cell_type":"code","source":"import keras.backend as K\nfrom keras.callbacks import CSVLogger, ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau\nfrom keras.optimizers import Adam\nfrom segmentation_models import Unet, FPN\n\nmodel = FPN(backbone_name='resnet34', encoder_weights='imagenet', activation='sigmoid', classes=1, dropout=0.5)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4a8ab86cda8227e7a43495d8b804f29217bf4fb6"},"cell_type":"markdown","source":"Trivial segmentation stuff. Dice coefficient and binary cross entropy loss united."},{"metadata":{"trusted":true,"_uuid":"e8f3c9d5260648b58f7a23f87ed208e7202e0a76"},"cell_type":"code","source":"from keras.losses import binary_crossentropy\nimport keras.backend as K\nimport tensorflow as tf\n\n\ndef dice_coeff_L(y_true, y_pred):\n    smooth = 1.\n    y_pred_sig = tf.nn.sigmoid(y_pred)\n\n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred_sig)\n    intersection = K.sum(y_true_f * y_pred_f)\n    score = (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n    return score\n\ndef dice_coeff(y_true, y_pred):\n    smooth = 1.\n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = K.sum(y_true_f * y_pred_f)\n    score = (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n    return score\n\n\ndef dice_loss(y_true, y_pred):\n    loss = 1 - dice_coeff(y_true, y_pred)\n    return loss\n\n\ndef bce_dice_loss(y_true, y_pred):\n    loss = binary_crossentropy(y_true, y_pred) + dice_loss(y_true, y_pred)\n    return loss","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9910d91b88dcadfc01aef06054cd40024ecaf185"},"cell_type":"markdown","source":"Let's train the model to see what we can get."},{"metadata":{"trusted":true,"_uuid":"cc8a39f3e4043cf9706e194dbb9aeb01824b6445"},"cell_type":"code","source":"from keras.callbacks import ReduceLROnPlateau\nfrom keras.optimizers import SGD\n\ntraining_generator = DataGenerator(X[:400], M[:400], batch_size=16,  dim=(384, 384), aug=aug, \n                                   preprocess_input=preprocess_input)\nvalid_genarator = DataGenerator(X[400:], M[400:], batch_size=16, dim=(384, 384), aug=aug_null, \n                                preprocess_input=preprocess_input, shuffle=False)\n\nmodel.compile(optimizer=Adam(lr=0.001),\n          loss=bce_dice_loss,\n          metrics=[dice_coeff])\n\nepochs = 40\n\nearly_stopping = EarlyStopping(patience=10, verbose=1, monitor='val_dice_coeff', mode='max')\nmodel_checkpoint = ModelCheckpoint(\"fpnresnet34_384_{epoch:02d}-{val_loss:.3f}-{val_dice_coeff:.3f}.hdf5\", \n                                   save_best_only=True, \n                                   save_weights_only=True, \n                                   monitor='val_dice_coeff', verbose=1, mode='max', period=2)\nreduce_lr = ReduceLROnPlateau(factor=0.5, patience=5, min_lr=0.000001, verbose=1, monitor='val_dice_coeff', mode='max')\n\n \nhistory = model.fit_generator( training_generator,\n                                     validation_data=valid_genarator,\n                                     epochs=epochs,\n                                     callbacks=[ reduce_lr, early_stopping, model_checkpoint], \n                                     verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0ed358c3d77f50ceba1f32b9bd45f2bf0e068c2e"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"27969e5492df73b64cbda4f1f9cbc356774712d4"},"cell_type":"markdown","source":"This is not good software engineering, but  I was in a hurry so I copied/modified `DataGenerator` to create the `TestDataGenerator` class that also extends `keras.utils.Sequence` but ignores masks. This will be used for inference only. "},{"metadata":{"trusted":true,"_uuid":"99f0a347ec3c100934e9dbd73f623176a6282885"},"cell_type":"code","source":"aug_null = Compose([])\n\n\nclass TestDataGenerator(keras.utils.Sequence):\n    'Generates data for Keras'\n    def __init__(self, X, batch_size=32,\n                 dim=(299,299),  shuffle=True, \n                 preprocess_input=preprocess_input, \n                 aug=aug_null, min_mask=2 ):\n        'Initialization'\n        self.X = X\n        self.batch_size = batch_size\n        self.n_classes = 1\n        self.shuffle = shuffle\n        self.preprocess_input = preprocess_input\n        self.aug = aug\n        self.dim = dim\n        self.on_epoch_end()\n\n    def set_aug(self, aug):\n        self.aug = aug\n        self.on_epoch_end()\n      \n    def __len__(self):\n        'Denotes the number of batches per epoch'\n        return int(np.floor((len(self.X) / self.batch_size) / 1) )\n\n    def __getitem__(self, index):\n        'Generate one batch of data'\n        # Generate indexes of the batch\n        \n        end_index = min((index+1)*self.batch_size, len(self.indexes))\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n\n        # Find list of IDs\n\n        # Generate data\n        xx = self.__data_generation(indexes)\n\n        return xx\n\n    def on_epoch_end(self):\n        'Updates indexes after each epoch'\n        self.indexes = np.arange(len(self.X))\n        if self.shuffle == True:\n            np.random.shuffle(self.indexes)\n\n    def __data_generation(self, indexes):\n        'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)\n        \n        batch_size = len(indexes)\n        \n        # Initialization\n        XX = np.empty((batch_size, self.dim[0], self.dim[1], 3), dtype='float32')\n\n        # Generate data\n        for i, ID in enumerate(indexes):\n            # Store sample\n            img = self.X[ID]\n            if img.shape[0] != self.dim[0]:\n                img = cv2.resize(img, self.dim, cv2.INTER_CUBIC)\n            \n            \n            # Store class\n            augmented = self.aug(image=img)\n            aug_img = augmented['image']\n           \n            XX[i,] = aug_img.astype('float32')\n    \n       \n        XX = self.preprocess_input(XX)\n\n        return XX","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c8c414addb1b91237ad70d9a2f3d8d87de0a26c6"},"cell_type":"markdown","source":"The model is trained... Let us see some validation cases. Using the handy generator with `shuffle=False` we can easily perform a simple test time augmentattion."},{"metadata":{"trusted":true,"_uuid":"cb58afc7e7216e9da1d7abdff85b8088d06aa9f8"},"cell_type":"code","source":"null_aug = Compose([])\nvalid_genarator = DataGenerator(X[400:], M[400:], batch_size=16, aug=null_aug, preprocess_input=preprocess_input, dim=(384, 384), \n                               shuffle=False)\npreds = model.predict_generator(valid_genarator, verbose=1)\n\nflip_aug = Compose([HorizontalFlip(p=1.0) ])\nvalid_genarator = DataGenerator(X[400:], M[400:], batch_size=16, aug=flip_aug, preprocess_input=preprocess_input,  dim=(384, 384), shuffle=False)\npreds_hflip = model.predict_generator(valid_genarator, verbose=1)\n\nblur_aug = Compose([Blur(p=1.0)])\nvalid_genarator = DataGenerator(X[400:], M[400:], batch_size=16, aug=blur_aug, preprocess_input=preprocess_input,  dim=(384, 384), shuffle=False)\npreds_blur = model.predict_generator(valid_genarator, verbose=1)\n\nTARGET_VAL = []\nfor i in range(len(preds)):\n  pp = (preds[i] + np.fliplr(preds_hflip[i]) + preds_blur[i])/3\n  TARGET_VAL.append(pp)\n\nTARGET_VAL = np.array(TARGET_VAL)  ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9aea6a136d400adfc01f8fec9d985ce599517dc5"},"cell_type":"markdown","source":"The nice side effect from segmenting the whale flukes is that we can get tight bounding boxes as well. In the figure bellow we keep a tight bounding box and only pixels that correspond to the mask. There are several ways to move from here:\n### 1. Draw colored flukes on white background"},{"metadata":{"trusted":true,"_uuid":"f34cf38f6c66602b55c9cdc527b480a9fe0639e5"},"cell_type":"code","source":"SIZE = 384\nf, axarr = plt.subplots(6, 6)\nf.set_figwidth(20)\nf.set_figheight(15)\nkernel = np.ones((3,3),np.uint8)\n\nfor i in range(0, 36):\n    img =  cv2.resize(X[400+i], (384, 384))\n    mask = ((TARGET_VAL[i, ..., 0]) > 0.25).astype('uint8')\n    back = ((TARGET_VAL[i, ..., 0]) <= 0.25).astype('uint8')\n\n    img = np.stack([img[..., j] * mask + back*255 for j in range(3)], axis=-1)\n\n    contours,hierarchy = cv2.findContours(mask, 1, 2)\n  # Cycle through contours and add area to array\n    areas = []\n    for c in contours:\n        areas.append(cv2.contourArea(c))\n\n    # Sort array of areas by size\n    sorted_areas = sorted(zip(areas, contours), key=lambda x: x[0], reverse=True)\n    title = str(len(sorted_areas)) \n    \n    cnt = sorted_areas[0][1]\n    x1,y1,w,h = cv2.boundingRect(cnt)\n    x2 = x1 + w\n    y2 = y1 + h\n    \n    for j in range(1, len(sorted_areas)):\n        cnt = sorted_areas[j][1]\n        tx1,ty1,tw,th = cv2.boundingRect(cnt)\n        tx2 = tx1 + tw\n        ty2 = ty1 + th\n        x1 = min(x1, tx1)\n        y1 = min(y1, ty1)\n        x2 = max(x2, tx2)\n        y2 = max(y2, ty2)\n    \n    x = x1\n    y = y1\n    w = x2-x1\n    h = y2-y1\n\n\n    img_cropped = img[y:y+h, x:x+w]\n    axarr[int(i/6), i%6].imshow(img_cropped, cmap='gray')\n    axarr[int(i/6), i%6].axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ac9d64f3794f676458ba5abf539eeff556739d4a"},"cell_type":"markdown","source":"### 2. Draw colored flukes on black background"},{"metadata":{"trusted":true,"_uuid":"246a213f55b9a01c6a47175239e221370459742b"},"cell_type":"code","source":"SIZE = 384\nf, axarr = plt.subplots(6, 6)\nf.set_figwidth(20)\nf.set_figheight(15)\nkernel = np.ones((3,3),np.uint8)\n\nfor i in range(0, 36):\n    img =  cv2.resize(X[400+i], (384, 384))\n    mask = ((TARGET_VAL[i, ..., 0]) > 0.25).astype('uint8')\n\n    img = np.stack([img[..., j] * mask  for j in range(3)], axis=-1)\n\n    contours,hierarchy = cv2.findContours(mask, 1, 2)\n  # Cycle through contours and add area to array\n    areas = []\n    for c in contours:\n        areas.append(cv2.contourArea(c))\n\n    # Sort array of areas by size\n    sorted_areas = sorted(zip(areas, contours), key=lambda x: x[0], reverse=True)\n    title = str(len(sorted_areas)) \n    \n    cnt = sorted_areas[0][1]\n    x1,y1,w,h = cv2.boundingRect(cnt)\n    x2 = x1 + w\n    y2 = y1 + h\n    \n    for j in range(1, len(sorted_areas)):\n        cnt = sorted_areas[j][1]\n        tx1,ty1,tw,th = cv2.boundingRect(cnt)\n        tx2 = tx1 + tw\n        ty2 = ty1 + th\n        x1 = min(x1, tx1)\n        y1 = min(y1, ty1)\n        x2 = max(x2, tx2)\n        y2 = max(y2, ty2)\n    \n    x = x1\n    y = y1\n    w = x2-x1\n    h = y2-y1\n\n\n    img_cropped = img[y:y+h, x:x+w]\n    axarr[int(i/6), i%6].imshow(img_cropped, cmap='gray')\n    axarr[int(i/6), i%6].axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"31dbd255425eff221974a342ce6e6e714640aa4d"},"cell_type":"markdown","source":"### 3. Draw colored flukes and keep the sea (tight bounding box)"},{"metadata":{"trusted":true,"_uuid":"7b97e9582eebd458618ebbd07c0ec4868507ca17"},"cell_type":"code","source":"SIZE = 384\nf, axarr = plt.subplots(6, 6)\nf.set_figwidth(20)\nf.set_figheight(15)\nkernel = np.ones((3,3),np.uint8)\n\nfor i in range(0, 36):\n    img =  cv2.resize(X[400+i], (384, 384))\n    mask = ((TARGET_VAL[i, ..., 0]) > 0.25).astype('uint8')\n\n    img = np.stack([img[..., j]  for j in range(3)], axis=-1)\n\n    contours,hierarchy = cv2.findContours(mask, 1, 2)\n  # Cycle through contours and add area to array\n    areas = []\n    for c in contours:\n        areas.append(cv2.contourArea(c))\n\n    # Sort array of areas by size\n    sorted_areas = sorted(zip(areas, contours), key=lambda x: x[0], reverse=True)\n    title = str(len(sorted_areas)) \n    \n    cnt = sorted_areas[0][1]\n    x1,y1,w,h = cv2.boundingRect(cnt)\n    x2 = x1 + w\n    y2 = y1 + h\n    \n    for j in range(1, len(sorted_areas)):\n        cnt = sorted_areas[j][1]\n        tx1,ty1,tw,th = cv2.boundingRect(cnt)\n        tx2 = tx1 + tw\n        ty2 = ty1 + th\n        x1 = min(x1, tx1)\n        y1 = min(y1, ty1)\n        x2 = max(x2, tx2)\n        y2 = max(y2, ty2)\n    \n    x = x1\n    y = y1\n    w = x2-x1\n    h = y2-y1\n\n\n    img_cropped = img[y:y+h, x:x+w]\n    axarr[int(i/6), i%6].imshow(img_cropped, cmap='gray')\n    axarr[int(i/6), i%6].axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fdd306d3cdc4f5096900bb7f654d049da79b4b10"},"cell_type":"markdown","source":"### 4. Draw grayscale flukes in white background"},{"metadata":{"trusted":true,"_uuid":"2e5c64ad551019fbc30e18dfa5cf6cfbc9afceec"},"cell_type":"code","source":"SIZE = 384\nf, axarr = plt.subplots(6, 6)\nf.set_figwidth(20)\nf.set_figheight(15)\nkernel = np.ones((3,3),np.uint8)\n\nfor i in range(0, 36):\n    img =  cv2.resize(X[400+i], (384, 384))\n    img =  cv2.cvtColor(img, cv2.cv2.COLOR_BGR2GRAY)\n    mask = ((TARGET_VAL[i, ..., 0]) > 0.25).astype('uint8')\n    back = ((TARGET_VAL[i, ..., 0]) <= 0.25).astype('uint8')\n    img = img * mask + back*255\n\n    contours,hierarchy = cv2.findContours(mask, 1, 2)\n  # Cycle through contours and add area to array\n    areas = []\n    for c in contours:\n        areas.append(cv2.contourArea(c))\n\n    # Sort array of areas by size\n    sorted_areas = sorted(zip(areas, contours), key=lambda x: x[0], reverse=True)\n    title = str(len(sorted_areas)) \n    \n    cnt = sorted_areas[0][1]\n    x1,y1,w,h = cv2.boundingRect(cnt)\n    x2 = x1 + w\n    y2 = y1 + h\n    \n    for j in range(1, len(sorted_areas)):\n        cnt = sorted_areas[j][1]\n        tx1,ty1,tw,th = cv2.boundingRect(cnt)\n        tx2 = tx1 + tw\n        ty2 = ty1 + th\n        x1 = min(x1, tx1)\n        y1 = min(y1, ty1)\n        x2 = max(x2, tx2)\n        y2 = max(y2, ty2)\n    \n    x = x1\n    y = y1\n    w = x2-x1\n    h = y2-y1\n\n\n    img_cropped = img[y:y+h, x:x+w]\n    axarr[int(i/6), i%6].imshow(img_cropped, cmap='gray')\n    axarr[int(i/6), i%6].axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"eac275763cc320bec28906073977f078d4d233fe"},"cell_type":"markdown","source":"A small explanation about this part of code\n   ```\n   contours,hierarchy = cv2.findContours(mask, 1, 2)\n    areas = []\n    for c in contours:\n        areas.append(cv2.contourArea(c))\n\n    # Sort array of areas by size\n    sorted_areas = sorted(zip(areas, contours), key=lambda x: x[0], reverse=True)\n    title = str(len(sorted_areas)) \n    \n    cnt = sorted_areas[0][1]\n    x1,y1,w,h = cv2.boundingRect(cnt)\n    x2 = x1 + w\n    y2 = y1 + h\n    \n    for j in range(1, len(sorted_areas)):\n        cnt = sorted_areas[j][1]\n        tx1,ty1,tw,th = cv2.boundingRect(cnt)\n        tx2 = tx1 + tw\n        ty2 = ty1 + th\n        x1 = min(x1, tx1)\n        y1 = min(y1, ty1)\n        x2 = max(x2, tx2)\n        y2 = max(y2, ty2)\n        ```"},{"metadata":{"_uuid":"21dc15308e38468d662cfba4dc9d1231645eea08"},"cell_type":"markdown","source":"I calculate the bouning box of the mask using `cv2.findContours`. In case we have multiple disconnected components in the same mask, the bounding box is calculated using the minimum `x` and `y` and the maximum `x` and `y` of all components, hence the loop. This is the meaning of:\n```\n    x1 = min(x1, tx1)\n    y1 = min(y1, ty1)\n    x2 = max(x2, tx2)\n    y2 = max(y2, ty2)\n   ```\n   within the loop."},{"metadata":{"trusted":true,"_uuid":"15718f0f6c1ce6b60eda2f29c4e0aaace4746f00"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}