{"cells":[{"metadata":{},"cell_type":"markdown","source":"   #  Understanding Clouds from Satellite Images\n## About this kernel: \n   This kernel is organized as walkthrough to demonstrate one of approches to solve this competation.\n   I am not expert in Computer vision so your feedback is so much appreciated.\n   I hope you upvote this kernel ,if you find it helpful .\n## The competition:\n   So in this competition the task is multiclass segmentation task: finding 4 different cloud                  patterns in the images.We have four patterns (Fish, Flower, Gravel, Sugar).We make predictions for each      pair of image and label separately, so this could be treated as 4 binary segmentation tasks.\n   The image size =(1400, 2100)\n## References:\n   * An overview of semantic image segmentation :[https://www.jeremyjordan.me/semantic-segmentation/](http://)\n   * Losses for Image Segmentation : [https://lars76.github.io/neural-networks/object-detection/losses-for-segmentation/](http://)\n   * Segmentation Models :   [https://github.com/qubvel/segmentation_models](http://)\n   * Albumentations :  [https://github.com/albumentations-team/albumentations](http://)\n   \n   ### kernels:\n   \n   * Understanding Clouds - EDA and Keras U-Net :[https://www.kaggle.com/dimitreoliveira/understanding-clouds-eda-and-keras-u-net](http://)\n   * Segmentation in PyTorch using convenient tools : [https://www.kaggle.com/artgor/segmentation-in-pytorch-using-convenient-tools](http://)\n   * Satellite Clouds: U-Net with ResNet Encoder : [https://www.kaggle.com/xhlulu/satellite-clouds-u-net-with-resnet-encoder/notebook](http://)\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n#for dirname, _, filenames in os.walk('/kaggle/input'):\n#    for filename in filenames:\n#        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install tensorflow==1.13.1\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#!pip install tensorflow-gpu==1.13.1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -U --pre segmentation-models --user","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -U albumentations --user \n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### First let's read the training file (train.csv) and submission file (sample_submission.csv) using Pandas"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom PIL import Image\nimport cv2\nimport keras\nimport albumentations as A\nimport segmentation_models as sm\npath=\"/kaggle/input/understanding_cloud_organization\"\ntrain = pd.read_csv(f'{path}/train.csv')\nsub = pd.read_csv(f'{path}/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* As you see in the table each entry contains two colume (Image_label,EncodedPixels) \n* The colume Image_label  is image_id concatenated with one of the labels(patterns (Fish, Flower, Gravel, Sugar)) and EncodedPixels colume contains the pixels of the specified label if you faltten the image and take the first item as the start of pattern and the second item the lenght from that pixel.\n* For example if the image is 10x10 matrix(ignore the number of channles for simplicity) when we flatten .it is going to 100 pixels array so the EncodedPixels will contain [0 100 150 50 ....][start lenght start lenght ...] so on "},{"metadata":{"trusted":true},"cell_type":"code","source":"n_train = len(os.listdir(f'{path}/train_images'))\nn_test = len(os.listdir(f'{path}/test_images'))\nprint(f'There are {n_train} images in train dataset')\nprint(f'There are {n_test} images in test dataset')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Separate the image id and the label in train datafram "},{"metadata":{"trusted":true},"cell_type":"code","source":"train['label'] = train['Image_Label'].apply(lambda x: x.split('_')[1])\ntrain['im_id'] = train['Image_Label'].apply(lambda x: x.split('_')[0])\n\nsub['label'] = sub['Image_Label'].apply(lambda x: x.split('_')[1])\nsub['im_id'] = sub['Image_Label'].apply(lambda x: x.split('_')[0])\n\nimage_id_list = train['im_id'].unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Some helpful function for building mask and visualization"},{"metadata":{"trusted":true},"cell_type":"code","source":"def visualize(**images):\n    \"\"\"PLot images in one row.\"\"\"\n    n = len(images)\n    plt.figure(figsize=(16, 5))\n    for i, (name, image) in enumerate(images.items()):\n        plt.subplot(1, n, i + 1)\n        plt.xticks([])\n        plt.yticks([])\n        plt.title(' '.join(name.split('_')).title())\n        plt.imshow(image)\n    plt.show()\n\ndef rle_decode(mask_rle: str = '', shape: tuple =(1400, 2100)):\n    '''\n    Decode rle encoded mask.\n    \n    :param mask_rle: run-length as string formatted (start length)\n    :param shape: (height, width) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    \n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape, order='F')\n\ndef mask2rle(img):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    '''\n    pixels= img.T.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    \n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\ndef bulid_mask(train_df,image_id,image_shape):\n    masks=np.zeros((*image_shape,4))\n    for i, (idx, row) in enumerate(train.loc[train['im_id'] == image_id].iterrows()):\n        \n        mask_rle = row['EncodedPixels']\n        \n        try: # label might not be there!\n            mask = rle_decode(mask_rle)\n        except:\n            mask = np.zeros(image_shape)\n        masks[:,:,i]=mask\n   \n    return masks \n\ndef show_image_mask(image_id):\n    image = Image.open(f\"{path}/train_images/{image_id}\")\n    print(\"actual mask\")\n    mask=bulid_mask(train,image_id ,(1400, 2100))\n    #mask=bulid_mask(train,image_id ,(350, 525))\n    \n    visualize(\n       image=image, \n       Fish_mask=mask[..., 0].squeeze(),\n       Flower_mask=mask[..., 1].squeeze(),\n       Gravel_mask=mask[..., 2].squeeze(),\n       Suger_mask=mask[..., 3].squeeze(),    \n    )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### visualization of images and their masks"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\nfor j, im_id in enumerate(np.random.choice(train['im_id'].unique(), 4)):\n    for i, (idx, row) in enumerate(train.loc[train['im_id'] == im_id].iterrows()):\n        ax = fig.add_subplot(5, 4, j * 4 + i + 1, xticks=[], yticks=[])\n        im = Image.open(f\"{path}/train_images/{row['Image_Label'].split('_')[0]}\")\n        plt.imshow(im)\n        mask_rle = row['EncodedPixels']\n        try: # label might not be there!\n            mask = rle_decode(mask_rle)\n        except:\n            mask = np.zeros((1400, 2100))\n        plt.imshow(mask, alpha=0.5, cmap='gray')\n        ax.set_title(f\"Image: {row['Image_Label'].split('_')[0]}. Label: {row['label']}\")\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(0)\nfor i in np.random.randint(0,len(image_id_list),size=5):\n    image_id=image_id_list[i]\n    show_image_mask(image_id)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Some helpful function to create training set of images and their masks "},{"metadata":{"trusted":true},"cell_type":"code","source":"\n    \nclass Dataset:\n    \"\"\"CamVid Dataset. Read images, apply augmentation and preprocessing transformations.\n    \n    Args:\n        images_dir (str): path to images folder\n        masks_dir (str): path to segmentation masks folder\n        class_values (list): values of classes to extract from segmentation mask\n        augmentation (albumentations.Compose): data transfromation pipeline \n            (e.g. flip, scale, etc.)\n        preprocessing (albumentations.Compose): data preprocessing \n            (e.g. noralization, shape manipulation, etc.)\n    \n    \"\"\"\n    \n    CLASSES = ['Fish', 'Flower','Gravel','Suger']\n    \n    def __init__(\n            self,\n            tain_df,\n            images_dir, \n            image_id_list, \n            classes=None, \n            augmentation=None, \n            preprocessing=None,\n    ):\n        self.ids = image_id_list\n        self.images_fps = [f\"{images_dir}/train_images/{image_id}\" for image_id in self.ids]\n\n        self.train_df= tain_df \n        self.augmentation = augmentation\n        self.preprocessing = preprocessing\n    \n    def __getitem__(self, i):\n        \n        # read data\n        image = cv2.imread(self.images_fps[i])\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n        \n        # apply augmentations\n        mask=bulid_mask(self.train_df,self.ids[i] ,(1400, 2100))\n        if self.augmentation:\n            sample = self.augmentation(image=image, mask=mask)\n            image, mask = sample['image'], sample['mask']\n        \n        # apply preprocessing\n        if self.preprocessing:\n            sample = self.preprocessing(image=image, mask=mask)\n            image, mask = sample['image'], sample['mask']\n            \n        return image, mask\n        \n    def __len__(self):\n        return len(self.ids)\n    \n    \n    \n    \nclass Dataloder(keras.utils.Sequence):\n    \"\"\"Load data from dataset and form batches\n    \n    Args:\n        dataset: instance of Dataset class for image loading and preprocessing.\n        batch_size: Integet number of images in batch.\n        shuffle: Boolean, if `True` shuffle image indexes each epoch.\n    \"\"\"\n    \n    def __init__(self, dataset, batch_size=1, shuffle=False):\n        self.dataset = dataset\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n        self.indexes = np.arange(len(dataset))\n\n        self.on_epoch_end()\n\n    def __getitem__(self, i):\n        \n        # collect batch data\n        start = i * self.batch_size\n        stop = (i + 1) * self.batch_size\n        data = []\n        for j in range(start, stop):\n            data.append(self.dataset[j])\n        \n        # transpose list of lists\n        batch = [np.stack(samples, axis=0) for samples in zip(*data)]\n        \n        return batch\n    \n    def __len__(self):\n        \"\"\"Denotes the number of batches per epoch\"\"\"\n        return len(self.indexes) // self.batch_size\n    \n    def on_epoch_end(self):\n        \"\"\"Callback function to shuffle indexes each epoch\"\"\"\n        if self.shuffle:\n            self.indexes = np.random.permutation(self.indexes)\n\n\n\ndef round_clip_0_1(x, **kwargs):\n    return x.round().clip(0, 1)\n\n# define heavy augmentations\ndef get_training_augmentation():\n    train_transform = [\n\n        A.HorizontalFlip(p=0.5),\n\n        A.ShiftScaleRotate(scale_limit=0.5, rotate_limit=0, shift_limit=0.1, p=1, border_mode=0),\n\n        A.PadIfNeeded(min_height=320, min_width=320, always_apply=True, border_mode=0),\n        A.RandomCrop(height=320, width=320, always_apply=True),\n\n        A.IAAAdditiveGaussianNoise(p=0.2),\n        A.IAAPerspective(p=0.5),\n\n        A.OneOf(\n            [\n                A.CLAHE(p=1),\n                A.RandomBrightness(p=1),\n                A.RandomGamma(p=1),\n            ],\n            p=0.9,\n        ),\n\n        A.OneOf(\n            [\n                A.IAASharpen(p=1),\n                A.Blur(blur_limit=3, p=1),\n                A.MotionBlur(blur_limit=3, p=1),\n            ],\n            p=0.9,\n        ),\n\n        A.OneOf(\n            [\n                A.RandomContrast(p=1),\n                A.HueSaturationValue(p=1),\n            ],\n            p=0.9,\n        ),\n        A.Lambda(mask=round_clip_0_1)\n    ]\n    return A.Compose(train_transform)\n\n\ndef get_validation_augmentation():\n    \"\"\"Add paddings to make image shape divisible by 32\"\"\"\n    test_transform = [\n        #A.PadIfNeeded(384, 480),\n        #A.PadIfNeeded(320, 320),\n        #A.PadIfNeeded(min_height=384, min_width=480, always_apply=True, border_mode=0)\n        A.Resize(320, 320, interpolation=1, always_apply=False, p=1)\n        \n    ]\n    \n    \n    return A.Compose(test_transform)\n\ndef get_preprocessing(preprocessing_fn):\n    \"\"\"Construct preprocessing transform\n    \n    Args:\n        preprocessing_fn (callbale): data normalization function \n            (can be specific for each pretrained neural network)\n    Return:\n        transform: albumentations.Compose\n    \n    \"\"\"\n    \n    _transform = [\n        A.Lambda(image=preprocessing_fn),\n    ]\n    return A.Compose(_transform)    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'''\n#BACKBONE = 'efficientnetb3'\nBACKBONE='resnet18'\nBATCH_SIZE = 8\nCLASSES = ['Fish', 'Flower','Gravel','Suger']\nLR = 0.001\nEPOCHS = 10\n\npreprocess_input = sm.get_preprocessing(BACKBONE)\n# define network parameters\nn_classes = 1 if len(CLASSES) == 1 else (len(CLASSES))  # case for binary and multiclass segmentation\nactivation = 'sigmoid' #if n_classes == 1 else 'softmax'\n\n#create model\nmodel = sm.Unet(BACKBONE, classes=n_classes, activation=activation)\n\n# define optomizer\noptim = keras.optimizers.Adam(LR)\n\n# Segmentation models losses can be combined together by '+' and scaled by integer or float factor\n# set class weights for dice_loss (car: 1.; pedestrian: 2.; background: 0.5;)\ndice_loss = sm.losses.DiceLoss(class_weights=np.array([1, 1, 1,1])) \nfocal_loss = sm.losses.BinaryFocalLoss() if n_classes == 1 else sm.losses.CategoricalFocalLoss()\ntotal_loss = dice_loss + (1 * focal_loss)\n\n# actulally total_loss can be imported directly from library, above example just show you how to manipulate with losses\n# total_loss = sm.losses.binary_focal_dice_loss # or sm.losses.categorical_focal_dice_loss \n\nmetrics = [sm.metrics.IOUScore(threshold=0.5), sm.metrics.FScore(threshold=0.5)]\n\n# compile keras model with defined optimozer, loss and metrics\nmodel.compile(optim, total_loss, metrics)\n'''","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Creating the model using Segmentation Models "},{"metadata":{"trusted":true},"cell_type":"code","source":"import keras.backend as K\nfrom keras.legacy import interfaces\nfrom keras.optimizers import Optimizer\nfrom keras.losses import binary_crossentropy\nimport tensorflow as tf\nclass AdamAccumulate(Optimizer):\n\n    def __init__(self, lr=0.001, beta_1=0.9, beta_2=0.999,\n                 epsilon=None, decay=0., amsgrad=False, accum_iters=1, **kwargs):\n        if accum_iters < 1:\n            raise ValueError('accum_iters must be >= 1')\n        super(AdamAccumulate, self).__init__(**kwargs)\n        with K.name_scope(self.__class__.__name__):\n            self.iterations = K.variable(0, dtype='int64', name='iterations')\n            self.lrr = K.variable(lr,name='lr')\n            self.beta_1 = K.variable(beta_1, name='beta_1')\n            self.beta_2 = K.variable(beta_2, name='beta_2')\n            self.decay = K.variable(decay, name='decay')\n        if epsilon is None:\n            epsilon = K.epsilon()\n        self.epsilon = epsilon\n        self.initial_decay = decay\n        self.amsgrad = amsgrad\n        self.accum_iters = K.variable(accum_iters, K.dtype(self.iterations))\n        self.accum_iters_float = K.cast(self.accum_iters, K.floatx())\n\n    @interfaces.legacy_get_updates_support\n    def get_updates(self, loss, params):\n        grads = self.get_gradients(loss, params)\n        self.updates = [K.update_add(self.iterations, 1)]\n\n        lr = self.lrr\n\n        completed_updates = K.cast(tf.math.floordiv(self.iterations, self.accum_iters), K.floatx())\n\n        if self.initial_decay > 0:\n            lr = lr * (1. / (1. + self.decay * completed_updates))\n\n        t = completed_updates + 1\n\n        lr_t = lr * (K.sqrt(1. - K.pow(self.beta_2, t)) / (1. - K.pow(self.beta_1, t)))\n\n        # self.iterations incremented after processing a batch\n        # batch:              1 2 3 4 5 6 7 8 9\n        # self.iterations:    0 1 2 3 4 5 6 7 8\n        # update_switch = 1:        x       x    (if accum_iters=4)  \n        update_switch = K.equal((self.iterations + 1) % self.accum_iters, 0)\n        update_switch = K.cast(update_switch, K.floatx())\n\n        ms = [K.zeros(K.int_shape(p), dtype=K.dtype(p)) for p in params]\n        vs = [K.zeros(K.int_shape(p), dtype=K.dtype(p)) for p in params]\n        gs = [K.zeros(K.int_shape(p), dtype=K.dtype(p)) for p in params]\n\n        if self.amsgrad:\n            vhats = [K.zeros(K.int_shape(p), dtype=K.dtype(p)) for p in params]\n        else:\n            vhats = [K.zeros(1) for _ in params]\n\n        self.weights = [self.iterations] + ms + vs + vhats\n\n        for p, g, m, v, vhat, tg in zip(params, grads, ms, vs, vhats, gs):\n\n            sum_grad = tg + g\n            avg_grad = sum_grad / self.accum_iters_float\n\n            m_t = (self.beta_1 * m) + (1. - self.beta_1) * avg_grad\n            v_t = (self.beta_2 * v) + (1. - self.beta_2) * K.square(avg_grad)\n\n            if self.amsgrad:\n                vhat_t = K.maximum(vhat, v_t)\n                p_t = p - lr_t * m_t / (K.sqrt(vhat_t) + self.epsilon)\n                self.updates.append(K.update(vhat, (1 - update_switch) * vhat + update_switch * vhat_t))\n            else:\n                p_t = p - lr_t * m_t / (K.sqrt(v_t) + self.epsilon)\n\n            self.updates.append(K.update(m, (1 - update_switch) * m + update_switch * m_t))\n            self.updates.append(K.update(v, (1 - update_switch) * v + update_switch * v_t))\n            self.updates.append(K.update(tg, (1 - update_switch) * sum_grad))\n            new_p = p_t\n\n            # Apply constraints.\n            if getattr(p, 'constraint', None) is not None:\n                new_p = p.constraint(new_p)\n\n            self.updates.append(K.update(p, (1 - update_switch) * p + update_switch * new_p))\n        return self.updates\n\n    def get_config(self):\n        config = {'lr': float(K.get_value(self.lrr)),\n                  'beta_1': float(K.get_value(self.beta_1)),\n                  'beta_2': float(K.get_value(self.beta_2)),\n                  'decay': float(K.get_value(self.decay)),\n                  'epsilon': self.epsilon,\n                  'amsgrad': self.amsgrad}\n        base_config = super(AdamAccumulate, self).get_config()\n        return dict(list(base_config.items()) + list(config.items()))\n\ndef dice_coef(y_true, y_pred, smooth=1):\n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = K.sum(y_true_f * y_pred_f)\n    return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n\ndef dice_loss(y_true, y_pred):\n    smooth = 1.\n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = y_true_f * y_pred_f\n    score = (2. * K.sum(intersection) + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n    return 1. - score\n\ndef bce_dice_loss(y_true, y_pred):\n    return binary_crossentropy(y_true, y_pred) + dice_loss(y_true, y_pred)\n    \npreprocess_input = sm.get_preprocessing('resnet18')    \nEPOCHS = 15\nBATCH_SIZE = 8\nCLASSES = ['Fish', 'Flower','Gravel','Suger']\nn_classes = len(CLASSES)\nopt = AdamAccumulate(lr=0.002, accum_iters=8)\n\nmodel = sm.Unet(\n    'resnet18', \n    classes=4,\n    input_shape=(320, 320, 3),\n    activation='sigmoid'\n)\nmodel.compile(optimizer=opt, loss=bce_dice_loss, metrics=[dice_coef])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Creating training dataset and validation dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\ntrain_id_list, val_id_list = train_test_split(\n    image_id_list, random_state=2019, test_size=0.1\n)\ntrain_dataset = Dataset(\n    train,\n    path, \n    train_id_list, \n    classes=['Fish', 'Flower','Gravel','Suger'],\n    augmentation=get_validation_augmentation(),#get_training_augmentation(),\n    preprocessing=get_preprocessing(preprocess_input),\n)\n\n# Dataset for validation images\nvalid_dataset = Dataset(\n    train,\n    path, \n    val_id_list, \n    classes=['Fish', 'Flower','Gravel','Suger'], \n    augmentation=get_validation_augmentation(),\n    preprocessing=get_preprocessing(preprocess_input),\n)\n\ntrain_dataloader = Dataloder(train_dataset, batch_size=BATCH_SIZE, shuffle=True)\nvalid_dataloader = Dataloder(valid_dataset, batch_size=1, shuffle=False)\n\n# check shapes for errors\nassert train_dataloader[0][0].shape == (BATCH_SIZE, 320, 320, 3)\nassert train_dataloader[0][1].shape == (BATCH_SIZE, 320, 320, n_classes)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training the model and saving its weights"},{"metadata":{"trusted":true},"cell_type":"code","source":"\n'''\n# define callbacks for learning rate scheduling and best checkpoints saving\ncallbacks = [\n    keras.callbacks.ModelCheckpoint('./best_model.h5', save_weights_only=True, save_best_only=True, mode='min'),\n    #keras.callbacks.ReduceLROnPlateau(),\n]\n# train model\nhistory = model.fit_generator(\n    train_dataloader, \n    steps_per_epoch=len(train_dataloader), \n    epochs=EPOCHS, \n    callbacks=callbacks, \n    validation_data=valid_dataloader, \n    validation_steps=len(valid_dataloader),\n)\n\n\n# Plot training & validation iou_score values\n#plt.figure(figsize=(30, 5))\n#plt.subplot(121)\n#plt.plot(history.history['iou_score'])\n#plt.plot(history.history['val_iou_score'])\n#plt.title('Model iou_score')\n#plt.ylabel('iou_score')\n#plt.xlabel('Epoch')\n#plt.legend(['Train', 'Test'], loc='upper left')\n\n# Plot training & validation loss values\n#plt.subplot(122)\n#plt.plot(history.history['loss'])\n#plt.plot(history.history['val_loss'])\n#plt.title('Model loss')\n#plt.ylabel('Loss')\n#plt.xlabel('Epoch')\n#plt.legend(['Train', 'Test'], loc='upper left')\n#plt.show()\n\nhistory_df = pd.DataFrame(history.history)\nhistory_df.to_csv('history.csv', index=False)\n\nhistory_df[['loss', 'val_loss']].plot()\nhistory_df[['dice_coef', 'val_dice_coef']].plot()\nhistory_df[['lr']].plot()\n'''","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loading the trained model "},{"metadata":{"trusted":true},"cell_type":"code","source":"model.load_weights('/kaggle/input/resnet18-trained-model/best_model.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Some helpful functions for creating the predicted mask from the model we have created"},{"metadata":{"trusted":true},"cell_type":"code","source":"def denormalize(x):\n    \"\"\"Scale image to range 0..1 for correct plot\"\"\"\n    x_max = np.percentile(x, 98)\n    x_min = np.percentile(x, 2)    \n    x = (x - x_min) / (x_max - x_min)\n    x = x.clip(0, 1)\n    return x\ndef predict_mask(image_file,threshold):\n\n    image = cv2.imread(image_file)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    width = 320#384 \n    height = 320#480\n    dim = (width, height)\n    #if image.shape[0]==width and image.shape[0]==width:\n        #augment=get_validation_augmentation()\n        #image=augment(image=image) \n    if image.shape[0]!=width and image.shape[0]!=width:\n        image = cv2.resize(image, dim, interpolation = cv2.INTER_AREA)\n        #augment=get_validation_augmentation()\n        #image=augment(image=image) \n\n    preprocss=get_preprocessing(preprocess_input)\n    image=preprocss(image=image)\n    image = np.expand_dims(image['image'], axis=0)\n    p_mask = model.predict(image)\n    p_mask=(p_mask > threshold).astype(np.float_)\n    image = cv2.resize(image.squeeze(axis=0),(2100,1400), interpolation = cv2.INTER_AREA)\n    p_mask =cv2.resize(p_mask.squeeze(axis=0), (2100,1400), interpolation = cv2.INTER_AREA)\n    #image = cv2.resize(image.squeeze(axis=0),(2100,1400), interpolation = cv2.INTER_AREA)\n    #p_mask =cv2.resize(p_mask.squeeze(axis=0), (350,525), interpolation = cv2.INTER_AREA)\n    \n    visualize(\n        image=denormalize(image),\n        Fish_mask=p_mask[..., 0],\n        Flower_mask=p_mask[..., 1],\n        Gravel_mask=p_mask[..., 2],\n        Suger_mask=p_mask[..., 3],  \n    )\n    \n    return image,p_mask\n\n\ndef post_process(mask, min_size):\n    \"\"\"\n    Post processing of each predicted mask, components with lesser number of pixels\n    than `min_size` are ignored\n    \"\"\"\n    \n    #mask = cv2.threshold(probability, threshold, 1, cv2.THRESH_BINARY)[1]\n    \n    num_component, component = cv2.connectedComponents(mask.astype(np.uint8))\n    #predictions = np.zeros((1400,2100), np.float32)\n    predictions = np.zeros((350,525), np.float32)\n    \n    num = 0\n    for c in range(1, num_component):\n        p = (component == c)\n        if p.sum() > min_size:\n            predictions[p] = 1\n            num += 1\n    return predictions, num\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualization of the images , actual masks and predicted masks from image in training set ater training for 10 epochs ,you will get better result it you increase number of epochs"},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(0)\nfor i in np.random.randint(0,len(image_id_list),size=5):\n    image_id=image_id_list[i]\n    show_image_mask(image_id)\n    print(\"predicted mask\")\n    image,p_mask=predict_mask(f\"{path}/train_images/{image_id}\",0.45)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Predicting masks from test set image "},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.head()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Number of test images \",len(sub['im_id'].unique()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'''\ntest_path='/kaggle/input/understanding_cloud_organization/test_images/'\nmini_number=15000\nencoded_pixels = []\ntest_images_list=sub['im_id'].unique()\nfor i in range(len(test_images_list)):\n    image,p_mask=predict_mask(f\"{test_path}{test_images_list[i]}\",0.45)\n    #p_mask =cv2.resize(p_mask, (350, 525), interpolation = cv2.INTER_AREA)\n    for m in range(p_mask.shape[-1]):\n        pred_mask= p_mask[...,m].astype('float32') \n        print(i,len(test_images_list))\n        num_predict=pred_mask.sum()            \n        pred_mask, num_predict = post_process(pred_mask, mini_number)\n            \n        if num_predict == 0:\n            encoded_pixels.append('')\n        else:\n            r = mask2rle(pred_mask)\n            encoded_pixels.append(r)\nsub['EncodedPixels'] = encoded_pixels\n'''","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Making the submission.csv"},{"metadata":{"trusted":true},"cell_type":"code","source":"#sub.to_csv('submission.csv', columns=['Image_Label', 'EncodedPixels'], index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#from IPython.display import FileLink\n#FileLink(r'submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#from IPython.display import FileLink\n#FileLink(r'best_model.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#os.listdir(path)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}