{"cells":[{"metadata":{},"cell_type":"markdown","source":"*Note: To explain more about the competition, I created a seperate notebook where I do exploratory data analysis (link: https://www.kaggle.com/ekhtiar/eda-find-me-in-the-clouds). I will be using this notebook to take a workshop. I am making it public so new Kaggler's can also follow along.*"},{"metadata":{},"cell_type":"markdown","source":"## What Is Semantic Segmentation?\n\nImage classification, semantic segmentation, object detection, and instance segmentation are four different types of problems we deal with in image data. Semantic segmentation is the task of classifying each and very pixel in an image into a class. For example, in the image below, you can see that all baloons are classified as once in blue. A more difficult task in image segementation is rather instance segmentation. Semantic segmentation is different from instance segmentation which is that different objects of the same class will have different labels as in ballon1, baloon2 and hence different colours. The picture below very crisply illustrates the difference between instance and semantic segmentation; as well as classification and object detection.\n\n![](https://miro.medium.com/max/548/1*OnuIJiFVpy7m83LSCUgi6w.png)\nSource: https://towardsdatascience.com/semantic-segmentation-popular-architectures-dff0a75f39d0\n\nIf you want to get more fundemental knowledge about semantic segmentation, [this presentation](http://www.cs.toronto.edu/~tingwuwang/semantic_segmentation.pdf) is also a great point to start. \n\nIn this competition, we are asked to segment different types of cloud formation appearing in the image. Our task for this competition is to label each pixel of the image to five classes: Fish, Flower, Gravel, Sugar, or Nothing! I am sure you already know that Convolutional Neural Networks (CNNs) are great at image related tasks. However, there are many great CNN architectures to handle semantic segmentation tasks. You can find a list of them with a link to practical implementation in [this link](https://github.com/mrgloom/awesome-semantic-segmentation). In this tutorial, I will pick up a much improved version of a very popular CNN architecture, UNet++ or Nested UNets. More about the architecture of our network later. First, let's set our notebook up by doing a few basic imports and creating a few configuration parameters."},{"metadata":{},"cell_type":"markdown","source":"## TensorFlow - A Few Words\n\nTensorFlow is an open source software library released in 2015 by Google. TensorFlow enables users to express arbitrary computation as a graph of data flows. Nodes in this graph represent mathematical operations, whereas edges represent data that is communicated from one node to another. Data in TensorFlow are represented as tensors, which are multidimensional arrays. Although this framework for thinking about computation is valuable in many different fields, TensorFlow is primarily used for deep learning in practice and research. [1]\n\nAlthough TensorFlow was always powerful, it was not always the most intuitive deep learning framework to use. However, the TensorFlow development team started addressing this issues by working toward a more stable and intuitive release of TensorFlow 2.0. One of the major change going forward is integration of Keras. Keras is an open-source neural-network library written in Python. It is capable of running on top of many other deep learning frameworks including TensorFlow. At the time TensorFlow was initially release, Keras was much more user-friendly, modular, and extensible. However, now you can use Keras as one of the TensorFlow APIs. Along with this, many other exciting improvements came in the much anticipated TensorFlow 2.0 release. The stable release was made on 30th September 2019, and you can read the official annoucement here: https://towardsdatascience.com/announcement-tensorflow-2-0-has-arrived-ee59283fd83a.\n\nTensorFlow is also becoming much more complete and end-to-end, with an emphasis on simplification of model deployment and productization. TensorFlow 2.0 standardized the SavedModel file format as the format accross all the deployment options accross various platform (cloud, web, browser, Node.js, mobile and embedded systems). It also supports high performance training, like multi-gpu training, by the [Distribution Strategy API](https://www.tensorflow.org/guide/distributed_training).\n\n![](https://miro.medium.com/max/960/0*C7GCWYlsMrhUYRYi)\n\nIn Kaggle, TensorFlow 1.14 is still the default version, so we will use that for this tutorial. However, the APIs that I will be using are all tensorflow.keras. So, this code would run without any change in TensorFlow 2.0.\n\n[1] TensorFlow for deep learning—implementing neural networks - by Nikhil Buduma Publisher: O'Reilly Media, Inc.\n\n"},{"metadata":{},"cell_type":"markdown","source":"## Import & Configurations \n\nEnough information, let's get started by importing some libraries and setting up some parameters for our network."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# basic imports\nimport pandas as pd\nimport numpy as np\nimport os\nimport cv2\nimport matplotlib.pylab as plt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# tensorflow imports\nfrom tensorflow import reduce_sum\nfrom tensorflow.keras.utils import Sequence\nfrom tensorflow.keras.models import Model\nfrom tensorflow.keras.layers import Input, Conv2D, Conv2DTranspose, MaxPool2D, Dropout, concatenate, Flatten\nfrom tensorflow.keras.losses import binary_crossentropy\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.regularizers import l2\nfrom sklearn.model_selection import train_test_split","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# image directory paths \ndata_path = '../input/understanding_cloud_organization'\ntrain_csv_path = os.path.join('../input/understanding_cloud_organization','train.csv')\ntrain_image_path = os.path.join('../input/understanding_cloud_organization','train_images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# network configuration parameters\n# original image is 1600x256, so we will resize it\nimg_w = 384 # resized weidth\nimg_h = 256 # resized height\nbatch_size = 10\nepochs = 25\n# batch size for training unet\nk_size = 3 # kernel size 3x3\nval_size = .20 # split of training set between train and validation set\n# network hyper parameters\nsmooth = 1.\ndropout_rate = 0.5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# saving and loading model\nload_pretrained_model = False # load a pre-trained model\nsave_model = True # save the model after training\npretrained_model_path = './nested_unet.h5' # path of pretrained model\nmodel_save_path = './nested_unet.h5' # path of model to save","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load Data & Utility Functions\n\nIn this section, we will load the metadata about the image into pandas dataframe. We will also process it a little bit to make our life easier."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load full data and label no mask as -1\ntrain_df = pd.read_csv(train_csv_path).fillna(-1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# image id and class id are two seperate entities and it makes it easier to split them up in two columns\ntrain_df['ImageId'] = train_df['Image_Label'].apply(lambda x: x.split('_')[0])\ntrain_df['Label'] = train_df['Image_Label'].apply(lambda x: x.split('_')[1])\n# lets create a dict with class id and encoded pixels and group all the defaults per image\ntrain_df['Label_EncodedPixels'] = train_df.apply(lambda row: (row['Label'], row['EncodedPixels']), axis = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# group together all masks for each image\ngrouped_EncodedPixels = train_df.groupby('ImageId')['Label_EncodedPixels'].apply(list)\ngrouped_EncodedPixels.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Utility Functions for RLE Encoding & Decoding "},{"metadata":{"trusted":true},"cell_type":"code","source":"# from https://www.kaggle.com/robertkag/rle-to-mask-converter\ndef rle_to_mask(rle_string,height,width):\n    '''\n    convert RLE(run length encoding) string to numpy array\n\n    Parameters: \n    rleString (str): Description of arg1 \n    height (int): height of the mask\n    width (int): width of the mask \n\n    Returns: \n    numpy.array: numpy array of the mask\n    '''\n    rows, cols = height, width\n    if rle_string == -1:\n        return np.zeros((height, width))\n    else:\n        rleNumbers = [int(numstring) for numstring in rle_string.split(' ')]\n        rlePairs = np.array(rleNumbers).reshape(-1,2)\n        img = np.zeros(rows*cols,dtype=np.uint8)\n        for index,length in rlePairs:\n            index -= 1\n            img[index:index+length] = 255\n        img = img.reshape(cols,rows)\n        img = img.T\n        return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Thanks to the authors of: https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode\ndef mask_to_rle(mask):\n    '''\n    Convert a mask into RLE\n    \n    Parameters: \n    mask (numpy.array): binary mask of numpy array where 1 - mask, 0 - background\n\n    Returns: \n    sring: run length encoding \n    '''\n    pixels= mask.T.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Generator\nTo push the data to our model, we will create a custom data generator. A generator lets us load data progressively, instead of loading it all into memory at once. A custom generator allows us to also fit in more customization during the time of loading the data. As the model is being procssed in the GPU, we can use a custom generator to pre-process images via a generator. At this time, we can also take advantage multiple processors to parallelize our pre-processing."},{"metadata":{"trusted":true},"cell_type":"code","source":"class DataGenerator(Sequence):\n    def __init__(self, list_ids, labels, image_dir, batch_size=32,\n                 img_h=256, img_w=512, shuffle=True):\n        \n        self.list_ids = list_ids\n        self.labels = labels\n        self.image_dir = image_dir\n        self.batch_size = batch_size\n        self.img_h = img_h\n        self.img_w = img_w\n        self.shuffle = shuffle\n        self.on_epoch_end()\n    \n    def __len__(self):\n        'denotes the number of batches per epoch'\n        return int(np.floor(len(self.list_ids)) / self.batch_size)\n    \n    def __getitem__(self, index):\n        'generate one batch of data'\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n        # get list of IDs\n        list_ids_temp = [self.list_ids[k] for k in indexes]\n        # generate data\n        X, y = self.__data_generation(list_ids_temp)\n        # return data \n        return X, y\n    \n    def on_epoch_end(self):\n        'update ended after each epoch'\n        self.indexes = np.arange(len(self.list_ids))\n        if self.shuffle:\n            np.random.shuffle(self.indexes)\n            \n    def __data_generation(self, list_ids_temp):\n        'generate data containing batch_size samples'\n        X = np.empty((self.batch_size, self.img_h, self.img_w, 1))\n        y = np.empty((self.batch_size, self.img_h, self.img_w, 4))\n        \n        for idx, id in enumerate(list_ids_temp):\n            file_path =  os.path.join(self.image_dir, id)\n            image = cv2.imread(file_path, 0)\n            image_resized = cv2.resize(image, (self.img_w, self.img_h))\n            image_resized = np.array(image_resized, dtype=np.float64)\n            # standardization of the image\n            image_resized -= image_resized.mean()\n            image_resized /= image_resized.std()\n            \n            mask = np.empty((img_h, img_w, 4))\n            \n            for idm, image_class in enumerate(['Fish', 'Flower', 'Gravel', 'Sugar']):\n                rle = self.labels.get(id + '_' + image_class)\n                # if there is no mask create empty mask\n                if rle is None:\n                    class_mask = np.zeros((2100, 1400))\n                else:\n                    class_mask = rle_to_mask(rle, width=2100, height=1400)\n             \n                class_mask_resized = cv2.resize(class_mask, (self.img_w, self.img_h))\n                mask[...,idm] = class_mask_resized\n            \n            X[idx,] = np.expand_dims(image_resized, axis=2)\n            y[idx,] = mask\n        \n        # normalize Y\n        y = (y > 0).astype(int)\n            \n        return X, y","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# split the training data into train and validation set (stratified)\ntrain_image_ids = train_df['ImageId'].unique()\nX_train, X_val = train_test_split(train_image_ids, test_size=val_size, random_state=42)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# create a dict of all the masks\nmasks = {}\nfor index, row in train_df[train_df['EncodedPixels']!=-1].iterrows():\n    masks[row['Image_Label']] = row['EncodedPixels']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"params = {'img_h': img_h,\n          'img_w': img_w,\n          'image_dir': train_image_path,\n          'batch_size': batch_size,\n          'shuffle': True}\n\n# Get Generators\ntraining_generator = DataGenerator(X_train, masks, **params)\nvalidation_generator = DataGenerator(X_val, masks, **params)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# check out the shapes\nx, y = training_generator.__getitem__(0)\nprint(x.shape, y.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize cloud image with four classes of faults in seperate columns\ndef viz_cloud_img_mask(img, masks):\n    img = cv2.cvtColor(img.astype('float32'), cv2.COLOR_BGR2RGB)\n    fig, ax = plt.subplots(nrows=1, ncols=4, sharey=True, figsize=(20,10))\n    cmaps = [\"Reds\", \"Blues\", \"Greens\", \"Purples\"]\n    for idx, mask in enumerate(masks):\n        ax[idx].imshow(img)\n        ax[idx].imshow(mask, alpha=0.3, cmap=cmaps[idx])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# lets visualize some images with their cloud formation mask to make sure our data generator is working like it should\nfor ix in range(0,batch_size):\n    if y[ix].sum() > 0:\n        img = x[ix]\n        masks_temp = [y[ix][...,i] for i in range(0,4)]\n        viz_cloud_img_mask(img, masks_temp)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## UNet++: A Nested U-Net Architecture\n\nTo detect the cloud formation in our image, we need a convolutional neural network. In this section we will write the code build this network or model. The most used architecture for (semantic segmentation) task is U-Net. U-Net++ makes significant improvements on this, and this will be our architecture of choice for this experiment. [This medium article](https://medium.com/@sh.tsang/review-unet-a-nested-u-net-architecture-biomedical-image-segmentation-57be56859b20) explains the differences and improvement's of U-Net++ architecture over U-Net. Furthermore, how the architecture with \"deep-supervision\" works, a term you will see as a configurable parameter in our model. So the article is a good read if you want to get more knowledge about U-Net++.\n\n![UNet++: A Nested U-Net Architecture](https://miro.medium.com/max/658/1*ExIkm6cImpPgpetFW1kwyQ.png)\n\nThe code below has been adopted from this GitHub repo: https://github.com/CarryHJR/Nested-UNet. So a big shoutout to the authors!"},{"metadata":{"trusted":true},"cell_type":"code","source":"def standard_unit(input_tensor, stage, nb_filter, kernel_size=3):\n\n    act = 'elu'\n\n    x = Conv2D(nb_filter, (kernel_size, kernel_size), activation=act, name='conv'+stage+'_1', kernel_initializer = 'he_normal', padding='same', kernel_regularizer=l2(1e-4))(input_tensor)\n    x = Dropout(dropout_rate, name='dp'+stage+'_1')(x)\n    x = Conv2D(nb_filter, (kernel_size, kernel_size), activation=act, name='conv'+stage+'_2', kernel_initializer = 'he_normal', padding='same', kernel_regularizer=l2(1e-4))(x)\n    x = Dropout(dropout_rate, name='dp'+stage+'_2')(x)\n\n    return x","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def Nest_Net(img_rows, img_cols, color_type=1, num_class=1, deep_supervision=False):\n\n    nb_filter = [32,64,128,256,512]\n    act = 'elu'\n\n    bn_axis = 3\n    img_input = Input(shape=(img_rows, img_cols, color_type), name='main_input')\n\n    conv1_1 = standard_unit(img_input, stage='11', nb_filter=nb_filter[0])\n    pool1 = MaxPool2D((2, 2), strides=(2, 2), name='pool1')(conv1_1)\n\n    conv2_1 = standard_unit(pool1, stage='21', nb_filter=nb_filter[1])\n    pool2 = MaxPool2D((2, 2), strides=(2, 2), name='pool2')(conv2_1)\n\n    up1_2 = Conv2DTranspose(nb_filter[0], (2, 2), strides=(2, 2), name='up12', padding='same')(conv2_1)\n    conv1_2 = concatenate([up1_2, conv1_1], name='merge12', axis=bn_axis)\n    conv1_2 = standard_unit(conv1_2, stage='12', nb_filter=nb_filter[0])\n\n    conv3_1 = standard_unit(pool2, stage='31', nb_filter=nb_filter[2])\n    pool3 = MaxPool2D((2, 2), strides=(2, 2), name='pool3')(conv3_1)\n\n    up2_2 = Conv2DTranspose(nb_filter[1], (2, 2), strides=(2, 2), name='up22', padding='same')(conv3_1)\n    conv2_2 = concatenate([up2_2, conv2_1], name='merge22', axis=bn_axis)\n    conv2_2 = standard_unit(conv2_2, stage='22', nb_filter=nb_filter[1])\n\n    up1_3 = Conv2DTranspose(nb_filter[0], (2, 2), strides=(2, 2), name='up13', padding='same')(conv2_2)\n    conv1_3 = concatenate([up1_3, conv1_1, conv1_2], name='merge13', axis=bn_axis)\n    conv1_3 = standard_unit(conv1_3, stage='13', nb_filter=nb_filter[0])\n\n    conv4_1 = standard_unit(pool3, stage='41', nb_filter=nb_filter[3])\n    pool4 = MaxPool2D((2, 2), strides=(2, 2), name='pool4')(conv4_1)\n\n    up3_2 = Conv2DTranspose(nb_filter[2], (2, 2), strides=(2, 2), name='up32', padding='same')(conv4_1)\n    conv3_2 = concatenate([up3_2, conv3_1], name='merge32', axis=bn_axis)\n    conv3_2 = standard_unit(conv3_2, stage='32', nb_filter=nb_filter[2])\n\n    up2_3 = Conv2DTranspose(nb_filter[1], (2, 2), strides=(2, 2), name='up23', padding='same')(conv3_2)\n    conv2_3 = concatenate([up2_3, conv2_1, conv2_2], name='merge23', axis=bn_axis)\n    conv2_3 = standard_unit(conv2_3, stage='23', nb_filter=nb_filter[1])\n\n    up1_4 = Conv2DTranspose(nb_filter[0], (2, 2), strides=(2, 2), name='up14', padding='same')(conv2_3)\n    conv1_4 = concatenate([up1_4, conv1_1, conv1_2, conv1_3], name='merge14', axis=bn_axis)\n    conv1_4 = standard_unit(conv1_4, stage='14', nb_filter=nb_filter[0])\n\n    conv5_1 = standard_unit(pool4, stage='51', nb_filter=nb_filter[4])\n\n    up4_2 = Conv2DTranspose(nb_filter[3], (2, 2), strides=(2, 2), name='up42', padding='same')(conv5_1)\n    conv4_2 = concatenate([up4_2, conv4_1], name='merge42', axis=bn_axis)\n    conv4_2 = standard_unit(conv4_2, stage='42', nb_filter=nb_filter[3])\n\n    up3_3 = Conv2DTranspose(nb_filter[2], (2, 2), strides=(2, 2), name='up33', padding='same')(conv4_2)\n    conv3_3 = concatenate([up3_3, conv3_1, conv3_2], name='merge33', axis=bn_axis)\n    conv3_3 = standard_unit(conv3_3, stage='33', nb_filter=nb_filter[2])\n\n    up2_4 = Conv2DTranspose(nb_filter[1], (2, 2), strides=(2, 2), name='up24', padding='same')(conv3_3)\n    conv2_4 = concatenate([up2_4, conv2_1, conv2_2, conv2_3], name='merge24', axis=bn_axis)\n    conv2_4 = standard_unit(conv2_4, stage='24', nb_filter=nb_filter[1])\n\n    up1_5 = Conv2DTranspose(nb_filter[0], (2, 2), strides=(2, 2), name='up15', padding='same')(conv2_4)\n    conv1_5 = concatenate([up1_5, conv1_1, conv1_2, conv1_3, conv1_4], name='merge15', axis=bn_axis)\n    conv1_5 = standard_unit(conv1_5, stage='15', nb_filter=nb_filter[0])\n\n    nestnet_output_1 = Conv2D(num_class, (1, 1), activation='sigmoid', name='output_1', kernel_initializer = 'he_normal', padding='same', kernel_regularizer=l2(1e-4))(conv1_2)\n    nestnet_output_2 = Conv2D(num_class, (1, 1), activation='sigmoid', name='output_2', kernel_initializer = 'he_normal', padding='same', kernel_regularizer=l2(1e-4))(conv1_3)\n    nestnet_output_3 = Conv2D(num_class, (1, 1), activation='sigmoid', name='output_3', kernel_initializer = 'he_normal', padding='same', kernel_regularizer=l2(1e-4))(conv1_4)\n    nestnet_output_4 = Conv2D(num_class, (1, 1), activation='sigmoid', name='output_4', kernel_initializer = 'he_normal', padding='same', kernel_regularizer=l2(1e-4))(conv1_5)\n\n    if deep_supervision:\n        model = Model(img_input, [nestnet_output_1,nestnet_output_2,nestnet_output_3,nestnet_output_4])\n    else:\n        model = Model(img_input, [nestnet_output_4])\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Loss Functions\n\nLoss Functions allows our network to measure the error and reduce the error by using gradient descent. This competition is evaluated on the mean [Dice coefficient](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient). The Dice coefficient can be used to compare the pixel-wise agreement between a predicted segmentation and its corresponding ground truth. Since dice coefficient is the evaluation metric, we will use dice loss function as our loss function for the model. However, there are loss functions like Tversky, and Focal Tversky that you can experiment with for a better result."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Dice similarity coefficient loss, brought to you by: https://github.com/nabsabraham/focal-tversky-unet\ndef dsc(y_true, y_pred):\n    smooth = 1.\n    y_true_f = Flatten()(y_true)\n    y_pred_f = Flatten()(y_pred)\n    intersection = reduce_sum(y_true_f * y_pred_f)\n    score = (2. * intersection + smooth) / (reduce_sum(y_true_f) + reduce_sum(y_pred_f) + smooth)\n    return score\n\ndef dice_loss(y_true, y_pred):\n    loss = 1 - dsc(y_true, y_pred)\n    return loss\n\ndef bce_dice_loss(y_true, y_pred):\n    loss = binary_crossentropy(y_true, y_pred) + dice_loss(y_true, y_pred)\n    return loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Compile & Fit The Model\nNow we have our data generator, network architecture, loss function defined, we will compile and train the model in this section. You will notice we are using Adam as our optimizer. If you want to read more about Adam, or understand more about what optimizers are and why they are needed, this is a good article: https://medium.com/datadriveninvestor/overview-of-different-optimizers-for-neural-networks-e0ed119440c3."},{"metadata":{"trusted":true},"cell_type":"code","source":"# get an instance of the model\nmodel = Nest_Net(img_h, img_w, color_type=1, num_class=4, deep_supervision=False)\n# define optimizer \nadam = Adam(lr = 0.05, epsilon = 0.1)\nmodel.compile(optimizer=adam, loss=bce_dice_loss, metrics=[dice_loss])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if load_pretrained_model:\n    try:\n        model.load_weights(pretrained_model_path)\n        print('pre-trained model loaded!')\n    except OSError:\n        print('You need to run the model and load the trained model')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(generator=training_generator, validation_data=validation_generator, epochs=epochs, verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if save_model: \n    model.save(model_save_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model Insights\nUsing the history object of the model, we can review how in each epoch we did (in terms of reducing error or improving accuracy). In this section we will do two plots to show model accuracy and loss for our training and validation set per epoch."},{"metadata":{"trusted":true},"cell_type":"code","source":"# summarize history for accuracy\nplt.figure(figsize=(20,5))\nplt.subplot(1,2,1)\nplt.plot(history.history['loss'])\nplt.plot(history.history['dice_loss'])\nplt.title('model accuracy')\nplt.ylabel('accuracy')\nplt.xlabel('epoch')\nplt.legend(['train', 'validation'], loc='upper left')\n\n# summarize history for loss\nplt.subplot(1,2,2)\nplt.plot(history.history['val_loss'])\nplt.plot(history.history['val_dice_loss'])\nplt.title('model loss')\nplt.ylabel('loss')\nplt.xlabel('epoch')\nplt.legend(['train', 'validation'], loc='upper left')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## TODO\n\nNow that we have our model you can extend this notebook to do the following:\n\n1. Visualize the segmentations for the validation set.\n2. Use the model to make segmentation for the training set.\n3. Use the mask to rle function to create a submission file.\n\nI have done the above for another challenge in this kernel: https://www.kaggle.com/ekhtiar/resunet-a-baseline-on-tensorflow. Use it for helping you solve the three challenges above. "}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}