{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Description \nThis notebook contains a basic implementation of a Keras UNet model for the [Sartorius - Cell Instance Segmentation competition](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/data). If you wish to run an inference and submit, only export your model using [tf.keras.callbacks.ModelCheckpoint()](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpoint). Then you can create a private dataset containing the model, load the model and make the predictions for the test set in another notebook (internet should be inactivated in notebook settings).\n\n## Problem definition\nData: \n\n[Phase-contrast microscopy](https://en.wikipedia.org/wiki/Phase-contrast_microscopy) images of human neuronal cell  along with annotations (labels) representing cell segmentations. \n\nAim: \n\nThe trained model should be able to predict the annotations for cell segmentation, including rare cell types (such as neuroblastoma cell line SH-SY5Y as discussed in the [competition description](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/overview)). Annotations should be provided in run-length format (see functions rle_decode() and rle_encode below).\n\n## Approach \n[U-Net](https://en.wikipedia.org/wiki/U-Net) as implemented in reference notebook 2. \n\n\"The model being used here is a modified U-Net. A U-Net consists of an encoder (downsampler) and decoder (upsampler). In-order to learn robust features and reduce the number of trainable parameters, you will use a pretrained model - MobileNetV2 - as the encoder. For the decoder, you will use the upsample block, which is already implemented in the pix2pix example in the TensorFlow Examples repo. (Check out the pix2pix: Image-to-image translation with a conditional GAN tutorial in a notebook.)\n\nAs mentioned, the encoder will be a pretrained MobileNetV2 model which is prepared and ready to use in tf.keras.applications. The encoder consists of specific outputs from intermediate layers in the model. Note that the encoder will not be trained during the training process.\n\nThe decoder/upsampler is simply a series of upsample blocks implemented in TensorFlow examples.\" [Tensorflow Tutorials: Image segmentation ](https://www.tensorflow.org/tutorials/images/segmentation).  \n\nReference notebooks:\n1. [Cell Segmentation - Run Length Decoding](https://www.kaggle.com/ihelon/cell-segmentation-run-length-decoding).\n2. [Sartors TF starter](https://www.kaggle.com/barteksadlej123/sartors-tf-starter).\n3. [Positive score with Detectron 3/3 - Inference](https://www.kaggle.com/dragonzhang/positive-score-with-detectron-3-3-inference).\n\nOther references: \n1. [Tensorflow Tutorials: Image segmentation ](https://www.tensorflow.org/tutorials/images/segmentation).\n2. [pix2pix: Image-to-image translation with a conditional GAN](https://www.tensorflow.org/tutorials/generative/pix2pix?hl=nb).\n","metadata":{}},{"cell_type":"markdown","source":"## Workflow\n\n[1. Libraries and paths](#section-1)\n\n[2. Functions](#section-2)\n\n[3. Constants](#section-3)\n\n[4. Generate train and validation data sets](#section-4)\n\n[5. Define the model](#section-5)\n\n[6. Training](#section-6)\n\n[7. Test set predictions](#section-7)\n\n","metadata":{}},{"cell_type":"markdown","source":"<a id=\"section-1\"></a>\n### 1. Libraries and paths","metadata":{}},{"cell_type":"code","source":"# required libraries \nimport numpy as np \nimport pandas as pd\nimport os\nfrom pathlib import Path\nimport cv2\n\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras.layers.experimental import preprocessing\n\nfrom IPython.display import clear_output\nimport matplotlib.pyplot as plt","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-10-27T03:21:08.287153Z","iopub.execute_input":"2021-10-27T03:21:08.287785Z","iopub.status.idle":"2021-10-27T03:21:13.385326Z","shell.execute_reply.started":"2021-10-27T03:21:08.287701Z","shell.execute_reply":"2021-10-27T03:21:13.384562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# paths\n# input\nDIR = '../input/sartorius-cell-instance-segmentation'\ntrain_csv = os.path.join(DIR,'train.csv') \ntrain_path =  os.path.join(DIR, 'train/')\ntest_path = os.path.join(DIR, 'test/')\n\n# output \ncsv_output = os.path.join('./', 'submission.csv') \nmodel_output = os.path.join('./', 'unet_keras_model.h5')","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:13.387024Z","iopub.execute_input":"2021-10-27T03:21:13.387264Z","iopub.status.idle":"2021-10-27T03:21:13.393452Z","shell.execute_reply.started":"2021-10-27T03:21:13.387232Z","shell.execute_reply":"2021-10-27T03:21:13.392347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-2\"></a>\n### 2. Functions","metadata":{}},{"cell_type":"code","source":"def rle_decode(mask_rle, shape, color=1):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height,width) of array to return \n    Returns numpy array, 1 - mask, 0 - background.\n    ref: https://www.kaggle.com/inversion/run-length-decoding-quick-start\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros((shape[0] * shape[1]), dtype=np.float32)\n    for lo, hi in zip(starts, ends):\n        img[lo : hi] = color\n    return img.reshape(shape)\n\n\ndef rle_encode(img):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    ref: https://www.kaggle.com/dragonzhang/positive-score-with-detectron-3-3-inference\n    '''\n    pixels = img.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\n\ndef get_mask(image_id, df):\n    '''\n    Uses rle_decode() to get ndarray from mask using image_id in dataframe (df).\n    ref: https://www.kaggle.com/barteksadlej123/sartors-tf-starter\n    '''\n    current = df[df[\"id\"] == image_id]\n    labels = current[\"annotation\"].tolist()\n    \n    mask = np.zeros((HEIGHT, WIDTH))\n    for label in labels:\n        mask += rle_decode(label, (HEIGHT, WIDTH))\n    mask = mask.clip(0, 1)\n    \n    return mask\n\n\n#  fix overlaps: \n\ndef check_overlap(msk):\n    '''\n    Checks if there are overlap in a mask (msk).\n    ref: https://www.kaggle.com/awsaf49/sartorius-fix-overlap\n    '''\n    msk = msk.astype(np.bool).astype(np.uint8)\n    return np.any(np.sum(msk, axis=-1)>1)\n\n\ndef fix_overlap(msk):\n    '''\n    Args:\n        mask: multi-channel mask, each channel is an instance of cell, shape:(520,704,None)\n    Returns:\n        multi-channel mask with non-overlapping values, shape:(520,704,None) \n    ref: https://www.kaggle.com/awsaf49/sartorius-fix-overlap\n    '''\n    msk = np.array(msk)\n    msk = np.pad(msk, [[0,0],[0,0],[1,0]])\n    ins_len = msk.shape[-1]\n    msk = np.argmax(msk,axis=-1)\n    msk = tf.keras.utils.to_categorical(msk, num_classes=ins_len)\n    msk = msk[...,1:]\n    msk = msk[...,np.any(msk, axis=(0,1))]\n    return msk\n\n\n# make predictions for test set: \n\ndef make_predictions(dataset, num, keras_model):\n    '''\n    For a tf.Dataset, makes predictions for n=num (num =-1 or all_images takes all images in the dataset), \n    images using a keras_model. Returns a list of predicted masks, each as ndarray. \n    '''\n    predictions = []\n    if dataset:\n        for image in dataset.take(num):\n            image = image[None]\n            pred_mask = keras_model.predict(image)\n            # changes shape from (1,512,512,1) to (512,512)\n            pred_mask = pred_mask[0, :, :, 0]\n            # fix overlaps\n            if check_overlap(msk=pred_mask)==True:\n                pred_mask = pred_mask[None]\n                pred_mask = fix_overlap(msk=pred_mask)\n            # transforms ndarray values to 0s and 1s\n            pred_mask =  np.where( pred_mask > 0.5, 1, 0)\n            predictions.append(pred_mask)\n    return predictions","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:13.395443Z","iopub.execute_input":"2021-10-27T03:21:13.395713Z","iopub.status.idle":"2021-10-27T03:21:13.415567Z","shell.execute_reply.started":"2021-10-27T03:21:13.395677Z","shell.execute_reply":"2021-10-27T03:21:13.414715Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-3\"></a>\n### 3. Constants ","metadata":{}},{"cell_type":"code","source":"# constants\n\nDEBUG = False\n\nSEED = 123\nWIDTH, HEIGHT = 704, 520\nRESIZE_WIDTH, RESIZE_HEIGHT = 512, 512\nBATCH_SIZE = 32\nBUFFER_SIZE = 32\n\nVAL_SPLIT = 0.2\n\nAUTO = tf.data.AUTOTUNE\n\nEPOCHS = 20\n","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:13.417868Z","iopub.execute_input":"2021-10-27T03:21:13.418412Z","iopub.status.idle":"2021-10-27T03:21:13.425183Z","shell.execute_reply.started":"2021-10-27T03:21:13.418366Z","shell.execute_reply":"2021-10-27T03:21:13.424526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-4\"></a>\n### 4. Generate train and validation data sets","metadata":{}},{"cell_type":"code","source":"# train and validation split\ntrain = pd.read_csv(train_csv)\ntrain.head()\n\nn_ids = train.id.nunique()\n\nif DEBUG:\n    unique_ids_train = list(set(train['id'].tolist()))[:BATCH_SIZE]\n    unique_ids_valid = list(set(train['id'].tolist()))[BATCH_SIZE:2*BATCH_SIZE]\nelse:\n    unique_ids_train = list(set(train['id'].tolist()))[:int(n_ids * (1 - VAL_SPLIT))]\n    unique_ids_valid = list(set(train['id'].tolist()))[int(n_ids * (1 - VAL_SPLIT)):]\n\n\ntemp = pd.DataFrame()\nfor sample_id in unique_ids_train:\n    query = train[train.id == sample_id]\n    temp = pd.concat([temp, query])\ntrain = temp\ntrain = train.reset_index(drop=True)\n\ntemp = pd.DataFrame()\nfor sample_id in unique_ids_valid:\n    query = train[train.id == sample_id]\n    temp = pd.concat([temp, query])\nvalid = temp\nvalid = train.reset_index(drop=True)\n    \nTRAIN_LENGTH = train['id'].nunique()\nSTEPS_PER_EPOCH = TRAIN_LENGTH // BATCH_SIZE\n\nVALID_LENGTH = valid['id'].nunique()\nVALIDATION_STEPS = VALID_LENGTH // BATCH_SIZE","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:13.426585Z","iopub.execute_input":"2021-10-27T03:21:13.427014Z","iopub.status.idle":"2021-10-27T03:21:21.490696Z","shell.execute_reply.started":"2021-10-27T03:21:13.426974Z","shell.execute_reply":"2021-10-27T03:21:21.489715Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# training data generator \ndef train_generator(df):\n    image_ids = set(df['id'].tolist())\n    \n    for image_id in image_ids:\n        image = cv2.imread(os.path.join(train_path, image_id) + '.png') \n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        \n        mask = get_mask(image_id, df)\n        \n        image = cv2.resize(image, (RESIZE_HEIGHT, RESIZE_WIDTH))\n        mask = cv2.resize(mask, (RESIZE_HEIGHT, RESIZE_WIDTH))\n        mask = mask.reshape((*mask.shape, 1))\n        \n        image = image.astype(np.float32)\n        mask = mask.astype(np.int32)\n        \n        yield image, mask","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:21.492263Z","iopub.execute_input":"2021-10-27T03:21:21.492560Z","iopub.status.idle":"2021-10-27T03:21:21.499799Z","shell.execute_reply.started":"2021-10-27T03:21:21.492523Z","shell.execute_reply":"2021-10-27T03:21:21.499009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# use the generator to get training and validation sets\ntrain_ds = tf.data.Dataset.from_generator(\n    lambda : train_generator(train), \n    output_types=(tf.float32, tf.int32),\n    output_shapes=((RESIZE_HEIGHT, RESIZE_WIDTH, 3), (RESIZE_HEIGHT, RESIZE_WIDTH, 1)))\n\nvalid_ds = tf.data.Dataset.from_generator(\n    lambda : train_generator(valid), \n    output_types=(tf.float32, tf.int32),\n    output_shapes=((RESIZE_HEIGHT, RESIZE_WIDTH, 3), (RESIZE_HEIGHT, RESIZE_WIDTH, 1)))\n","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:21.501134Z","iopub.execute_input":"2021-10-27T03:21:21.501401Z","iopub.status.idle":"2021-10-27T03:21:24.091873Z","shell.execute_reply.started":"2021-10-27T03:21:21.501365Z","shell.execute_reply":"2021-10-27T03:21:24.090219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# \"the following class performs a simple augmentation by randomly-flipping an image\"\nclass Augment(tf.keras.layers.Layer):\n    def __init__(self, seed=SEED):\n        super().__init__()\n        \n        self.augment_inputs = preprocessing.RandomFlip('horizontal', seed=seed)\n        self.augment_labels = preprocessing.RandomFlip('horizontal', seed=seed)\n        \n    def call(self, inputs, labels):\n        inputs = self.augment_inputs(inputs)\n        labels = self.augment_labels(labels)\n        return inputs, labels","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:24.093410Z","iopub.execute_input":"2021-10-27T03:21:24.093684Z","iopub.status.idle":"2021-10-27T03:21:24.100984Z","shell.execute_reply.started":"2021-10-27T03:21:24.093646Z","shell.execute_reply":"2021-10-27T03:21:24.099184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# \"build the input pipeline, applying the augmentation after batching the inputs\"\n\ntrain_ds = (\n    train_ds\n    .shuffle(BUFFER_SIZE)\n    .batch(BATCH_SIZE)\n    .repeat()\n    .map(Augment())\n    .prefetch(AUTO))\n\nvalid_ds = (\n    valid_ds\n    .batch(BATCH_SIZE)\n    .repeat()\n    .prefetch(AUTO))","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:24.102279Z","iopub.execute_input":"2021-10-27T03:21:24.102845Z","iopub.status.idle":"2021-10-27T03:21:24.268277Z","shell.execute_reply.started":"2021-10-27T03:21:24.102802Z","shell.execute_reply":"2021-10-27T03:21:24.267510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-5\"></a>\n### 5. Define the model ","metadata":{}},{"cell_type":"code","source":"# \"visualize an image example and its corresponding mask from the dataset\"\n\ndef display(display_list):\n    plt.figure(figsize=(20, 20))\n\n    title = ['Input Image', 'True Mask', 'Predicted Mask']\n\n    for i in range(len(display_list)):\n        plt.subplot(1, len(display_list), i+1)\n        plt.title(title[i])\n        plt.imshow(tf.keras.preprocessing.image.array_to_img(display_list[i]))\n        plt.axis('off')\n    plt.show()\n    \nfor images, masks in train_ds.take(2):\n    sample_image, sample_mask = images[0], masks[0]\n    display([sample_image, sample_mask])","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:24.271434Z","iopub.execute_input":"2021-10-27T03:21:24.271707Z","iopub.status.idle":"2021-10-27T03:21:36.238076Z","shell.execute_reply.started":"2021-10-27T03:21:24.271670Z","shell.execute_reply":"2021-10-27T03:21:36.237280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# base model and encoder (down_stack)\n\nbase_model = tf.keras.applications.MobileNetV2(input_shape=[RESIZE_HEIGHT, RESIZE_WIDTH, 3], include_top=False)\n\n# use the activations of these layers\nlayer_names = [\n    'block_1_expand_relu',   # 64x64\n    'block_3_expand_relu',   # 32x32\n    'block_6_expand_relu',   # 16x16\n    'block_13_expand_relu',  # 8x8\n    'block_16_project',      # 4x4\n]\n\nbase_model_outputs = [base_model.get_layer(name).output for name in layer_names]\n\n# create the feature extraction model\ndown_stack = tf.keras.Model(inputs=base_model.input, outputs=base_model_outputs)\ndown_stack.trainable = False","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:36.239888Z","iopub.execute_input":"2021-10-27T03:21:36.240158Z","iopub.status.idle":"2021-10-27T03:21:37.468043Z","shell.execute_reply.started":"2021-10-27T03:21:36.240130Z","shell.execute_reply":"2021-10-27T03:21:37.467113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# decoder / upsampler\n\ndef upsample(filters, size, apply_dropout=False):\n    '''\n    filters: Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution). \n    size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. \n    Can be a single integer to specify the same value for all spatial dimensions. \n    return: 4+D tensor with shape: batch_shape + (channels, rows, cols) if data_format='channels_first' \n    or 4+D tensor with shape: batch_shape + (rows, cols, channels) if data_format='channels_last\n    ref: https://www.tensorflow.org/tutorials/generative/pix2pix?hl=nb\n    '''\n    initializer = tf.random_normal_initializer(0., 0.02)\n    result = tf.keras.Sequential()\n    result.add(\n    tf.keras.layers.Conv2DTranspose(filters, size, strides=2,\n                                    padding='same',\n                                    kernel_initializer=initializer,\n                                    use_bias=False))\n    result.add(tf.keras.layers.BatchNormalization())\n    if apply_dropout:\n        result.add(tf.keras.layers.Dropout(0.5))\n    result.add(tf.keras.layers.ReLU())\n    return result\n\n\nup_stack = [\n    upsample(512, 3),  # 4x4 -> 8x8\n    upsample(256, 3),  # 8x8 -> 16x16\n    upsample(128, 3),  # 16x16 -> 32x32\n    upsample(64, 3),   # 32x32 -> 64x64\n]","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:37.469479Z","iopub.execute_input":"2021-10-27T03:21:37.469850Z","iopub.status.idle":"2021-10-27T03:21:37.499505Z","shell.execute_reply.started":"2021-10-27T03:21:37.469809Z","shell.execute_reply":"2021-10-27T03:21:37.498790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# U-Net model \ndef unet_model(output_channels : int):\n    inputs = tf.keras.layers.Input(shape=[RESIZE_HEIGHT , RESIZE_WIDTH, 3])\n    # \"downsampling through the model\"\n    skips = down_stack(inputs)\n    x = skips[-1]\n    skips = reversed(skips[:-1])\n    # \"sampling and establishing the skip connections\"\n    for up, skip in zip(up_stack, skips):\n        x = up(x)\n        concat = tf.keras.layers.Concatenate()\n        x = concat([x, skip])\n    # \"this is the last layer of the model\"\n    last = tf.keras.layers.Conv2DTranspose(\n        filters=output_channels, kernel_size=3, strides=2,\n        padding='same', activation='sigmoid') #64x64 -> 128x128\n    \n    x = last(x)\n    \n    return tf.keras.Model(inputs=inputs, outputs=x)","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:37.500619Z","iopub.execute_input":"2021-10-27T03:21:37.501007Z","iopub.status.idle":"2021-10-27T03:21:37.508391Z","shell.execute_reply.started":"2021-10-27T03:21:37.500968Z","shell.execute_reply":"2021-10-27T03:21:37.507420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# compile model \n## in a segmentation task each pixel is given a class \n## OUTPUT_CLASSES: number of classes that can be assigned to each pixel \n\nOUTPUT_CLASSES = 1\n\nmodel = unet_model(output_channels=OUTPUT_CLASSES)\n\nopt = keras.optimizers.Adam(learning_rate=0.01)\n\nmodel.compile(optimizer=opt,\n              loss=tf.keras.losses.BinaryCrossentropy(from_logits=False),\n              metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:37.509523Z","iopub.execute_input":"2021-10-27T03:21:37.510356Z","iopub.status.idle":"2021-10-27T03:21:37.969434Z","shell.execute_reply.started":"2021-10-27T03:21:37.510318Z","shell.execute_reply":"2021-10-27T03:21:37.968655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# visualize model architecture \ntf.keras.utils.plot_model(model, show_shapes=True)","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:37.970575Z","iopub.execute_input":"2021-10-27T03:21:37.970837Z","iopub.status.idle":"2021-10-27T03:21:38.840625Z","shell.execute_reply.started":"2021-10-27T03:21:37.970801Z","shell.execute_reply":"2021-10-27T03:21:38.839876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# try out the model to check what it predicts before training\n\ndef create_mask(pred_mask):\n    pred_mask = tf.where(pred_mask > 0.5,1,0)\n    return pred_mask\n\n\ndef show_predictions(dataset=None, num=1):\n    if dataset:\n        for image, mask in dataset.take(num):\n            pred_mask = model.predict(image)\n            display([image[0], mask[0], create_mask(pred_mask[0])])\n    else:\n        display([sample_image, sample_mask,\n                 create_mask(model.predict(sample_image[tf.newaxis, ...])[0])])\n\n        \nshow_predictions(train_ds)","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:38.842141Z","iopub.execute_input":"2021-10-27T03:21:38.844160Z","iopub.status.idle":"2021-10-27T03:21:59.365193Z","shell.execute_reply.started":"2021-10-27T03:21:38.844115Z","shell.execute_reply":"2021-10-27T03:21:59.364322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-6\"></a>\n### 6. Training","metadata":{}},{"cell_type":"code","source":"# training\n\n# \"observe how the model improves while it is training\"\nclass DisplayCallback(tf.keras.callbacks.Callback):\n    def __init__(self):\n        super().__init__()\n    def on_epoch_end(self, epoch, logs=None):\n        clear_output(wait=False)\n        show_predictions()\n        print ('\\nSample Prediction after epoch {}\\n'.format(epoch+1))\n# display callback defined above\ndisplay_cb = DisplayCallback()\n\n# \"save the Keras model or model weights at some frequency\"\nmodel_checkpoint = tf.keras.callbacks.ModelCheckpoint(\n    model_output,\n    save_best_only=True,\n    save_weights_only=True,\n)\n\n# \"reduce learning rate when a metric has stopped improving\"\n# documentation: https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ReduceLROnPlateau\nlr_reduce = tf.keras.callbacks.ReduceLROnPlateau( monitor='val_loss', factor=0.1, patience=10, verbose=0,\n    mode='auto', min_delta=0.0001, cooldown=0, min_lr=0)\n\nmodel_history = model.fit(train_ds, epochs=EPOCHS,\n                          steps_per_epoch=STEPS_PER_EPOCH,\n                          validation_steps=VALIDATION_STEPS,\n                          validation_data=valid_ds,\n                          callbacks=[display_cb, model_checkpoint, lr_reduce])","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:21:59.367111Z","iopub.execute_input":"2021-10-27T03:21:59.367386Z","iopub.status.idle":"2021-10-27T03:45:38.778941Z","shell.execute_reply.started":"2021-10-27T03:21:59.367351Z","shell.execute_reply":"2021-10-27T03:45:38.778114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot training curve\nloss = model_history.history['loss']\nval_loss = model_history.history['val_loss']\nplt.figure()\nplt.plot(model_history.epoch, loss, 'r', label='Training loss')\nplt.plot(model_history.epoch, val_loss, 'bo', label='Validation loss')\nplt.title('Training and Validation Loss')\nplt.xlabel('Epoch')\nplt.ylabel('Loss Value')\nplt.ylim([0, 1])\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:38.781006Z","iopub.execute_input":"2021-10-27T03:45:38.781788Z","iopub.status.idle":"2021-10-27T03:45:38.993624Z","shell.execute_reply.started":"2021-10-27T03:45:38.781747Z","shell.execute_reply":"2021-10-27T03:45:38.992838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-7\"></a>\n### 7. Test set predictions","metadata":{}},{"cell_type":"code","source":"# test data generator \ntest_ids = [  os.path.join(test_path, each)  for each in os.listdir(test_path) if each.endswith('.png')]\ndef test_generator(image_ids):\n    for image_id in image_ids:\n        image = cv2.imread(image_id) \n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)        \n        image = cv2.resize(image, (RESIZE_HEIGHT, RESIZE_WIDTH))\n        image = image.astype(np.float32)\n        yield image","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:38.994890Z","iopub.execute_input":"2021-10-27T03:45:38.995163Z","iopub.status.idle":"2021-10-27T03:45:39.004586Z","shell.execute_reply.started":"2021-10-27T03:45:38.995128Z","shell.execute_reply":"2021-10-27T03:45:39.003796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test dataset from test data generator \ntest_ds = tf.data.Dataset.from_generator(\n    lambda : test_generator(test_ids), \n    output_types=(tf.float32),\n    output_shapes=((RESIZE_HEIGHT, RESIZE_WIDTH, 3)) )","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:39.006187Z","iopub.execute_input":"2021-10-27T03:45:39.007361Z","iopub.status.idle":"2021-10-27T03:45:39.032063Z","shell.execute_reply.started":"2021-10-27T03:45:39.007316Z","shell.execute_reply":"2021-10-27T03:45:39.031240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test image ids and predictions\ntest_predictions = make_predictions(dataset=test_ds, num=len(test_ids), keras_model=model)","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:39.033466Z","iopub.execute_input":"2021-10-27T03:45:39.033792Z","iopub.status.idle":"2021-10-27T03:45:39.267612Z","shell.execute_reply.started":"2021-10-27T03:45:39.033752Z","shell.execute_reply":"2021-10-27T03:45:39.266860Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# encode predections in the RL format\ntest_predictions = [rle_encode(mask) for mask in test_predictions] ","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:39.269178Z","iopub.execute_input":"2021-10-27T03:45:39.269455Z","iopub.status.idle":"2021-10-27T03:45:39.279448Z","shell.execute_reply.started":"2021-10-27T03:45:39.269419Z","shell.execute_reply":"2021-10-27T03:45:39.278695Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# transform full image paths to ids \ntest_ids = [Path(ID).stem for ID in test_ids]","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:39.282250Z","iopub.execute_input":"2021-10-27T03:45:39.282444Z","iopub.status.idle":"2021-10-27T03:45:39.288335Z","shell.execute_reply.started":"2021-10-27T03:45:39.282420Z","shell.execute_reply":"2021-10-27T03:45:39.287645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# generate submission data frame \nsubmisssion = pd.DataFrame.from_dict({'id': test_ids, 'predicted': test_predictions} )\nsubmisssion = submisssion.sort_values( ['id'], ascending=True )\nprint(submisssion.head(), 'n')\nsubmisssion.to_csv(csv_output, index=False)","metadata":{"execution":{"iopub.status.busy":"2021-10-27T03:45:39.289450Z","iopub.execute_input":"2021-10-27T03:45:39.289714Z","iopub.status.idle":"2021-10-27T03:45:39.304877Z","shell.execute_reply.started":"2021-10-27T03:45:39.289679Z","shell.execute_reply":"2021-10-27T03:45:39.304145Z"},"trusted":true},"execution_count":null,"outputs":[]}]}