{"cells":[{"metadata":{},"cell_type":"markdown","source":"# ***Disclaimer:*** \nHello Kagglers! I am a Solution Architect with the Google Cloud Platform. I am a coach for this competition, the focus of my contributions is on helping users to leverage GCP components (GCS, TPUs, BigQueryetc..) in order to solve large problems. My ideas and contributions represent my own opinion, and are not representative of an official recommendation by Google. Also, I try to develop notebooks quickly in order to help users early in competitions. There may be better ways to solving particular problems, I welcome comments and suggestions. Use my contributions at your own risk, I don't garantee that they will help on winning any competition, but I am hoping to learn by collaborating with everyone.\n"},{"metadata":{},"cell_type":"markdown","source":"# Objective:\n\n\nThe objective of this notebook is to demonstrate how to feed a TFRecord dataset to a Keras Unet model for image segmentation. The advantage of using a TFRecord dataset is that you can then train using TPUs, as it will be explained in the next Notebook -- and you get 100x performance. \n\nIn previous notebooks, I demonstrated how to read the competition data and produce a TFRecord dataset. This Notebook will use this dataset as input:\n--> [Link to the TFRecord Dataset Used by this Notebook.](https://www.kaggle.com/marcosnovaes/hubmap-tfrecord-512)\n\nPrevious Notebooks in this competition: \n\n[https://www.kaggle.com/marcosnovaes/hubmap-read-data-and-build-tfrecords/](https://www.kaggle.com/marcosnovaes/hubmap-read-data-and-build-tfrecords/): Demonstrates how the TFRecord Dataset was built\n\n[https://www.kaggle.com/marcosnovaes/hubmap-looking-at-tfrecords/](https://www.kaggle.com/marcosnovaes/hubmap-looking-at-tfrecords/): Explains how to read the data using the TFRecord Dataset\n\n"},{"metadata":{},"cell_type":"markdown","source":"# Setup\n1) Add the TFRecord Dataset as input to the notebook: Go to the Data section at the right, click \"add data\" and lof for the dataset: \"hubmap_train_test\"\n2) This Notebook also shows how to access a Kaggle dataset directly from Google Cloud Storage (GCS). To enable this feature, you need to link the Notebook to a GCS project, by going to the menu Add-ons-->Cloud SDK"},{"metadata":{},"cell_type":"markdown","source":"My import section is a little messy as I imported snippets from several sources. It will be cleaned eventually."},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport sys\nimport random\nimport warnings\n\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\n\nfrom tqdm import tqdm\nfrom itertools import chain\nfrom skimage.io import imread, imshow, imread_collection, concatenate_images\nfrom skimage.transform import resize\nfrom skimage.morphology import label\n\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.layers import Input\n\nfrom keras.layers.core import Dropout, Lambda\nfrom tensorflow.keras.layers import Conv2D, Conv2DTranspose\nfrom tensorflow.keras.layers import MaxPooling2D, UpSampling2D\nfrom tensorflow.keras.layers import concatenate\n\nfrom tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\nfrom tensorflow.keras import backend as K\n\nfrom tensorflow.keras import layers\n\nfrom keras.engine.topology import Layer\n#from tensorflow.keras.layers.merge import concatenate, add\n\n\nfrom tensorflow.keras.optimizers import Adam\nfrom keras.utils.generic_utils import get_custom_objects\n\n\nfrom kaggle_datasets import KaggleDatasets\nfrom kaggle_secrets import UserSecretsClient\n\nimport tensorflow as tf","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"user_secrets = UserSecretsClient()\nuser_credential = user_secrets.get_gcloud_credential()\nuser_secrets.set_tensorflow_credential(user_credential)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# IMPORTANT SECTION: \nThe code below shows how to read TFRecords as in the previous notebook. However, the read function is altered here to return only the image and mask arrays. \n\nNotice the inclusion of a tf.reshape statement using literal dimensions. i.e, tf.reshape(1,512,512,3) for images and tf.reshape(1,512,512) for masks. We were previously reading these values dynamically, but when using TPUs this lead to a compilation error (\"Dynamic Shape xxxx not support in function XXX). This is because the TPU compilation does not support dynamic shapes yet, so the shapes must be \"baked\" into the code with literals as shown below. I learned this the hard way..."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a dictionary describing the features.\nimage_feature_description = {\n    'img_index': tf.io.FixedLenFeature([], tf.int64),\n    'height': tf.io.FixedLenFeature([], tf.int64),\n    'width': tf.io.FixedLenFeature([], tf.int64),\n    'num_channels': tf.io.FixedLenFeature([], tf.int64),\n    'img_bytes': tf.io.FixedLenFeature([], tf.string),\n    'mask': tf.io.FixedLenFeature([], tf.string),\n    'tile_id': tf.io.FixedLenFeature([], tf.int64),\n    'tile_col_pos': tf.io.FixedLenFeature([], tf.int64),\n    'tile_row_pos': tf.io.FixedLenFeature([], tf.int64),\n}\n\ndef _parse_image_and_masks_function(example_proto):\n    single_example = tf.io.parse_single_example(example_proto, image_feature_description)\n    img_height = single_example['height']\n    img_width = single_example['width']\n    num_channels = single_example['num_channels']\n    \n    img_bytes =  tf.io.decode_raw(single_example['img_bytes'],out_type='uint8')\n    #img_array = tf.reshape( img_bytes, (img_height, img_width, num_channels))\n    # Need to define array shape with literals to avoid dynamic shape errors\n    img_array = tf.reshape( img_bytes, (1,512, 512, 3))\n    \n    mask_bytes =  tf.io.decode_raw(single_example['mask'],out_type='bool')\n    \n    #mask = tf.reshape(mask_bytes, (1,img_height,img_width,1))\n    #mask = tf.reshape(mask_bytes, (img_height,img_width))\n    # Need to define array shape with literals to avoid dynamic shape errors\n    mask = tf.reshape(mask_bytes, (1,512,512))\n    \n    #cast to float 32\n    img_array = tf.cast(img_array, tf.float32) / 255.0\n    mask = tf.cast(mask, tf.float32)\n    return img_array, mask\n\ndef _parse_mask_function(example_proto):\n    single_example = tf.io.parse_single_example(example_proto, image_feature_description)\n    img_height = single_example['height']\n    img_width = single_example['width']\n    num_channels = single_example['num_channels']   \n    mask_bytes =  tf.io.decode_raw(single_example['mask'],out_type='bool') \n    mask = tf.reshape(mask_bytes, (img_height,img_width))\n    return mask\n\ndef read_tf_dataset(storage_file_path):\n    encoded_image_dataset = tf.data.TFRecordDataset(storage_file_path, compression_type=\"GZIP\")\n    parsed_image_dataset = encoded_image_dataset.map(_parse_image_data_function)\n    return parsed_image_dataset\n\ndef read_images_and_masks(storage_file_path):\n    encoded_image_dataset = tf.data.TFRecordDataset(storage_file_path, compression_type=\"GZIP\")\n    parsed_image_dataset = encoded_image_dataset.map(_parse_image_and_masks_function)\n    return parsed_image_dataset\n\ndef read_masks(storage_file_path):\n    encoded_image_dataset = tf.data.TFRecordDataset(storage_file_path, compression_type=\"GZIP\")\n    parsed_image_dataset = encoded_image_dataset.map(_parse_mask_function)\n    return parsed_image_dataset\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now look for the dataset \"hubmap-tfrecord-512\" in your input directory. If it is note there, add it as described in the Setup section."},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls /kaggle/input","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls /kaggle/input/hubmap-tfrecord-512","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I have recently added two CSV files that have a lot of useful metadata for each tile. The file \"train_all_tiles.csv\" has the data for all images tiles in the train set and \"test_all_tiles.csv\" for the test set."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_tiles_csv = '/kaggle/input/hubmap-tfrecord-512/train_all_tiles.csv'\ntest_tiles_csv = '/kaggle/input/hubmap-tfrecord-512/test_all_tiles.csv'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_tiles_df = pd.read_csv(train_tiles_csv)\ntrain_tiles_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Notice above that I have include both the local path and the gcs path for each tile. This is super handy. If you are using a GPU you can use the local file paths, but TPUs require the GCS file path. There is also metadata on some metrics for each tile, such as the mask density and lowband_density (as explained in the previous notebook). We can use these values to filters tiles of interest. For example, if we are interested in getting all tiles that have gloms, we select mask_density > 0 as below:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# build a dataset of all images tiles from the train set that have gloms in them\n#for csv_file in file_list:\n\ngloms_df = train_tiles_df.loc[train_tiles_df[\"mask_density\"]  > 0]\n\ngloms_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gloms_df.__len__()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, we have a total of 3384 tiles that have gloms. Selecting \"lowband_density > 1000\" will exclude all the \"black\" and \"gray\" tiles from the border, which have no tissue. "},{"metadata":{"trusted":true},"cell_type":"code","source":"cropped_df = train_tiles_df.loc[train_tiles_df[\"lowband_density\"]  > 1000]\ncropped_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cropped_df.__len__()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, there are 15957 tiles with tissue, 3384 of them have gloms. We may want to produce a balanced mix for training."},{"metadata":{},"cell_type":"markdown","source":"# Selecting a Unet Model\nI found several references to a [popular paper in biomedical image segementation](https://arxiv.org/abs/1505.04597), by (Olaf Ronneberger, Philipp Fischer, Thomas Brox).\n\nI found a few implementations with slight differences in downsampling and upsampling layers.\n\n1) Unet1: [Kaggle Notebook](https://www.kaggle.com/keegil/keras-u-net-starter-lb-0-277) by @keegil. That notebook has some useful background. \n\n2) Unet2: [Github site](https://github.com/jocicmarko/ultrasound-nerve-segmentation/blob/master/train.py) by jocicmarko, built for the ultrasound-nerve-segmentation competition\n\n3) Unet3: [The Magician's Corner repository](https://github.com/RSNA/MagiciansCorner/blob/master/UNetWithTensorflow.ipynb), maintained by [Dr. Bradley Erickson](https://github.com/slowvak). \n\nI modified slightly all three, so that they can run both in CPU and TPUs. They seem to work, but I could note get (2) to produce good results. The main difference between (1) and (3) is that (3) uses normalization layers, which make a huge difference. I also noticed that (3) is much faster, because it that fewer layers. I am showing all three here as a curiosity, and to encourage discussion (feel free to comment to this notebook)."},{"metadata":{},"cell_type":"markdown","source":"# Loss Function\nI use the dice_coefficient for loss function, the exact same function was used in (2) and (3). \n\nAlso notice the LayerNormalization here, which is a subclass of keras.engine.topology.Layer. \n\nLayerNormalization is the original one used in (3)\n\nLayerNormalization2 is the slight modification I made using tensorflow.math functions so that it can run on a TPU."},{"metadata":{"trusted":true},"cell_type":"code","source":"def dice_coeff(y_true, y_pred):\n    # add epsilon to avoid a divide by 0 error in case a slice has no pixels set\n    # we only care about relative value, not absolute so this alteration doesn't matter\n    _epsilon = 10 ** -7\n    intersections = tf.reduce_sum(y_true * y_pred)\n    unions = tf.reduce_sum(y_true + y_pred)\n    dice_scores = (2.0 * intersections + _epsilon) / (unions + _epsilon)\n    return dice_scores\n\ndef dice_loss(y_true, y_pred):\n    loss = 1 - dice_coeff(y_true, y_pred)\n    return loss\n  \nget_custom_objects().update({\"dice\": dice_loss})\n\nclass LayerNormalization (Layer) :\n    \n    def call(self, x, mask=None, training=None) :\n        axis = list (range (1, len (x.shape)))\n        x /= K.std (x, axis = axis, keepdims = True) + K.epsilon()\n        x -= K.mean (x, axis = axis, keepdims = True)\n        return x\n        \n    def compute_output_shape(self, input_shape):\n        return input_shape\n    \nclass LayerNormalization2 (Layer) :\n    \n    def call(self, x, mask=None, training=None) :\n        axis = list (range (1, len (x.shape)))\n        _epsilon = 10 ** -7\n        x /= tf.math.reduce_std (x, axis = axis, keepdims = True) + _epsilon\n        x -= tf.math.reduce_mean (x, axis = axis, keepdims = True)\n        return x\n        \n    def compute_output_shape(self, input_shape):\n        return input_shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def build_unet1( img_height, img_width, img_channels):\n    # Build U-Net model\n    inputs = Input((img_height, img_width, img_channels))\n    #s = Lambda(lambda x: x / 255) (inputs)\n    \n    #c1 = Conv2D(16, (3, 3), activation='elu', kernel_initializer='he_normal', padding='same') (s)\n    \n    c1 = Conv2D(16, (3, 3), activation='linear', kernel_initializer='he_normal', padding='same') (inputs)\n    c1 = Dropout(0.1) (c1)\n    c1 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c1)\n    p1 = MaxPooling2D((2, 2)) (c1)\n\n    c2 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (p1)\n    c2 = Dropout(0.1) (c2)\n    c2 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c2)\n    p2 = MaxPooling2D((2, 2)) (c2)\n\n    c3 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (p2)\n    c3 = Dropout(0.2) (c3)\n    c3 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c3)\n    p3 = MaxPooling2D((2, 2)) (c3)\n\n    c4 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (p3)\n    c4 = Dropout(0.2) (c4)\n    c4 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c4)\n    p4 = MaxPooling2D(pool_size=(2, 2)) (c4)\n\n    c5 = Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (p4)\n    c5 = Dropout(0.3) (c5)\n    c5 = Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c5)\n\n    u6 = Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same') (c5)\n    u6 = concatenate([u6, c4])\n    c6 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (u6)\n    c6 = Dropout(0.2) (c6)\n    c6 = Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c6)\n\n    u7 = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same') (c6)\n    u7 = concatenate([u7, c3])\n    c7 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (u7)\n    c7 = Dropout(0.2) (c7)\n    c7 = Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c7)\n\n    u8 = Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same') (c7)\n    u8 = concatenate([u8, c2])\n    c8 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (u8)\n    c8 = Dropout(0.1) (c8)\n    c8 = Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c8)\n\n    u9 = Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same') (c8)\n    u9 = concatenate([u9, c1], axis=3)\n    c9 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (u9)\n    c9 = Dropout(0.1) (c9)\n    c9 = Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same') (c9)\n\n    outputs = Conv2D(1, (1, 1), activation='sigmoid') (c9)\n\n    model = Model(inputs=[inputs], outputs=[outputs])\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def build_unet2(img_rows, img_cols, img_channels):\n    #inputs = Input((img_rows, img_cols, img_channels))\n    inputs = Input((512, 512, 3))\n    act_fn = 'relu'\n    \n    #conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)\n    conv1 = Conv2D(32, (3, 3), activation='linear', padding='same')(inputs)\n    conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n\n    conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1)\n    #conv2 = Conv2D(64, (3, 3), activation='linear', padding='same')(pool1)\n    conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv2)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n\n    conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool2)\n    #conv3 = Conv2D(128, (3, 3), activation='linear', padding='same')(pool2)\n    conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n\n    conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(pool3)\n    #conv4 = Conv2D(256, (3, 3), activation='linear', padding='same')(pool3)\n    conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv4)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(conv4)\n\n    conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool4)\n    #conv5 = Conv2D(512, (3, 3), activation='linear', padding='same')(pool4)\n    conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv5)\n\n    up6 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(conv5), conv4], axis=3)\n    conv6 = Conv2D(256, (3, 3), activation=act_fn, padding='same')(up6)\n    conv6 = Conv2D(256, (3, 3), activation=act_fn, padding='same')(conv6)\n\n    up7 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(conv6), conv3], axis=3)\n    conv7 = Conv2D(128, (3, 3), activation=act_fn, padding='same')(up7)\n    conv7 = Conv2D(128, (3, 3), activation=act_fn, padding='same')(conv7)\n\n    up8 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(conv7), conv2], axis=3)\n    conv8 = Conv2D(64, (3, 3), activation=act_fn, padding='same')(up8)\n    conv8 = Conv2D(64, (3, 3), activation=act_fn, padding='same')(conv8)\n\n    up9 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv8), conv1], axis=3)\n    conv9 = Conv2D(32, (3, 3), activation=act_fn, padding='same')(up9)\n    conv9 = Conv2D(32, (3, 3), activation=act_fn, padding='same')(conv9)\n\n    conv10 = Conv2D(1, (1, 1), activation='sigmoid')(conv9)\n\n    model = Model(inputs=[inputs], outputs=[conv10])\n    #model.compile(optimizer = Adam(lr = 1e-4), loss = 'dice', metrics=[dice_coeff])\n    \n    return model\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def build_unet3(act_fn = 'relu', init_fn = 'he_normal', width=512, height = 512, channels = 3): \n    inputs = Input((512,512,3))\n    act_fn = 'relu'\n    init_fn = 'he_normal'\n\n    # note we use linear function before layer normalization\n    conv1 = Conv2D(8, 5, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(inputs)\n    conv1 = LayerNormalization()(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2 = Conv2D(16, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(pool1)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(32, 3, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(pool2)\n    conv3 = LayerNormalization()(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n    conv4 = Conv2D(64, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(pool3)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(conv4)\n\n    conv5 = Conv2D(72, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(pool4)\n\n    up6 = Conv2D(64, 2, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv5))\n    up6 = LayerNormalization()(up6)\n    merge6 = concatenate([conv4,up6], axis = 3)\n    conv6 = Conv2D(64, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge6)\n\n    up7 = Conv2D(32, 2, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv6))\n    merge7 = concatenate([conv3,up7], axis = 3)\n    conv7 = Conv2D(32, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge7)\n\n    up8 = Conv2D(16, 2, activation = 'linear', padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv7))\n    up8 = LayerNormalization()(up8)\n    merge8 = concatenate([conv2,up8], axis = 3)\n    conv8 = Conv2D(16, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge8)\n\n    up9 = Conv2D(8, 2, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(UpSampling2D(size = (2,2))(conv8))\n    merge9 = concatenate([conv1,up9], axis = 3)\n    conv9 = Conv2D(8, 3, activation = act_fn, padding = 'same', kernel_initializer = init_fn)(merge9)\n    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)\n    model = Model(inputs = inputs, outputs = conv10)\n\n    return model\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Choose the model you want to try by moving the comments below. I am using (3) going forward"},{"metadata":{"trusted":true},"cell_type":"code","source":"#unet_model = build_unet1( 512, 512, 3)\n#unet_model = get_unet2(512,512,3)\nunet_model = build_unet3(512,512,3)\n\nunet_model.compile(optimizer = Adam(lr = 1e-4), loss = 'dice', metrics=[dice_coeff])\nunet_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tf.keras.utils.plot_model(unet_model, show_shapes=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build train and validation datasets\nFor testing purposes, I am using a very small dataset, only 10 images for test and 5 for validation. Notice that I first build file lists that have the tiles we want. Then we create tf.data.datasets using these lists. It is that simple! "},{"metadata":{"trusted":true},"cell_type":"code","source":"small_train = gloms_df[0:10]['gcs_path']\nsmall_test = gloms_df[10:15]['gcs_path']\n\nsmall_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dataset = read_images_and_masks(small_train)\ntest_dataset = read_images_and_masks(small_test)\n\nfor image, mask in train_dataset.take(1):\n    sample_image, sample_mask = image, mask\n\nfig, ax = plt.subplots(1,2,figsize=(20,3))\nax[0].imshow(sample_image[0,:,:,:])\nax[1].imshow(sample_mask[0,:,:])\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#Using keras model.fit\nNow we can use model.fit to train a model."},{"metadata":{"trusted":true},"cell_type":"code","source":"earlystopper = EarlyStopping(patience=5, verbose=1)\ncheckpointer = ModelCheckpoint('/kaggle/working/model-hubmap.h5', verbose=1)\n# run 1 epoch\nresults = unet_model.fit(train_dataset, batch_size=1, epochs=1, callbacks=[checkpointer])\n#results = unet_model.fit(train_dataset, batch_size=1, epochs=5, validation_data=test_dataset,callbacks=[checkpointer])\n\n#results = model.fit(X_train, Y_train, validation_split=0.1, batch_size=16, epochs=50, \n#                    callbacks=[earlystopper, checkpointer])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls /kaggle/working","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"small_test = gloms_df[1000:1005]['gcs_path']\nsmall_dataset = read_images_and_masks(small_test)\n\nfor image, mask in test_dataset.take(1):\n    test_image, test_mask = image, mask\ntest_image.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(test_image[0,:,:,:])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"unet_model.load_weights(\"/kaggle/working/model-hubmap.h5\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#reshaped = tf.reshape(test_image,(1,512,512,3))\npred_mask = unet_model.predict(test_image, verbose=1)\npred_mask.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's have a look at the mask returned"},{"metadata":{"trusted":true},"cell_type":"code","source":"pred_mask[0,:,:,0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(pred_mask[0,:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"By looking at the numbers, we see that 0.45 is an interesting threshold to produce a boolean mask. "},{"metadata":{"trusted":true},"cell_type":"code","source":"#bool_mask = pred_mask[0,:,:,0] > 0.9\nbool_mask = pred_mask[0,:,:,0] > 0.45\nplt.imshow(bool_mask)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is pretty good for a single epoch with only 10 images!!! But we tried with an image from the train set, so it seems to be overfitting already. Let's try with a image from the test set:"},{"metadata":{"trusted":true},"cell_type":"code","source":"#unet_model.load_weights(\"/kaggle/working/model-hubmap.h5\")\n\nsmall_test = gloms_df[1000:1005]['gcs_path']\nsmall_dataset = read_images_and_masks(small_test)\n\ntest_image = []\ntest_mask = []\npred_mask = []\nfor image, mask in small_dataset.take(1):\n    test_image, test_mask = image, mask\n    pred_mask = unet_model.predict(test_image, verbose=1)\n    pred_mask = pred_mask[0,:,:,0] > 0.5\n    #pred_mask = (pred_mask > 0.3).astype(np.uint8)\n    #pred_mask = (pred_mask > 0.702)\n    \nfig, ax = plt.subplots(1,3,figsize=(20,3))\nax[0].imshow(test_image[0,:,:,:])\nax[1].imshow(test_mask[0,:,:])\nax[2].imshow(pred_mask)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mask_density = np.count_nonzero(pred_mask)\nmask_density","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It does pick up the tissue inside the glom, but also some outside -- but again we only trained with 10 images. But it is a good start. In my next notebooks we will scale up the training to thousands of images using TPUs. "}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}