{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"collapsed":true},"cell_type":"markdown","source":"# Foreword\n\nHello,\n\nIn this kernel, I would be using the UNet architecture to classify image pixels as belongong to a particular class (ship in this competitions context) or background. UNet is a very popular image segmentation archutecture initially designed for biomedical image processing but later adapted by practitionars from other fields as well. \n\nLink to the original UNet paper: https://arxiv.org/pdf/1505.04597.pdf\n\nI would be using Keras, a very easy to use high level deep learning library built on top of Tenforflow and Theano. Keras is especially well suited for beginners who are acquainted with the basics of neural networks and want to try different neural network archtectures without gong into too much details about the model.\n\nI will try and explain small details into why I am doing what I am doing. I hope some of you will be able to follow the tutorial and learn something substantial, just like I did, from other awesome Kaggle Kernels. Please remember, this is a very basic model and is not designed/optimized to perform well in the competition. So feel free to use this kernel to come up with more interesting ideas/architectures. \n\nIf you have any questions, comment and let me know. I will try to clear up any doubts to the best of my knowledge and capability.\nThanks,\nKrishanu"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"## Importing all the basic python libraries"},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"dcc17e89371c598e57ddcd8d7e88f24a488064e7"},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"afe3ca8845d1d0a50f014ee2b68cb063eadc3dd5"},"cell_type":"code","source":"print(os.listdir(\"../input\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1cdf4b9747b0332a6eacdbb7f1c47b9e7ac7684f"},"cell_type":"markdown","source":"## Reading in the Images"},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"9abc21e86c1a16a79967d6140ac6d21fdd4dcf97"},"cell_type":"code","source":"ship_dir = '../input'\ntrain_image_dir = os.path.join(ship_dir, 'train')\ntest_image_dir = os.path.join(ship_dir, 'test')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"22ebe26a5fdb882fa796da55c67521316ee5dac0"},"cell_type":"markdown","source":"### Deciphering the Run Length Encoding\nThe technique has been taken from the following kernel: https://www.kaggle.com/paulorzp/run-length-encode-and-decode"},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"4dca44992d7671c291d26c81e76aea927e9b0672"},"cell_type":"code","source":"def multi_rle_encode(img):\n    labels = label(img[:, :, 0])\n    return [rle_encode(labels==k) for k in np.unique(labels[labels>0])]\n\ndef rle_encode(img):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    '''\n    pixels = img.T.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\ndef rle_decode(mask_rle, shape=(768, 768)):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height,width) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape).T  # Needed to align to RLE direction\n\ndef masks_as_image(in_mask_list, all_masks=None):\n    # Take the individual ship masks and create a single mask array for all ships\n    if all_masks is None:\n        all_masks = np.zeros((768, 768), dtype = np.int16)\n    #if isinstance(in_mask_list, list):\n    for mask in in_mask_list:\n        if isinstance(mask, str):\n            all_masks += rle_decode(mask)\n    return np.expand_dims(all_masks, -1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"375e951f55c1ce2102de5f1a4243cbba927316e6"},"cell_type":"markdown","source":"## We take a look at the masks csv file, and read their summary information"},{"metadata":{"trusted":true,"_uuid":"2bac08982e16a1df60ddd9156ddb78f4ea065ae9"},"cell_type":"code","source":"masks = pd.read_csv(os.path.join('../input/', 'train_ship_segmentations.csv'))\nprint(masks.shape[0], ' masks found')\nprint(masks['ImageId'].value_counts().shape[0], ' images found')\nmasks.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"70e33c087f1b064faa894d337f074ba5dd32d274"},"cell_type":"markdown","source":"## Reading in the training images\n\nReading all the training images in one go is not possible with the computation power Kaggle has to offer. However, we can smartly overcome this problem by partially reading in some of the images temporarily, training the neural network on the random set of images and then again deleting the set and reading in a new set and continuing this."},{"metadata":{"trusted":true,"_uuid":"b005756b8391fba0d6149a906a5587e1c9469916"},"cell_type":"code","source":"# First need to import some libararies\nimport skimage\nfrom skimage.io import imread\n\ntrain_images = os.listdir(train_image_dir)\ntrain_temp = np.random.choice(train_images, 2000) # We choose 2000 random images every time\n\ntrain_temp_img = []\ntrain_temp_mask = []\n\nfor img in train_temp:\n    train_temp_img.append(imread(train_image_dir + '/' + img).astype('uint8'))\n    train_temp_mask.append(masks_as_image(masks.query('ImageId==\"'+img+'\"')['EncodedPixels']))\n    \ntrain_temp_img = np.array(train_temp_img)\ntrain_temp_mask = np.array(train_temp_mask)\n\nprint(train_temp_img.shape, train_temp_mask.shape)\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"2c6aaa0bc24651b82eb78d85e39850decfa9d1ab"},"cell_type":"markdown","source":"## So we have just read 2000 images from the training set. Now we will have a look at some random images"},{"metadata":{"trusted":true,"_uuid":"7b9b1bcf47187d263e261406fce522b84c20969e"},"cell_type":"code","source":"random = np.random.choice(2000, 1)\n\nfig, axes = plt.subplots(1, 2, figsize=(20, 10))\naxes[0].imshow(train_temp_img[random[0]])\naxes[1].imshow(train_temp_mask[random[0]][:, :, 0])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"41850afa2a7c1cc5d311093a46bbbe7329e5e6ef","scrolled":false},"cell_type":"code","source":"fg, axes = plt.subplots(5, 2, figsize=(15, 50))\n\nrandoms = np.random.choice(range(2000), 5)\n\naxes[0, 0].imshow(train_temp_img[randoms[0]])\naxes[0, 1].imshow(train_temp_mask[randoms[0]][:, :, 0])\naxes[1, 0].imshow(train_temp_img[randoms[1]])\naxes[1, 1].imshow(train_temp_mask[randoms[1]][:, :, 0])\naxes[2, 0].imshow(train_temp_img[randoms[2]])\naxes[2, 1].imshow(train_temp_mask[randoms[2]][:, :, 0])\naxes[3, 0].imshow(train_temp_img[randoms[3]])\naxes[3, 1].imshow(train_temp_mask[randoms[3]][:, :, 0])\naxes[4, 0].imshow(train_temp_img[randoms[4]])\naxes[4, 1].imshow(train_temp_mask[randoms[4]][:, :, 0])\n\nplt.show()\n        ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"f9a70960b395640c7e5c850d71085d413d53f455"},"cell_type":"markdown","source":"# Model Construction"},{"metadata":{"_uuid":"c2a3c51dbba70cafdee5d3b333275ee5a03ebfde"},"cell_type":"markdown","source":"### First we will construct the loss functions. We will construct the loss function according to the Keras documentation\n\nThe 2 losses we will try out are the Jaccard Distance Loss and the Dice Coefficient Loss"},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"be576d97674f8a948c95fecb397bee2398033d72"},"cell_type":"code","source":"def jaccard_distance_loss(y_true, y_pred, smooth=100):\n    intersection = K.sum(K.abs(y_true * y_pred), axis=-1)\n    sum_ = K.sum(K.abs(y_true) + K.abs(y_pred), axis=-1)\n    jac = (intersection + smooth) / (sum_ - intersection + smooth)\n    loss = (1 - jac) * smooth\n#     print(loss.shape)\n#     print(loss)\n    return loss\n\ndef dice_coef(y_true, y_pred, smooth=1):\n    intersection = K.sum(K.abs(y_true * y_pred), axis=-1)\n    return (2. * intersection + smooth) / (K.sum(K.square(y_true),-1) + K.sum(K.square(y_pred),-1) + smooth)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cfcca62fa7a60e2ada3a4d52475a3d919dc6733a"},"cell_type":"markdown","source":"### Model Constructor Functions\nNow the model construction functions will be implemented. Since there are several units that are repeated, we will write functions that help us in reducing repetitive coding"},{"metadata":{"trusted":true,"_uuid":"2168be094089bf664a9e2eabcd7b14ca1900d1ab"},"cell_type":"code","source":"from keras import regularizers\nfrom keras.layers import BatchNormalization as BatchNorm\nfrom keras.models import Model, load_model\nfrom keras.layers import Input, Reshape, Activation\nfrom keras.layers.core import Lambda\nfrom keras.layers.convolutional import Conv2D, Conv2DTranspose, UpSampling2D\nfrom keras.layers.pooling import MaxPooling2D\nfrom keras.layers.merge import Concatenate\nfrom keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\nfrom keras import backend as K\nfrom keras import layers","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"8b47a0396987308cd0df11b1923e2f7e53b7bcf4"},"cell_type":"code","source":"def downsampling(x, level, filters, kernel_size, num_convs, conv_strides=1, activation = 'relu', batch_norm = False, pool_size = 2, pool_strides = 2, regularizer = None, regularizer_param = 0.001):\n    if regularizer is not None:\n        if regularizer == 'l2':\n            reg = regularizers.l2(regularizer_param)\n        elif regularizer == 'l1':\n            reg = regularizers.l1(regularizer_param)\n    else:\n        reg = None\n    for i in range(num_convs):\n        x = Conv2D(filters=filters, kernel_size=kernel_size, strides=conv_strides, padding='same', kernel_regularizer=reg, bias_regularizer=reg, name = 'downsampling_' + str(level) + '_conv_' + str(i))(x)\n        if batch_norm:\n            x = BatchNorm(name = 'downsampling_' + str(level) + '_batchnorm_' + str(i))(x)\n        x = Activation(activation, name = 'downsampling_' + str(level) + '_activation_' + str(i))(x)\n    skip = x\n    x = MaxPooling2D(pool_size=2, strides=2)(x)\n    return x, skip\n\ndef bottleneck_dilated(x, filters, kernel_size, num_convs = 6, activation = 'relu', batch_norm = False, last_activation = False, regularizer = None, regularizer_param = 0.001):\n#     assert num_convs == len(conv_strides)\n    if regularizer is not None:\n        if regularizer == 'l2':\n            reg = regularizers.l2(regularizer_param)\n        elif regularizer == 'l1':\n            reg = regularizers.l1(regularizer_param)\n    else:\n        reg = None\n    skips = []\n    for i in range(num_convs):\n        x = Conv2D(filters=filters, kernel_size=kernel_size, strides=1, dilation_rate = 2 ** i, activation='relu', padding='same', kernel_regularizer=reg, bias_regularizer=reg, name = 'bottleneck_skip_' + str(i))(x)\n        skips.append(x)\n    x = layers.add(skips)\n    if last_activation:\n        x = Activation('relu')(x)\n    return x\n    \ndef bottleneck(x, filters, kernel_size, num_convs, conv_strides=1, activation = 'relu', batch_norm = False, pool_size = 2, pool_strides = 2, regularizer = None, regularizer_param = 0.001):\n    if regularizer is not None:\n        if regularizer == 'l2':\n            reg = regularizers.l2(regularizer_param)\n        elif regularizer == 'l1':\n            reg = regularizers.l1(regularizer_param)\n    else:\n        reg = None\n    for i in range(num_convs):\n        x = Conv2D(filters=filters, kernel_size=3, strides=1, padding='same', kernel_regularizer=reg, bias_regularizer=reg, name = 'bottleneck_' + str(i))(x)\n        if batch_norm:\n            x = BatchNorm()(x)\n        x = Activation(activation)(x)\n    return x\n\ndef upsampling(x, level, skip, filters, kernel_size, num_convs, conv_strides=1, activation = 'relu', batch_norm = False, conv_transpose = True, upsampling_size = 2, upsampling_strides = 2, regularizer = None, regularizer_param = 0.001):\n    if regularizer is not None:\n        if regularizer == 'l2':\n            reg = regularizers.l2(regularizer_param)\n        elif regularizer == 'l1':\n            reg = regularizers.l1(regularizer_param)\n    else:\n        reg = None\n    if conv_transpose:\n        x = Conv2DTranspose(filters=filters, kernel_size = upsampling_size, strides=upsampling_strides, name = 'upsampling_' + str(level) + '_conv_trans_' + str(level))(x)\n    else:\n        x = UpSampling2D((upsampling_size), name = 'upsampling_' + str(level) + '_ups_' + str(i))(x)\n    x = Concatenate()([x, skip])\n    for i in range(num_convs):\n        x = Conv2D(filters=filters, kernel_size=3, strides=1, padding='same', kernel_regularizer=reg, bias_regularizer=reg, name = 'upsampling_' + str(level) + '_conv_' + str(i))(x)\n        if batch_norm:\n            x = BatchNorm(name = 'upsampling_' + str(level) + '_batchnorm_' + str(i))(x)\n        x = Activation(activation, name = 'upsampling_' + str(level) + '_activation_' + str(i))(x)\n    return x\n\ndef model_simple_unet_initializer(num_classes, num_levels, num_layers = 2, num_bottleneck = 2, filter_size_start = 16, batch_norm = False, kernel_size = 3, bottleneck_dilation = False, bottleneck_sum_activation = False, regularizer = None, regularizer_param = 0.001):\n    inputs = Input((img_shape))\n    x = inputs\n    skips = []\n    for i in range(num_levels):\n        x, skip = downsampling(x, i, filter_size_start * (2 ** i), kernel_size, num_layers, batch_norm=True, regularizer= regularizer, regularizer_param=regularizer_param)\n        skips.append(skip)\n    if bottleneck_dilation:\n        x = bottleneck_dilated(x, filter_size_start * (2 ** num_levels), kernel_size, num_bottleneck, batch_norm=True, last_activation=bottleneck_sum_activation, regularizer= regularizer, regularizer_param=regularizer_param)\n    else:\n        x = bottleneck(x, filter_size_start * (2 ** num_levels), kernel_size, num_bottleneck, batch_norm=True, regularizer=regularizer, regularizer_param=regularizer_param)\n    for j in range(num_levels):\n        x = upsampling(x, j, skips[num_levels - j - 1], filter_size_start * (2 ** (num_levels - j - 1)), kernel_size, num_layers, batch_norm=True, regularizer= regularizer, regularizer_param=regularizer_param)\n    outputs = Conv2D(filters=num_classes, kernel_size=1, strides=1, padding='same', activation='softmax', name = 'output_softmax')(x)\n    \n    model = Model(inputs = inputs, outputs = outputs)\n    model.compile(optimizer='adam', loss=jaccard_distance_loss, metrics=[dice_coef])\n    model.summary()\n    return model\n\ndef model_train(model, x, y, epochs, num_test, early_stopper, patience_lr, model_name):\n    num_data = x.shape[0]\n    num_train = num_data - num_test\n    early_stopper = EarlyStopping(patience=early_stopper, verbose=1)\n    reduce_learning_rate = ReduceLROnPlateau(monitor='loss', factor = 0.75, patience = patience_lr, verbose=1)\n    checkpointer = ModelCheckpoint(model_name + '.h5', verbose=1, save_best_only=True)\n    checkpointer_train = ModelCheckpoint(model_name + 'best_train.h5', monitor='loss', verbose=1, save_best_only=True)\n    results = model.fit(x[0:num_train], y[0:num_train], validation_data=(x[num_train:], y[num_train:]), batch_size=5, epochs=epochs, callbacks=[early_stopper, checkpointer, checkpointer_train, reduce_learning_rate])\n    return model, results","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"caaf9bde2f062cddbaa3f88e998c86a1012aa776"},"cell_type":"code","source":"img_height = 768\nimg_width = 768\nnum_channels = 3\nimg_shape = (img_height, img_width, num_channels)\nnum_classes = 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7a31dfdfc85bc2f29497c901fa07a570ee600bd9"},"cell_type":"code","source":"model1 = model_simple_unet_initializer(1, 4, 2, 5, 16, True, 3, True, False, 'l2', 0.001)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bd499b1367e7ac1f4af2afa38c8788d8de69f2d0"},"cell_type":"code","source":"model1, results1 = model_train(model1, train_temp_img, train_temp_mask, 30, 50, 8, 6, 'model1')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"c842349bb478e501b52dc93824c4eb54d7df617a"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}