{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# imports\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# import basic libraries\nimport os\nfrom glob import glob\n\n# import plotting\nfrom matplotlib import pyplot as plt\nimport matplotlib.patches as patches\nimport matplotlib\nimport seaborn as sns\n\n# import image manipulation\nfrom PIL import Image\nimport imageio","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"! pip install --upgrade imgaug","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# import data augmentation\nimport imgaug as ia\nfrom imgaug import augmenters as iaa\n# import segmentation maps from imgaug\nfrom imgaug.augmentables.segmaps import SegmentationMapOnImage\nimport imgaug.imgaug","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![image](https://github.com/Lexie88rus/understanding_clouds/raw/master/assets/bay-beach-beautiful-2876737.jpg)"},{"metadata":{},"cell_type":"markdown","source":"Photo by [Vincent Rivaud from Pexels](https://www.pexels.com/@vince?utm_content=attributionCopyText&utm_medium=referral&utm_source=pexels)"},{"metadata":{},"cell_type":"markdown","source":"# Understanding Clouds EDA\n*Comprehensive overview of the [competition](https://www.kaggle.com/c/understanding_cloud_organization/data) data*"},{"metadata":{},"cell_type":"markdown","source":"## About the Competition"},{"metadata":{},"cell_type":"markdown","source":"The challenge is to __segment satellite images into one of four classes: Sugar, Flower, Fish and Gravel.__ These clouds look benign compared to big thunderstorms but, in fact, for the Earth’s climate they play a huge role. The reason is that they reflect a lot of sunlight back into space, thereby cooling our planet, while only contributing marginally to the greenhouse effect. This means that it’s really important to figure out how these clouds will change as our planet warms."},{"metadata":{},"cell_type":"markdown","source":"## Load Data"},{"metadata":{},"cell_type":"markdown","source":"Firts, let's define the paths to train and test images and load the dataframe with train images:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# set paths to train and test image datasets\nTRAIN_PATH = '../input/understanding_cloud_organization/train_images/'\nTEST_PATH = '../input/understanding_cloud_organization/test_images/'\n\n# load dataframe with train labels\ntrain_df = pd.read_csv('../input/understanding_cloud_organization/train.csv')\ntrain_fns = sorted(glob(TRAIN_PATH + '*.jpg'))\n\nprint('There are {} images in the train set.'.format(len(train_df)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's load explore the test set a little:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the filenames for test images\ntest_fns = sorted(glob(TEST_PATH + '*.jpg'))\n\nprint('There are {} images in the test set.'.format(len(test_fns)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plot the pie chart for the train and test datasets:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# plotting a pie chart which demonstrates train and test sets\nlabels = 'Train', 'Test'\nsizes = [len(train_fns), len(test_fns)]\nexplode = (0, 0.1)\n\nfig, ax = plt.subplots(figsize=(6, 6))\nax.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)\nax.axis('equal')\nax.set_title('Train and Test Sets')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Explore Labels from Train Set"},{"metadata":{},"cell_type":"markdown","source":"Look how the dataframe with train labels looks like:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that:\n* For each image from the training dataset there are __4 lines for each type of clouds__.\n* `Image_Label` is a __contatenation of the image filename and a cloud type__.\n* If a certain type of clouds in present on the image, the `EncodedPixels` column is non-null and contains the __segmentation map for the corresponding cloud type__."},{"metadata":{},"cell_type":"markdown","source":"`1.` Explore null values:\n\nNow let's see how many null values are there:"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('There are {} rows with empty segmentation maps.'.format(len(train_df) - train_df.EncodedPixels.count()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# plotting a pie chart\nlabels = 'Non-empty', 'Empty'\nsizes = [train_df.EncodedPixels.count(), len(train_df) - train_df.EncodedPixels.count()]\nexplode = (0, 0.1)\n\nfig, ax = plt.subplots(figsize=(6, 6))\nax.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)\nax.axis('equal')\nax.set_title('Non-empty and Empty Masks')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looks like almost __half of the lines is empty__."},{"metadata":{},"cell_type":"markdown","source":"`2.` Explore the labels:"},{"metadata":{},"cell_type":"markdown","source":"Let's split the `Image_Label` into two columns and analyze the labels:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# split column\nsplit_df = train_df[\"Image_Label\"].str.split(\"_\", n = 1, expand = True)\n# add new columns to train_df\ntrain_df['Image'] = split_df[0]\ntrain_df['Label'] = split_df[1]\n\n# check the result\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can count the number of labels of each cloud type:"},{"metadata":{"trusted":true},"cell_type":"code","source":"fish = train_df[train_df['Label'] == 'Fish'].EncodedPixels.count()\nflower = train_df[train_df['Label'] == 'Flower'].EncodedPixels.count()\ngravel = train_df[train_df['Label'] == 'Gravel'].EncodedPixels.count()\nsugar = train_df[train_df['Label'] == 'Sugar'].EncodedPixels.count()\n\nprint('There are {} fish clouds'.format(fish))\nprint('There are {} flower clouds'.format(flower))\nprint('There are {} gravel clouds'.format(gravel))\nprint('There are {} sugar clouds'.format(sugar))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# plotting a pie chart\nlabels = 'Fish', 'Flower', 'Gravel', 'Sugar'\nsizes = [fish, flower, gravel, sugar]\n\nfig, ax = plt.subplots(figsize=(6, 6))\nax.pie(sizes, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)\nax.axis('equal')\nax.set_title('Cloud Types')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that at least the dataset is somewhat __balanced__, which is great and makes are task way more easier."},{"metadata":{},"cell_type":"markdown","source":"`3.` Explore the number of labels per image:"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_per_image = train_df.groupby('Image')['EncodedPixels'].count()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('The mean number of labels per image is {}'.format(labels_per_image.mean()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(6, 6))\nax.hist(labels_per_image)\nax.set_title('Number of Labels per Image')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__So most of the images have 2 labels.__"},{"metadata":{},"cell_type":"markdown","source":"`4.` Explore the correlation between different cloud types.\n\nUsing the dataframe with labels, we can try to find the correlation between different types of clouds."},{"metadata":{"trusted":true},"cell_type":"code","source":"# create dummy columns for each cloud type\ncorr_df = pd.get_dummies(train_df, columns = ['Label'])\n# fill null values with '-1'\ncorr_df = corr_df.fillna('-1')\n\n# define a helper function to fill dummy columns\ndef get_dummy_value(row, cloud_type):\n    ''' Get value for dummy column '''\n    if cloud_type == 'fish':\n        return row['Label_Fish'] * (row['EncodedPixels'] != '-1')\n    if cloud_type == 'flower':\n        return row['Label_Flower'] * (row['EncodedPixels'] != '-1')\n    if cloud_type == 'gravel':\n        return row['Label_Gravel'] * (row['EncodedPixels'] != '-1')\n    if cloud_type == 'sugar':\n        return row['Label_Sugar'] * (row['EncodedPixels'] != '-1')\n    \n# fill dummy columns\ncorr_df['Label_Fish'] = corr_df.apply(lambda row: get_dummy_value(row, 'fish'), axis=1)\ncorr_df['Label_Flower'] = corr_df.apply(lambda row: get_dummy_value(row, 'flower'), axis=1)\ncorr_df['Label_Gravel'] = corr_df.apply(lambda row: get_dummy_value(row, 'gravel'), axis=1)\ncorr_df['Label_Sugar'] = corr_df.apply(lambda row: get_dummy_value(row, 'sugar'), axis=1)\n\n# check the result\ncorr_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# group by the image\ncorr_df = corr_df.groupby('Image')['Label_Fish', 'Label_Flower', 'Label_Gravel', 'Label_Sugar'].max()\ncorr_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can explore the correlation between `Label_Fish, Label_Flower, Label_Gravel, Label_Sugar` columns:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#Find out correlation between columns and plot\ncorrs = np.corrcoef(corr_df.values.T)\nsns.set(font_scale=1)\nsns.set(rc={'figure.figsize':(7,7)})\nhm=sns.heatmap(corrs, cbar = True, annot=True, square = True, fmt = '.2f',\n              yticklabels = ['Fish', 'Flower', 'Gravel', 'Sugar'], \n               xticklabels = ['Fish', 'Flower', 'Gravel', 'Sugar']).set_title('Cloud type correlation heatmap')\n\nfig = hm.get_figure()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can observe, there is __no strong correlation between the types of the clouds__ on one image (all the correlation coefficients are close to zero)."},{"metadata":{},"cell_type":"markdown","source":"## Explore the Images\n\nHere goes the most exciting part of the EDA: exploring the images themselves."},{"metadata":{},"cell_type":"markdown","source":"`1.` Explore image sizes:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_image_sizes(train = True):\n    '''\n    Function to get sizes of images from test and train sets.\n    INPUT:\n        train - indicates whether we are getting sizes of images from train or test set\n    '''\n    if train:\n        path = TRAIN_PATH\n    else:\n        path = TEST_PATH\n        \n    widths = []\n    heights = []\n    \n    images = sorted(glob(path + '*.jpg'))\n    \n    max_im = Image.open(images[0])\n    min_im = Image.open(images[0])\n        \n    for im in range(0, len(images)):\n        image = Image.open(images[im])\n        width, height = image.size\n        \n        if len(widths) > 0:\n            if width > max(widths):\n                max_im = image\n\n            if width < min(widths):\n                min_im = image\n\n        widths.append(width)\n        heights.append(height)\n        \n    return widths, heights, max_im, min_im","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get sizes of images from test and train sets\ntrain_widths, train_heights, max_train, min_train = get_image_sizes(train = True)\ntest_widths, test_heights, max_test, min_test = get_image_sizes(train = False)\n\nprint('Maximum width for training set is {}'.format(max(train_widths)))\nprint('Minimum width for training set is {}'.format(min(train_widths)))\nprint('Maximum height for training set is {}'.format(max(train_heights)))\nprint('Minimum height for training set is {}'.format(min(train_heights)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Maximum width for test set is {}'.format(max(test_widths)))\nprint('Minimum width for test set is {}'.format(min(test_widths)))\nprint('Maximum height for test set is {}'.format(max(test_heights)))\nprint('Minimum height for test set is {}'.format(min(test_heights)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that __all images have the same size__. That's great!"},{"metadata":{},"cell_type":"markdown","source":"`2.` Plot sample images from training set:"},{"metadata":{},"cell_type":"markdown","source":"At first, I will prepare some helper functions for visualization:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# helper function to get a string of labels for the picture\ndef get_labels(image_id):\n    ''' Function to get the labels for the image by name'''\n    im_df = train_df[train_df['Image'] == image_id].fillna('-1')\n    im_df = im_df[im_df['EncodedPixels'] != '-1'].groupby('Label').count()\n    \n    index = im_df.index\n    all_labels = ['Fish', 'Flower', 'Gravel', 'Sugar']\n    \n    labels = ''\n    \n    for label in all_labels:\n        if label in index:\n            labels = labels + ' ' + label\n    \n    return labels\n\n# function to plot a grid of images and their labels\ndef plot_training_images(width = 5, height = 2):\n    \"\"\"\n    Function to plot grid with several examples of cloud images from train set.\n    INPUT:\n        width - number of images per row\n        height - number of rows\n\n    OUTPUT: None\n    \"\"\"\n    \n    # get a list of images from training set\n    images = sorted(glob(TRAIN_PATH + '*.jpg'))\n    \n    fig, axs = plt.subplots(height, width, figsize=(width * 3, height * 3))\n    \n    # create a list of random indices \n    rnd_indices = rnd_indices = [np.random.choice(range(0, len(images))) for i in range(height * width)]\n    \n    for im in range(0, height * width):\n        # open image with a random index\n        image = Image.open(images[rnd_indices[im]])\n        \n        i = im // width\n        j = im % width\n        \n        # plot the image\n        axs[i,j].imshow(image) #plot the data\n        axs[i,j].axis('off')\n        axs[i,j].set_title(get_labels(images[rnd_indices[im]].split('/')[-1]))\n\n    # set suptitle\n    plt.suptitle('Sample images from the train set')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's plot sample images:"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_training_images()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`3.` Visualize segmentation maps:"},{"metadata":{},"cell_type":"markdown","source":"I will use a function from [this great EDA kernel](https://www.kaggle.com/ekhtiar/eda-find-me-in-the-clouds). I upvoted it and encourage you to do so too."},{"metadata":{"trusted":true},"cell_type":"code","source":"def rle_to_mask(rle_string, width, height):\n    '''\n    convert RLE(run length encoding) string to numpy array\n\n    Parameters: \n    rle_string (str): string of rle encoded mask\n    height (int): height of the mask\n    width (int): width of the mask\n\n    Returns: \n    numpy.array: numpy array of the mask\n    '''\n    \n    rows, cols = height, width\n    \n    if rle_string == -1:\n        return np.zeros((height, width))\n    else:\n        rle_numbers = [int(num_string) for num_string in rle_string.split(' ')]\n        rle_pairs = np.array(rle_numbers).reshape(-1,2)\n        img = np.zeros(rows*cols, dtype=np.uint8)\n        for index, length in rle_pairs:\n            index -= 1\n            img[index:index+length] = 255\n        img = img.reshape(cols,rows)\n        img = img.T\n        return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I will use __[imgaug](https://imgaug.readthedocs.io/en/latest/index.html) library__ to visualize the segmentation maps. This library has special helpers for visualization and augmentation of images with segmentation maps. You will see how easy it is to work with segmentation maps with __imgaug__."},{"metadata":{"trusted":true},"cell_type":"code","source":"from __future__ import print_function\nimport numpy as np\n\ndef valid_imshow_data(data):\n    data = np.asarray(data)\n    if data.ndim == 2:\n        return True\n    elif data.ndim == 3:\n        if 3 <= data.shape[2] <= 4:\n            return True\n        else:\n            print('The \"data\" has 3 dimensions but the last dimension '\n                  'must have a length of 3 (RGB) or 4 (RGBA), not \"{}\".'\n                  ''.format(data.shape[2]))\n            return False\n    else:\n        print('To visualize an image the data must be 2 dimensional or '\n              '3 dimensional, not \"{}\".'\n              ''.format(data.ndim))\n        return False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_mask(line_id, shape = (2100, 1400)):\n    '''\n    Function to visualize the image and the mask.\n    INPUT:\n        line_id - id of the line to visualize the masks\n        shape - image shape\n    RETURNS:\n        np_mask - numpy segmentation map\n    '''\n    # replace null values with '-1'\n    im_df = train_df.fillna('-1')\n    \n    # convert rle to mask\n    rle = im_df.loc[line_id]['EncodedPixels']\n    if rle != '-1':\n        np_mask = rle_to_mask(rle, shape[0], shape[1])\n        np_mask = np.clip(np_mask, 0, 1)\n    else:\n        # empty mask\n        np_mask = np.zeros((shape[0],shape[1]), dtype=np.uint8)\n        \n    return np_mask\n\n# helper function to get segmentation mask for an image by filename\ndef get_mask_by_image_id(image_id, label):\n    '''\n    Function to visualize several segmentation maps.\n    INPUT:\n        image_id - filename of the image\n    RETURNS:\n        np_mask - numpy segmentation map\n    '''\n    im_df = train_df[train_df['Image'] == image_id.split('/')[-1]].fillna('-1')\n\n    image = np.asarray(Image.open(image_id))\n\n    rle = im_df[im_df['Label'] == label]['EncodedPixels'].values[0]\n    if rle != '-1':\n        np_mask = rle_to_mask(rle, np.asarray(image).shape[1], np.asarray(image).shape[0])\n        np_mask = np.clip(np_mask, 0, 1)\n    else:\n        # empty mask\n        np_mask = np.zeros((np.asarray(image).shape[0], np.asarray(image).shape[1]), dtype=np.uint8)\n        \n    return np_mask\n\ndef visualize_image_with_mask(line_id):\n    '''\n    Function to visualize the image and the mask.\n    INPUT:\n        line_id - id of the line to visualize the masks\n    '''\n    # replace null values with '-1'\n    im_df = train_df.fillna('-1')\n    \n    # get segmentation mask\n    np_mask = get_mask(line_id)\n    \n    # open the image\n    image = Image.open(TRAIN_PATH + im_df.loc[line_id]['Image'])\n\n    # create segmentation map\n    segmap = SegmentationMapOnImage(np_mask, np_mask.shape, nb_classes=2)\n    \n    # visualize the image and map\n    side_by_side = np.hstack([\n        segmap.draw_on_image(np.asarray(image))\n    ]).reshape(np.asarray(image).shape)\n\n    fig, ax = plt.subplots(figsize=(6, 4))\n    ax.axis('off')\n    plt.title(im_df.loc[line_id]['Label'])\n    \n    ax.imshow(side_by_side)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Visualize sample masks:"},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_image_with_mask(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_image_with_mask(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# empty mask:\nvisualize_image_with_mask(2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Visualize image grids:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def plot_training_images_and_masks(n_images = 3):\n    '''\n    Function to plot several random images with segmentation masks.\n    INPUT:\n        n_images - number of images to visualize\n    '''\n    \n    # get a list of images from training set\n    images = sorted(glob(TRAIN_PATH + '*.jpg'))\n    \n    fig, ax = plt.subplots(n_images, 4, figsize=(20, 10))\n    \n    # create a list of random indices \n    rnd_indices = [np.random.choice(range(0, len(images))) for i in range(n_images)]\n    \n    for im in range(0, n_images):\n        # open image with a random index\n        image = Image.open(images[rnd_indices[im]])\n        \n        # get segmentation masks\n        fish = get_mask_by_image_id(images[rnd_indices[im]], 'Fish')\n        flower = get_mask_by_image_id(images[rnd_indices[im]], 'Flower')\n        gravel = get_mask_by_image_id(images[rnd_indices[im]], 'Gravel')\n        sugar = get_mask_by_image_id(images[rnd_indices[im]], 'Sugar')\n        \n        # draw masks on images\n        shape = (np.asarray(image).shape[0], np.asarray(image).shape[1])\n        if np.sum(fish) > 0:\n            segmap_fish = SegmentationMapOnImage(fish, shape=shape, nb_classes=2)\n            im_fish = np.array(segmap_fish.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_fish = np.asarray(image)\n        \n        if np.sum(flower) > 0:\n            segmap_flower = SegmentationMapOnImage(flower, shape=shape, nb_classes=2)\n            im_flower = np.array(segmap_flower.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_flower = np.asarray(image)\n        \n        if np.sum(gravel) > 0:\n            segmap_gravel = SegmentationMapOnImage(gravel, shape=shape, nb_classes=2)\n            im_gravel = np.array(segmap_gravel.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_gravel = np.asarray(image)\n        \n        if np.sum(sugar) > 0:\n            segmap_sugar = SegmentationMapOnImage(sugar, shape=shape, nb_classes=2)\n            im_sugar = np.array(segmap_sugar.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_sugar = np.asarray(image)\n        \n        # plot images and masks\n        ax[im, 0].imshow(im_fish)\n        ax[im, 0].axis('off')\n        ax[im, 0].set_title('Fish')\n        \n        # plot images and masks\n        ax[im, 1].imshow(im_flower)\n        ax[im, 1].axis('off')\n        ax[im, 1].set_title('Flower')\n        \n        # plot images and masks\n        ax[im, 2].imshow(im_gravel)\n        ax[im, 2].axis('off')\n        ax[im, 2].set_title('Gravel')\n        \n        # plot images and masks\n        ax[im, 3].imshow(im_sugar)\n        ax[im, 3].axis('off')\n        ax[im, 3].set_title('Sugar')\n        \n    plt.suptitle('Sample images from the train set')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_training_images_and_masks(n_images = 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`4.` With __imgaug__ we can visualize several segmentation maps on one image:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def create_segmap(image_id):\n    '''\n    Helper function to create a segmentation map for an image by image filename\n    '''\n    # open the image\n    image = np.asarray(Image.open(image_id))\n    \n    # get masks for different classes\n    fish_mask = get_mask_by_image_id(image_id, 'Fish')\n    flower_mask = get_mask_by_image_id(image_id, 'Flower')\n    gravel_mask = get_mask_by_image_id(image_id, 'Gravel')\n    sugar_mask = get_mask_by_image_id(image_id, 'Sugar')\n    \n    # label numpy map with 4 classes\n    segmap = np.zeros((image.shape[0], image.shape[1]), dtype=np.int32)\n    segmap = np.where(fish_mask == 1, 1, segmap)\n    segmap = np.where(flower_mask == 1, 2, segmap)\n    segmap = np.where(gravel_mask == 1, 3, segmap)\n    segmap = np.where(sugar_mask == 1, 4, segmap)\n    \n    # create a segmantation map\n    segmap = SegmentationMapOnImage(segmap, shape=image.shape, nb_classes=5)\n    \n    return segmap\n\ndef draw_labels(image, np_mask, label):\n    '''\n    Function to add labels to the image.\n    '''\n    if np.sum(np_mask) > 0:\n        x,y = 0,0\n        x,y = np.argwhere(np_mask==1)[0]\n                \n        image = imgaug.imgaug.draw_text(image, x, y, label, color=(255, 255, 255), size=50)\n    return image\n\ndef draw_segmentation_maps(image_id):\n    '''\n    Helper function to draw segmantation maps and text.\n    '''\n    # open the image\n    image = np.asarray(Image.open(image_id))\n    \n    # get masks for different classes\n    fish_mask = get_mask_by_image_id(image_id, 'Fish')\n    flower_mask = get_mask_by_image_id(image_id, 'Flower')\n    gravel_mask = get_mask_by_image_id(image_id, 'Gravel')\n    sugar_mask = get_mask_by_image_id(image_id, 'Sugar')\n    \n    # label numpy map with 4 classes\n    segmap = create_segmap(image_id)\n    \n    # draw the map on image\n    image = np.asarray(segmap.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n    \n    image = draw_labels(image, fish_mask, 'Fish')\n    image = draw_labels(image, flower_mask, 'Flower')\n    image = draw_labels(image, gravel_mask, 'Gravel')\n    image = draw_labels(image, sugar_mask, 'Sugar')\n    \n    return image\n\n# helper function to visualize several segmentation maps on a single image\ndef visualize_several_maps(image_id):\n    '''\n    Function to visualize several segmentation maps.\n    INPUT:\n        image_id - filename of the image\n    '''\n    # open the image\n    image = np.asarray(Image.open(image_id))\n    \n    # draw segmentation maps and labels on image\n    image = draw_segmentation_maps(image_id)\n    \n    # visualize the image and map\n    side_by_side = np.hstack([\n        image\n    ])\n    \n    labels = get_labels(image_id.split('/')[-1])\n\n    fig, ax = plt.subplots(figsize=(15, 7))\n    ax.axis('off')\n    plt.title('Segmentation maps:' + labels)\n    plt.legend()\n    \n    ax.imshow(side_by_side)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# create list of all training images filenames\ntrain_fns = sorted(glob(TRAIN_PATH + '*.jpg'))\n\n# generate random index for an image\nnp.random.seed(41)\nrnd_index = np.random.choice(range(len(train_fns)))\n\n# call helper function to visualize the image\nvisualize_several_maps(train_fns[rnd_index])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can create a function to plot sample images with segmentation maps:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# function to plot a grid of images and their labels and segmantation maps\ndef plot_training_images_and_masks(width = 2, height = 3):\n    \"\"\"\n    Function to plot grid with several examples of cloud images from train set.\n    INPUT:\n        width - number of images per row\n        height - number of rows\n\n    OUTPUT: None\n    \"\"\"\n    \n    # get a list of images from training set\n    images = sorted(glob(TRAIN_PATH + '*.jpg'))\n    \n    fig, axs = plt.subplots(height, width, figsize=(20, 20))\n    \n    # create a list of random indices \n    rnd_indices = rnd_indices = [np.random.choice(range(0, len(images))) for i in range(height * width)]\n    \n    for im in range(0, height * width):\n        # open image with a random index\n        image = Image.open(images[rnd_indices[im]])\n        # draw segmentation maps and labels on image\n        image = draw_segmentation_maps(images[rnd_indices[im]])\n        \n        i = im // width\n        j = im % width\n        \n        # plot the image\n        axs[i,j].imshow(image) #plot the data\n        axs[i,j].axis('off')\n        axs[i,j].set_title(get_labels(images[rnd_indices[im]].split('/')[-1]))\n\n    # set suptitle\n    plt.suptitle('Sample images from the train set')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(42)\nplot_training_images_and_masks()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`4.` Add data augmentation:\n\nNow we can easily add data augmentation to our images and segmentation maps with __imgaug__."},{"metadata":{"trusted":true},"cell_type":"code","source":"# initialize augmentations\nseq = iaa.Sequential([\n    iaa.Affine(rotate=(-30, 30)),\n    iaa.Fliplr(0.5),\n    iaa.ElasticTransformation(alpha=10, sigma=1)\n])\n\n# generate random index for an image\nrnd_index = np.random.choice(range(len(train_fns)))\nimg_id = train_fns[rnd_index]\n\nimage = Image.open(img_id)\nsegmap = create_segmap(img_id)\n\n# apply augmentation for image and mask\nimage_aug, segmap_aug = seq(image=np.asarray(image), segmentation_maps=segmap)\n\n# visualize the image and map\nside_by_side = np.hstack([\n    draw_segmentation_maps(img_id),\n    np.asarray(segmap_aug.draw_on_image(image_aug)).reshape(np.asarray(image).shape)\n])\n\nlabels = get_labels(img_id.split('/')[-1])\n\nfig, ax = plt.subplots(figsize=(15, 7))\nax.axis('off')\nplt.title('Segmentation maps (original and augmented image):' + labels)\nplt.legend()\n\nax.imshow(side_by_side)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`5.` Distribution of mask area sizes\n\nThat's an interesting question. I'll observe the mask area sizes distribution for each label."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def add_mask_areas(train_df):\n    '''\n    Helper function to add mask area as a new column to the dataframe\n    INPUT:\n        train_df - dataset with training labels\n    '''\n    masks_df = train_df.copy()\n    masks_df['Area'] = 0\n        \n    for i, row in masks_df.iterrows():\n        masks_df['Area'].loc[i] = np.sum(get_mask(i))\n    \n    return masks_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Add the column `Area` to the dataframe with the segmentation mask area:"},{"metadata":{"trusted":true},"cell_type":"code","source":"masks_df = add_mask_areas(train_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plot the distribution of segmentation area masks for each label:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Plot Histograms and KDE plots\nplt.figure(figsize=(15,7))\n\nplt.subplot(221)\nsns.distplot(masks_df[masks_df['Label'] == 'Fish']['Area'].values, kde=False, label='Fish')\nplt.legend()\nplt.title('Mask Area Histogram : Fish', fontsize=15)\n\nplt.subplot(222)\nsns.distplot(masks_df[masks_df['Label'] == 'Gravel']['Area'].values, kde=False, label='Gravel')\nplt.legend()\nplt.title('Mask Area Histogram: Gravel', fontsize=15)\n\nplt.subplot(223)\nsns.distplot(masks_df[masks_df['Label'] == 'Flower']['Area'].values, kde=False, label='Flower')\nplt.legend()\nplt.title('Mask Area Histogram : Flower', fontsize=15)\n\nplt.subplot(224)\nsns.distplot(masks_df[masks_df['Label'] == 'Sugar']['Area'].values, kde=False, label='Sugar')\nplt.legend()\nplt.title('Mask Area Histogram: Sugar', fontsize=15)\n\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(15,4))\n\nplt.subplot(111)\nsns.kdeplot(masks_df[masks_df['Label'] == 'Fish']['Area'].values, label='Fish')\nsns.kdeplot(masks_df[masks_df['Label'] == 'Flower']['Area'].values, label='Flower')\nsns.kdeplot(masks_df[masks_df['Label'] == 'Gravel']['Area'].values, label='Gravel')\nsns.kdeplot(masks_df[masks_df['Label'] == 'Sugar']['Area'].values, label='Sugar')\nplt.legend()\n\nplt.title('Mask Area KDE Plot', fontsize=15)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`6.` Number of masks per image:"},{"metadata":{"trusted":true},"cell_type":"code","source":"from scipy.ndimage import label, generate_binary_structure\n\ndef add_mask_number(train_df):\n    '''\n    Helper function to add mask area as a new column to the dataframe\n    INPUT:\n        train_df - dataset with training labels\n    '''\n    masks_df = train_df.copy()\n    masks_df['NumMasks'] = 0\n    \n    s = generate_binary_structure(2,2)\n        \n    for i, row in masks_df.iterrows():\n        mask = get_mask(i)\n        \n        if np.sum(mask) > 0:\n            labeled_array, labels = label(mask, structure=s)\n            masks_df['NumMasks'].loc[i] = labels\n        else:\n            masks_df['NumMasks'].loc[i] = 0\n    \n    return masks_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"masks_df = add_mask_number(masks_df)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Plot Histograms and KDE plots\nplt.figure(figsize=(15,7))\n\nplt.subplot(221)\nsns.distplot(masks_df[masks_df['Label'] == 'Fish']['NumMasks'].values, kde=False, label='Fish')\nplt.legend()\nplt.title('Number of Masks Histogram : Fish', fontsize=15)\n\nplt.subplot(222)\nsns.distplot(masks_df[masks_df['Label'] == 'Gravel']['NumMasks'].values, kde=False, label='Gravel')\nplt.legend()\nplt.title('Number of Masks Histogram: Gravel', fontsize=15)\n\nplt.subplot(223)\nsns.distplot(masks_df[masks_df['Label'] == 'Flower']['NumMasks'].values, kde=False, label='Flower')\nplt.legend()\nplt.title('Number of Masks Histogram : Flower', fontsize=15)\n\nplt.subplot(224)\nsns.distplot(masks_df[masks_df['Label'] == 'Sugar']['NumMasks'].values, kde=False, label='Sugar')\nplt.legend()\nplt.title('Number of Masks Histogram: Sugar', fontsize=15)\n\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(15,4))\n\nplt.subplot(111)\nsns.kdeplot(masks_df[masks_df['Label'] == 'Fish']['NumMasks'].values, label='Fish')\nsns.kdeplot(masks_df[masks_df['Label'] == 'Flower']['NumMasks'].values, label='Flower')\nsns.kdeplot(masks_df[masks_df['Label'] == 'Gravel']['NumMasks'].values, label='Gravel')\nsns.kdeplot(masks_df[masks_df['Label'] == 'Sugar']['NumMasks'].values, label='Sugar')\nplt.legend()\n\nplt.title('Mask Area KDE Plot', fontsize=15)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Conclusion\nIn this kernel:\n* I analyzed training and testing data for the competition.\n* I used __imgaug__ package to demonstrate code for visualization and augmenting the images from the training dataset.\n\n__Please, leave your comments on how to improve this kernel and follow the updates.__"},{"metadata":{},"cell_type":"markdown","source":"## Credits and References:\n1. [Article on Medium](https://towardsdatascience.com/sugar-flower-fish-or-gravel-now-a-kaggle-competition-8d2b6b3b118) from competition organizers.\n2. [Original paper](https://arxiv.org/pdf/1906.01906.pdf) for the competition.\n3. [EDA: Find Me In The Clouds](https://www.kaggle.com/ekhtiar/eda-find-me-in-the-clouds) kernel. I took `rle_to_mask` function from there.\n4. [My kernel on data augmentation packages](https://www.kaggle.com/aleksandradeis/data-augmentation-packages-overview) for those who want to learn more about different data augmantation packages."},{"metadata":{},"cell_type":"markdown","source":"## Updates:\n1. Added the analysis of mask area distribution for each label.\n2. Added the analysis for number of masks per image for each label.\n3. Corrected issues."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}