{"cells":[{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install networkx\n!pip install regex\n!pip install nltk","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**UNDERSTANDING THE NATURE OF THE COMPETITION BY ANALYZING THE SOURCE PAPER WHICH IS PUBLISHED ON CORNELL UNIVERSITY'S WEBSITE - https://arxiv.org/abs/1906.01906**"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.cm as cm\nimport nltk\nimport nltk.corpus\nimport os\nimport networkx as nx\nfrom networkx.algorithms import community\nimport community\nfrom collections import Counter\nfrom heapq import nlargest \nimport matplotlib.pyplot as plt\nimport re #for tokenizing the file\nnltk.download('punkt')\nfrom nltk.tokenize import RegexpTokenizer\n\ntokenizer = RegexpTokenizer(r'\\w+')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Reading the files\nfile1= open('../input/paperforanalysis/paper.txt','r')\nstopwrd = open('../input/stopwords/stopwords_en.txt','r')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Converting the input from the files to lower case\nread1 = file1.read().lower()\nread2 = stopwrd.read().lower()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Replacing the new line (\"\\n\") to space (\" \")\nread1 = read1.replace(\"\\n\", \" \")\nread2= read2.replace(\"\\n\", \" \")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Tokenizing and removing the punctuations\nread1 = tokenizer.tokenize(read1)\nread2 = tokenizer.tokenize(read2)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Removing numerial data from the files\nalpha1 = []\n#alpha2 = []\n\nfor i in read1:\n\tif i.isdigit() == False:\n\t\talpha1.append(i)\n\n#for j in read2:\n#\tif j.isdigit() == False:\n#\t\talpha2.append(j)\n\n#Removing stopwords from the files\nclean1 = []\n#clean2 = []\nfor i in alpha1:\n\tif i not in read2:\n\t\tclean1.append(i)\n\n#for j in alpha2:\n#\tif j not in read2:\n#\t\tclean2.append(j)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#creating bigrams\nbigram1 = list(nltk.bigrams(clean1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"column_2 = bigram1[0:201]\ncolumn_2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"column_1 = clean1[0:201]\ncolumn_1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#creating a dataframe\ndf = pd.DataFrame(list(zip(column_1, column_2)), \n               columns =['Source', 'Target'])\n\ndf ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#creating a network\nnetwork_1 = nx.Graph()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#populating the graph\nfor row in df.iterrows():\n    network_1.add_edge(row[1]['Source'], row[1]['Target'])\ndf_1 = [network_1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#now let's take a look at the edges\nprint(list(network_1.edges(data=True))[3])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#let's find the most important node in the network\ndeg_centrality = nx.degree_centrality(df_1[0])\n#deg_centrality\nmax(deg_centrality, key=deg_centrality.get)\nxy = sorted(deg_centrality, key=deg_centrality.get, reverse=True)[:10]\nxy\nprint('\\n')\nc = sorted(deg_centrality.items(), key=lambda x:x[1], reverse = True)[0:10] #using lambda function to print the list of nodes in reverse order \nprint (c)\nprint('\\n')\n#creating this into a dataframe\ndf_2 = pd.DataFrame(list(deg_centrality.items()), columns=['Words', 'Degree Centrality Values'])\ndegree_centrality_head = df_2.head(50)\nprint(df_2.head(6))\nprint('\\n')\nnetwork_2 = nx.DiGraph()\nfor row in degree_centrality_head.iterrows():\n    network_2.add_edge(row[1]['Words'], row[1]['Degree Centrality Values'])\ndf_3 = [network_2]\nprint(df_3)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"These words are the most important words in the file. Hence, we can say that most of the paper revolves around these words only"},{"metadata":{},"cell_type":"markdown","source":"We are also analyzing the Betweenness Centrality and Page Rank to understand the most important word"},{"metadata":{"trusted":true},"cell_type":"code","source":"###looking into the betweenness centrality of the graph\n\nk = sorted(nx.betweenness_centrality(network_1).items(), key = lambda x:x[1], reverse = True)[0:10]\nprint (k) \nprint('\\n')\n#analyzing with the PageRank Algorithm. The PageRank algorithm works on an underlying assumption that more important the website, the website will have more links\ne = sorted(nx.pagerank_numpy(network_1, weight = 'weight').items(), key = lambda x:x[1], reverse = True)[0:10]\nprint (e)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now it is important to form the communities of word. Hence, we are looking into the modularity class of the text data that we are dealing with. We are identifying the various words that belong to a similar community."},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(10, 10))\n\npartition = community.best_partition(network_1)\nsize = (len(set(partition.values())))\npos = nx.spring_layout(network_1)\ncount = 0\ncolors = [cm.jet(x) for x in np.linspace(0, 1, size)]\nfor com in set(partition.values()):\n    list_nodes = [nodes for nodes in partition.keys()\n                                if partition[nodes] == com]\n    nx.draw_networkx_nodes(network_1, pos, list_nodes, node_size = 100, node_color=colors[count])\n    count = count + 1\n#nx.draw_networkx_edges(network_1, pos, alpha=0.2)\nnx.draw_circular(network_2,with_labels=True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see a lot of words that belong to the same community. This means all these words belong to a similar class words."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('The pairs of the community is as follows:')\nprint('\\n')\nd = {}\nfor character, par in partition.items():\n    if par in d:\n        d[par].append(character)\n    else:\n        d[par] = [character]\nd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**YAJNIK-BARABASSI ALGORITHM FOR PREFERENTIAL ATTACHMENT **\n\nIt is very important that for future study prospects, we find the probabilistic answer as to in which topic a study might occur by the researchers. This algorithm is based on the Albert-Barabassi Model for preferential attachment. The main idea of the model says that:\n\nThe recognition that growth and preferential attachment coexist in real networks has inspired a minimal model called the Barabási-Albert model, which can generate scale-free networks. The network develops following two steps which are known as Growth and Preferential attachment phenomenan.\n\nSource : http://barabasi.com/f/622.pdf\n\nHence, by using the top 10 values of the words taken from degree centrality, I created this function which will tell us the probability as to where the researchers might think to invest their area of research in future."},{"metadata":{"trusted":true},"cell_type":"code","source":"def prob(m,a):\n  try:\n    if m<len(a):\n      print(\"You have qualified the conditions. The answer that you are seeking is:\")\n      xy = sorted(a,reverse=True)\n      \n      b = xy[0:m+1]\n      print (b)\n    else:\n      \n      if m == len(a):\n        print (\"You are not putting the right option. The value has to be less than the value of the length of the original list.\")\n      if m > len(a):\n        print (\"You have put an invalid option. The value has to be less than the value of the length of the original list.\")\n    y = max(b)\n    x = sum(b)\n    b.sort()\n    length = len(b)\n    second_largest = b[length-2]\n    third_largest = b[length-3]\n      \n    a = y/x\n    print (\"The probability of attaching to the most famous point is,\", a)\n    if a<0.45:\n      print(\"The probability is too less to rely on that data point, so we choose the second largest data point.\")\n      c = second_largest/x\n      print (\"The probability of attaching to the second most famous point is,\", c)\n      if c<0.45:\n             print(\"The probability is too less to rely on that data point, so we choose the third largest data point.\")\n             dc = third_largest/x\n             print (\"The probability of attaching to the third most famous point is,\", dc)\n\n             \n      \n  except:\n      print (\"There can be some problem in the new section of the code. Please check them.\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"parameter_1 = int(input(\"Please enter a subset from the inital list:\"))\nparameter_2 = [0.014749262536873156,0.011799410029498525, 0.011799410029498525, 0.011799410029498525, 0.011799410029498525, 0.008849557522123894,0.008849557522123894, 0.008849557522123894,  0.008849557522123894,0.008849557522123894]\nresult_1 = prob(parameter_1,parameter_2)\nprint(result_1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As the word series was already in the order of preference, so there is a probability of 24%  that the researchers might include the topic related to 'mcc' or the whole area of research might circke around the topic 'mcc'.  "},{"metadata":{},"cell_type":"markdown","source":"**Now we will look into Data Exploration and main model building**"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# imports\n# import basic libraries\nimport os\nfrom glob import glob\n\n# import plotting\nfrom matplotlib import pyplot as plt\nimport matplotlib.patches as patches\nimport matplotlib\nimport seaborn as sns\n\n# import image manipulation\nfrom PIL import Image\nimport imageio","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"! pip install --upgrade imgaug","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# import data augmentation\nimport imgaug as ia\nfrom imgaug import augmenters as iaa\n# import segmentation maps from imgaug\nfrom imgaug.augmentables.segmaps import SegmentationMapOnImage\nimport imgaug.imgaug","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load Data"},{"metadata":{},"cell_type":"markdown","source":"Firts, let's define the paths to train and test images and load the dataframe with train images:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# set paths to train and test image datasets\nTRAIN_PATH = '../input/understanding_cloud_organization/train_images/'\nTEST_PATH = '../input/understanding_cloud_organization/test_images/'\nsubmission = '../input/understanding_cloud_organization/sample_submission.csv'\npath = '../input/understanding_cloud_organization'\n\n# load dataframe with train labels\ntrain_df = pd.read_csv('../input/understanding_cloud_organization/train.csv')\ntrain_fns = sorted(glob(TRAIN_PATH + '*.jpg'))\n\nprint('There are {} images in the train set.'.format(len(train_df)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's load explore the test set a little:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the filenames for test images\ntest_fns = sorted(glob(TEST_PATH + '*.jpg'))\n\nprint('There are {} images in the test set.'.format(len(test_fns)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plot the pie chart for the train and test datasets:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# plotting a pie chart which demonstrates train and test sets\nlabels = 'Train', 'Test'\nsizes = [len(train_fns), len(test_fns)]\nexplode = (0, 0.1)\n\nfig, ax = plt.subplots(figsize=(6, 6))\nax.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)\nax.axis('equal')\nax.set_title('Train and Test Sets')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Explore Labels from Train Set"},{"metadata":{},"cell_type":"markdown","source":"Look how the dataframe with train labels looks like:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('There are {} rows with empty segmentation maps.'.format(len(train_df) - train_df.EncodedPixels.count()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# plotting a pie chart\nlabels = 'Non-empty', 'Empty'\nsizes = [train_df.EncodedPixels.count(), len(train_df) - train_df.EncodedPixels.count()]\nexplode = (0, 0.1)\n\nfig, ax = plt.subplots(figsize=(6, 6))\nax.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)\nax.axis('equal')\nax.set_title('Non-empty and Empty Masks')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looks like almost __half of the lines is empty__."},{"metadata":{},"cell_type":"markdown","source":"`2.` Explore the labels:"},{"metadata":{},"cell_type":"markdown","source":"Let's split the `Image_Label` into two columns and analyze the labels:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# split column\nsplit_df = train_df[\"Image_Label\"].str.split(\"_\", n = 1, expand = True)\n# add new columns to train_df\ntrain_df['Image'] = split_df[0]\ntrain_df['Label'] = split_df[1]\n\n# check the result\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can count the number of labels of each cloud type:"},{"metadata":{"trusted":true},"cell_type":"code","source":"fish = train_df[train_df['Label'] == 'Fish'].EncodedPixels.count()\nflower = train_df[train_df['Label'] == 'Flower'].EncodedPixels.count()\ngravel = train_df[train_df['Label'] == 'Gravel'].EncodedPixels.count()\nsugar = train_df[train_df['Label'] == 'Sugar'].EncodedPixels.count()\n\nprint('There are {} fish clouds'.format(fish))\nprint('There are {} flower clouds'.format(flower))\nprint('There are {} gravel clouds'.format(gravel))\nprint('There are {} sugar clouds'.format(sugar))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# plotting a pie chart\nlabels = 'Fish', 'Flower', 'Gravel', 'Sugar'\nsizes = [fish, flower, gravel, sugar]\n\nfig, ax = plt.subplots(figsize=(6, 6))\nax.pie(sizes, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)\nax.axis('equal')\nax.set_title('Cloud Types')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that at least the dataset is somewhat __balanced__, which is great and makes are task way more easier."},{"metadata":{},"cell_type":"markdown","source":"`3.` Explore the number of labels per image:"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_per_image = train_df.groupby('Image')['EncodedPixels'].count()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('The mean number of labels per image is {}'.format(labels_per_image.mean()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(6, 6))\nax.hist(labels_per_image)\nax.set_title('Number of Labels per Image')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"__So most of the images have 2 labels.__"},{"metadata":{},"cell_type":"markdown","source":"`4.` Explore the correlation between different cloud types.\n\nUsing the dataframe with labels, we can try to find the correlation between different types of clouds."},{"metadata":{"trusted":true},"cell_type":"code","source":"# create dummy columns for each cloud type\ncorr_df = pd.get_dummies(train_df, columns = ['Label'])\n# fill null values with '-1'\ncorr_df = corr_df.fillna('-1')\n\n# define a helper function to fill dummy columns\ndef get_dummy_value(row, cloud_type):\n    ''' Get value for dummy column '''\n    if cloud_type == 'fish':\n        return row['Label_Fish'] * (row['EncodedPixels'] != '-1')\n    if cloud_type == 'flower':\n        return row['Label_Flower'] * (row['EncodedPixels'] != '-1')\n    if cloud_type == 'gravel':\n        return row['Label_Gravel'] * (row['EncodedPixels'] != '-1')\n    if cloud_type == 'sugar':\n        return row['Label_Sugar'] * (row['EncodedPixels'] != '-1')\n    \n# fill dummy columns\ncorr_df['Label_Fish'] = corr_df.apply(lambda row: get_dummy_value(row, 'fish'), axis=1)\ncorr_df['Label_Flower'] = corr_df.apply(lambda row: get_dummy_value(row, 'flower'), axis=1)\ncorr_df['Label_Gravel'] = corr_df.apply(lambda row: get_dummy_value(row, 'gravel'), axis=1)\ncorr_df['Label_Sugar'] = corr_df.apply(lambda row: get_dummy_value(row, 'sugar'), axis=1)\n\n# check the result\ncorr_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# group by the image\ncorr_df = corr_df.groupby('Image')['Label_Fish', 'Label_Flower', 'Label_Gravel', 'Label_Sugar'].max()\ncorr_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can explore the correlation between `Label_Fish, Label_Flower, Label_Gravel, Label_Sugar` columns:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#Find out correlation between columns and plot\ncorrs = np.corrcoef(corr_df.values.T)\nsns.set(font_scale=1)\nsns.set(rc={'figure.figsize':(7,7)})\nhm=sns.heatmap(corrs, cbar = True, annot=True, square = True, fmt = '.2f',\n              yticklabels = ['Fish', 'Flower', 'Gravel', 'Sugar'], \n               xticklabels = ['Fish', 'Flower', 'Gravel', 'Sugar']).set_title('Cloud type correlation heatmap')\n\nfig = hm.get_figure()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can observe, there is __no strong correlation between the types of the clouds__ on one image (all the correlation coefficients are close to zero)."},{"metadata":{},"cell_type":"markdown","source":"## Explore the Images\n\nHere goes the most exciting part of the EDA: exploring the images themselves."},{"metadata":{},"cell_type":"markdown","source":"`1.` Explore image sizes:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_image_sizes(train = True):\n    '''\n    Function to get sizes of images from test and train sets.\n    INPUT:\n        train - indicates whether we are getting sizes of images from train or test set\n    '''\n    if train:\n        path = TRAIN_PATH\n    else:\n        path = TEST_PATH\n        \n    widths = []\n    heights = []\n    \n    images = sorted(glob(path + '*.jpg'))\n    \n    max_im = Image.open(images[0])\n    min_im = Image.open(images[0])\n        \n    for im in range(0, len(images)):\n        image = Image.open(images[im])\n        width, height = image.size\n        \n        if len(widths) > 0:\n            if width > max(widths):\n                max_im = image\n\n            if width < min(widths):\n                min_im = image\n\n        widths.append(width)\n        heights.append(height)\n        \n    return widths, heights, max_im, min_im","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get sizes of images from test and train sets\ntrain_widths, train_heights, max_train, min_train = get_image_sizes(train = True)\ntest_widths, test_heights, max_test, min_test = get_image_sizes(train = False)\n\nprint('Maximum width for training set is {}'.format(max(train_widths)))\nprint('Minimum width for training set is {}'.format(min(train_widths)))\nprint('Maximum height for training set is {}'.format(max(train_heights)))\nprint('Minimum height for training set is {}'.format(min(train_heights)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Maximum width for test set is {}'.format(max(test_widths)))\nprint('Minimum width for test set is {}'.format(min(test_widths)))\nprint('Maximum height for test set is {}'.format(max(test_heights)))\nprint('Minimum height for test set is {}'.format(min(test_heights)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that __all images have the same size__. That's great!"},{"metadata":{},"cell_type":"markdown","source":"`2.` Plot sample images from training set:"},{"metadata":{},"cell_type":"markdown","source":"At first, I will prepare some helper functions for visualization:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# helper function to get a string of labels for the picture\ndef get_labels(image_id):\n    ''' Function to get the labels for the image by name'''\n    im_df = train_df[train_df['Image'] == image_id].fillna('-1')\n    im_df = im_df[im_df['EncodedPixels'] != '-1'].groupby('Label').count()\n    \n    index = im_df.index\n    all_labels = ['Fish', 'Flower', 'Gravel', 'Sugar']\n    \n    labels = ''\n    \n    for label in all_labels:\n        if label in index:\n            labels = labels + ' ' + label\n    \n    return labels\n\n# function to plot a grid of images and their labels\ndef plot_training_images(width = 5, height = 2):\n    \"\"\"\n    Function to plot grid with several examples of cloud images from train set.\n    INPUT:\n        width - number of images per row\n        height - number of rows\n\n    OUTPUT: None\n    \"\"\"\n    \n    # get a list of images from training set\n    images = sorted(glob(TRAIN_PATH + '*.jpg'))\n    \n    fig, axs = plt.subplots(height, width, figsize=(width * 3, height * 3))\n    \n    # create a list of random indices \n    rnd_indices = rnd_indices = [np.random.choice(range(0, len(images))) for i in range(height * width)]\n    \n    for im in range(0, height * width):\n        # open image with a random index\n        image = Image.open(images[rnd_indices[im]])\n        \n        i = im // width\n        j = im % width\n        \n        # plot the image\n        axs[i,j].imshow(image) #plot the data\n        axs[i,j].axis('off')\n        axs[i,j].set_title(get_labels(images[rnd_indices[im]].split('/')[-1]))\n\n    # set suptitle\n    plt.suptitle('Sample images from the train set')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's plot sample images:"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_training_images()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`3.` Visualize segmentation maps:"},{"metadata":{},"cell_type":"markdown","source":"I will use a function from [this great EDA kernel](https://www.kaggle.com/ekhtiar/eda-find-me-in-the-clouds). I upvoted it and encourage you to do so too."},{"metadata":{"trusted":true},"cell_type":"code","source":"def rle_to_mask(rle_string, width, height):\n    '''\n    convert RLE(run length encoding) string to numpy array\n\n    Parameters: \n    rle_string (str): string of rle encoded mask\n    height (int): height of the mask\n    width (int): width of the mask\n\n    Returns: \n    numpy.array: numpy array of the mask\n    '''\n    \n    rows, cols = height, width\n    \n    if rle_string == -1:\n        return np.zeros((height, width))\n    else:\n        rle_numbers = [int(num_string) for num_string in rle_string.split(' ')]\n        rle_pairs = np.array(rle_numbers).reshape(-1,2)\n        img = np.zeros(rows*cols, dtype=np.uint8)\n        for index, length in rle_pairs:\n            index -= 1\n            img[index:index+length] = 255\n        img = img.reshape(cols,rows)\n        img = img.T\n        return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I will use __[imgaug](https://imgaug.readthedocs.io/en/latest/index.html) library__ to visualize the segmentation maps. This library has special helpers for visualization and augmentation of images with segmentation maps. You will see how easy it is to work with segmentation maps with __imgaug__."},{"metadata":{"trusted":true},"cell_type":"code","source":"from __future__ import print_function\nimport numpy as np\n\ndef valid_imshow_data(data):\n    data = np.asarray(data)\n    if data.ndim == 2:\n        return True\n    elif data.ndim == 3:\n        if 3 <= data.shape[2] <= 4:\n            return True\n        else:\n            print('The \"data\" has 3 dimensions but the last dimension '\n                  'must have a length of 3 (RGB) or 4 (RGBA), not \"{}\".'\n                  ''.format(data.shape[2]))\n            return False\n    else:\n        print('To visualize an image the data must be 2 dimensional or '\n              '3 dimensional, not \"{}\".'\n              ''.format(data.ndim))\n        return False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_mask(line_id, shape = (2100, 1400)):\n    '''\n    Function to visualize the image and the mask.\n    INPUT:\n        line_id - id of the line to visualize the masks\n        shape - image shape\n    RETURNS:\n        np_mask - numpy segmentation map\n    '''\n    # replace null values with '-1'\n    im_df = train_df.fillna('-1')\n    \n    # convert rle to mask\n    rle = im_df.loc[line_id]['EncodedPixels']\n    if rle != '-1':\n        np_mask = rle_to_mask(rle, shape[0], shape[1])\n        np_mask = np.clip(np_mask, 0, 1)\n    else:\n        # empty mask\n        np_mask = np.zeros((shape[0],shape[1]), dtype=np.uint8)\n        \n    return np_mask\n\n# helper function to get segmentation mask for an image by filename\ndef get_mask_by_image_id(image_id, label):\n    '''\n    Function to visualize several segmentation maps.\n    INPUT:\n        image_id - filename of the image\n    RETURNS:\n        np_mask - numpy segmentation map\n    '''\n    im_df = train_df[train_df['Image'] == image_id.split('/')[-1]].fillna('-1')\n\n    image = np.asarray(Image.open(image_id))\n\n    rle = im_df[im_df['Label'] == label]['EncodedPixels'].values[0]\n    if rle != '-1':\n        np_mask = rle_to_mask(rle, np.asarray(image).shape[1], np.asarray(image).shape[0])\n        np_mask = np.clip(np_mask, 0, 1)\n    else:\n        # empty mask\n        np_mask = np.zeros((np.asarray(image).shape[0], np.asarray(image).shape[1]), dtype=np.uint8)\n        \n    return np_mask\n\ndef visualize_image_with_mask(line_id):\n    '''\n    Function to visualize the image and the mask.\n    INPUT:\n        line_id - id of the line to visualize the masks\n    '''\n    # replace null values with '-1'\n    im_df = train_df.fillna('-1')\n    \n    # get segmentation mask\n    np_mask = get_mask(line_id)\n    \n    # open the image\n    image = Image.open(TRAIN_PATH + im_df.loc[line_id]['Image'])\n\n    # create segmentation map\n    segmap = SegmentationMapOnImage(np_mask, np_mask.shape, nb_classes=2)\n    \n    # visualize the image and map\n    side_by_side = np.hstack([\n        segmap.draw_on_image(np.asarray(image))\n    ]).reshape(np.asarray(image).shape)\n\n    fig, ax = plt.subplots(figsize=(6, 4))\n    ax.axis('off')\n    plt.title(im_df.loc[line_id]['Label'])\n    \n    ax.imshow(side_by_side)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Visualize sample masks:"},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_image_with_mask(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"visualize_image_with_mask(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# empty mask:\nvisualize_image_with_mask(2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Visualize image grids:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def plot_training_images_and_masks(n_images = 3):\n    '''\n    Function to plot several random images with segmentation masks.\n    INPUT:\n        n_images - number of images to visualize\n    '''\n    \n    # get a list of images from training set\n    images = sorted(glob(TRAIN_PATH + '*.jpg'))\n    \n    fig, ax = plt.subplots(n_images, 4, figsize=(20, 10))\n    \n    # create a list of random indices \n    rnd_indices = [np.random.choice(range(0, len(images))) for i in range(n_images)]\n    \n    for im in range(0, n_images):\n        # open image with a random index\n        image = Image.open(images[rnd_indices[im]])\n        \n        # get segmentation masks\n        fish = get_mask_by_image_id(images[rnd_indices[im]], 'Fish')\n        flower = get_mask_by_image_id(images[rnd_indices[im]], 'Flower')\n        gravel = get_mask_by_image_id(images[rnd_indices[im]], 'Gravel')\n        sugar = get_mask_by_image_id(images[rnd_indices[im]], 'Sugar')\n        \n        # draw masks on images\n        shape = (np.asarray(image).shape[0], np.asarray(image).shape[1])\n        if np.sum(fish) > 0:\n            segmap_fish = SegmentationMapOnImage(fish, shape=shape, nb_classes=2)\n            im_fish = np.array(segmap_fish.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_fish = np.asarray(image)\n        \n        if np.sum(flower) > 0:\n            segmap_flower = SegmentationMapOnImage(flower, shape=shape, nb_classes=2)\n            im_flower = np.array(segmap_flower.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_flower = np.asarray(image)\n        \n        if np.sum(gravel) > 0:\n            segmap_gravel = SegmentationMapOnImage(gravel, shape=shape, nb_classes=2)\n            im_gravel = np.array(segmap_gravel.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_gravel = np.asarray(image)\n        \n        if np.sum(sugar) > 0:\n            segmap_sugar = SegmentationMapOnImage(sugar, shape=shape, nb_classes=2)\n            im_sugar = np.array(segmap_sugar.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n        else:\n            im_sugar = np.asarray(image)\n        \n        # plot images and masks\n        ax[im, 0].imshow(im_fish)\n        ax[im, 0].axis('off')\n        ax[im, 0].set_title('Fish')\n        \n        # plot images and masks\n        ax[im, 1].imshow(im_flower)\n        ax[im, 1].axis('off')\n        ax[im, 1].set_title('Flower')\n        \n        # plot images and masks\n        ax[im, 2].imshow(im_gravel)\n        ax[im, 2].axis('off')\n        ax[im, 2].set_title('Gravel')\n        \n        # plot images and masks\n        ax[im, 3].imshow(im_sugar)\n        ax[im, 3].axis('off')\n        ax[im, 3].set_title('Sugar')\n        \n    plt.suptitle('Sample images from the train set')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_training_images_and_masks(n_images = 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`4.` With __imgaug__ we can visualize several segmentation maps on one image:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def create_segmap(image_id):\n    '''\n    Helper function to create a segmentation map for an image by image filename\n    '''\n    # open the image\n    image = np.asarray(Image.open(image_id))\n    \n    # get masks for different classes\n    fish_mask = get_mask_by_image_id(image_id, 'Fish')\n    flower_mask = get_mask_by_image_id(image_id, 'Flower')\n    gravel_mask = get_mask_by_image_id(image_id, 'Gravel')\n    sugar_mask = get_mask_by_image_id(image_id, 'Sugar')\n    \n    # label numpy map with 4 classes\n    segmap = np.zeros((image.shape[0], image.shape[1]), dtype=np.int32)\n    segmap = np.where(fish_mask == 1, 1, segmap)\n    segmap = np.where(flower_mask == 1, 2, segmap)\n    segmap = np.where(gravel_mask == 1, 3, segmap)\n    segmap = np.where(sugar_mask == 1, 4, segmap)\n    \n    # create a segmantation map\n    segmap = SegmentationMapOnImage(segmap, shape=image.shape, nb_classes=5)\n    \n    return segmap\n\ndef draw_labels(image, np_mask, label):\n    '''\n    Function to add labels to the image.\n    '''\n    if np.sum(np_mask) > 0:\n        x,y = 0,0\n        x,y = np.argwhere(np_mask==1)[0]\n                \n        image = imgaug.imgaug.draw_text(image, x, y, label, color=(255, 255, 255), size=50)\n    return image\n\ndef draw_segmentation_maps(image_id):\n    '''\n    Helper function to draw segmantation maps and text.\n    '''\n    # open the image\n    image = np.asarray(Image.open(image_id))\n    \n    # get masks for different classes\n    fish_mask = get_mask_by_image_id(image_id, 'Fish')\n    flower_mask = get_mask_by_image_id(image_id, 'Flower')\n    gravel_mask = get_mask_by_image_id(image_id, 'Gravel')\n    sugar_mask = get_mask_by_image_id(image_id, 'Sugar')\n    \n    # label numpy map with 4 classes\n    segmap = create_segmap(image_id)\n    \n    # draw the map on image\n    image = np.asarray(segmap.draw_on_image(np.asarray(image))).reshape(np.asarray(image).shape)\n    \n    image = draw_labels(image, fish_mask, 'Fish')\n    image = draw_labels(image, flower_mask, 'Flower')\n    image = draw_labels(image, gravel_mask, 'Gravel')\n    image = draw_labels(image, sugar_mask, 'Sugar')\n    \n    return image\n\n# helper function to visualize several segmentation maps on a single image\ndef visualize_several_maps(image_id):\n    '''\n    Function to visualize several segmentation maps.\n    INPUT:\n        image_id - filename of the image\n    '''\n    # open the image\n    image = np.asarray(Image.open(image_id))\n    \n    # draw segmentation maps and labels on image\n    image = draw_segmentation_maps(image_id)\n    \n    # visualize the image and map\n    side_by_side = np.hstack([\n        image\n    ])\n    \n    labels = get_labels(image_id.split('/')[-1])\n\n    fig, ax = plt.subplots(figsize=(15, 7))\n    ax.axis('off')\n    plt.title('Segmentation maps:' + labels)\n    plt.legend()\n    \n    ax.imshow(side_by_side)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# create list of all training images filenames\ntrain_fns = sorted(glob(TRAIN_PATH + '*.jpg'))\n\n# generate random index for an image\nnp.random.seed(41)\nrnd_index = np.random.choice(range(len(train_fns)))\n\n# call helper function to visualize the image\nvisualize_several_maps(train_fns[rnd_index])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can create a function to plot sample images with segmentation maps:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# function to plot a grid of images and their labels and segmantation maps\ndef plot_training_images_and_masks(width = 2, height = 3):\n    \"\"\"\n    Function to plot grid with several examples of cloud images from train set.\n    INPUT:\n        width - number of images per row\n        height - number of rows\n\n    OUTPUT: None\n    \"\"\"\n    \n    # get a list of images from training set\n    images = sorted(glob(TRAIN_PATH + '*.jpg'))\n    \n    fig, axs = plt.subplots(height, width, figsize=(20, 20))\n    \n    # create a list of random indices \n    rnd_indices = rnd_indices = [np.random.choice(range(0, len(images))) for i in range(height * width)]\n    \n    for im in range(0, height * width):\n        # open image with a random index\n        image = Image.open(images[rnd_indices[im]])\n        # draw segmentation maps and labels on image\n        image = draw_segmentation_maps(images[rnd_indices[im]])\n        \n        i = im // width\n        j = im % width\n        \n        # plot the image\n        axs[i,j].imshow(image) #plot the data\n        axs[i,j].axis('off')\n        axs[i,j].set_title(get_labels(images[rnd_indices[im]].split('/')[-1]))\n\n    # set suptitle\n    plt.suptitle('Sample images from the train set')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(42)\nplot_training_images_and_masks()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`4.` Add data augmentation:\n\nNow we can easily add data augmentation to our images and segmentation maps with __imgaug__."},{"metadata":{"trusted":true},"cell_type":"code","source":"# initialize augmentations\nseq = iaa.Sequential([\n    iaa.Affine(rotate=(-30, 30)),\n    iaa.Fliplr(0.5),\n    iaa.ElasticTransformation(alpha=10, sigma=1)\n])\n\n# generate random index for an image\nrnd_index = np.random.choice(range(len(train_fns)))\nimg_id = train_fns[rnd_index]\n\nimage = Image.open(img_id)\nsegmap = create_segmap(img_id)\n\n# apply augmentation for image and mask\nimage_aug, segmap_aug = seq(image=np.asarray(image), segmentation_maps=segmap)\n\n# visualize the image and map\nside_by_side = np.hstack([\n    draw_segmentation_maps(img_id),\n    np.asarray(segmap_aug.draw_on_image(image_aug)).reshape(np.asarray(image).shape)\n])\n\nlabels = get_labels(img_id.split('/')[-1])\n\nfig, ax = plt.subplots(figsize=(15, 7))\nax.axis('off')\nplt.title('Segmentation maps (original and augmented image):' + labels)\nplt.legend()\n\nax.imshow(side_by_side)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def add_mask_areas(train_df):\n    '''\n    Helper function to add mask area as a new column to the dataframe\n    INPUT:\n        train_df - dataset with training labels\n    '''\n    masks_df = train_df.copy()\n    masks_df['Area'] = 0\n        \n    for i, row in masks_df.iterrows():\n        masks_df['Area'].loc[i] = np.sum(get_mask(i))\n    \n    return masks_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = pd.read_csv(submission)\nsub.head(5)\n\nsub.to_csv(f'submission.csv', columns=['Image_Label', 'EncodedPixels'], index=False)\n\nsub.to_csv(f'submission1.csv', columns=['Image_Label', 'EncodedPixels'], index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Conclusion\nIn this kernel:\n* I analyzed training and testing data for the competition.\n* I used __imgaug__ package to demonstrate code for visualization and augmenting the images from the training dataset.\n\n__Please, leave your comments on how to improve this kernel and follow the updates.__"},{"metadata":{},"cell_type":"markdown","source":"## Credits and References:\n1. [Article on Medium](https://towardsdatascience.com/sugar-flower-fish-or-gravel-now-a-kaggle-competition-8d2b6b3b118) from competition organizers.\n2. [Original paper](https://arxiv.org/pdf/1906.01906.pdf) for the competition.\n3. [EDA: Find Me In The Clouds](https://www.kaggle.com/ekhtiar/eda-find-me-in-the-clouds) kernel. I took `rle_to_mask` function from there.\n4. [My kernel on data augmentation packages](https://www.kaggle.com/aleksandradeis/data-augmentation-packages-overview) for those who want to learn more about different data augmantation packages."},{"metadata":{},"cell_type":"markdown","source":"## Updates:\n1. Added the analysis of mask area distribution for each label.\n2. Added the analysis for number of masks per image for each label.\n3. Corrected issues."},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}