{"cells":[{"metadata":{"_uuid":"8ff62ec93b70fd9ddcab85caf317b4b272785706","_cell_guid":"4a31a06e-67e1-46c1-8e36-bb8602bf9970"},"cell_type":"markdown","source":"Hello Kagglers!! I hope you are busy doing analysis and building models on Kaggle.  This competition is hosted by the 2018 CVPR workshop on autonomous driving (WAD). Here Kagglers are challenged to develop a robust segmentation algorithm that can be used for self-driving cars. I have been working in this field since last year, so I know how crucial this part is. Before we dive into the data analysis part, I would like to elaborate on some important aspects regarding segmentation:\n\n* For a self-driving car, segmentation or to be more precise semantic segmentation gives your car a view of what lies ahead of it. Though little bit computationally expensive compared to the algorithms generating bounding boxes, semantic segmentation gives a car much more idea about scene understanding. Also, it overcomes all those critical situations where bounding box fails miserably.\n\n* It is not possible to label each and every type of object/instance on the road. This poses another challenge i.e. how to interpret something that is labeled by the model as an `unknown` instance. How do you interpret the label map generated in this case?\n\n* **Most important point** Self-driving cars run with chipsets integrated within the car system. Though most people use `Drive PX2` but not all of them. Self-driving car makes sense only when your model is able to run on an embedded device and you get efficient, if not almost real-time, FPS.  So, don't ensemble first. Try to develop something that is lightweight and is capable of running on an embedded device because that is the model for which this competition is hosted \n\n\nLet's dive into the dataset!!\n\n![](https://media.giphy.com/media/mlBDoVLOGidEc/giphy.gif)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \nimport os\nimport io\nimport glob\nimport cv2\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom pathlib import Path\nfrom time import time\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mimg\nfrom concurrent.futures import ProcessPoolExecutor\nfrom os import listdir, makedirs, getcwd, remove\nfrom os.path import isfile, join, abspath, exists, isdir, expanduser\nfrom skimage.io import imread\nfrom PIL import Image\nfrom pathlib import Path\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler\ncolor = sns.color_palette()\n%matplotlib inline\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":1,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# Define some paths first\ninput_dir = Path('../input')\nimages_dir = input_dir / 'train_color'\nlabels_dir = input_dir / 'train_label'","execution_count":2,"outputs":[]},{"metadata":{"_uuid":"a481a294e51844e29431b8d2cacb79ce5d7e901c","_cell_guid":"bcb9dab0-539c-43b8-aba6-5b5b7a80c0af","trusted":true},"cell_type":"code","source":"# Hoe many samples are there in the training dataset?\ntrain_images = sorted(os.listdir(images_dir))\ntrain_labels = sorted(os.listdir(labels_dir))\n\nprint(\"Number of images and labels in the training data: {}  and {} respectively\".format(len(train_images), len(train_labels)))","execution_count":3,"outputs":[]},{"metadata":{"_uuid":"b8fd7653c21c956a7f49fbfdb3c3d147ac07af04","_cell_guid":"3444800a-74f9-419c-b418-1958f8e16b38"},"cell_type":"markdown","source":"### How the dataset is arranged?\n\nIt is actually very interesting how the information is encoded in label corresponding to an image. The data comes from a video stream which is a very common data collection technique when it comes to self-driving cars. Each image contains certain objects. Within an image, there can be a single object or multiple objects of same/different type. for example within a frame, there may be a single car or three cars or a car, a person and a bus.\n\nNow each label map is an image of the same size as the original image. Here is the interesting part. All pixels except for the pixels belonging to a certain object we want to coonisder for our dataset, has a value of 255. All the pixels that belong to an object has a value greater tha 255 and is given a value in such a way that when you divide that value by 1000, you get the class of the object to which this pixel belongs to and doing a mod by 1000 will give you the instance number. \n\nFor example, a pixel value of 33000 means it belongs to label 33 (a car), is instance #0, while the pixel value of 33001 means it also belongs to class 33 (a car) , and is instance #1. These represent two different cars in an image."},{"metadata":{"_uuid":"587cea568f80635286949f0fd9923aed0a175039","collapsed":true,"_cell_guid":"7af3426c-744b-4a85-a8a9-14cd221b3df2","trusted":true},"cell_type":"code","source":"# Define the label mappings \nlabelmap = {0:'others', \n            1:'rover', \n            17:'sky', \n            33:'car', \n            34:'motorbicycle', \n            35:'bicycle', \n            36:'person', \n            37:'rider', \n            38:'truck', \n            39:'bus', \n            40:'tricycle', \n            49:'road', \n            50:'siderwalk', \n            65:'traffic_cone', \n            66:'road_pile', \n            67:'fence', \n            81:'traffic_light', \n            82:'pole', \n            83:'traffic_sign', \n            84:'wall', \n            85:'dustbin', \n            86:'billboard', \n            97:'building', \n            98:'bridge', \n            99:'tunnel', \n            100:'overpass', \n            113:'vegatation', \n            161:'car_groups', \n            162:'motorbicycle_group', \n            163:'bicycle_group', \n            164:'person_group', \n            165:'rider_group', \n            166:'truck_group', \n            167:'bus_group', \n            168:'tricycle_group'}","execution_count":4,"outputs":[]},{"metadata":{"_uuid":"bfbe1baeede5348a15ac085c981a6a6db57469fd","_cell_guid":"922834ae-7a52-4019-a679-6fb5729517da","trusted":true},"cell_type":"code","source":"# Create an empty dataframe\ndata_df = pd.DataFrame()\ndf_list = []\n\n# Iterate over data. I have just shown it for 500 images just to save time \nfor idx in range(500):\n    # Get the image name and corresponding label\n    img_name = train_images[idx]\n    label_name = train_labels[idx]\n    label = imread(labels_dir / train_labels[idx])\n    pixel_classes = np.unique(label//1000)\n    classes, instance_count = np.unique(pixel_classes, return_counts=True) # Courtesy:https://www.kaggle.com/jpmiller/cvpr-eda\n    data_dict = dict(zip(classes, instance_count))\n    df = pd.DataFrame.from_dict(data_dict, orient='index').transpose()\n    df.rename(columns=labelmap, inplace=True)\n    df['img'] = img_name\n    df['label'] = label_name\n    \n    # Concate to the final dataframe\n    #data_df = pd.concat([data_df, df], copy=False)\n    # append to the list of intermediate df list\n    df_list.append(df)\n    \ndata_df = pd.concat(df_list, axis=0)\ndel df_list\n\n# Fill the NaN with zero\ndata_df = data_df.fillna(0)\n\n# Rearrange the columns\ncols = data_df.columns.tolist()\ncols = [x for x in cols if x not in ['img', 'label']]\ncols = ['img', 'label'] + cols\ndata_df = data_df[cols]\n\n# Display the results\ndata_df = data_df.reset_index(drop=True)\ndata_df.head(10)    ","execution_count":6,"outputs":[]},{"metadata":{"_uuid":"2bf23ef7f9b13ec17f2d7cd27d5838d94fde56fa","_cell_guid":"2f99d724-1aed-47a9-a5dd-8e8baaa7786c","trusted":true},"cell_type":"code","source":"# Let's have a look at some of the images \nsample_images = (data_df['img'][300:305]).reset_index(drop=True)\nsample_labels = (data_df['label'][300:305]).reset_index(drop=True)\n\nf, ax = plt.subplots(5,3, figsize=(20,20))\nfor i in range(5):\n    img = imread(images_dir / sample_images[i])\n    label = imread(labels_dir / sample_labels[i]) // 1000\n    label[label!=0] = 255\n    blended_image = Image.blend(Image.fromarray(img), Image.fromarray(label).convert('RGB'), alpha=0.8)\n    \n    ax[i, 0].imshow(img, aspect='auto')\n    ax[i, 0].axis('off')\n    ax[i, 1].imshow(label, aspect='auto')\n    ax[i, 1].axis('off')\n    ax[i, 2].imshow(blended_image, aspect='auto')\n    ax[i, 2].axis('off')\nplt.show()","execution_count":7,"outputs":[]},{"metadata":{"_uuid":"7ffe0a9072cdd541ce6980ab95df7d873be6fffd","collapsed":true,"_cell_guid":"4146a506-f348-4065-90e0-4434c11ed8aa","trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d7c15f21c75cacce27de6b4361c886f17ab98e01","collapsed":true,"_cell_guid":"35668e76-f75f-4382-88cf-092ec2a2cb43"},"cell_type":"markdown","source":"That's it folks!! I hope you enjoyed it. I will try to update this as soon as possible. Please upvote if you liked it!! "},{"metadata":{"_uuid":"c51fbfe7750de098a0ba34672463a8d1bf53f61a","collapsed":true,"_cell_guid":"3407a0f5-0e0e-4bd4-9e01-b3afb231cfe4","trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}