{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Landmark Recognition\n\n**In this notebook, I will try to perform exploratory data analysis (EDA) on this dataset. As I am a beginner myself, I will try to explain the findings as much as possible. Please give your valuable opinions and suggestions in the comments.**\n\nThe following notebooks helped me a lot to write this notebook:\n1. https://www.kaggle.com/chirag9073/landmark-recognition-exploratory-data-analysis/notebook\n2. https://www.kaggle.com/azaemon/mura-classification\n\nCheck out my other notebooks if interested: https://www.kaggle.com/azaemon/notebooks \n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**At first, Let's import the modules.**","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport plotly.offline as py\nimport plotly.express as px\nimport plotly.graph_objs as go\nimport plotly.tools as tls#visualization\nimport plotly.figure_factory as ff#visualization\nimport matplotlib.image as mpimg\n\n# Set Color Palettes for the notebook (https://color.adobe.com/)\ncolors_nude = ['#FFE61A','#B2125F','#FF007B','#14B4CC','#099CB3']\nsns.palplot(sns.color_palette(colors_nude))\n\n# Set Style\nsns.set_style(\"whitegrid\")\nsns.despine(left=True, bottom=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Part 1: EDA","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Load the training csv file and look at the first 5 entries**","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train_data= pd.read_csv(\"../input/landmark-recognition-2020/train.csv\")\nprint(train_data.head())\nprint()\nprint(\"Here, id means Image Id\\n      landmark_id points to a specific ID of the landmark \")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Let's see the summary of the loaded data**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now, Let's see if the data contains any missing value in the csv","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(train_data.isna().sum())\nprint()\nprint('Here, we can see there is no missing data in any of the columns.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Now let's do an EDA by using the *basic_image_eda* library**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install basic_image_eda\nfrom basic_image_eda import BasicImageEDA","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are total 1580470 images in the train folder. To run the following operation, it estimated about 9 hours which we do not have. That's why I am applying this only for one of the subfolders.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"data_dir = \"../input/landmark-recognition-2020/train/0\"\nextensions = ['png', 'jpg', 'jpeg']\nthreads = 0\ndimension_plot = True\nchannel_hist = True\nnonzero = False\nhw_division_factor = 1.0\n\nBasicImageEDA.explore(data_dir, extensions, threads, dimension_plot, channel_hist, nonzero, hw_division_factor)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now, let's analyze the number of landmark types and their distributions.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data['landmark_id'].value_counts()\nprint(\"Types of Landmarks: 81313\")\nprint(\"Landmark ID: 138982 has the highest number of images (6272)\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Most frequent landmark counts (Top 10)**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Occurance of landmark_id in decreasing order(Top categories)\ntemp = pd.DataFrame(train_data.landmark_id.value_counts().head(10))\ntemp.reset_index(inplace=True)\ntemp.columns = ['Landmark ID','Number of Images']\n\n# Plot the most frequent landmark_ids\nplt.figure(figsize = (9, 10))\nplt.title('Top 10 the mostfrequent landmarks')\nsns.set_color_codes(\"deep\")\nsns.barplot(x=\"Landmark ID\", y=\"Number of Images\", data=temp,\n            label=\"Count\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Least frequent landmark counts (Top 10)**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"temp = pd.DataFrame(train_data.landmark_id.value_counts().tail(10))\ntemp.reset_index(inplace=True)\ntemp.columns = ['Landmark ID','Number of Images']\n# Plot the least frequent landmark_ids\nplt.figure(figsize = (9, 10))\nplt.title('Top 10 the least frequent landmarks')\nsns.set_color_codes(\"deep\")\nsns.barplot(x=\"Landmark ID\", y=\"Number of Images\", data=temp,\n            label=\"Count\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Let's see another distribution plot**\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import cufflinks as cf\ncf.go_offline()\ncf.set_config_file(offline=False, world_readable=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data['landmark_id'].value_counts(normalize=True).sort_values().iplot(kind='barh',\n                                                      xTitle='Percentage', \n                                                      linecolor='black', \n                                                      opacity=0.7,\n                                                      color='blue',\n                                                      theme='pearl',\n                                                      bargap=0.2,\n                                                      gridcolor='white',\n                                                      title='Distribution in the training set')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's a KDE (Kernel Density Estimate) plot for the landmark_ids.\n\nA kernel density estimate plot shows the distribution of a single variable and can be thought of as a smoothed histogram","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.distplot(temp['Landmark ID'], hist=True, rug=True);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's plot some random images","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from random import randrange\nfig= plt.figure(figsize=(20,10))\nindex= '../input/landmark-recognition-2020/train/7/0/4/704001a0be55059a.jpg'\na= fig.add_subplot(2,3,1)\na.set_title(index.split(\"/\")[-1])\nplt.imshow(plt.imread(index))\n\nindex= '../input/landmark-recognition-2020/train/7/0/4/7040a5cfa43e0633.jpg'\na= fig.add_subplot(2,3,2)\na.set_title(index.split(\"/\")[-1])\nplt.imshow(plt.imread(index))\n\nindex= '../input/landmark-recognition-2020/train/7/0/0/7000542ecac029aa.jpg'\na= fig.add_subplot(2,3,3)\na.set_title(index.split(\"/\")[-1])\nplt.imshow(plt.imread(index))\n\nindex= '../input/landmark-recognition-2020/train/7/0/5/7050308f31e8f117.jpg'\na= fig.add_subplot(2,3,4)\na.set_title(index.split(\"/\")[-1])\nplt.imshow(plt.imread(index))\n\nindex= '../input/landmark-recognition-2020/train/7/0/a/70a039ff5015a267.jpg'\na= fig.add_subplot(2,3,5)\na.set_title(index.split(\"/\")[-1])\nplt.imshow(plt.imread(index))\n\nindex= '../input/landmark-recognition-2020/train/7/0/f/70f0333a732c666d.jpg'\na= fig.add_subplot(2,3,6)\na.set_title(index.split(\"/\")[-1])\nplt.imshow(plt.imread(index))\n\nplt.show()\n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Part 2: Data augmentation\n\n**Data augmentation is a strategy that enables practitioners to significantly increase the diversity of data available for training models, without actually collecting new data. In this notebook we will use *albumentations* data augmentation**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's import the modules","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from albumentations import (\n    Compose, HorizontalFlip, CLAHE, HueSaturationValue,\n    RandomBrightness, RandomContrast, RandomGamma,\n    ToFloat, ShiftScaleRotate, ElasticTransform,ChannelShuffle\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**There are various types of augmentation techniques available. In this case,  we applied:**\n\n1. Horizontal Flip\n2. Random contrast\n3. Random Gamma\n4. Random Brightness\n5. Shift Scale Rotate\n6. Channel Shuffle\n7. Elastic Transform\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"albumentation_list =  [\n    HorizontalFlip(p=0.5),\n    RandomContrast(limit=0.2, p=1),\n    RandomGamma(gamma_limit=(80, 120), p=1),\n    RandomBrightness(limit=0.2, p=0.5),\n    ShiftScaleRotate(\n        shift_limit=0.0625, scale_limit=0.1, \n        rotate_limit=15, border_mode=cv2.BORDER_REFLECT_101, p=0.8), \n    ChannelShuffle(p=1),\n    ElasticTransform(p=1,border_mode=cv2.BORDER_REFLECT_101,alpha_affine=40)\n]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Apply the augmentations on a Random image**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"chosen_image= plt.imread('../input/landmark-recognition-2020/train/0/4/3/04305edef6cf2186.jpg')\nimg_matrix_list = []\nbboxes_list = []\nfor aug_type in albumentation_list:\n    img = aug_type(image = chosen_image)['image']\n    img_matrix_list.append(img)\n    \nimg_matrix_list.insert(0,chosen_image)    \n\ntitles_list = [\"Original\",\"Horizontal Flip\",\"Random Contrast\",\"Random Gamma\",\"RandomBrightness\",\n               \"Shift Scale Rotate\",\"Channel Shuffle\", \"Elastic Transform\"]\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Function for plotting images**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_multiple_img(img_matrix_list, title_list, ncols, main_title=\"Data Augmentation\"):\n    fig, myaxes = plt.subplots(figsize=(20, 15), nrows=2, ncols=ncols, squeeze=True)\n    fig.suptitle(main_title, fontsize = 30)\n    #fig.subplots_adjust(wspace=0.3)\n    #fig.subplots_adjust(hspace=0.3)\n    for i, (img, title) in enumerate(zip(img_matrix_list, title_list)):\n        myaxes[i // ncols][i % ncols].imshow(img)\n        myaxes[i // ncols][i % ncols].set_title(title, fontsize=15)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Plot the augmented images**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_multiple_img(img_matrix_list, titles_list, ncols = 4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}