{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <center> 🐳 Happy Whale Kernel 🐬 </center>\n<center><img src  = \"https://thumbs.dreamstime.com/b/colorful-underwater-world-whales-dolphin-colorful-underwater-world-whales-dolphin-swimming-octopus-176245123.jpg\" height = 600 width = 600 ></center>","metadata":{}},{"cell_type":"markdown","source":"# Introduction\n\nThis competition is a Vision Based Classification Competition . We are given the individual id's and species along with their images for Dolphins and Whales and we have to predict 5 individual id's for each image and the evaluation metric for the competition is MAP@5 (Mean Average Precision) .  \n\n<div class=\"alert alert-success\">\n <strong> Problem Statement :</strong> This is  quite a different competition than usual so let's understand why. So how do you classify between a bird, cat or a dog it's simple our eyes can differentiate it right away and they may consider features for differentition but now let's change the question now we have dogs 🐶 but of different breeds ok then still two breeds 🐩🐕‍may have some different attributes . Now let's take another scenario you and your friend had same breed dogs how you will differ between them .We may look for different patches , markings etc like in the case of human beings if they have same face even they will have difference in fingerprints.Let's pick up our Binoculars🔭 now. Let's explore the world of Whales 🐳 and Dolphins 🐬 . So whales and dolphins can be identified using shape and markings on their tails, dorsal fins, heads and other body parts.<br>\n </div>\n<center><img src = \"https://storage.googleapis.com/kaggle-competitions/kaggle/3333/media/happy-whale.jpg\" width = 400 height = 200></center><center>Credits : Humpback Whale Identification</center>","metadata":{}},{"cell_type":"markdown","source":"# 📚 Loading Libraries 📚 ","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport pandas as pd\nimport warnings\nfrom termcolor import colored\nwarnings.filterwarnings(\"ignore\")\nfrom PIL import Image\n\n%matplotlib inline","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-02-15T06:58:41.159030Z","iopub.execute_input":"2022-02-15T06:58:41.159312Z","iopub.status.idle":"2022-02-15T06:58:41.166396Z","shell.execute_reply.started":"2022-02-15T06:58:41.159283Z","shell.execute_reply":"2022-02-15T06:58:41.165845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading Data","metadata":{}},{"cell_type":"code","source":"train_data = '../input/happy-whale-and-dolphin/train_images'\ntest_data = '../input/happy-whale-and-dolphin/test_images'","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:41.516495Z","iopub.execute_input":"2022-02-15T06:58:41.517124Z","iopub.status.idle":"2022-02-15T06:58:41.521268Z","shell.execute_reply.started":"2022-02-15T06:58:41.517067Z","shell.execute_reply":"2022-02-15T06:58:41.520365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv('../input/happy-whale-and-dolphin/train.csv')\nprint(\"Shape of train Dataframe is : {} \".format(train_df.shape))\ntrain_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:41.682434Z","iopub.execute_input":"2022-02-15T06:58:41.683310Z","iopub.status.idle":"2022-02-15T06:58:41.755437Z","shell.execute_reply.started":"2022-02-15T06:58:41.683266Z","shell.execute_reply":"2022-02-15T06:58:41.754449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df['Path'] = '../input/happy-whale-and-dolphin/train_images/' + train_df['image']\ntrain_df","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:41.823819Z","iopub.execute_input":"2022-02-15T06:58:41.824091Z","iopub.status.idle":"2022-02-15T06:58:41.846728Z","shell.execute_reply.started":"2022-02-15T06:58:41.824061Z","shell.execute_reply":"2022-02-15T06:58:41.845821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.info()","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:42.000853Z","iopub.execute_input":"2022-02-15T06:58:42.001137Z","iopub.status.idle":"2022-02-15T06:58:42.024012Z","shell.execute_reply.started":"2022-02-15T06:58:42.001095Z","shell.execute_reply":"2022-02-15T06:58:42.023148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploratory Data Analysis\n\n### Let's explore the species first!","metadata":{}},{"cell_type":"code","source":"\nprint(len(train_df['species'].value_counts()))\ntrain_df['species'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:42.294143Z","iopub.execute_input":"2022-02-15T06:58:42.294382Z","iopub.status.idle":"2022-02-15T06:58:42.308315Z","shell.execute_reply.started":"2022-02-15T06:58:42.294356Z","shell.execute_reply":"2022-02-15T06:58:42.307217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### So there are 30 species in total in the training images\n<ul>\n<li>Beluga is a whale species here\n<li>Globis is also a whale specie\n<li>There are duplicates so we have to remove the duplicate species","metadata":{}},{"cell_type":"code","source":"print(colored('Before fixing duplicate labels : ','green'))\nprint(\"Number of unique species : \", train_df['species'].nunique())\nprint(\"\\nSpecies names:\" ,train_df['species'].unique())\n\n\n#fix duplicates\ntrain_df['species']  = train_df['species'].str.replace('bottlenose_dolpin','bottlenose_dolphin')\ntrain_df['species']  = train_df['species'].str.replace('kiler_whale','killer_whale')\nprint(colored('\\nAfter fixing duplicate labels : ','green'))\nprint(\"Number of unique species : \", train_df['species'].nunique())\nprint(\"\\nSpecies names:\" ,train_df['species'].unique())\n\ntrain_df['species'].replace({'beluga' : 'beluga_whale','globis' : 'globis_whale'},inplace =True)\n\n","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:42.540277Z","iopub.execute_input":"2022-02-15T06:58:42.541208Z","iopub.status.idle":"2022-02-15T06:58:42.625294Z","shell.execute_reply.started":"2022-02-15T06:58:42.541150Z","shell.execute_reply":"2022-02-15T06:58:42.624278Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (7,7))\nsns.countplot(data = train_df , y = 'species',palette = 'plasma');","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:42.676706Z","iopub.execute_input":"2022-02-15T06:58:42.677531Z","iopub.status.idle":"2022-02-15T06:58:43.076480Z","shell.execute_reply.started":"2022-02-15T06:58:42.677477Z","shell.execute_reply":"2022-02-15T06:58:43.075949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### The data contains the pictures of bottlenose dolphins the highest","metadata":{}},{"cell_type":"code","source":"train_df['D_or_W'] = ([i[-1] for i in train_df['species'].str.split('_')])\ntrain_df['D_or_W'].head()","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:43.077549Z","iopub.execute_input":"2022-02-15T06:58:43.078318Z","iopub.status.idle":"2022-02-15T06:58:43.135744Z","shell.execute_reply.started":"2022-02-15T06:58:43.078287Z","shell.execute_reply":"2022-02-15T06:58:43.135045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"explode = (0.05, 0.05)\nplt.pie(x  = train_df['D_or_W'].value_counts(),labels = ['Whale','Dolphin'],colors=[\"#0077b6\",\"#90e0ef\"],\n        textprops={'fontsize': 13},autopct = '%1.1f%%',explode = explode)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:43.136815Z","iopub.execute_input":"2022-02-15T06:58:43.137562Z","iopub.status.idle":"2022-02-15T06:58:43.221458Z","shell.execute_reply.started":"2022-02-15T06:58:43.137527Z","shell.execute_reply":"2022-02-15T06:58:43.220214Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### So there are more Whales in the dataset as compared to Dolphins","metadata":{}},{"cell_type":"code","source":"### Show one image for each species\nplt.figure(figsize = (15,12))\nfor idx,i in enumerate(train_df.species.unique()):\n    plt.subplot(4,7,idx+1)\n    df = train_df[train_df['species'] ==i].reset_index(drop = True)\n    image_path = df.loc[0,'Path']\n    img = Image.open(image_path)\n    img = img.resize((224,224))\n    plt.imshow(img)\n    plt.axis('off')\n    plt.title(i)\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:43.483741Z","iopub.execute_input":"2022-02-15T06:58:43.484036Z","iopub.status.idle":"2022-02-15T06:58:48.403602Z","shell.execute_reply.started":"2022-02-15T06:58:43.484002Z","shell.execute_reply":"2022-02-15T06:58:48.402747Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_species(df,species_name):\n    plt.figure(figsize = (10,10))\n    species_df = df[df['species'] ==species_name].reset_index(drop = True)\n    plt.suptitle(species_name)\n    for idx,i in enumerate(np.random.choice(species_df['Path'],9)):\n        plt.subplot(3,3,idx+1)\n        image_path = i\n        img = Image.open(image_path)\n        img = img.resize((224,224))\n        plt.imshow(img)\n        plt.axis('off')\n    plt.tight_layout()\n    plt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:48.405258Z","iopub.execute_input":"2022-02-15T06:58:48.405485Z","iopub.status.idle":"2022-02-15T06:58:48.411265Z","shell.execute_reply.started":"2022-02-15T06:58:48.405455Z","shell.execute_reply":"2022-02-15T06:58:48.410321Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_species(train_df , 'pygmy_killer_whale')","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:48.412343Z","iopub.execute_input":"2022-02-15T06:58:48.412561Z","iopub.status.idle":"2022-02-15T06:58:49.293038Z","shell.execute_reply.started":"2022-02-15T06:58:48.412526Z","shell.execute_reply":"2022-02-15T06:58:49.291915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_species(train_df , 'gray_whale')","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:49.295390Z","iopub.execute_input":"2022-02-15T06:58:49.295697Z","iopub.status.idle":"2022-02-15T06:58:52.345321Z","shell.execute_reply.started":"2022-02-15T06:58:49.295652Z","shell.execute_reply":"2022-02-15T06:58:52.344630Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Let's see the individual id's now!","metadata":{}},{"cell_type":"code","source":"print(\"Number of unique individual whales and dolphins : \",len(train_df['individual_id'].value_counts()))\ntrain_df['individual_id'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:52.346255Z","iopub.execute_input":"2022-02-15T06:58:52.346772Z","iopub.status.idle":"2022-02-15T06:58:52.374835Z","shell.execute_reply.started":"2022-02-15T06:58:52.346734Z","shell.execute_reply":"2022-02-15T06:58:52.373996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_individual(df,individual_id):\n    plt.figure(figsize = (10,10))\n    species_df = df[df['individual_id'] ==individual_id].reset_index(drop = True)\n    plt.suptitle(\"ID : {} specie : {}\".format(individual_id,species_df.species[0]))\n    for idx,i in enumerate(np.random.choice(species_df['Path'],9)):\n        plt.subplot(3,3,idx+1)\n        image_path = i\n        img = Image.open(image_path)\n        img = img.resize((224,224))\n        plt.imshow(img)\n        plt.axis('off')\n    plt.tight_layout()\n    plt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:52.376054Z","iopub.execute_input":"2022-02-15T06:58:52.376456Z","iopub.status.idle":"2022-02-15T06:58:52.382416Z","shell.execute_reply.started":"2022-02-15T06:58:52.376426Z","shell.execute_reply":"2022-02-15T06:58:52.381832Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_individual(train_df , '37c7aba965a5')","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:52.383555Z","iopub.execute_input":"2022-02-15T06:58:52.383958Z","iopub.status.idle":"2022-02-15T06:58:55.247820Z","shell.execute_reply.started":"2022-02-15T06:58:52.383915Z","shell.execute_reply":"2022-02-15T06:58:55.246996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_individual(train_df , 'a6e325d8e924')","metadata":{"execution":{"iopub.status.busy":"2022-02-15T06:58:55.249027Z","iopub.execute_input":"2022-02-15T06:58:55.249259Z","iopub.status.idle":"2022-02-15T06:58:58.244222Z","shell.execute_reply.started":"2022-02-15T06:58:55.249230Z","shell.execute_reply":"2022-02-15T06:58:58.243543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## <center> Work in Progress!🌊</center>\n## <center> Do Upvote! if you like the Notebook</center>\n## <center>Share your thoughts in comments!🤗</center>","metadata":{}}]}