{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Happy Whalphins\n\nCurrently, most research institutions rely on time-intensive—and sometimes inaccurate—manual matching by the human eye. Thousands of hours go into manual matching, which involves staring at photos to compare one individual to another, finding matches, and identifying new individuals. While researchers enjoy looking at a whale photo or two, manual matching limits the scope and reach.\n\nIn this competition, we are asked to develop a model to match individual whales and dolphins by unique—but often subtle—characteristics of their natural markings. You'll pay particular attention to dorsal fins and lateral body views in image sets from a multi-species dataset built by 28 research institutions. The best submissions will suggest photo-ID solutions that are fast and accurate.","metadata":{}},{"cell_type":"markdown","source":"# Load the Dataset","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport cv2\n# import imagesize","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-02-06T04:42:53.673800Z","iopub.execute_input":"2022-02-06T04:42:53.675072Z","iopub.status.idle":"2022-02-06T04:42:55.235697Z","shell.execute_reply.started":"2022-02-06T04:42:53.674963Z","shell.execute_reply":"2022-02-06T04:42:55.234674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/happy-whale-and-dolphin/train.csv')\nsample_submission_df = pd.read_csv('/kaggle/input/happy-whale-and-dolphin/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:44:15.366398Z","iopub.execute_input":"2022-02-06T04:44:15.367146Z","iopub.status.idle":"2022-02-06T04:44:15.483206Z","shell.execute_reply.started":"2022-02-06T04:44:15.367101Z","shell.execute_reply":"2022-02-06T04:44:15.482498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:43:36.893318Z","iopub.execute_input":"2022-02-06T04:43:36.893690Z","iopub.status.idle":"2022-02-06T04:43:36.916260Z","shell.execute_reply.started":"2022-02-06T04:43:36.893650Z","shell.execute_reply":"2022-02-06T04:43:36.915622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:44:25.530694Z","iopub.execute_input":"2022-02-06T04:44:25.531050Z","iopub.status.idle":"2022-02-06T04:44:25.541921Z","shell.execute_reply.started":"2022-02-06T04:44:25.531008Z","shell.execute_reply":"2022-02-06T04:44:25.540827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Whale and Dholpins\nExplore the data of whales and dholpins.","metadata":{}},{"cell_type":"code","source":"train_df.loc[train_df.species.str.contains('beluga'), 'species'] = 'beluga_whale'\ntrain_df.loc[train_df.species.str.contains('globis'), 'species'] = 'globis_whale'\ntrain_df['class'] = train_df.species.map(lambda x: 'whale' if 'whale' in x else 'dolphin')\ntrain_df['species'] = train_df['species'].str.replace('bottlenose_dolpin','bottlenose_dolphin')\ntrain_df['species'] = train_df['species'].str.replace('kiler_whale','killer_whale')","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:47:28.870258Z","iopub.execute_input":"2022-02-06T04:47:28.870935Z","iopub.status.idle":"2022-02-06T04:47:29.055509Z","shell.execute_reply.started":"2022-02-06T04:47:28.870888Z","shell.execute_reply":"2022-02-06T04:47:29.054701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = train_df.groupby([\"class\"])[\"species\"].nunique()\ndf = pd.DataFrame({'Classes': temp.index,\n                   'Species': temp.values\n                  })\ndf = df.sort_values(['Species'], ascending=False)\nplt.figure(figsize = (6,6))\nplt.title('Species distribution - grouped on Dolphins and Whales - train dataset')\nsns.set_color_codes(\"pastel\")\ns = sns.barplot(x = 'Classes', y=\"Species\", data=df)\ns.set_xticklabels(s.get_xticklabels(),rotation=90)\nlocs, labels = plt.xticks()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:47:42.974873Z","iopub.execute_input":"2022-02-06T04:47:42.975331Z","iopub.status.idle":"2022-02-06T04:47:43.184675Z","shell.execute_reply.started":"2022-02-06T04:47:42.975298Z","shell.execute_reply":"2022-02-06T04:47:43.183806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize=(7, 7))\nsns.kdeplot(np.log(train_df.loc[train_df[\"class\"]=='whale'].individual_id.value_counts()))\nsns.kdeplot(np.log(train_df.loc[train_df[\"class\"]=='dolphin'].individual_id.value_counts()))\nax.legend(labels=['whale', 'dolphin'])\nplt.title(\"Logaritmic distribution of individual_id frequency in images\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:56:04.559368Z","iopub.execute_input":"2022-02-06T04:56:04.559771Z","iopub.status.idle":"2022-02-06T04:56:04.969272Z","shell.execute_reply.started":"2022-02-06T04:56:04.559728Z","shell.execute_reply":"2022-02-06T04:56:04.968294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Samples Images","metadata":{}},{"cell_type":"code","source":"def plot_image_samples(species):\n    root_path = \"/kaggle/input/happy-whale-and-dolphin/\"\n    fig.subplots_adjust(hspace = .1, wspace=.1)\n    images_folder=\"train_images/\"\n    df = train_df[train_df['species']==species].copy()\n    df.index = range(len(df.index))\n\n    f, ax = plt.subplots(4, 4, figsize=(16,16))\n\n    for i in range(16):\n        file = df.loc[i, 'image']\n        species = df.loc[i, 'species']\n        identifier = df.loc[i, 'individual_id']\n        img = cv2.imread(root_path+images_folder+file)\n        ax[i//4, i%4].imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))\n        ax[i//4, i%4].set_title(identifier+\" (\"+species+\")\")\n        ax[i//4, i%4].axis('off')","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:56:10.468768Z","iopub.execute_input":"2022-02-06T04:56:10.469459Z","iopub.status.idle":"2022-02-06T04:56:10.480617Z","shell.execute_reply.started":"2022-02-06T04:56:10.469382Z","shell.execute_reply":"2022-02-06T04:56:10.479145Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_image_samples(\"bottlenose_dolphin\")","metadata":{"execution":{"iopub.status.busy":"2022-02-06T04:56:13.622350Z","iopub.execute_input":"2022-02-06T04:56:13.623220Z","iopub.status.idle":"2022-02-06T04:56:30.902989Z","shell.execute_reply.started":"2022-02-06T04:56:13.623165Z","shell.execute_reply":"2022-02-06T04:56:30.901681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Thank you for visiting this notebook!\n\nIf you like this notebook, you know it is <b>FREE</b> to click the upvote button.\n\nThanks for reading this notebook. If you have any feedback or comments please write it down the comment section below.","metadata":{}},{"cell_type":"markdown","source":"# References\n\n[Happy Whales and Dolphins](https://www.kaggle.com/gpreda/happy-whales-and-dolphins/notebook)</br>\n[Read, Display and Write an Image using OpenCV](https://learnopencv.com/read-display-and-write-an-image-using-opencv/)","metadata":{}}]}