\nHello! This notebook is made to help you explore the data easily and also for me to share some observations. \n \nIt is a good idea to fork, change the seed below and view more samples in plotly! 🍻\n
","metadata":{"execution":{"iopub.status.busy":"2022-02-13T09:26:28.236706Z","iopub.execute_input":"2022-02-13T09:26:28.237032Z","iopub.status.idle":"2022-02-13T09:26:28.244676Z","shell.execute_reply.started":"2022-02-13T09:26:28.237Z","shell.execute_reply":"2022-02-13T09:26:28.243353Z"}}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport plotly.express as px\nimport plotly.graph_objects as go\nfrom skimage import io\nfrom skimage.color import gray2rgb\nfrom skimage.transform import resize\nfrom rich.jupyter import print\n\nimport os\nimport torch\nfrom torch import nn\nimport torch.nn.functional as F\nfrom torch.utils.data import DataLoader, random_split, Dataset\nfrom torchvision import transforms\nimport pytorch_lightning as pl\nfrom typing import Tuple, List\n\nseed=400 # to make the notebook reproducible (including images)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":false,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-13T16:09:36.865961Z","iopub.execute_input":"2022-02-13T16:09:36.866535Z","iopub.status.idle":"2022-02-13T16:09:47.956664Z","shell.execute_reply.started":"2022-02-13T16:09:36.866408Z","shell.execute_reply":"2022-02-13T16:09:47.955556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Meta Data 📄","metadata":{}},{"cell_type":"code","source":"def get_train_csv(\n path:str=\"/kaggle/input/happy-whale-and-dolphin/train.csv\"\n )->Tuple[pd.DataFrame, pd.DataFrame]:\n # fix spelling of two species and merging subspecies, reduces the unique species from 30 -> 26\n train_df = pd.read_csv(path)\n train_df['species'].replace({'bottlenose_dolpin': 'bottlenose_dolphin', \n 'kiler_whale': 'killer_whale',\n 'pilot_whale': 'short_finned_pilot_whale',\n 'globis': 'short_finned_pilot_whale'\n }, inplace=True)\n\n # assign id to species\n train_df['species_id'], species_index = train_df['species'].factorize()\n return train_df, species_index\n\n# the extended csv is just original csv and shape info\ntrain_df, species_index = get_train_csv(path=\"/kaggle/input/happywhale-extended-meta/train_extended.csv\")\ntrain_df.describe(include='all').loc[['count', 'unique', 'freq', 'top']]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-13T16:09:47.958691Z","iopub.execute_input":"2022-02-13T16:09:47.958977Z","iopub.status.idle":"2022-02-13T16:09:48.209587Z","shell.execute_reply.started":"2022-02-13T16:09:47.958939Z","shell.execute_reply":"2022-02-13T16:09:48.208767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"# Sample distributions 📈","metadata":{}},{"cell_type":"code","source":"# gets nice gradient, also avoid repeating colros\ncolors = ['hsl('+str(h)+',50%'+',50%)' for h in np.linspace(0, 320, len(species_index))] # https://plotly.com/python/box-plots/\n\n# samples per species\nfig = px.histogram(train_df, 'species', color='d2', labels={'d2': '# channels'},\n title='samples per species').update_xaxes(categoryorder='total descending')\nfig.show()\n\nprint(f\"[blue]🌟 {len(train_df[train_df['d2']==1])} out of {len(train_df)} images are single channel\", justify='center')\nn_only_single_channel = train_df['individual_id'].nunique() - train_df[train_df['d2'] == 3]['individual_id'].nunique()\nprint(f\"[blue]🌟 {n_only_single_channel} individuals have only single channel samples\", justify='center')\n\n# individuals per species\nuniques_df = train_df.groupby('species', sort=False)['individual_id'].nunique().reset_index(name='uniques')\nfig = px.histogram(uniques_df, x='species', y='uniques', color_discrete_sequence=['#FF6692'], title='individual per species').update_xaxes(categoryorder='total descending')\nfig.show()\n\n# individual frequency distribution\ncounts_df = train_df.groupby('species', sort=False)['individual_id'].value_counts().reset_index(name='frequency')\nfig = px.histogram(counts_df, x='frequency', color_discrete_sequence=['#00CC96'], title='individual frequency distribution', log_y=True)\nfig.show()\n\n# individual frequencies across species\nfn = lambda s: pd.Series({\"frequency\": list(s[\"frequency\"]),\"id\": list(s[\"individual_id\"])})\nspecies_freq = counts_df.groupby(['species'], sort=False).apply(fn).reset_index()\n\nfig = go.Figure()\nfor i in range(len(species_freq)):\n box = go.Box(y=species_freq['frequency'][i],\n name=species_freq['species'][i],\n hovertext=species_freq['id'][i],\n jitter=1,\n marker=dict(color = colors[i], size=2),\n boxpoints='all', \n pointpos=0, # hide box\n fillcolor='rgba(0,0,0,0)',\n line_width=0)\n fig.add_trace(box)\n \nfig.update_layout(\n yaxis_type=\"log\",\n title=\"individual frequencies across species\",\n yaxis_title=\"frequency of an individual\",\n showlegend=False)\nfig.show()\n\ndel uniques_df","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-13T16:09:48.212519Z","iopub.execute_input":"2022-02-13T16:09:48.212996Z","iopub.status.idle":"2022-02-13T16:09:50.250120Z","shell.execute_reply.started":"2022-02-13T16:09:48.212957Z","shell.execute_reply":"2022-02-13T16:09:50.249438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### To keep in mind:\n- Should handle the observed class imbalance\n- ~ 17% of the individuals have only one instance, i.e no other positive sample","metadata":{"execution":{"iopub.status.busy":"2022-02-05T10:17:09.5886Z","iopub.execute_input":"2022-02-05T10:17:09.588857Z","iopub.status.idle":"2022-02-05T10:17:09.59461Z","shell.execute_reply.started":"2022-02-05T10:17:09.588824Z","shell.execute_reply":"2022-02-05T10:17:09.593579Z"}}},{"cell_type":"markdown","source":"# Species images - Plotly 🐬 🐋","metadata":{}},{"cell_type":"code","source":"def get_image_array(paths: List[str], shape: Tuple=(300,300), image_folder:str=\"/kaggle/input/happy-whale-and-dolphin/train_images\") -> np.array:\n \"\"\"read all images as a single array for plotly.\"\"\"\n read_img = lambda file: io.imread(f\"{image_folder}/{file}\")\n images = []\n for x in paths:\n img = read_img(x)\n if len(img.shape) == 2: \n img = gray2rgb(img)\n images.append(resize(img, shape))\n return np.asarray(images)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-13T16:09:50.251279Z","iopub.execute_input":"2022-02-13T16:09:50.252005Z","iopub.status.idle":"2022-02-13T16:09:50.258732Z","shell.execute_reply.started":"2022-02-13T16:09:50.251962Z","shell.execute_reply":"2022-02-13T16:09:50.257887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get a sample from all species\nsamples = train_df.groupby('species', sort=False).apply(lambda df: df.sample(1, random_state=seed)).droplevel(0)\nsample_images = get_image_array(samples['image'])\n\n# plot image grid\nfig = px.imshow(sample_images, facet_col=0, binary_string=True, facet_col_wrap=10, facet_row_spacing=0.0, facet_col_spacing=0)\nfig.update_xaxes(showticklabels=False).update_yaxes(showticklabels=False)\n\n# add species name and individual_id\nfor i, a in enumerate(fig.layout.annotations):\n a.text = samples.iloc[i].species + \" \" + samples.iloc[i].individual_id + \" \" + samples.iloc[i].image + \" \" + str(samples.iloc[i][['d0', 'd1', 'd2']].tolist())\nfig.update_layout(title=\"Samples from all the species\", autosize=True, hovermode=False, height=900, width=1500, margin=dict(l=0, r=0, t=150, b=80))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-13T16:09:50.259972Z","iopub.execute_input":"2022-02-13T16:09:50.260193Z","iopub.status.idle":"2022-02-13T16:10:10.626978Z","shell.execute_reply.started":"2022-02-13T16:09:50.260165Z","shell.execute_reply":"2022-02-13T16:10:10.625920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Individual samples 🔍🕵\n\nHere we see all samples of an individual with fewer total samples. This might help us a better understanding on what information we have about an individual and what the model should focus on during training to achieve better results. \n\nWe can encourage that at the data creation step by answering if it makes more sense to \n- use the whole image \n- crop the whole animal, using shark/dolhin detector\n- crop the fins alone, by training a fin detector (seems like what the competition description points to)\n- ignore the images that doesnt have a nice view of the fins \n- ignore images that are too big for individuals with very high samples and so on.. 🤷🏻♂️\n- Too many samples for an individual might hurt as they maybe from various time periods, thus confusing the models! 🙀","metadata":{}},{"cell_type":"code","source":"def samples_with_freq(df:pd.DataFrame, freq:int, seed:int, n_samples:int=1, verbose:bool=True):\n \"\"\"Get samples of an id with particular frequency from every species\"\"\"\n samples = []\n n_total = 0\n n_sampled = 0\n for name, g in df.groupby('species'):\n subset = g[g['frequency']==freq]\n n_total += len(subset)\n if len(subset) >= n_samples: \n n_sampled += n_samples\n samples.append(subset.sample(n_samples, random_state=seed))\n \n if verbose: print(f\"['info'] samples_with_freq {freq} - sampled {n_sampled}/{n_total} [{100*n_sampled/n_total:.2f}%]\")\n \n return pd.concat(samples)","metadata":{"execution":{"iopub.status.busy":"2022-02-13T16:10:10.628237Z","iopub.execute_input":"2022-02-13T16:10:10.628562Z","iopub.status.idle":"2022-02-13T16:10:10.634336Z","shell.execute_reply.started":"2022-02-13T16:10:10.628532Z","shell.execute_reply":"2022-02-13T16:10:10.633551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ⭐️ Single instance 🐬","metadata":{}},{"cell_type":"code","source":"# ids with frequncy 1\nn_display = 10\nfreq = 1\n# one sample per species\nids = samples_with_freq(counts_df, freq=freq, seed=seed, n_samples=1)['individual_id']\nsamples = train_df.loc[train_df['individual_id'].isin(ids.sample(n_display, random_state=seed).tolist())]\nprint(f\"['info'] displaying {n_display} ids\")\nsample_images = get_image_array(samples['image'])\n\n# plot image grid\nfig = px.imshow(sample_images, facet_col=0, binary_string=True, facet_row_spacing=0.0, facet_col_spacing=0)\nfig.update_xaxes(showticklabels=False).update_yaxes(showticklabels=False)\n\n# add species name and individual_id\nfor i, a in enumerate(fig.layout.annotations):\n a.text = samples.iloc[i].species + \" \" + samples.iloc[i].individual_id + \" \" + samples.iloc[i].image + \" \" + str(samples.iloc[i][['d0', 'd1', 'd2']].tolist())\nfig.update_layout(title=f\"Individuals with {freq} instance\", autosize=True, hovermode=False, height=400, width=1500, margin=dict(l=40, r=0, t=140, b=80))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-13T16:10:10.635654Z","iopub.execute_input":"2022-02-13T16:10:10.636325Z","iopub.status.idle":"2022-02-13T16:10:18.440426Z","shell.execute_reply.started":"2022-02-13T16:10:10.636275Z","shell.execute_reply":"2022-02-13T16:10:18.439755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
Scroll right to see more images!\n
👉🏼
\n
","metadata":{}},{"cell_type":"markdown","source":"### Observations\n- As the output suggests, all the species have an idividual with 1 instance.\n- We should learn to identify these individuals without any positive pair! 😥\n- There is good view of the fin! 😇\n- It seems hard to identidy the species by just looking at the fin 😕, for example, spotted dolphin doesnt seem to have spotted fin. \n - Important, if you want to do candidate selection based on initial species classification. ⚠️\n- Super zoomed out images, blind downsampling would hurt picking the individual's features ✂️\n- Let's strengthen these observations after analyzing the test images as well! (in the later sections)","metadata":{}},{"cell_type":"markdown","source":"
\nI would encourage you to fork, change the seed or remove the random_state in the `samples_with_freq` method and explore more samples to see if these observations hold and make sure that they are generalizable
","metadata":{}},{"cell_type":"markdown","source":"# ⭐️ Many (10) instances 🐬🐬🐬","metadata":{}},{"cell_type":"code","source":"# ids with frequncy 10\nn_display = 10\nfreq = 10\n# one sample per species\nids = samples_with_freq(counts_df, freq=freq, seed=seed, n_samples=1)['individual_id']\nsamples = train_df.loc[train_df['individual_id'].isin(ids.sample(n_display, random_state=seed).tolist())]\nprint(f\"['info'] displaying {n_display} ids\")\n\nid_groups = samples.groupby('individual_id')\nif len(id_groups) != n_display: \n print(\"!!! Something is wrong when grouping same individuals, plot is wrong!\")\n \n# plot image grid\nfor individual_id, samples in id_groups: # samples is no longer the whole df\n sample_images = get_image_array(samples['image'])\n \n fig = px.imshow(sample_images, facet_col=0, binary_string=True, facet_row_spacing=0.0, facet_col_spacing=0)\n fig.update_xaxes(showticklabels=False).update_yaxes(showticklabels=False)\n \n # add species name and individual_id\n for i, a in enumerate(fig.layout.annotations):\n a.text = samples.iloc[i].image + \" \" + str(samples.iloc[i][['d0', 'd1', 'd2']].tolist())\n fig.update_layout(title=f\"Instances of {individual_id} - {samples['species'].iloc[0]}\", autosize=True, hovermode=False, height=350, width=1500, margin=dict(l=40, r=0, t=140, b=0))\n fig.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-13T16:10:18.441440Z","iopub.execute_input":"2022-02-13T16:10:18.441959Z","iopub.status.idle":"2022-02-13T16:11:59.692241Z","shell.execute_reply.started":"2022-02-13T16:10:18.441923Z","shell.execute_reply":"2022-02-13T16:11:59.691280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations \n- Adding to the observations from the single instance section, its safe to say just cropping the fins (contrary to the initial assumption made in the initial sections), might be a bad idea.\n- Cues to easily identify the individual can be seen on the body (color) and not just the fins (pattern)\n- Varying colors of the water and the image tone\n - Less likely that the water color would be leak or clue to identify the individual\n- It wouldnt be a bad idea to not crop at all. As some individuals seems to be shot at the same distance. \n - Probably all shot in a day? 😅","metadata":{}},{"cell_type":"markdown","source":"# ⭐️ *Way too many (50!)* instances 🥴","metadata":{"execution":{"iopub.status.busy":"2022-02-11T06:42:24.856538Z","iopub.execute_input":"2022-02-11T06:42:24.857032Z","iopub.status.idle":"2022-02-11T06:42:24.862806Z","shell.execute_reply.started":"2022-02-11T06:42:24.856994Z","shell.execute_reply":"2022-02-11T06:42:24.862019Z"}}},{"cell_type":"code","source":"# ids with frequncy 50\nn_display = 1\nfreq = 50\n# one sample per species\nids = samples_with_freq(counts_df, freq=freq, seed=seed, n_samples=1)['individual_id']\nsamples = train_df.loc[train_df['individual_id'].isin(ids.sample(n_display, random_state=seed).tolist())]\nprint(f\"['info'] displaying {n_display} ids\")\n\nid_groups = samples.groupby('individual_id')\nif len(id_groups) != n_display: \n print(\"!!! Something is wrong when grouping same individuals, plot is wrong!\")\n \n# plot image grid\nfor individual_id, samples in id_groups: # samples is no longer the whole df\n sample_images = get_image_array(samples['image'])\n \n fig = px.imshow(sample_images, facet_col=0, facet_col_wrap=10, binary_string=True, facet_row_spacing=0.0, facet_col_spacing=0)\n fig.update_xaxes(showticklabels=False).update_yaxes(showticklabels=False)\n \n # add species name and individual_id\n for i, a in enumerate(fig.layout.annotations):\n a.text = samples.iloc[i].image + \" \" + str(samples.iloc[i][['d0', 'd1', 'd2']].tolist())\n fig.update_layout(title=f\"Instances of {individual_id} - {samples.iloc[0].species}\", autosize=True, hovermode=False, height=1000, width=1500, margin=dict(l=40, r=0, t=120, b=10))\n fig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-13T16:11:59.693484Z","iopub.execute_input":"2022-02-13T16:11:59.694173Z","iopub.status.idle":"2022-02-13T16:12:52.021349Z","shell.execute_reply.started":"2022-02-13T16:11:59.694133Z","shell.execute_reply":"2022-02-13T16:12:52.020270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations\n- Why not visualize individuals with 100 or 400 samples?\n - 1. too long to viz :) 2. They are outliers, but there are many ids with freq 20-50.\n- Obviously there are some extremely zoomed out images of the whale. ID-ing it aside, there is no way to even classify it as a whale or a duck! 😅","metadata":{}},{"cell_type":"markdown","source":"# Test dataset","metadata":{}},{"cell_type":"code","source":"# read test images\nn_samples = 50\ntest_image_folder = \"/kaggle/input/happy-whale-and-dolphin/test_images/\"\nsubmission_df = pd.read_csv(\"/kaggle/input/happywhale-extended-meta/sample_submission_extended.csv\")\nsamples = submission_df.sample(n_samples, random_state=seed)\n\nprint(f\"[blue]🌟 {len(submission_df[submission_df['d2']==1])} out of {len(submission_df)} images are single channel\", justify='center')","metadata":{"execution":{"iopub.status.busy":"2022-02-13T16:16:06.793392Z","iopub.execute_input":"2022-02-13T16:16:06.794127Z","iopub.status.idle":"2022-02-13T16:16:06.839436Z","shell.execute_reply.started":"2022-02-13T16:16:06.794081Z","shell.execute_reply":"2022-02-13T16:16:06.838876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_images = get_image_array(paths=samples['image'], image_folder=test_image_folder)\n\n# plot image grid\nfig = px.imshow(sample_images, facet_col=0, binary_string=True, facet_col_wrap=10, facet_row_spacing=0.0, facet_col_spacing=0)\nfig.update_xaxes(showticklabels=False).update_yaxes(showticklabels=False)\n\n# add species name and individual_id\nfor i, a in enumerate(fig.layout.annotations):\n a.text = samples.image.iloc[i] + \" \" + str(samples.iloc[i][['d0', 'd1', 'd2']].tolist())\nfig.update_layout(title=f\"Random test images\", autosize=True, hovermode=False, height=1000, width=1500, margin=dict(l=40, r=0, t=120, b=10))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-13T16:12:52.099235Z","iopub.execute_input":"2022-02-13T16:12:52.099859Z","iopub.status.idle":"2022-02-13T16:13:51.272139Z","shell.execute_reply.started":"2022-02-13T16:12:52.099802Z","shell.execute_reply":"2022-02-13T16:13:51.270919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Observations\n- From the random samples we can mostly see decent images but few are too far away and cant be seen at all? Please explore more here.","metadata":{}},{"cell_type":"markdown","source":"
\nNow its time to put these insights into good use. Thanks for reading, good luck! 🍀\n