{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![header](https://storage.googleapis.com/kaggle-competitions/kaggle/22962/logos/header.png?t=2021-03-17-22-44-09)","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n Introduction \n

\n
\n\nData in this competition contains images of over 15,000 unique individual marine mammals from 30 different species collected from 28 different research organizations. Individuals have been manually identified and given an individual_id by marine researches.\n
\n\n

\nObjective\n

\n
\n\nYour task is to correctly identify these individuals in images. It's a challenging task that has the potential to drive significant advancements in understanding and protecting marine mammals across the globe.\n
\n\n

\n Methodology \n

\n
\n\nIntroducing Species\n\nRead Data & View Images\n\nData Cleaning\n\nExploratory Data Analysis\n\nImage Transformation\n\nImage Augmentation\n\n***In Progress...***","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n Import Libraries \n

\n
","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport cv2\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport os\nfrom tqdm import tqdm\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Used Functions \n

\n
","metadata":{}},{"cell_type":"code","source":"# Function from https://www.kaggle.com/heye0507/happy-whale-eda-baseline\nPATH = '../input/happy-whale-and-dolphin'\ndef plot_n_samples(species,row=5,path=PATH,folder='train_images'):\n path = PATH + '/' + folder\n n = row * row\n df1 = df[df['species'] == species]\n \n _,ax = plt.subplots(row,row,figsize=(15,10))\n \n for i in range(n):\n idx = np.random.randint(len(df1))\n fname = df1.iloc[idx]['image']\n label = df1.iloc[idx]['individual_id']\n img = cv2.imread(path + '/' + fname)\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n ax[i//row][i%row].imshow(img)\n ax[i//row][i%row].set_title(label + '-' + species)\n ax[i//row][i%row].axis('off')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Read Data \n

\n
","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(\"../input/happy-whale-and-dolphin/train.csv\")\ndf","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Number of Unique Individual IDs: \", df.individual_id.nunique())\nprint(\"Number of Unique Species: \", df.species.nunique())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.individual_id.value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.species.value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Getting To Know Every Specie! \n

\n
\n\n
\n\n

\n \nWe need to pay particular attention to dorsal fins and lateral body views (to the side of, or away from, or the middle of the body) \n

\n
","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n 🐬 Bottlenose Dolphin \n

\n
\n\n![bottlenose](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-dolphin_bottlenose_nb_w.png?itok=e6t8QRC9)\n\n▪️ They are generally gray in color. They can range from light gray to almost black on top near their dorsal fin and light gray to almost white on their belly.\n\n▪️ They are easy to view in the wild because they live close to shore and are distributed throughout coastal and estuarine waters.\n\nhttps://www.fisheries.noaa.gov/species/common-bottlenose-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n 🐳 Beluga \n

\n
\n\n![beluga](https://media.istockphoto.com/illustrations/watercolor-beluga-whale-illustration-isolated-on-white-background-illustration-id1227138495?k=20&m=1227138495&s=170667a&w=0&h=SlYhWGMejmAixWtRzo68wtU5EGuAe9Mnqa0ysIBIVzo=)\n\n▪️ The beluga, or white whale, is one of the smallest species of whale. Their distinctive color and prominent foreheads make them easily identifiable.\n\n▪️ The beluga has a very flexible neck that enables it to nod and turn its head in all directions.\n\n\nhttps://kids.nationalgeographic.com/animals/mammals/facts/beluga-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n 🐬Dusky Dolphin \n

\n
\n\n![dusky](https://ars.els-cdn.com/content/image/3-s2.0-B9780128043271001114-f04-24-9780128043271.jpg)\n\n▪️ Often several dusky dolphins jump together in the same patch of the ocean but facing in multiple directions,\n\n▪️ Dusky’s look like they are wearing eyemasks as they have distinctive dark lips, snout tip, and patch around each eye which stand out on the lighter face. \n\n▪️ The back is dark grey to bluish-black, the belly is white, and the sides are grey. \n\nhttps://uk.whales.org/whales-dolphins/species-guide/dusky-dolphin/","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Spinner Dolphin \n

\n
\n \n![spinnner](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-spinner-dolphin.png?itok=vN0dNT-x)\n\n▪️ Spinner dolphins received their common name because they are often seen leaping and spinning out of the water.\n\n▪️ Spinner dolphins are relatively small compared with other species of oceanic dolphins. They are slender, with thin, recurved flippers, and dorsal fins that usually range from slightly curved to erect and triangular.\n\nhttps://www.fisheries.noaa.gov/species/spinner-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Spotted Dolphin \n

\n
\n\n![spotted](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-atlantic-spotted-dolphin.png?itok=I6ISA-rw)\n\n▪️ There are two species of spotted dolphins the pantropical spotted dolphin and atlantic spotted dolphin, probably in this atlantic spotted dolphin is meant as pantropical dolphin is already in different category.\n\n▪️ Young Atlantic spotted dolphins do not have spots. As a result, they can look like slender bottlenose dolphins\n\n▪️ They have a robust body with a tall, curved dorsal fin located midway down their back.\n\n\nhttps://uk.whales.org/whales-dolphins/species-guide/pantropical-spotted-dolphin/\n\nhttps://www.fisheries.noaa.gov/species/atlantic-spotted-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Common Dolphin \n

\n
\n \n![common](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-short-beaked-common-dolphin.png?itok=aZiaoEmW)\n\n▪️ Short-beaked common dolphins are one of the most abundant and familiar dolphins in the world. \n\n▪️ Their body is sleek with a relatively tall, triangular dorsal fin in the middle of their back.\n\n\n\nhttps://www.fisheries.noaa.gov/species/short-beaked-common-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬White Sided Dolphin \n

\n
\n \n***1. Atlantic White Sided Dolphin***\n![atlantic](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-atlantic-white-sided-dolphin.jpg?itok=2t_0VVSy)\n\n▪️ Atlantic white-sided dolphins are relatively small delphinds.\n\n▪️ They have a distinct prominent, relatively large, tall, falcate dorsal fin, located midway down the back.\n\n***2. Pacific White Sided Dolphin***\n![pacific](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-pacific-white-sided-dolphin.png?itok=-RaS_tlw)\n\n▪️ These dolphins have a robust body, short rostrum (snout), and large dorsal fin compared to their overall body size. \n\n▪️ Pacific white-sided dolphins are most likely to be confused with common dolphins\n\nhttps://www.fisheries.noaa.gov/species/atlantic-white-sided-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Pantropic Spotted Dolphin \n

\n
\n \n![pantropic](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-pantropical-spotted-dolphin.png?itok=5XTCLTnu)\n\n▪️ Adults are spottier than younger dolphins, as more and more spots appear as they get older.\n\n▪️ The dorsal fin is strongly curved, tall and narrow. \n \n▪️ They are dark grey on top with light spots and lighter grey on the belly and sides with dark spots. \n\nhttps://www.fisheries.noaa.gov/species/pantropical-spotted-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Commersons Dolphin \n

\n
\n \n![commersons](http://whaleopedia.org/animalfund/wp-content/uploads/2013/10/Commersons-Dolphin1.png)\n\n▪️ Commerson’s dolphins are small, chubby dolphins with cone-shaped heads and no beak. \n\n▪️ The dorsal fin is large and rounded; it looks like one of Mickey Mouse’s ears.\n\n\nhttps://uk.whales.org/whales-dolphins/species-guide/commersons-dolphin/","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Rough Toothed Dolphin \n

\n
\n\n![rough](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-rough-toothed-dolphin.png?itok=WvxCx5oj)\n\n▪️ Rough-toothed dolphins are relatively small compared to other dolphins. \n\n▪️ Their dorsal fin and pectoral fins or flippers are distinctively large, which is characteristic of this species. They have a “reptilian” appearance that is also distinct and unique among dolphins.\n\nhttps://www.fisheries.noaa.gov/species/rough-toothed-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Frasiers Dolphin \n

\n
\n \n![fraser](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-frasers-dolphin.png?itok=mAHHyXOJ)\n\n▪️ Their dorsal fin—located midway down their back—is small and triangular.\n\n▪️ All Fraser's dolphins have a dark stripe that extends down their side from eye to pectoral flipper\n\n\nhttps://www.fisheries.noaa.gov/species/frasers-dolphin","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Globis/Pilot Whale/Short Finned Pilot Whale \n

\n
\n\n![pilot](https://media.fisheries.noaa.gov/styles/original/s3/2021-08/640x427-Whale_Short-Finned_Pilot.png?itok=jmCRoGqj)\n\n▪️ They are one of two species of pilot whale, along with the long-finned pilot whale. The two species differ slightly in size, features, coloration, and pattern. In the field and at sea, it is very difficult to tell the difference between the two species.\n\n▪️ The short-finned pilot whale has a bulbous melon head with no obvious rostrum. Its dorsal fin is far forward on its body and has a relatively long base. The body is black or dark brown, with a large gray saddle behind the dorsal fin.\n\n\nhttps://www.fisheries.noaa.gov/species/short-finned-pilot-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Humpback Whale \n

\n
\n\n![humpback](https://media.fisheries.noaa.gov/styles/original/s3/2021-08/640x427-Whale-Humpback-watermark.jpg?itok=3RBJ1uQq)\n\n▪️ The humpback whale gets its common name from the distinctive hump on its back.\n\nhttps://www.fisheries.noaa.gov/species/humpback-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Blue Whale \n

\n
\n \n \n![blue](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-blue-whale.jpg?itok=Ffb4BA78)\n\n▪️ Blue whales are the largest animals ever to live on our planet. Today, blue whales are listed as endangered.\n\n▪️ Its dorsal (top) fin is small and triangular or falcate (curved) in shape, and is located three-fourths of the way back on the body. \n\nhttps://www.fisheries.noaa.gov/species/blue-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬False Killer Whale \n

\n
\n \n![falsekiller](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-false-killer-whale.png?itok=v9Lw8QPd)\n\n▪️ The false killer whale’s entire body is black or dark gray, although lighter areas may occur ventrally (on its underside) between the flippers or on the sides of the head\n\n▪️ The dorsal fin is located in the middle of the back and generally curves backward. In Hawaiian waters, dorsal fin shapes show a lot of variability, often caused by injury from fishery interactions!\n\nhttps://www.fisheries.noaa.gov/species/false-killer-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Melon Headed Whale \n

\n
\n \n![melon](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-melon-headed-whale.png?itok=Cg3fSKwZ)\n\n▪️ Melon-headed whales have a small head with a rounded melon and no discernible beak.\n\n▪️ Their dorsal fin is relatively large and they have pointed, tapering flippers (pectoral fins).\n\nhttps://www.fisheries.noaa.gov/species/melon-headed-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Minke Whale \n

\n
\n \n \n![MINKE](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-minke-whale.png?itok=lb2-uZjw)\n\n▪️ Minke whales have a fairly tall, sickle-shaped dorsal fin located about two-thirds down their back. \n\n▪️ Their body is black to dark grayish/brownish, with a pale chevron on the back behind the head and above the flippers, as well as a white underside. \n\nhttps://www.fisheries.noaa.gov/species/minke-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Killer Whale \n

\n
\n \n![killer](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-killer-whale.png?itok=mpHhEa6Y)\n\n\n▪️ It is the largest member of the Delphinidae family, or dolphins\n\n▪️ Killer whales are mostly black on top with white undersides and white patches near the eyes.\n\n▪️ They also have a gray or white saddle patch behind the dorsal fin. \n\n▪️ Adult males develop disproportionately larger pectoral flippers, dorsal fins, tail flukes, and girths than females.\n\nhttps://www.fisheries.noaa.gov/species/killer-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Fin Whale \n

\n
\n \n![fin](https://media.fisheries.noaa.gov/styles/original/s3/2020-09/640x4270-fin-whale-v2.jpg?itok=q6QDTYa_)\n\n▪️ The fin whale is the second-largest whale species on earth, second only to the blue whale. \n\n▪️ It is found throughout the world’s oceans. It gets its name from an easy-to-spot fin on its back, near its tail. They have a tall, hooked dorsal fin, about two-thirds of the way back on the body, that rises at a shallow angle from the back. \n\nhttps://www.fisheries.noaa.gov/species/fin-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Gray Whale \n

\n
\n \n![gray](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-gray-whale.png?itok=r5uMVbmg)\n\n▪️ Gray whales have a mottled gray body with small eyes located just above the corners of the mouth. Their pectoral flippers are broad, paddle-shaped, and pointed at the tips. \n\n▪️ They lack a dorsal fin! Instead Gray whales have a mottled gray body with small eyes located just above the corners of the mouth. Their pectoral flippers are broad, paddle-shaped, and pointed at the tips. \n\nhttps://www.fisheries.noaa.gov/species/gray-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Southern Right Whale \n

\n
\n \n![southern](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-southern-right-whale.jpg?itok=Xb4SDpxg)\n\n▪️ Southern right whales have a stocky, black body often with white belly and chin patches and a large head covered in callosities.\n\n▪️ They lack a dorsal fin and have wide, paddle-shaped flippers!\n\nhttps://www.fisheries.noaa.gov/species/southern-right-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Sei Whale \n

\n
\n \n \n![sei](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-sei-whale.png?itok=aV7zzhmQ)\n\n▪️ Sei whales have a long, sleek body that is dark bluish-gray to black in color and white or cream-colored on the underside. The body is often covered in oval-shaped scars.\n\n▪️ Sei whales have a tall, hooked dorsal fin located about two-thirds down their back.\n\n▪️ At the water's surface, sei whales can be recognized by a columnar or bushy blow that is about 10 to 13 feet high. The dorsal fin usually appears at the same time as the blowhole when the animal surfaces to breathe.\n\nhttps://www.fisheries.noaa.gov/species/sei-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Cuviers Beaked Whale \n

\n
\n\n\n![](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-cuviers-beaked-whale.png?itok=YoK45CGr)\n\n▪️ A Cuvier’s beaked whale’s body has variable coloration that ranges from dark gray to a reddish-brown, with a paler counter-shaded underside. \n\n▪️ Cuvier's beaked whale is medium-sized with a round and robust body and a triangular falcate dorsal fin located far down the animal’s back. \n\nhttps://www.fisheries.noaa.gov/species/cuviers-beaked-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐬Long Finned Pilot Whale \n

\n
\n \n \n![](https://media.fisheries.noaa.gov/styles/original/s3/dam-migration/640x427-long-finned-pilot-whale.png?itok=pFxt6472)\n\n▪️ Long-finned pilot whales are one of two species of pilot whale, along with short-finned pilot whales. In the field and at sea, it is very difficult to distinguish the two species, which differ only slightly in physical size, features, coloration, and pattern.\n\n▪️ Their thick dorsal fin is located about a third of the body length behind the head. As they mature, their dorsal fin becomes broader and rounder.\n\n▪️ This species gets its common name from the pair of long, tapered, sickle-shaped flippers on either side of its body.\n\nhttps://www.fisheries.noaa.gov/species/long-finned-pilot-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n🐳Brydes Whale \n

\n
\n\n![](https://media.fisheries.noaa.gov/styles/original/s3/2021-07/640x427-Whale_Bryde%27s.jpg?itok=wxoXJVQy)\n \n \n▪️ They are considered one of the \"great whales,\" or rorquals, which is a group that also includes blue whales and humpback whales. \n\n▪️ The head of a Bryde's whale makes up about one quarter of its entire body length. The whales have a broad fluke, or tail, and a pointed and strongly hooked dorsal fin located about two-thirds back on the body. \n\nhttps://www.fisheries.noaa.gov/species/brydes-whale","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n Data Cleaning \n

\n
","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n Replace Misspellings \n

\n
","metadata":{}},{"cell_type":"code","source":"df.species.replace({'kiler_whale': 'killer_whale'}, inplace = True)\ndf.species.replace({'bottlenose_dolpin': 'bottlenose_dolphin'}, inplace = True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"According to the following links ***globis*** and ***beluga*** are types of whales so we will add whale to it. This is important as I'll create new column called class based on parsing the specie name.\n","metadata":{}},{"cell_type":"code","source":"df.species.replace({'globis': 'globis_whale'}, inplace = True)\ndf.species.replace({'beluga': 'beluga_whale'}, inplace = True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Merge Columns \n

\n
","metadata":{}},{"cell_type":"markdown","source":"According to the discussion [here](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305468) from competition host ***pilot_whale*** and ***globis_whale*** are types of short_finned_pilot_whale, so the three can be merged together.","metadata":{}},{"cell_type":"code","source":"df.species.replace({'pilot_whale': 'short_finned_pilot_whale',\n 'globis_whale' : 'short_finned_pilot_whale'}, inplace = True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Whale or Dolphin? \n

\n
","metadata":{}},{"cell_type":"markdown","source":"According to this [notebook](https://www.kaggle.com/kwentar/not-every-whale-is-a-whale). Some species are named as whales but they are technically dolphins so that intrigued me to research more about this as it affects classes distribution drastically. \n\n🔴 ***False Killer Whale***. \"False killer whales are large members of the dolphin family\". https://oceana.org/marine-life/false-killer-whale/\n\n🔴 ***Killer Whale***. \"Orcas, or killer whales, are the largest of the dolphins\". https://www.nationalgeographic.com/animals/mammals/facts/orca\n\n🔴 ***Pygmy Killer Whale***. \"pygmy killer whales don’t actually look anything like killer whales. In fact, like killer whales, they’re not even whales: they’re dolphins.\" https://uk.whales.org/whales-dolphins/species-guide/pygmy-killer-whale/\n\n🔴 ***Pilot Whales***. \"The two species of pilot whales are actually dolphins, not whales\" https://oceana.org/marine-life/long-finned-pilot-whale/\n\n🔴 ***Melon Headed Whale***. \"Not actually a whale, and with no actual melons. Well, not of the fruit variety anyway\" https://uk.whales.org/whales-dolphins/species-guide/melon-headed-whale/\n\n\nAccordingly I will change their class from *whale* to *dolphin*.","metadata":{}},{"cell_type":"code","source":"# Create a new column for class\ndf['class'] = df['species'].map(lambda x: 'whale' if 'whale' in x else 'dolphin')\n\nfor i in range(len(df)):\n if df.species[i] in ['false_killer_whale', 'killer_whale',\n 'pygmy_killer_whale', 'long_finned_pilot_whale',\n 'short_finned_pilot_whale', 'melon_headed_whale']:\n df['class'][i] = 'dolphin'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Total number of species after cleaning: \", df.species.nunique())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df.isnull().sum().sum())\nprint(df.duplicated().sum())\nprint(\"There are no nulls or duplicates in the data\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Exploratory Data Analysis \n

\n
","metadata":{}},{"cell_type":"code","source":"plot = sns.countplot(x = df['class'], color = '#2596be')\nsns.despine()\nplot.set_title('Class Distribution\\n', font = 'serif', x = 0.1, y=1, fontsize = 16);\nplot.set_ylabel(\"Count\", x = 0.02, font = 'serif', fontsize = 12)\nplot.set_xlabel(\"Specie\", fontsize = 12, font = 'serif')\n\nfor p in plot.patches:\n plot.annotate(format(p.get_height(), '.0f'), (p.get_x() + p.get_width() / 2, p.get_height()), \n ha = 'center', va = 'center', xytext = (0, -20),font = 'serif', textcoords = 'offset points', size = 15)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can note from the previous plot that the classes are almost similarily distributed.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,7))\n\n# define plot\nplot = sns.countplot(y = df.species,\n color = '#2596be',\n order = df['species'].value_counts().index)\n\n# Add titles\nplot.set_title('Species Distributions\\n\\n', font = 'serif', x = 0.1, y=1, fontsize = 16);\nplot.set_xlabel(\"Count\", x = 0.02, font = 'serif', fontsize = 12)\nplot.set_ylabel(\"Species\", fontsize = 12, font = 'serif')\n# Remove unneccessary gridlines\nsns.despine(top = False, bottom = True)\n# Move X Axis to top\nplot.xaxis.set_ticks_position(\"top\")\nplot.xaxis.set_label_position('top')\n","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Within whales and dolphin species the distribution is different so I might consider data augmentation for the species with least frequency.","metadata":{}},{"cell_type":"markdown","source":"
\n\n

\n Most Frequent Whales \n

\n
","metadata":{}},{"cell_type":"code","source":"whales = df.loc[df['class'] == 'whale', 'species']\nplt.figure(figsize=(15,7))\n\nplot = sns.countplot(y = df.loc[df['class'] == 'whale', 'species'],\n color = 'grey',\n order = df.loc[df['class'] == 'whale', 'species'].value_counts().index)\n\nsns.despine(bottom = True, left = True, top = False)\n\nplot.set_title('Most Frequent Whales\\n\\n', size = 20,x = 0.13, y=1, font = 'serif');\nplt.xlabel(\"Count\", font = 'serif', size = 12, x = 0.018)\nplot.xaxis.set_ticks_position(\"top\")\nplot.xaxis.set_label_position('top')\nplt.ylabel(\"Whale Specie\", font = 'serif', size = 12)\n\nx = np.sort(whales.value_counts())[-3:]\nfor bar in plot.patches:\n if bar.get_width() in x:\n bar.set_color('#2596be') \n else:\n bar.set_color('grey')\n\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('beluga_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('humpback_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('blue_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Least Frequent Whales \n

\n
","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,7))\n\nplot = sns.countplot(y = df.loc[df['class'] == 'whale', 'species'],\n color = 'grey',\n order = df.loc[df['class'] == 'whale', 'species'].value_counts(ascending = True).index)\n\nsns.despine(bottom = True, left = True, top = False)\n\nplot.set_title('Least Frequent Whales\\n\\n', x = 0.13, y = 1, size = 20, font = 'serif');\nplt.xlabel(\"Count\", font = 'serif', x = 0.018, size = 12)\nplot.xaxis.set_ticks_position(\"top\")\nplot.xaxis.set_label_position('top')\nplt.ylabel(\"Whale Specie\", font = 'serif', size = 12)\n\nx = np.sort(whales.value_counts())[0:3]\nfor bar in plot.patches:\n if bar.get_width() in x:\n bar.set_color('#2596be') \n else:\n bar.set_color('grey')\n\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('pygmy_killer_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('cuviers_beaked_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('sei_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Most Frequent Dolphins \n

\n
","metadata":{}},{"cell_type":"code","source":"dolphins = df.loc[df['class'] == 'dolphin', 'species']\nplt.figure(figsize=(15,7))\n\nplot = sns.countplot(y = df.loc[df['class'] == 'dolphin', 'species'],\n color = 'grey',\n order = df.loc[df['class'] == 'dolphin', 'species'].value_counts().index)\n\nsns.despine(bottom = True, left = True, top = False)\n\nplot.set_title('Most Frequent Dolphins\\n\\n', x = 0.14, y = 1, size = 20, font = 'serif');\nplt.xlabel(\"Count\", font = 'serif', x = 0.018, size = 12)\nplot.xaxis.set_ticks_position(\"top\")\nplot.xaxis.set_label_position('top')\nplt.ylabel(\"Dolphin Specie\", font = 'serif', size = 12)\n\nx = np.sort(dolphins.value_counts())[-3:]\nfor bar in plot.patches:\n if bar.get_width() in x:\n bar.set_color('#2596be') \n else:\n bar.set_color('grey')\n\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('bottlenose_dolphin', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('false_killer_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('dusky_dolphin', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Least Frequent Dolphins \n

\n
","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,7))\n\nplot = sns.countplot(y = df.loc[df['class'] == 'dolphin', 'species'],\n color = 'grey',\n order = df.loc[df['class'] == 'dolphin', 'species'].value_counts(ascending = True).index)\n\nsns.despine(bottom = True, left = True, top = False)\n\nplot.set_title('Least Frequent Dolphins\\n\\n', x = 0.14, y =1, size = 20, font = 'serif');\nplt.xlabel(\"Count\", font = 'serif', x = 0.013, size = 12)\nplot.xaxis.set_ticks_position(\"top\")\nplot.xaxis.set_label_position('top')\nplt.ylabel(\"Dolphin Specie\", font = 'serif', size = 12)\n\nx = np.sort(dolphins.value_counts())[0:3]\nfor bar in plot.patches:\n if bar.get_width() in x:\n bar.set_color('#2596be') \n else:\n bar.set_color('grey')\n\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('frasiers_dolphin', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('rough_toothed_dolphin', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_n_samples('pygmy_killer_whale', 2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Most Frequent Whale ID \n

\n
","metadata":{}},{"cell_type":"code","source":"# Function from https://www.kaggle.com/heye0507/happy-whale-eda-baseline\nPATH = '../input/happy-whale-and-dolphin'\ndef plot_id_whale_samples(row=5,path=PATH,folder='train_images'):\n path = PATH + '/' + folder\n n = row * row\n df1 = df.loc[df['class'] == 'whale']\n df1 = df1.loc[df1['individual_id'] == df1['individual_id'].value_counts().index[0]]\n _,ax = plt.subplots(row,row,figsize=(15,10))\n \n for i in range(n):\n idx = np.random.randint(len(df1))\n fname = df1.iloc[idx]['image']\n label = df1.iloc[idx]['individual_id']\n img = cv2.imread(path + '/' + fname)\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n ax[i//row][i%row].imshow(img)\n ax[i//row][i%row].set_title(label)\n ax[i//row][i%row].axis('off')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_id_whale_samples(2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Most Frequent Dolphin ID \n

\n
","metadata":{}},{"cell_type":"code","source":"# Function from https://www.kaggle.com/heye0507/happy-whale-eda-baseline\nPATH = '../input/happy-whale-and-dolphin'\ndef plot_id_dolphin_samples(row=5,path=PATH,folder='train_images'):\n path = PATH + '/' + folder\n n = row * row\n df1 = df.loc[df['class'] == 'dolphin']\n df1 = df1.loc[df1['individual_id'] == df1['individual_id'].value_counts().index[0]]\n _,ax = plt.subplots(row,row,figsize=(15,10))\n \n for i in range(n):\n idx = np.random.randint(len(df1))\n fname = df1.iloc[idx]['image']\n label = df1.iloc[idx]['individual_id']\n img = cv2.imread(path + '/' + fname)\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n ax[i//row][i%row].imshow(img)\n ax[i//row][i%row].set_title(label)\n ax[i//row][i%row].axis('off')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_id_dolphin_samples(2)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T14:25:28.398107Z","iopub.execute_input":"2022-02-09T14:25:28.398621Z","iopub.status.idle":"2022-02-09T14:25:35.150231Z","shell.execute_reply.started":"2022-02-09T14:25:28.398591Z","shell.execute_reply":"2022-02-09T14:25:35.149168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Image Transformation \n\n In Progress...\n

\n
","metadata":{}},{"cell_type":"code","source":"# Function from https://www.kaggle.com/gpreda/resize-images-of-happy-whales-and-dolphins/notebook\ndef create_resized_images(image_path, image_output_path):\n image_list = os.listdir(image_path)\n for image in tqdm(image_list):\n img = cv2.imread(os.path.join(image_path, image))\n img = cv2.resize(img, NEW_SIZE[::-1], interpolation=INTERPOLATION)\n cv2.imwrite(os.path.join(image_output_path , image), img)\n \ndef plot_image_samples_test(images_folder):\n root_path = \"/kaggle/working/\"\n images_folder=images_folder\n\n f, ax = plt.subplots(4, 4, figsize=(16,16))\n file_list = list(os.listdir(root_path+images_folder))\n for i in range(16):\n file = file_list[i]\n img = cv2.imread(root_path+images_folder+file)\n ax[i//4, i%4].imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))\n ax[i//4, i%4].set_title(\"Test image: \"+file+f\"\\nsize: {img.shape}\")\n ax[i//4, i%4].axis('off')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T14:25:35.151464Z","iopub.execute_input":"2022-02-09T14:25:35.151685Z","iopub.status.idle":"2022-02-09T14:25:35.16168Z","shell.execute_reply.started":"2022-02-09T14:25:35.151659Z","shell.execute_reply":"2022-02-09T14:25:35.160506Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Resize Images 128x128 \n

\n
\n","metadata":{"_kg_hide-input":true}},{"cell_type":"code","source":"# Code from https://www.kaggle.com/gpreda/resize-images-of-happy-whales-and-dolphins/notebook\n\n# os.makedirs('/kaggle/working/train_images_128')\n# # os.makedirs('/kaggle/working/test_images_128')\n\n# NEW_SIZE = (128, 128) # (height, width)\n# INTERPOLATION = cv2.INTER_CUBIC\n\n# train_image_path = \"../input/happy-whale-and-dolphin/train_images\"\n# train_image_output_path = \"/kaggle/working/train_images_128\"\n# create_resized_images(train_image_path, train_image_output_path)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T14:25:38.833824Z","iopub.execute_input":"2022-02-09T14:25:38.834235Z","iopub.status.idle":"2022-02-09T15:20:33.101356Z","shell.execute_reply.started":"2022-02-09T14:25:38.834209Z","shell.execute_reply":"2022-02-09T15:20:33.099629Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot_image_samples_test('train_images_128/')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T15:27:41.623469Z","iopub.execute_input":"2022-02-09T15:27:41.623811Z","iopub.status.idle":"2022-02-09T15:27:43.075651Z","shell.execute_reply.started":"2022-02-09T15:27:41.623775Z","shell.execute_reply":"2022-02-09T15:27:43.074852Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Data Augmentation \nIn Progress...\n

\n
\n\nCode taken from https://www.kaggle.com/kohjiahng/whaleeda-aug","metadata":{"_kg_hide-input":true}},{"cell_type":"code","source":"# def to_tensor(arr):\n# return torch.tensor(arr).permute(2,0,1)\n# def to_array(tensor):\n# return np.array(tensor).transpose((1, 2, 0))\n# def plot_augmentations(paths, aug_transform, figsize, n_augs=4):\n# aug_transform = transforms.Compose([\n# transforms.RandomResizedCrop((256,256)),\n# aug_transform\n# ])\n# fig, axes = plt.subplots(len(paths), n_augs + 1, figsize=figsize)\n# plt.tight_layout(rect=[0, 0.03, 1, 0.94])\n \n# axes[0][0].set_title('Original', fontsize=30)\n# for i in range(n_augs):\n# axes[0][i+1].set_title(f\"Augmentation {i+1}\", fontsize=30)\n \n# for row, path in enumerate(paths):\n# img = read_image(path)\n# axes[row][0].imshow(img)\n# axes[row][0].axis('off')\n\n# for aug in range(1, n_augs+1):\n# img_t = to_tensor(img)\n# axes[row][aug].imshow(to_array(aug_transform(img_t)))\n# axes[row][aug].axis('off')","metadata":{"execution":{"iopub.status.busy":"2022-02-09T17:33:53.256522Z","iopub.execute_input":"2022-02-09T17:33:53.256898Z","iopub.status.idle":"2022-02-09T17:33:53.291653Z","shell.execute_reply.started":"2022-02-09T17:33:53.256805Z","shell.execute_reply":"2022-02-09T17:33:53.290957Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Flipping Images \n

\n
","metadata":{"_kg_hide-input":true}},{"cell_type":"code","source":"# random_paths = df['path'].sample(4, random_state = 0)\n\n# hflip_transform = transforms.RandomHorizontalFlip(p=0.5)\n# plot_augmentations(random_paths, hflip_transform, (40,20))\n\n# plt.suptitle('Horizontal Flip with probability 0.5', fontsize=35)","metadata":{"execution":{"iopub.status.busy":"2022-02-09T17:35:40.903426Z","iopub.execute_input":"2022-02-09T17:35:40.903726Z","iopub.status.idle":"2022-02-09T17:35:41.056105Z","shell.execute_reply.started":"2022-02-09T17:35:40.903691Z","shell.execute_reply":"2022-02-09T17:35:41.054168Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Changing Brightness\n

\n
","metadata":{"_kg_hide-input":true,"_kg_hide-output":true}},{"cell_type":"code","source":"# brightness_transform = transforms.ColorJitter(brightness=0.5)\n# plot_augmentations(random_paths, brightness_transform, (40,20))\n\n# plt.suptitle('Random brightness change with brightness=0.5', fontsize=35)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# all_aug = transforms.Compose([\n# hflip_transform,\n# brightness_transform,\n# pov_transform,\n# rot_transform\n# ])\n# plot_augmentations(random_paths, all_aug, (40,20))\n# plt.suptitle('All Augmentations combined', fontsize=35)","metadata":{"_kg_hide-output":true,"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

\n Modeling \nIn Progress...\n

\n
","metadata":{"_kg_hide-input":true}}]}