{ "cells": [ { "cell_type": "markdown", "metadata": { "_cell_guid": "7daa5a37-ea6b-2274-4bc1-fadb0a2fc41a" }, "source": [ "# Count the Sea Lions in the first image\n", "\n", "- [Firs part:][1] \n", " - I am using the first picture to extract Sea Lion coordinates using blob detection\n", "\n", "- [Second part:][2]\n", " - I extract 64 by 64 images centered on the extracted coordinates\n", " - In addition to the previous script, I add negative examples to the training data\n", " - I train a simple keras model on the full data\n", " - In the third part I will test on the training data, so overfitting is not a bug, is a feature\n", "\n", "- Third part:\n", " - Built a test set by tiling the first image\n", " - Use the previously built model to decide if in each tile there is a Sea Lion\n", " - **I am aware that I am training and testing on the same data**\n", "\n", " [1]: https://www.kaggle.com/radustoicescu/noaa-fisheries-steller-sea-lion-population-count/get-coordinates-using-blob-detection\n", " [2]: https://www.kaggle.com/radustoicescu/noaa-fisheries-steller-sea-lion-population-count/use-keras-to-classify-sea-lions-0-91-accuracy" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "34be1096-5a5d-1bfa-ec25-65fd57f78ae9" }, "source": [ "# First Part" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "35cabbd8-8e7b-5be7-bd00-61b1addc2d75" }, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import os\n", "import cv2\n", "import matplotlib.pyplot as plt\n", "import skimage.feature\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.preprocessing import LabelBinarizer\n", "import keras\n", "from keras.models import Sequential, load_model\n", "from keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, Lambda, Cropping2D\n", "from keras.utils import np_utils\n", "\n", "from collections import Counter\n", "\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "39c70a8d-7645-f2f2-8e09-7d528eb48e57" }, "source": [ "### Initialize variables" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "3a6329c7-aa3b-b576-b6e1-d675e6e8fbf2" }, "outputs": [], "source": [ "class_names = ['adult_females', 'adult_males', 'juveniles', 'pups', 'subadult_males']\n", "\n", "file_names = os.listdir(\"../input/Train/\")\n", "file_names = sorted(file_names, key=lambda \n", " item: (int(item.partition('.')[0]) if item[0].isdigit() else float('inf'), item)) \n", "\n", "# select a subset of files to run on\n", "file_names = file_names[0:1]\n", "\n", "# dataframe to store results in\n", "coordinates_df = pd.DataFrame(index=file_names, columns=class_names)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "f6aecb87-1c65-5c28-12c8-d86c42ae223b" }, "source": [ "### Extract coordinates" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "7755c681-04df-368a-aca2-f099dd9ce805" }, "outputs": [], "source": [ "for filename in file_names:\n", " \n", " # read the Train and Train Dotted images\n", " image_1 = cv2.imread(\"../input/TrainDotted/\" + filename)\n", " image_2 = cv2.imread(\"../input/Train/\" + filename)\n", " \n", " cut = np.copy(image_2)\n", " \n", " # absolute difference between Train and Train Dotted\n", " image_3 = cv2.absdiff(image_1,image_2)\n", " \n", " # mask out blackened regions from Train Dotted\n", " mask_1 = cv2.cvtColor(image_1, cv2.COLOR_BGR2GRAY)\n", " mask_1[mask_1 < 20] = 0\n", " mask_1[mask_1 > 0] = 255\n", " \n", " mask_2 = cv2.cvtColor(image_2, cv2.COLOR_BGR2GRAY)\n", " mask_2[mask_2 < 20] = 0\n", " mask_2[mask_2 > 0] = 255\n", " \n", " image_3 = cv2.bitwise_or(image_3, image_3, mask=mask_1)\n", " image_3 = cv2.bitwise_or(image_3, image_3, mask=mask_2) \n", " \n", " # convert to grayscale to be accepted by skimage.feature.blob_log\n", " image_3 = cv2.cvtColor(image_3, cv2.COLOR_BGR2GRAY)\n", " \n", " # detect blobs\n", " blobs = skimage.feature.blob_log(image_3, min_sigma=3, max_sigma=4, num_sigma=1, threshold=0.02)\n", " \n", " adult_males = []\n", " subadult_males = []\n", " pups = []\n", " juveniles = []\n", " adult_females = [] \n", " \n", " image_circles = image_1\n", " \n", " for blob in blobs:\n", " # get the coordinates for each blob\n", " y, x, s = blob\n", " # get the color of the pixel from Train Dotted in the center of the blob\n", " g,b,r = image_1[int(y)][int(x)][:]\n", " \n", " # decision tree to pick the class of the blob by looking at the color in Train Dotted\n", " if r > 200 and g < 50 and b < 50: # RED\n", " adult_males.append((int(x),int(y)))\n", " cv2.circle(image_circles, (int(x),int(y)), 20, (0,0,255), 10) \n", " elif r > 200 and g > 200 and b < 50: # MAGENTA\n", " subadult_males.append((int(x),int(y))) \n", " cv2.circle(image_circles, (int(x),int(y)), 20, (250,10,250), 10)\n", " elif r < 100 and g < 100 and 150 < b < 200: # GREEN\n", " pups.append((int(x),int(y)))\n", " cv2.circle(image_circles, (int(x),int(y)), 20, (20,180,35), 10)\n", " elif r < 100 and 100 < g and b < 100: # BLUE\n", " juveniles.append((int(x),int(y))) \n", " cv2.circle(image_circles, (int(x),int(y)), 20, (180,60,30), 10)\n", " elif r < 150 and g < 50 and b < 100: # BROWN\n", " adult_females.append((int(x),int(y)))\n", " cv2.circle(image_circles, (int(x),int(y)), 20, (0,42,84), 10) \n", " \n", " cv2.rectangle(cut, (int(x)-112,int(y)-112),(int(x)+112,int(y)+112), 0,-1)\n", " \n", " coordinates_df[\"adult_males\"][filename] = adult_males\n", " coordinates_df[\"subadult_males\"][filename] = subadult_males\n", " coordinates_df[\"adult_females\"][filename] = adult_females\n", " coordinates_df[\"juveniles\"][filename] = juveniles\n", " coordinates_df[\"pups\"][filename] = pups" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "d5f4e244-3b12-4470-c262-15502af83e04" }, "outputs": [], "source": [ "f, ax = plt.subplots(1,1,figsize=(10,16))\n", "ax.imshow(cv2.cvtColor(image_circles, cv2.COLOR_BGR2RGB))\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "d1f098cf-4e42-64e2-df13-f20e75ee9288" }, "source": [ "# Second Part" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "d03424c1-b12b-ae53-2fed-1dff86398164" }, "source": [ "### Extract 32 by 32 images of Sea Lions" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "7001891b-daf5-976f-0c30-47edb34afc21" }, "source": [ "Use the previosly created image of the landscape with all the SeaLions cut out as template to extract negative examples" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "bbc83919-f2e9-1c92-df4e-2b6090cff4a0" }, "outputs": [], "source": [ "f, ax = plt.subplots(1,1,figsize=(10,16))\n", "ax.imshow(cv2.cvtColor(cut, cv2.COLOR_BGR2RGB))\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "c9549615-3a64-2ef2-2be0-1946e07c2ee2" }, "outputs": [], "source": [ "x = []\n", "y = []\n", "\n", "for filename in file_names: \n", " image = cv2.imread(\"../input/Train/\" + filename)\n", " for lion_class in class_names:\n", " for coordinates in coordinates_df[lion_class][filename]:\n", " thumb = image[coordinates[1]-32:coordinates[1]+32,coordinates[0]-32:coordinates[0]+32,:]\n", " if np.shape(thumb) == (64, 64, 3):\n", " x.append(thumb)\n", " y.append(lion_class)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "8d2bce79-3d13-38c0-9e05-cdc6e9fda3d4" }, "source": [ "### Add negative examples to the dataset" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "dbc255f0-9d70-31f9-cd20-c625f27e3645" }, "outputs": [], "source": [ "for i in range(0,np.shape(cut)[0],224):\n", " for j in range(0,np.shape(cut)[1],224): \n", " thumb = cut[i:i+64,j:j+64,:]\n", " if np.amin(cv2.cvtColor(thumb, cv2.COLOR_BGR2GRAY)) != 0:\n", " if np.shape(thumb) == (64,64,3):\n", " x.append(thumb)\n", " y.append(\"negative\") \n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "02e1e16e-976c-df5b-b623-92ed12a18a8f" }, "outputs": [], "source": [ "class_names.append(\"negative\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "5ccfa849-28bd-bf58-033d-315af4eac61e" }, "outputs": [], "source": [ "x = np.array(x)\n", "y = np.array(y)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "75016379-58d6-1ff4-d411-71bb29eec895" }, "source": [ "### Plot examples" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "981bcb37-d353-dcd1-fc99-ff178605811c" }, "outputs": [], "source": [ "for lion_class in class_names:\n", " f, ax = plt.subplots(1,10,figsize=(12,1.5))\n", " f.suptitle(lion_class)\n", " axes = ax.flatten()\n", " j = 0\n", " for a in axes:\n", " a.set_xticks([])\n", " a.set_yticks([])\n", " for i in range(j,len(x)):\n", " if y[i] == lion_class:\n", " j = i+1\n", " a.imshow(cv2.cvtColor(x[i], cv2.COLOR_BGR2RGB))\n", " break" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "3ae5856f-c680-e4e3-459a-fa764559938a" }, "source": [ "### One hot encoding" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "c1438c4d-e0d4-812a-adff-6f5db4c32a81" }, "outputs": [], "source": [ "encoder = LabelBinarizer()\n", "encoder.fit(y)\n", "y = encoder.transform(y).astype(float)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "7c0dfde1-a1b3-a181-bcaa-58455025138a" }, "source": [ "### Build Keras model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "25ecbbe7-cae5-a6fa-3ab0-943fe10f580a" }, "outputs": [], "source": [ "model = Sequential()\n", "\n", "model.add(Lambda(lambda x: (x / 255.0) - 0.5, input_shape=(64,64,3)))\n", "\n", "\n", "model.add(Conv2D(32, (5, 5), activation='relu', padding='same'))\n", "model.add(MaxPooling2D(pool_size=(2,2)))\n", "model.add(Conv2D(64, (5, 5), activation='relu', padding='same'))\n", "model.add(MaxPooling2D(pool_size=(2,2)))\n", "model.add(Conv2D(128, (5, 5), activation='relu', padding='same'))\n", "\n", "model.add(Flatten())\n", "\n", "model.add(Dense(512, activation='relu'))\n", "model.add(Dropout(0.5))\n", "model.add(Dense(6, activation='softmax'))\n", "\n", "model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "8f78d810-3f33-b73a-eaf9-426315561e82" }, "source": [ "### Training" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "e9447e9b-92f7-aab7-bd03-03a3aa82b816" }, "outputs": [], "source": [ "history = model.fit(x, y, epochs=10, verbose=0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "5782174e-3d6f-c3ce-be48-d8095a07c35e" }, "outputs": [], "source": [ "plt.plot(history.history['acc'])\n", "plt.title('model accuracy')\n", "plt.ylabel('accuracy')\n", "plt.xlabel('epoch')\n", "plt.legend(['train', 'test'], loc='upper left')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "9103e964-ee7a-64b0-f456-3510ecf7d301" }, "source": [ "# Third Part" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "466b1323-c4f8-f21d-f0a9-eff19bcf6ebd" }, "source": [ "### Build the test " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "b1d0eca0-e93b-fb07-c30d-fcf7eaa05022" }, "outputs": [], "source": [ "img = cv2.imread(\"../input/Train/\" + filename)\n", "\n", "x_test = []\n", "\n", "for i in range(0,np.shape(img)[0],64):\n", " for j in range(0,np.shape(img)[1],64): \n", " thumb = img[i:i+64,j:j+64,:] \n", " if np.shape(thumb) == (64,64,3):\n", " x_test.append(thumb)\n", "\n", "x_test = np.array(x_test)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "ed947adb-41dc-a62e-9ebf-b17368d63017" }, "source": [ "### Predict" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "d036eb7a-db49-f2c8-c86b-09141fb7cd7b" }, "outputs": [], "source": [ "y_predicted = model.predict(x_test, verbose=0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "60ab0e44-1c68-572d-a568-1aa2ac654e43" }, "outputs": [], "source": [ "y_predicted = encoder.inverse_transform(y_predicted)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "5df73b5a-3edd-716b-06f4-893ef38c431d" }, "outputs": [], "source": [ "print(Counter(y_predicted).items())" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "6ae51dba-c173-9d51-9e36-233e8b6f4779" }, "source": [ "### Correct numbers" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "09d81b32-7264-821b-30dc-066ef47e791e" }, "outputs": [], "source": [ "reference = pd.read_csv('../input/Train/train.csv')\n", "reference.ix[0:0]" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "5e03d63b-8362-879d-a89d-933324c52bcc" }, "source": [ "# Conclusions" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "1208e13d-ed90-da69-bf2b-08541f0c181e" }, "source": [ "The numbers are way off, not what I expected. Especially since I tested on the same data I trained.\n", "\n", "For some reason it detects a lot of pups.\n", "\n", "The bad results could be in part a consequence of the huge number of negative examples in the test set." ] } ], "metadata": { "_change_revision": 0, "_is_fork": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 0 }