{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "deca8809-b4e9-a559-26d2-e59001a5540c"
      },
      "source": [
        "# Goal of this kernel (Simply detect Sea Lions in test images)\n",
        " - **Initial Exploration**\n",
        " - **Extract individual images of Sea Lions from Training Set**\n",
        " - **Train a classifier on these images**\n",
        " - **Test on Test Set** (By sliding window)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "2d5ade65-1289-b965-8735-c5a42b0ea932"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import glob\n",
        "import os\n",
        "import cv2\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "train_data = pd.read_csv('../input/Train/train.csv')\n",
        "train_imgs = sorted(glob.glob('../input/Train/*.jpg'), key=lambda name: int(os.path.basename(name)[:-4]))\n",
        "train_dot_imgs = sorted(glob.glob('../input/TrainDotted/*.jpg'), key=lambda name: int(os.path.basename(name)[:-4]))\n",
        "\n",
        "submission = pd.read_csv('../input/sample_submission.csv')\n",
        "\n",
        "\n",
        "print(train_data.shape)\n",
        "print('Number of Train Images: {:d}'.format(len(train_imgs)))\n",
        "print('Number of Dotted-Train Images: {:d}'.format(len(train_dot_imgs)))\n",
        "\n",
        "\n",
        "\n",
        "print(train_data.head(6))\n",
        "\n",
        "#test_imgs = glob.glob('../input/Test/*.jpg')\n",
        "#print('Number of Test Images: {:d}'.format(len(test_imgs)))\n",
        "#from subprocess import check_output\n",
        "#print(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "eed090bb-ff3e-5b96-81b7-1782f2c73c1f"
      },
      "outputs": [],
      "source": [
        "submission.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "ef3e87dd-7a61-3a2b-cc18-245183aa62c4"
      },
      "source": [
        " - **Let's check how sea lions are distributed in training images:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "98f5151c-0917-226b-32e0-5681aa2021cd"
      },
      "outputs": [],
      "source": [
        "# Count of each type\n",
        "hist = train_data.sum(axis=0)\n",
        "print(hist)\n",
        "\n",
        "\n",
        "sea_lions_types = hist[1:]\n",
        "f, ax1 = plt.subplots(1,1,figsize=(5,5))\n",
        "sea_lions_types.plot(kind='bar', title='Count of Sea Lion Types (Train)', ax=ax1)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "fa5a1fef-ac56-59b2-1443-eb8e13674ecd"
      },
      "source": [
        "- **Let's plot One Image**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "12b80960-3d12-9c1b-d599-82e8f3e0cd82"
      },
      "outputs": [],
      "source": [
        "index = 5\n",
        "sl_counts = train_data.iloc[index]\n",
        "print(sl_counts)\n",
        "\n",
        "plt.figure()\n",
        "sl_counts.plot(kind='bar', title='Count of Sea Lion Types')\n",
        "plt.show()\n",
        "\n",
        "print(train_imgs[index])\n",
        "img = cv2.cvtColor(cv2.imread(train_imgs[index]), cv2.COLOR_BGR2RGB)\n",
        "img_dot = cv2.cvtColor(cv2.imread(train_dot_imgs[index]), cv2.COLOR_BGR2RGB)\n",
        "\n",
        "crop_img = img[200:2000, 2600:3500]\n",
        "crop_img_dot = img_dot[200:2000, 2600:3500]\n",
        "\n",
        "f, ax = plt.subplots(1,2,figsize=(16,8))\n",
        "(ax1, ax2) = ax.flatten()\n",
        "\n",
        "ax1.imshow(img)\n",
        "ax2.imshow(img_dot)\n",
        "\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "a3c25b55-8558-f45e-d76c-386d9a082e3d"
      },
      "source": [
        "**Let's zoom in on a cluster of Sea Lions**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "4b5a2b1e-2171-8066-cd04-b83fc6dc3978"
      },
      "outputs": [],
      "source": [
        "crop_img = img[1350:1900, 3000:3400]\n",
        "crop_img_dot = img_dot[1350:1900, 3000:3400]\n",
        "\n",
        "f, ax = plt.subplots(1,2,figsize=(16,8))\n",
        "(ax1, ax2) = ax.flatten()\n",
        "\n",
        "ax1.imshow(crop_img)\n",
        "ax2.imshow(crop_img_dot)\n",
        "\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "0b9a2605-3ceb-59af-9913-139910134339"
      },
      "source": [
        "Alright, I can see **1 adult male**, **12 adult females**, **3 juveniles** & **14 pups**.\n",
        "\n",
        " - Adult male looks whitish and females look more of brownish.\n",
        " - Let's see where are rest of adult males hanging around\n",
        " - Pups are mostly closer to adult females"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "2e1a6f2e-deb0-f84f-e91d-271c5e4366c5"
      },
      "source": [
        "# National Geographic Nostalgia\n",
        "\n",
        "I believe I have seen a documentary on Sea Lions quite a long time ago. Like most of the animals, male adult sea lions have to fight to mate and to keep their territories secure from other adult sea lions.\n",
        " \n",
        "It would be interesting to explore further the distances between male adult sea lions. Let's explore where adult male lions are in the image.\n",
        "\n",
        "# Where are the red dots in dotted-images ?\n",
        "\n",
        " - I think first task should be to identify all the dots.\n",
        " - Once locations of dots are known, check colors of those locations to classify sea lion.\n",
        " - Extract sea lion image by making a bounding box "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "b3cbea9f-4802-9da3-8511-4ada884bbeed"
      },
      "outputs": [],
      "source": [
        "index = 5\n",
        "\n",
        "image = cv2.cvtColor(cv2.imread(train_imgs[index]), cv2.COLOR_BGR2RGB)\n",
        "image_dot = cv2.cvtColor(cv2.imread(train_dot_imgs[index]), cv2.COLOR_BGR2RGB)\n",
        "\n",
        "img = image[1350:1900, 3000:3400]\n",
        "img_dot = image_dot[1350:1900, 3000:3400]\n",
        "\n",
        "#img_c = np.copy(img)\n",
        "\n",
        "diff = cv2.absdiff(img_dot, img)\n",
        "gray = cv2.cvtColor(diff, cv2.COLOR_RGB2GRAY)\n",
        "ret,th1 = cv2.threshold(gray,0,255,cv2.THRESH_BINARY | cv2.THRESH_OTSU)\n",
        "#plt.figure(figsize=(16,8))\n",
        "#plt.imshow(th1, 'gray')\n",
        "\n",
        "cnts = cv2.findContours(th1.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2]\n",
        "print(\"Sea Lions Found: {}\".format(len(cnts)))\n",
        "\n",
        "for (i, c) in enumerate(cnts):\n",
        "\t((x, y), _) = cv2.minEnclosingCircle(c)\n",
        "\tcv2.putText(diff, \"{}\".format(i + 1), (int(x) - 10, int(y)),cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255, 255, 255), 2)\n",
        "\tcv2.drawContours(diff, [c], -1, (0, 255, 0), 2)\n",
        "\n",
        "#plt.figure(figsize=(16,8))\n",
        "#plt.imshow(diff)\n",
        "\n",
        "f, ax = plt.subplots(3,1,figsize=(18,35))\n",
        "(ax1, ax2, ax3) = ax.flatten()\n",
        "ax1.imshow(img_dot)\n",
        "ax2.imshow(th1, 'gray')\n",
        "ax3.imshow(diff)\n",
        "#plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "863e03e8-f6ba-334a-a3fa-838a8b4fd928"
      },
      "source": [
        "**That's pretty accurate**\n",
        "Let's check on the whole image, if we get the correct count\n",
        "\n",
        "## Problem 1:\n",
        "In dotted images there are areas that are blackened. It will mess up our contour detection when run on on whole image\n",
        "## Solution:\n",
        "add the missing area (blackened) from original training image.\n",
        "## Problem 2:\n",
        "Even after adding the missing area (in dotted images), when we will use absdiff() there will some artefacts i.e. we will see differences in two images (new-dotted and original image) at the boundary of blackened areas. This will result in more contours detected.\n",
        "## Solution:\n",
        "Prune contours by radius. Prune all contours that have very very small radius."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "70e2f810-e82d-d3ba-cd5b-e7b818ec9c43"
      },
      "outputs": [],
      "source": [
        "index = 1\n",
        "print(index)\n",
        "sl_counts = train_data.iloc[index]\n",
        "print(sl_counts)\n",
        "print('[Ground Truth] Sea Lion Count: {}'.format(sum(sl_counts[1:])))\n",
        "\n",
        "img = cv2.cvtColor(cv2.imread(train_imgs[index]), cv2.COLOR_BGR2RGB)\n",
        "img_dot = cv2.cvtColor(cv2.imread(train_dot_imgs[index]), cv2.COLOR_BGR2RGB)\n",
        "\n",
        "\n",
        "# Now create a mask of missing area in dotted image and create its inverse mask also\n",
        "img_dot_gray = cv2.cvtColor(img_dot,cv2.COLOR_BGR2GRAY)\n",
        "ret, mask = cv2.threshold(img_dot_gray, 10, 255, cv2.THRESH_BINARY)\n",
        "mask_inv = cv2.bitwise_not(mask)\n",
        "\n",
        "img_dot_missing_area = cv2.bitwise_and(img,img,mask = mask_inv)\n",
        "fnl_dot = cv2.add(img_dot,img_dot_missing_area)\n",
        "\n",
        "###################################################\n",
        "\n",
        "\"\"\"\n",
        "diff = cv2.absdiff(img_dot, img)\n",
        "diff[img_dot==0] = 255\n",
        "diff = np.max(diff, axis=-1)\n",
        "f, ax = plt.subplots(1,1,figsize=(16,30))\n",
        "ax.imshow(diff, 'gray')\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "diff = cv2.absdiff(fnl_dot, img)\n",
        "gray = cv2.cvtColor(diff, cv2.COLOR_RGB2GRAY)\n",
        "\n",
        "\n",
        "ret,th1 = cv2.threshold(gray,10,255,cv2.THRESH_BINARY | cv2.THRESH_OTSU)\n",
        "\n",
        "# All contours\n",
        "cnts = cv2.findContours(th1.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2]\n",
        "print(\"[Initial Counted] Sea Lions Found: {}\".format(len(cnts)))\n",
        "\n",
        "# Remove contours where radius is very very small\n",
        "pruned_contours = []\n",
        "min_radius_threshold = 1.5\n",
        "max_radius_threshold = 5.0\n",
        "for (i, c) in enumerate(cnts):\n",
        "    ((x, y), r) = cv2.minEnclosingCircle(c)\n",
        "    if ((r>=min_radius_threshold) & (r<=max_radius_threshold)):\n",
        "        pruned_contours.append(c)\n",
        "print(\"[Final Counted] Sea Lions Found: {}\".format(len(pruned_contours)))\n",
        "\n",
        "#for (i, c) in enumerate(pruned_contours):\n",
        "#    ((x, y), _) = cv2.minEnclosingCircle(c)\n",
        "#    cv2.putText(diff, \"{}\".format(i + 1), (int(x) - 10, int(y)),cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2)\n",
        "#    cv2.drawContours(diff, [c], -1, (0, 255, 0), 2)\n",
        "\n",
        "\n",
        "\n",
        "#f, ax = plt.subplots(1,1,figsize=(16,30))\n",
        "#(ax1,ax2) = ax.flatten()\n",
        "#ax.imshow(diff)\n",
        "#ax.imshow(gray,'gray')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "5cea1d73-0318-370f-4627-88f7683fbaf2"
      },
      "source": [
        "**Still in progress, there is some problem when I use rest of the training images, need to figure out why it is detecting very high number of contours.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "5f741010-2803-419a-2fad-f450ed2ab2af"
      },
      "outputs": [],
      "source": [
        ""
      ]
    }
  ],
  "metadata": {
    "_change_revision": 0,
    "_is_fork": false,
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}