{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "b878500d-ac91-574f-fb85-af99a66ff166"
      },
      "source": [
        "## Reading WKT and plotting with Pyplot"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "b9bc9c0c-0fa1-da59-73cf-ed39517994ef"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from shapely.wkt import loads\n",
        "from matplotlib.patches import Polygon\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "df = pd.read_csv('../input/train_wkt_v2.csv')\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "c233d5d6-68a0-2ae8-5757-d58291245163"
      },
      "outputs": [],
      "source": [
        "# Use just first image\n",
        "polygonsList = {}\n",
        "image = df[df.ImageId == '6100_1_3']\n",
        "for cType in image.ClassType.unique():\n",
        "    polygonsList[cType] = loads(image[image.ClassType == cType].MultipolygonWKT.values[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "1fcab23c-506c-736d-86a4-32644a7226ce"
      },
      "outputs": [],
      "source": [
        "# plot using matplotlib\n",
        "fig, ax = plt.subplots(figsize=(8, 8))\n",
        "\n",
        "# plotting, color by class type\n",
        "for p in polygonsList:\n",
        "    for polygon in polygonsList[p]:\n",
        "        mpl_poly = Polygon(np.array(polygon.exterior), color=plt.cm.Set1(p*10), lw=0, alpha=0.3)\n",
        "        ax.add_patch(mpl_poly)\n",
        "\n",
        "ax.relim()\n",
        "ax.autoscale_view()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "7a4b1b5a-43bf-911f-fbe3-5ad5e8e937d7"
      },
      "outputs": [],
      "source": [
        "# number of objects on the image by type\n",
        "'''\n",
        "1. Buildings\n",
        "2. Misc. Manmade structures \n",
        "3. Road \n",
        "4. Track - poor/dirt/cart track, footpath/trail\n",
        "5. Trees - woodland, hedgerows, groups of trees, standalone trees\n",
        "6. Crops - contour ploughing/cropland, grain (wheat) crops, row (potatoes, turnips) crops\n",
        "7. Waterway \n",
        "8. Standing water\n",
        "9. Vehicle Large - large vehicle (e.g. lorry, truck,bus), logistics vehicle\n",
        "10. Vehicle Small - small vehicle (car, van), motorbike\n",
        "'''\n",
        "for p in polygonsList:\n",
        "    print(\"Type: {:4d}, objects: {}\".format(p,len(polygonsList[p].geoms)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "e245177a-5548-bdbb-2c52-f552a7f3093b"
      },
      "source": [
        "## Exploring available dataset\n",
        "\n",
        "Let's look at all available images and see how many objects of which type are there, what is there areas and how these vary from image to image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "ea7c4f5d-76ab-3b89-2eb4-b42c713d55ee"
      },
      "outputs": [],
      "source": [
        "# number of images in available kernel dataset?\n",
        "df.ImageId.unique()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "79374cf5-b772-754b-b6ce-8227bdafacb4"
      },
      "outputs": [],
      "source": [
        "# convert to shapely, get geometries and pivot\n",
        "df['polygons'] = df.apply(lambda row: loads(row.MultipolygonWKT),axis=1)\n",
        "df['nPolygons'] = df.apply(lambda row: len(row['polygons'].geoms),axis=1)\n",
        "\n",
        "pvt = df.pivot(index='ImageId', columns='ClassType', values='nPolygons')\n",
        "pvt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "a965f287-e53c-0cd4-591e-bbe428277701"
      },
      "outputs": [],
      "source": [
        "fig, ax = plt.subplots(figsize=(10, 4))\n",
        "ax.set_aspect('equal')\n",
        "plt.imshow(pvt.T, interpolation='nearest', cmap=plt.cm.Blues, extent=[0,22,10,1])\n",
        "plt.yticks(np.arange(1, 11, 1.0))\n",
        "plt.title('Number of objects by type')\n",
        "plt.ylabel('Class Type')\n",
        "plt.xlabel('Image')\n",
        "plt.colorbar()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "4e4cbeae-b332-2173-a14d-65bbbd48905e"
      },
      "source": [
        "Numbers-wise trees are leading by far, being the only major object type on some of the images. They are followed by buildings and other man structures, which are actually not present on every image, as well as vehicles. Waterways are present on only one image, which actually was added in v2 of WKT file, added after the first version of this kernel was published. \n",
        "\n",
        "Visually there also appears to be a correlation between the number of trees and buildings/structures."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "8d1a6812-884a-3258-f1db-c540796fc11b"
      },
      "outputs": [],
      "source": [
        "from scipy.stats import pearsonr\n",
        "print(\"Trees vs Buildings: {:5.4f}\".format(pearsonr(pvt[1],pvt[5])[0]))\n",
        "print(\"Trees vs Buildings and Structures: {:5.4f}\".format(pearsonr(pvt[1]+pvt[2],pvt[5])[0]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "7bccf7a9-81ec-ef27-0d56-9ea66ca05885"
      },
      "source": [
        "Indeed there is a medium strength negative correlation, which however may be specific to the given training dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "d8983f10-2771-c202-51b3-2ea2dbe51950"
      },
      "source": [
        "## Plot all images\n",
        "\n",
        "There are 22 images in train_wkt_v2.csv file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "0d44ec60-1f74-854e-5077-3b0a001da484"
      },
      "outputs": [],
      "source": [
        "for im in df.ImageId.unique():\n",
        "    image = df[df.ImageId == im]\n",
        "    for cType in image.ClassType.unique():\n",
        "        polygonsList[cType] = loads(image[image.ClassType == cType].MultipolygonWKT.values[0])\n",
        "    \n",
        "    # plot using matplotlib\n",
        "    fig, ax = plt.subplots(figsize=(8, 8))\n",
        "\n",
        "    # plotting, color by class type\n",
        "    for p in polygonsList:\n",
        "        for polygon in polygonsList[p]:\n",
        "            mpl_poly = Polygon(np.array(polygon.exterior), color=plt.cm.Set1(p*10), lw=0, alpha=0.3)\n",
        "            ax.add_patch(mpl_poly)\n",
        "\n",
        "    ax.relim()\n",
        "    ax.autoscale_view()\n",
        "    plt.show()"
      ]
    }
  ],
  "metadata": {
    "_change_revision": 0,
    "_is_fork": false,
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}