{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "482f4a00-26df-bc88-fd79-57da8b0f1d0c"
      },
      "source": [
        "#Inspiration\n",
        "This kernel is inspired by [David Thaler][1] idea to estimate the distribution of LB. My idea is to exploit the fact that log function exacerbates the value of x near 0 as opposite to its value at 1 (which is exactly zero). In order to deal with computational stability, log loss function utilizes a small coefficient epsilon = 1e-15. Fortunately, this small additive coefficient still reserve the great disparency between log(1-epsilon) and log(epsilon).\n",
        "\n",
        "I will first present my estimation method first then do all the computation at the end.\n",
        "\n",
        "  [1]: https://www.kaggle.com/davidthaler/quora-question-pairs/how-many-1-s-are-in-the-public-lb"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "7133bc16-ecea-83cb-71a8-b06a2020b9b9"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "epsilon = 1e-15\n",
        "print('log(epsilon):',-np.log(epsilon))\n",
        "print('log(1-epsilon):',-np.log(1-epsilon))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "9d3af2f2-c760-451c-293f-e816df54cff4"
      },
      "source": [
        "Here is a [quick description][1] of log loss function.\n",
        "\n",
        "#Motivation\n",
        "\n",
        "Firstly, what happened when we label all test data as 0. This obviously gives best loss score on negative label data and huge penalty on positive test data. In order word, this the log loss score would consist of many scores from positive label data (which is a constant **log(epsilon)** for each positive data point in LB). Thus we can ignore the log loss score from negative data point (**log(1-epsilon)** per negative data point).\n",
        "\n",
        "[1]:https://www.kaggle.com/wiki/LogarithmicLoss"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "238e543f-bd36-a91a-ea42-716104e25927"
      },
      "outputs": [],
      "source": [
        "df_test = pd.read_csv('../input/test.csv')\n",
        "n = len(df_test)\n",
        "print('n:',n)\n",
        "\n",
        "const = (n/n)*-np.log(1-epsilon)\n",
        "print('const:',const)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "ca3f65b6-20d6-8c3e-3c81-250b89ebbe9b"
      },
      "source": [
        "#Estimate portion of n1 (n1/n)\n",
        "\n",
        "Let define n1, n2, n3 as the number of positive data point, negative data point, and computer generated data point. Let call alpha as the log loss of test data label all 0. We have:\n",
        "alpha = (n1/n)*-log(epsilon)+(n2/n)*-log(1-epsilon)\n",
        "\n",
        "##Lower bound of n1/n:\n",
        "\n",
        "Since we only have n=2345796 data row in the testing data set, n2 < 2345796. Thus:\n",
        "\n",
        "(n2/n)*-log(1-epsilon) < const, with const = (n/n)*-log(1-epsilon) = 9.99200722163e-16\n",
        "Remember that: -log(1-epsilon) > 0\n",
        "\n",
        "Then alpha < (n1/n)*-log(epsilon)+ const. As a result: **(n1/n) > (alpha - const)/(-log(epsilon))**\n",
        "\n",
        "##Upper bound of n1/n:\n",
        "(n1/n)*-log(epsilon) < alpha Then: **(n1/n) < alpha/(-log(epsilon))**\n",
        "\n",
        "Thus we have: **(alpha - const)/(-log(epsilon)) < (n1/n) < alpha/(-log(epsilon))**\n",
        "\n",
        "#Estimating portion of n2 (n2/n):\n",
        "The same goes for n2. Let call beta = (n1/n)*-log(1-epsilon)+(n2/n)*-log(epsilon)\n",
        "\n",
        "Then **(beta - const)/(-log(epsilon)) < (n2/n) < beta/(-log(epsilon))**\n",
        "\n",
        "\n",
        "  [1]: https://www.kaggle.com/wiki/LogarithmicLoss"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "8d349131-d354-8f44-0ed5-42c902864228"
      },
      "outputs": [],
      "source": [
        "test = pd.read_csv('../input/test.csv')\n",
        "sub = test[['test_id']].copy()\n",
        "sub['is_duplicate'] = 0\n",
        "sub.to_csv('submission_alpha.csv', index=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "40592147-d6e9-887f-3373-6390af230202"
      },
      "outputs": [],
      "source": [
        "test = pd.read_csv('../input/test.csv')\n",
        "sub = test[['test_id']].copy()\n",
        "sub['is_duplicate'] = 1\n",
        "sub.to_csv('submission_beta.csv', index=False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "0b9569e8-19eb-65f1-d945-3f90729e25db"
      },
      "source": [
        " This result alpha = 6.0188 and beta = 28.52056"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "48dfd13a-a29a-06fc-69af-7ebf122f1786"
      },
      "outputs": [],
      "source": [
        "alpha = 6.0188 \n",
        "beta = 28.52056\n",
        "\n",
        "print ((alpha - const)/(-np.log(epsilon)),'< (n1/n) <',alpha/(-np.log(epsilon)))\n",
        "print ((beta - const)/(-np.log(epsilon)),'< (n2/n) <',beta/(-np.log(epsilon)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "0af02a90-fc87-87d0-5875-155fd96ec70d"
      },
      "source": [
        "Then we can conclude that Public leaderboard consist of 17.5% of positive data point and 82.5% of negative data point."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "022b690a-f9f0-a4e3-211a-34f337044818"
      },
      "source": [
        "Now can we estimate the exact number of positive/negative data point. In order word, can we know how many computer generated data point in test set ??\n",
        "To be continue.... \n",
        "\n",
        "Peace :)"
      ]
    }
  ],
  "metadata": {
    "_change_revision": 0,
    "_is_fork": false,
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}