{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Choosing the right Eps\nMany Kagglers clip right before Submit. What is the appropriate clipping width?\n\nThe appropriate epsilon value depends on the accuracy of the model. This Kernel visualizes its dependencies."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Import libs\nimport math\nimport numpy as np\nimport matplotlib.pyplot as plt\n\nfrom pylab import rcParams","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Define the function to calculate Logloss from gt and labels."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"def logloss(true_label, predicted, eps=1e-15):\n    p = np.clip(predicted, eps, 1 - eps)\n    if true_label == 1:\n        return -math.log(p)\n    else:\n        return -math.log(1 - p)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Define a function to calculate Logloss from Accuracy."},{"metadata":{"trusted":true},"cell_type":"code","source":"def acc2logloss(acc, size, e):\n    p = int(acc * size)\n    true_l = [1 for i in range(size)]\n    pred_l = [1 for i in range(p)] + [0 for i in range(size - p)]\n    ll = 0\n    for t, p in zip(true_l, pred_l):\n        ll += logloss(t, p, eps=e)\n    return ll / size","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualize the Logloss value when EPS is changed while fixing Accuracy."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_logloss(acc, size=4000):\n    eps = np.arange(0.01, 0.2, 0.01)\n    res = []\n    for e in eps:\n        res.append(acc2logloss(acc, size, e))\n    min_logloss_ind = np.argmin(res)\n    plt.plot(eps, res, label=str(acc))\n    plt.plot(eps[min_logloss_ind], res[min_logloss_ind], marker='*')\n    plt.text(eps[min_logloss_ind], res[min_logloss_ind], str((np.round(eps[min_logloss_ind], 2), np.round(res[min_logloss_ind], 2))), size=15, color=\"black\")\n    plt.xlabel(\"Epsilon\")\n    plt.ylabel(\"Logloss\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualize for some representative accuracy."},{"metadata":{"trusted":true},"cell_type":"code","source":"rcParams['figure.figsize'] = 10,10\naccs = np.arange(0.8, 0.9, 0.01)\nfor acc in accs:\n    plot_logloss(np.round(acc, 2))\nplt.legend()\nplt.title(\"0.80-0.90\")\nplt.show()\n\naccs = np.arange(0.9, 1.0, 0.01)\nfor acc in accs:\n    plot_logloss(np.round(acc, 2))\nplt.legend()\nplt.title(\"0.90-1.00\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nYou can see that choosing the right Eps for you can have a big impact on your Logloss Score."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}