{ "cells": [ { "cell_type": "markdown", "metadata": { "_cell_guid": "e1c128f6-6e0a-3454-b13e-b5da245ef11b" }, "source": [ "# Identifying Duplicate Questions\n", "\n", "Welcome to the Quora Question Pairs competition! Here, our goal is to identify which questions asked on [Quora](https://www.quora.com/), a quasi-forum website with over 100 million visitors a month, are duplicates of questions that have already been asked. This could be useful, for example, to instantly provide answers to questions that have already been answered. We are tasked with predicting whether a pair of questions are duplicates or not, and submitting a binary prediction against the logloss metric.\n", "\n", "Let's dive right into the data!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "e5115d83-b4aa-efff-4618-8ea7dc847e52" }, "outputs": [], "source": [ "import numpy as np # linear algebra\n", "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", "import os\n", "import gc\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "%matplotlib inline\n", "\n", "p = sns.color_palette()\n", "\n", "print('# File sizes')\n", "for f in os.listdir('../input'):\n", " if 'zip' not in f:\n", " print(f.ljust(30) + str(round(os.path.getsize('../input/' + f) / 1000000, 2)) + 'MB')" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "34497938-e316-f9b3-c647-80a155ea6644" }, "source": [ "Looks like we are simply given two files this time round, one for the training set and one for the test set. They are relatively small compared to other recent competitions, weighing in at less than 400MB total.\n", "\n", "It's worth noting that there is a lot more testing data than training data. This could be a sign that some of the test data is dummy data designed to deter hand-labelling, and not included in the calculations, like we recently saw in the [DSTL competition](https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection/leaderboard).\n", "\n", "Let's open up one of the datasets.\n", "\n", "## Training set" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "d0a16a30-ebd4-06ac-f116-aec6cf6446d9" }, "outputs": [], "source": [ "df_train = pd.read_csv('../input/train.csv')\n", "df_train.head()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "0e5880fd-ce7f-9a33-3542-7047c93571e1" }, "source": [ "We are given a minimal number of data fields here, consisting of:\n", "\n", "**`id`:** Looks like a simple rowID \n", "**`qid{1, 2}`:** The unique ID of each question in the pair \n", "**`question{1, 2}`:** The actual textual contents of the questions. \n", "**`is_duplicate`:** The **label** that we are trying to predict - whether the two questions are duplicates of each other." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "67c0cca7-2e41-2340-eacc-7e7a40001f08" }, "outputs": [], "source": [ "print('Total number of question pairs for training: {}'.format(len(df_train)))\n", "print('Duplicate pairs: {}%'.format(round(df_train['is_duplicate'].mean()*100, 2)))\n", "qids = pd.Series(df_train['qid1'].tolist() + df_train['qid2'].tolist())\n", "print('Total number of questions in the training data: {}'.format(len(\n", " np.unique(qids))))\n", "print('Number of questions that appear multiple times: {}'.format(np.sum(qids.value_counts() > 1)))\n", "\n", "plt.hist(qids.value_counts(), bins=50)\n", "plt.yscale('log', nonposy='clip')\n", "plt.title('Log-Histogram of question appearance counts')\n", "plt.xlabel('Number of occurences of question')\n", "plt.ylabel('Number of questions')\n", "print()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "f84ed854-c070-d946-c6c0-4a6d39eceb7b" }, "source": [ "In terms of questions, everything looks as I would expect here. Most questions only appear a few times, with very few questions appearing several times (and a few questions appearing many times). One question appears more than 160 times, but this is an outlier.\n", "\n", "We can see that we have a 37% positive class in this dataset. Since we are using the [LogLoss](https://www.kaggle.com/wiki/LogarithmicLoss) metric, and LogLoss looks at the actual predicts as opposed to the order of predictions, we should be able to get a decent score by creating a submission predicting the mean value of the label.\n", "\n", "## Test Submission" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "9743a85f-1e46-9615-32b7-51b254f4567d" }, "outputs": [], "source": [ "from sklearn.metrics import log_loss\n", "\n", "p = df_train['is_duplicate'].mean() # Our predicted probability\n", "print('Predicted score:', log_loss(df_train['is_duplicate'], np.zeros_like(df_train['is_duplicate']) + p))\n", "\n", "df_test = pd.read_csv('../input/test.csv')\n", "sub = pd.DataFrame({'test_id': df_test['test_id'], 'is_duplicate': p})\n", "sub.to_csv('naive_submission.csv', index=False)\n", "sub.head()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "3807679d-4e93-69bf-63e2-761c383f51da" }, "source": [ "**0.55 on the leaderboard! Score!**\n", "\n", "However, not all is well. The discrepancy between our local score and the LB one indicates that the distribution of values on the leaderboard is very different to what we have here, which could cause problems with validation later on in the competition" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "507bfa42-9d0c-3f7d-dec0-191ae4280650" }, "source": [ "" ] } ], "metadata": { "_change_revision": 0, "_is_fork": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 0 }