{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Riiid! Answer Correctness Prediction"},{"metadata":{},"cell_type":"markdown","source":"![Riid](https://www.riiid.co/assets/opengraph.png)\n\nDiscussion with a good intro to the competition - https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/189409  \nAnd the questions are from these 7 parts of the TOEIC test - Test of English for International Communication\n\n### 7 Parts of a TOEIC test\nhttps://www.ets.org/toeic/organizations/listening-reading/about/content-format/\n\n### Listening\n\n- Part 1: Photographs\n- Part 2: Question-Response\n- Part 3: Conversations\n- Part 4: Short Talks\n\n### Reading\n\n- Part 5: Incomplete Sentences\n- Part 6: Error Recognition or Text Completion\n- Part 7: Reading Comprehension\n"},{"metadata":{},"cell_type":"markdown","source":"# Data description"},{"metadata":{},"cell_type":"markdown","source":"Modified with changes from the competition hosts.\n\n### **train.csv**\n\n- `row_id`: (int64) ID code for the row.\n- `timestamp`: (int64) the time between this user interaction and the first event completion from that user. This is the timestamp of when the user started anwsering the question\n- `user_id`: (int32) ID code for the user.\n- `content_id`: (int16) ID code for the user interaction\n- `content_type_id`: (int8) 0 if the event was a question being posed to the user, 1 if the event was the user watching a lecture.\n- `task_container_id`: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id.\n- `user_answer`: (int8) the user's answer to the question, if any. Read -1 as null, for lectures.\n- `answered_correctly`: (int8) if the user responded correctly. Read -1 as null, for lectures.\n- `prior_question_elapsed_time`: (float32) How long it took a user to answer their all of the questions in the previous question bundle, ignoring any lectures in between. The value is shared across a single question bundle, and is null for a user's first question bundle or lecture.\n   CORRECT: Note that the time is the average time a user took to solve each question in the previous bundle\n   \n> **Suppose a user spent 60 seconds answering their first question at 10:00. The timestamp for that row would read 10:00 (we'll skip the normalization for simplicity), and the prior_question_elapsed_time would be null since it's the first question. If they took 30 seconds to answer their next question at 11:00 the second row's timestamp would be 11:00 and the prior_question_elapsed_time would be 60 seconds.**\n   \n- `prior_question_had_explanation`: (bool) Whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle, ignoring any lectures in between. The value is shared across a single question bundle, and is null for a user's first question bundle or lecture. Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback.\n\n### **questions.csv: metadata for the questions posed to users.**\n\n- `question_id`: foreign key for the train/test content_id column, when the content type is question (0).\n- `bundle_id`: code for which questions are served together.\n- `correct_answer`: the answer to the question. Can be compared with the train user_answer column to check if the user was right.\n- `part`: the relevant section of the TOEIC test.\n     - **What is TOEIC test?** The Test of English for International Communication is an international standardized test of English language proficiency for non-native speakers. parts mentioned above.\n- `tags`: one or more detailed tag codes for the question. The meaning of the tags will not be provided, but these codes are sufficient for clustering the questions together.\n\n### **lectures.csv: metadata for the lectures watched by users as they progress in their education.**\n\n- `lecture_id`: foreign key for the train/test content_id column, when the content type is lecture (1).\n- `part`: top level category code for the lecture.\n- `tag`: one tag codes for the lecture. The meaning of the tags will not be provided, but these codes are sufficient for clustering the lectures together.\n- `type_of`: brief description of the core purpose of the lecture\n\n### **example_test_rows.csv** \n\nThree sample groups of the test set data as it will be delivered by the time-series API. The format is largely the same as train.csv. There are two different columns that mirror what information the AI tutor actually has available at any given time, but with the user interactions grouped together for the sake of API performance rather than strictly showing information for a single user at a time. Some questions will appear in the hidden test set that have NOT been presented in the train set, emulating the challenge of quickly adapting to modeling newly introduced questions. Their metadata is still in question.csv as usual.\n\n`prior_group_responses` (string) provides all of the user_answer entries for previous group in a string representation of a list in the first row of the group. All other rows in each group are null. If you are using Python, you will likely want to call eval on the non-null rows. Some rows may be null, or empty lists.\n\n`prior_group_answers_correct` (string) provides all the answered_correctly field for previous group, with the same format and caveats as prior_group_responses. Some rows may be null, or empty lists."},{"metadata":{},"cell_type":"markdown","source":"# EDA - Q/A index"},{"metadata":{},"cell_type":"markdown","source":"I find it easier to do EDA when I break it down into specific questions.  \nThe answers to few specific questions will help in discovering features\n\n## **train.csv**\n\n- [Basic EDA](#train_eda)\n\n- [How many users' data are we given?](#q1)\n-  [How do users interact with different content-types?](#qd)\n- [How many questions did each user answer?](#q2)\n- [How many lectures did each student see?](#q3)\n- [Whats the correlation between Questions count and Number of lectures?](#q4)\n- [How long do users use the App?](#qtt)\n\n## **Questions.csv**\n\n- [How many questions are there in train data, and in questions meta-data?](#q5)\n- [How are questions split into each part of the TOEIC test?](#q6)\n- [How are questions split into bundles?](#q7)\n8. [What is the probability of getting a question right?](#q8)\n9. [What is the probability of a user getting any question right](#q9)\n10. [Should we predict answers for the same students in train? Will new students be added in test data?](#q10)\n11. [Do students reattempt questions?](#q11)\n12. [How are questions split into tags?](#q12)"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"scrolled":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom pathlib import Path\nimport plotly_express as px\nimport matplotlib.pyplot as plt\nfrom plotly.subplots import make_subplots\nimport plotly.graph_objects as go","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# from https://www.kaggle.com/rohanrao/riiid-with-blazing-fast-rid\n!pip install ../input/python-datatable/datatable-0.11.0-cp37-cp37m-manylinux2010_x86_64.whl","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"def reduce_mem_usage(df, verbose=True):\n    numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']\n    start_mem = df.memory_usage().sum() / 1024**2\n    for col in df.columns:\n        col_type = df[col].dtypes\n        if col_type in numerics:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)\n            else:\n                c_prec = df[col].apply(lambda x: np.finfo(x).precision).max()\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max and c_prec == np.finfo(np.float16).precision:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max and c_prec == np.finfo(np.float32).precision:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n    end_mem = df.memory_usage().sum() / 1024**2\n    if verbose: print('Mem. usage decreased to {:5.2f} Mb ({:.1f}% reduction)'.format(end_mem, 100 * (start_mem - end_mem) / start_mem))\n    return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# plotly helpers\n\n# plots a histogram and a box plot of a column in dataframe\ndef distribution_plot(df, column, min_quantile=0, max_quantile=1):\n    display(pd.DataFrame(df[column].describe(percentiles=np.arange(.1, 1, .1))).T)\n    min_value = df[column].quantile(min_quantile)\n    max_value = df[column].quantile(max_quantile)\n    df = df[(df[column] >= min_value) & (df[column] <= max_value)]\n    fig = make_subplots(rows=1, cols=2)\n    fig.add_trace(go.Histogram(x=df[column], nbinsx=100), row=1, col=1)\n    fig.add_trace(go.Box(y=df[column], orientation='v', name=column), row=1, col=2)\n    fig.update_layout(title_text=f'{column} Distribution | min quantile {min_quantile} | max quantile {max_quantile}', showlegend=False)\n    fig.show()\n    \ndef p_line(y, x=None, title=None):\n    if x is None:\n        if hasattr(y, 'index'):\n            x = y.index\n        else:\n            x = list(range(0, len(y)))\n    fig = go.Figure(data=go.Scatter(x=x, y=y))\n    if title is not None:\n        fig.update_layout(title_text=title)\n    fig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"data_dir = Path('../input/riiid-test-answer-prediction')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# import data files\nimport datatable as dt\n\ntrain = dt.fread(\"../input/riiid-test-answer-prediction/train.csv\").to_pandas()\nprint(train.shape)\ntrain = reduce_mem_usage(train)\nquestions = pd.read_csv(data_dir/'questions.csv')\nlectures = pd.read_csv(data_dir/'lectures.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"example_test = pd.read_csv(data_dir/'example_test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"set(example_test['content_id']) - set(questions['question_id'])\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id='train_eda'/>\n"},{"metadata":{},"cell_type":"markdown","source":"\n# Train.csv\n## Basic EDA"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"pd.DataFrame(train.isnull().sum(), columns=['null_count'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id='q1'/>"},{"metadata":{},"cell_type":"markdown","source":"## How many users' data are we given?\n"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"print(f\"Number of users in train - {train['user_id'].nunique()}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id='qd'/>"},{"metadata":{},"cell_type":"markdown","source":"## How do users interact with different content-types?\n\n0 - Questions\n1 - Lectures"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"content_count = pd.DataFrame(train['content_type_id'].value_counts()).reset_index()\ncontent_count.columns = ['type', 'count']\ncontent_count['type'] = content_count.type.replace({0: 'questions', 1: 'lectures'})\npx.pie(content_count, values='count', names='type')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id='q2'/>"},{"metadata":{},"cell_type":"markdown","source":"## How many questions did each user answer?"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"user_interactions_count = train[['row_id','user_id', 'content_type_id']].groupby(['user_id', 'content_type_id'], as_index=False).count()\nuser_interactions_count = user_interactions_count.rename(columns={'row_id': 'count'})\nuser_interactions_count = user_interactions_count.pivot(index='user_id', columns='content_type_id', values=['count'])\nuser_interactions_count = user_interactions_count.fillna(0)\nuser_interactions_count.columns = ['questions_count', 'lectures_count']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"distribution_plot(user_interactions_count,'questions_count', min_quantile=0, max_quantile=.9)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### What are top 10 counts of number of answered questions"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"qc_count = user_interactions_count['questions_count'].value_counts(normalize=True)\nqc_count.index = pd.Series(qc_count.index).apply(lambda x: f'count_{x}')\np_line(qc_count.head(10), title='percentage of users vs questions count')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- 15% of the users have 30 questions\n- There are spikes in the number of questions. This is worth exploring more. There might specific usage patterns in the app"},{"metadata":{},"cell_type":"markdown","source":"<a id='q3'/>"},{"metadata":{},"cell_type":"markdown","source":"## How many lectures did each student see?"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"distribution_plot(user_interactions_count,'lectures_count', min_quantile=0, max_quantile=.95)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Lectures count cumulative sum"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"p_line(user_interactions_count['lectures_count'].value_counts(normalize=True).cumsum(), title='User percentage cumsum vs number of lectures')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- This is unexpected, 62% of the users saw no lectures. So large number of users are here for the questions."},{"metadata":{},"cell_type":"markdown","source":"<a id='q4'/>"},{"metadata":{},"cell_type":"markdown","source":"## Whats the correlation between number of questions each user has answered and the number of lectures he has seen?"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"user_interactions_count.corr()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"user_interactions_count['questions_lectures'] = user_interactions_count['questions_count'].astype(str) + '_' + user_interactions_count['lectures_count'].astype(str)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Top 20 combinations of questions count and lectures count"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"p_line(user_interactions_count['questions_lectures'].value_counts(normalize=True).head(20), title='Percentage of users vs questions_lectures combo count')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"user_interactions_count = user_interactions_count.sort_values('questions_count')\nuser_interactions_count['user_index'] = range(0, len(user_interactions_count))\n\nfig = make_subplots(specs=[[{\"secondary_y\": True}]])\nfig.add_trace(\n    go.Scatter(x=user_interactions_count['user_index'], y=user_interactions_count['questions_count'], name=\"questions_count\"),\n    secondary_y=False,\n)\nfig.add_trace(\n    go.Scatter(x=user_interactions_count['user_index'], y=user_interactions_count['lectures_count'], name=\"lectures_count\", opacity=.5),\n    secondary_y=True,\n)\nfig.update_xaxes(title_text=\"user index\")\nfig.update_yaxes(title_text=\"number of Questions\", secondary_y=False)\nfig.update_yaxes(title_text=\"number of Lectures\", secondary_y=True)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"user_interactions_count = user_interactions_count.sort_values('lectures_count')\nuser_interactions_count['user_index'] = range(0, len(user_interactions_count))\n\nfig = make_subplots(specs=[[{\"secondary_y\": True}]])\nfig.add_trace(\n    go.Scatter(x=user_interactions_count['user_index'], y=user_interactions_count['questions_count'], name=\"questions_count\", opacity=.5),\n    secondary_y=False,\n)\nfig.add_trace(\n    go.Scatter(x=user_interactions_count['user_index'], y=user_interactions_count['lectures_count'], name=\"lectures_count\"),\n    secondary_y=True,\n)\nfig.update_xaxes(title_text=\"User index\")\nfig.update_yaxes(title_text=\"number of Questions\", secondary_y=False)\nfig.update_yaxes(title_text=\"number of Lectures\", secondary_y=True)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- There is a 80% correlation between number of questions, number of lectures seen be a user\n- There is also a huge variability, there are users with 14K questions but no lectures, there are users with 150 lectures but only 1000 questions  \n  This can be taken as the variability in user behaviour in the app, some want the app for questions, some want it for the lectures.\n  But in general a person whose watched more lectures, should have attemted more questions"},{"metadata":{},"cell_type":"markdown","source":"<a id='qtt'/>"},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"## How long do users use the App?"},{"metadata":{},"cell_type":"markdown","source":"#### Checking if each users' timestamp are monotonously increasing"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"pd.Series.is_monotonic_increasing","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"is_increasing = train[['timestamp', 'user_id']].groupby('user_id').agg(lambda x: x.is_monotonic_increasing)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id='q5'/>"},{"metadata":{},"cell_type":"markdown","source":"# Questions.csv\n\n## Basic EDA"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"questions.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"pd.DataFrame(questions.isnull().sum(), columns=['null_count'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### One question without any tag :P"},{"metadata":{},"cell_type":"markdown","source":"<a id='q5' />"},{"metadata":{},"cell_type":"markdown","source":"## How many questions are there in train data, and in questions meta-data?"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"print(f'Number of unique questions in train data - {train.loc[train[\"content_type_id\"] == 0, \"content_id\"].nunique()}')\nprint(f'Number of unique questions in questions metadata - {questions[\"question_id\"].nunique()}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"q_not_in_train_data = set(questions['question_id']) - set(train.loc[train['content_type_id'] == 0, 'content_id'])\nprint(f'Questions in Metadata, but not in train data - {q_not_in_train_data}')\n\nq_not_in_metadata = set(train.loc[train['content_type_id'] == 0, 'content_id']) - set(questions['question_id'])\nprint(f'Questions in train, but not in metadata - {q_not_in_metadata}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" - That's weird, from the description I got that the test set might have a few questions that are in questions meta-data but not in train data.\n - But all the questions in meta-data are in train data\n - TODO - this will impact feature engineering, get this clarified"},{"metadata":{},"cell_type":"markdown","source":"## 6. How are questions split into each part of the TOEIC test?\n<a id='q6'/>"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"questions.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"questions['part'].unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"parts = questions[['question_id', 'part']].groupby('part', as_index=False).count().rename(columns={'question_id': 'number_of_questions'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"px.bar(parts, x='part', y='number_of_questions')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Part 5: Incomplete Sentences has a large number of unique questions"},{"metadata":{},"cell_type":"markdown","source":"## 7. How are questions split into bundles?\n<a id='q7'/>"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"questions['bundle_id'].nunique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"bundle_count = questions[['question_id', 'bundle_id']].groupby('bundle_id', as_index=False).count().rename(columns={'question_id': 'number_of_questions'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"scrolled":true},"cell_type":"code","source":"distribution_plot(bundle_count, 'number_of_questions')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Most questions are just given alone"},{"metadata":{},"cell_type":"markdown","source":"## 8. What is the probability of getting a question right?\n<a id='q8'/>"},{"metadata":{"trusted":true,"collapsed":true,"scrolled":true},"cell_type":"code","source":"question_answers = train.loc[train['content_type_id'] == 0, ['content_id', 'answered_correctly']].groupby('content_id', as_index=False).mean()\ndistribution_plot(question_answers, 'answered_correctly')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Normal distribution with Right skew, smart students, or easy questions"},{"metadata":{},"cell_type":"markdown","source":"## 9. What is the probability of a user getting any question right?\n<a id='q9'/>"},{"metadata":{"trusted":true,"collapsed":true,"scrolled":true},"cell_type":"code","source":"user_answers = train.loc[train['content_type_id'] == 0, ['user_id', 'answered_correctly']].groupby('user_id', as_index=False).mean()\ndistribution_plot(user_answers, 'answered_correctly')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"scrolled":true},"cell_type":"code","source":"px.histogram(user_answers, 'answered_correctly', nbins=100)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Normal distribution with right skew\n- Does the peak have any meaning? don't think so"},{"metadata":{},"cell_type":"markdown","source":"## 10. Should we predict answers for the same students in train? Will new students be added in test data?\n<a id='q10'/>"},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"test = pd.read_csv(data_dir/'example_test.csv')\ntest.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"set(test['user_id']) - set(train['user_id'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- There is one extra student, I'm assuming there will be new users.\n- TODO - check if this is clearly stated somewhere - Done, there will be new students"},{"metadata":{},"cell_type":"markdown","source":"## 11. Do students reattempt questions? \n<a id='q11'/>"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"user_question_count = train.loc[train['content_type_id'] == 0, ['row_id', 'user_id', 'content_id']].groupby(['user_id', 'content_id']).count()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"scrolled":true},"cell_type":"code","source":"p_line(user_question_count['row_id'].value_counts(normalize=True))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- 89% of the questions are attempted only once\n- There are good number of questions that are reattempted upto 4 times"},{"metadata":{},"cell_type":"markdown","source":"## 12. How are questions split into tags?\n<a id='q12'/>"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"questions.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"print(f'{questions[\"tags\"].nunique()} unique combinations')\nquestions['tags'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"questions['n_tags'] = questions['tags'].fillna(\"\").apply(lambda x: len(x.split()))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"all_tags = pd.Series(np.concatenate(questions['tags'].fillna(\"\").apply(lambda x: x.split()).values)).apply(lambda x: f'tag_{x}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"all_tags.nunique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# this is number of question in questions meta-data,not in train data\np_line(all_tags.value_counts(), all_tags.value_counts().index, title='Unique question tag vs question count')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- 40% questions have a single tag associated with them\n- can have a maximum of 6 tags\n- There are 188 unique tags, arranged into 1519 combinations"},{"metadata":{},"cell_type":"markdown","source":"## 13. How much time does the user take to answer a question?"},{"metadata":{},"cell_type":"markdown","source":"## 14. How much time does a user spend on the App?"},{"metadata":{},"cell_type":"markdown","source":"# EDA - To Be Continued....."},{"metadata":{},"cell_type":"markdown","source":"# Baseline submission"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"question_answers = question_answers.rename(columns={'answered_correctly': 'question_score'})\nuser_answers = user_answers.rename(columns={'answered_correctly': 'user_score'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"question_answers['content_type_id'] = 0\nuser_answers['content_type_id'] = 0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"import riiideducation\nenv = riiideducation.make_env()\niter_test = env.iter_test()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"def harmonic_mean(a, b):\n    return (2 * a * b) / (a + b + 1e-6)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"for (test_df, sample_prediction_df) in iter_test:\n    test_df = test_df.merge(question_answers, on=['content_id', 'content_type_id'], how='left')\n    test_df = test_df.merge(user_answers, on=['user_id', 'content_type_id'], how='left')\n    test_df['question_score'].fillna(.5, inplace=True)\n    test_df['user_score'].fillna(.5, inplace=True)\n    test_df['prediction'] = test_df.apply(lambda row: harmonic_mean(row['question_score'], row['user_score']), axis=1)\n    test_df['answered_correctly'] = test_df['prediction']\n    # display(test_df)\n    env.predict(test_df.loc[test_df['content_type_id'] == 0, ['row_id', 'answered_correctly']])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}