{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# A Tale of Persistence and Heavy Clickers","metadata":{}},{"cell_type":"markdown","source":"This Notebook implements two simple ideas. First I will show you that users that have high average scores in the first quiz also tend to have high average scores in the second and third quiz. The second idea builds around the definition of our target variable. The organizer clearified that a \"correct\" answer means that the player got the answer correct on their first attempt [here](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384801#2137152). Now imagine a setting, in which there are two different kinds of players. The first carefully reads the dialogues and thinks before acting. The second one is impatient and ratherly plays the game with trial and error with many clicks (heavy clickers). From a pure player perspective, the second player experiences similar success throughout the game as the first player. It takes similar effort to play through the game and he experieriences similar levels of success. From our **kaggle competition perspective** both players anyhow are extremly different in their success. A heavy clicker is prone to failure, as it will be less likely that he clicks correctly on the first click. This notebook shows that heavyclickers indeed tend to underperform their counterparts. Heavy clicking is an easy to compute feature that will likely boost your model performance.","metadata":{}},{"cell_type":"markdown","source":"## Part 1: Persistence","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:31:36.048104Z","iopub.execute_input":"2023-03-18T16:31:36.050075Z","iopub.status.idle":"2023-03-18T16:31:36.056290Z","shell.execute_reply.started":"2023-03-18T16:31:36.050026Z","shell.execute_reply":"2023-03-18T16:31:36.054909Z"}}},{"cell_type":"code","source":"# read libraries\nimport numpy as np \nimport pandas as pd\n\n\n#compute average \"correctness among level groups\"\n# read labels\ntrain_labels=pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train_labels.csv\")\ntrain_labels=train_labels.dropna(axis=1,thresh=1)\n\n# split questionNo from ID\ntrain_labels[\"question\"]=train_labels[\"session_id\"].str[-3:].replace(\"_\",\"\",regex=True)\ntrain_labels[\"session_id\"]=train_labels[\"session_id\"].str[:-3].replace(\"_\",\"\",regex=True)\n\n# assign level groups\ntrain_labels.loc[train_labels[\"question\"].isin([\"q1\",\"q2\",\"q3\"]),\"level_group\"]=\"0-4\"\ntrain_labels.loc[train_labels[\"question\"].isin([\"q4\",\"q5\",\"q6\",\"q7\",\"q8\",\"q9\",\"q10\",\"q11\",\"q12\",\"q13\"]),\"level_group\"]=\"5-12\"\ntrain_labels.loc[train_labels[\"question\"].isin([\"q14\",\"q15\",\"q16\",\"q17\",\"q18\"]),\"level_group\"]=\"13-22\"\nlabels_agg=train_labels.groupby([\"session_id\",\"level_group\"])[\"correct\"].mean().unstack()\n","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:46:56.120821Z","iopub.execute_input":"2023-03-18T16:46:56.121271Z","iopub.status.idle":"2023-03-18T16:46:57.439014Z","shell.execute_reply.started":"2023-03-18T16:46:56.121232Z","shell.execute_reply":"2023-03-18T16:46:57.437309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Correlation\nlabels_agg[[\"0-4\",\"5-12\",\"13-22\"]].corr()","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:47:08.073580Z","iopub.execute_input":"2023-03-18T16:47:08.074026Z","iopub.status.idle":"2023-03-18T16:47:08.090606Z","shell.execute_reply.started":"2023-03-18T16:47:08.073989Z","shell.execute_reply":"2023-03-18T16:47:08.089550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# mutual information\nfrom sklearn.feature_selection import mutual_info_regression\nprint(mutual_info_regression(labels_agg[[\"0-4\"]], labels_agg[\"5-12\"],random_state=1))\nprint(mutual_info_regression(labels_agg[[\"0-4\",\"5-12\"]], labels_agg[\"13-22\"],random_state=1))","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:52:39.066886Z","iopub.execute_input":"2023-03-18T16:52:39.067329Z","iopub.status.idle":"2023-03-18T16:52:39.296457Z","shell.execute_reply.started":"2023-03-18T16:52:39.067286Z","shell.execute_reply":"2023-03-18T16:52:39.295474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we see, Quiz performance is persistent. High average correct answers in Quiz 1 imply high average correct answers in Quiz 2 and Quiz 3. Especially correct answers in Quiz 2 tend to be strongly correlated with correct answers with Quiz 1 and Quiz 3.","metadata":{}},{"cell_type":"markdown","source":"## Part 2: Heavy Clickers","metadata":{}},{"cell_type":"markdown","source":"Next, we will test our heavy clicker hypothesis. If it holds, we would expect heavy clickers to have lower average correct answers in all three Quizzes.","metadata":{}},{"cell_type":"code","source":"# read train data\ntrain=pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train.csv\")\n# drop all NA features\ntrain=train.dropna(axis=1,thresh=1)\n\n# count the number of clicks\nheavyclickers=train.groupby([\"session_id\",\"level_group\"])[\"elapsed_time\"].count().reset_index()\nheavyclickers.columns=[\"session_id\",\"level_group\",\"heavyclicker\"]\n\n# rearrange average correct answers\nlabels_agg=labels_agg.unstack().reset_index()\nlabels_agg.columns=[\"level_group\",\"session_id\",\"labels_agg\"]\nlabels_agg.session_id=labels_agg.session_id.astype(int)\n\n# and merge with the heavy clicker variable\nlabels_agg=labels_agg.merge(heavyclickers,left_on=[\"level_group\",\"session_id\"],right_on=[\"level_group\",\"session_id\"])","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:53:28.448455Z","iopub.execute_input":"2023-03-18T16:53:28.448872Z","iopub.status.idle":"2023-03-18T16:54:20.357502Z","shell.execute_reply.started":"2023-03-18T16:53:28.448835Z","shell.execute_reply":"2023-03-18T16:54:20.356513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# compute Correlation\nlabels_agg.iloc[:,2:].corr()","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:54:20.359292Z","iopub.execute_input":"2023-03-18T16:54:20.359873Z","iopub.status.idle":"2023-03-18T16:54:20.372374Z","shell.execute_reply.started":"2023-03-18T16:54:20.359836Z","shell.execute_reply":"2023-03-18T16:54:20.371222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# mutual information\nfrom sklearn.feature_selection import mutual_info_regression\nprint(mutual_info_regression(labels_agg[[\"heavyclicker\"]], labels_agg[\"labels_agg\"],random_state=1))","metadata":{"execution":{"iopub.status.busy":"2023-03-18T16:54:20.373937Z","iopub.execute_input":"2023-03-18T16:54:20.374516Z","iopub.status.idle":"2023-03-18T16:54:20.630339Z","shell.execute_reply.started":"2023-03-18T16:54:20.374473Z","shell.execute_reply":"2023-03-18T16:54:20.629230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As predicted, Heavy Clickers indeed underperform their peers. This is potentially not only due to the definition of our label in the kaggle competition, but might also imply different personality types that are differently successful in the game on average. ","metadata":{}},{"cell_type":"markdown","source":"If you liked this notebook, do not forget to **upvote**. I will continue the investigation of player clicking characteristics in future notebooks.","metadata":{}}]}