{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🎓[Detailed EDA] Predict Student Performance from Game Play\n---\nThis competition organized by [The Learning Agency Lab](https://www.kaggle.com/competitions/predict-student-performance-from-game-play) aims to predict the **performance of students while playing an educational game**. The data available to train a model is a large game log. The development of such a tool will assist developers in creating a more effective learning experience for students.\n\nThe peculiarities of this competition are the imposed constraints:\n- The virtual machine only has 2 cores and 8GB of RAM\n- The use of GPU is prohibited.\n\nFirst, let's start this journey by **exploring the data**. I don't know yet what it is all about but I am going to discover it with you in this EDA.\n\n# ✈️ Let's get started\nWe need to predict whether each student answered each question correctly, which involves binary classification for each question in each student session.\n\n## Competition metric\nThe evaluation metric is the F1-score, which is a common metric used for classification tasks. Its formula is as follow:\n\n$$ F_{1} = 2 \\cdot \\frac{precision \\cdot recall}{precision + recall} $$\nWhere,\n$$ precision = \\frac{TP}{TP + FP} \\hspace{1cm} recall = \\frac{TP}{TP + FN} $$\n\n## Efficiency prize\nThe efficiency prize combines both the F1-score and runtime. Just because a model has the highest leaderboard score, it doesn't necessarily mean it's the best in the real world. For example, a more accurate model may take days to run, while a less accurate one could run in a fraction of seconds. In such cases, which model is the best depends on a trade-off between accuracy and efficiency. The efficiency score is computed using the following formula:\n\nBenchmark is the score of the benchmark `sample_submission.csv` and maxF1 is the best LB score.\n\n\\begin{equation}\nEfficiency = \\frac{1}{Benchmark - maxF1}.F1 + \\frac{1}{32400}.RuntimeSeconds\n\\end{equation}\n\n\n\n## Data\nThe primary components of the data package are various CSV files. There is the standard `sample_submission.csv` file, which provides an example of the required submission format. Additionally, there are the `train.csv` and `test.csv` sets, as well as a separate file that contains labels for the training set. There is also a folder named `jo_wilder` that sets up the new environment.\n\n# 🛠️ Imports\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as mpatches\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2023-03-24T03:40:43.329288Z","iopub.execute_input":"2023-03-24T03:40:43.330225Z","iopub.status.idle":"2023-03-24T03:40:44.378305Z","shell.execute_reply.started":"2023-03-24T03:40:43.330138Z","shell.execute_reply":"2023-03-24T03:40:44.377164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 💾 Read the files\nWe load the csv files with pandas.","metadata":{}},{"cell_type":"code","source":"PATH = '/kaggle/input/predict-student-performance-from-game-play'\n\ntrain_df = pd.read_csv(f'{PATH}/train.csv')\ntest_df = pd.read_csv(f'{PATH}/test.csv')\nlabels_df = pd.read_csv(f'{PATH}/train_labels.csv')\nsubmission_df = pd.read_csv(f'{PATH}/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2023-03-24T03:40:44.381437Z","iopub.execute_input":"2023-03-24T03:40:44.381866Z","iopub.status.idle":"2023-03-24T03:42:35.397467Z","shell.execute_reply.started":"2023-03-24T03:40:44.381826Z","shell.execute_reply":"2023-03-24T03:42:35.387931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 👁️ Exploring the data\nLet's look at the dimensions of the train and test dataframes. We also display the first 2 rows of the train set.","metadata":{}},{"cell_type":"code","source":"print(\"Train: rows\", len(train_df), \"| columns\", len(train_df.columns))\nprint(\"Test:  rows    \", len(test_df), \"| columns\", len(test_df.columns))\n\ntrain_df.head(2)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:35.406322Z","iopub.execute_input":"2023-03-24T03:42:35.406870Z","iopub.status.idle":"2023-03-24T03:42:35.459631Z","shell.execute_reply.started":"2023-03-24T03:42:35.406838Z","shell.execute_reply":"2023-03-24T03:42:35.458690Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- It is quite a big dataframe (the train.csv file is more than 2GB large). It has more than 13 millions rows and a few features.\n- The test dataframe has an extra columns called `session_level` which will be used to create the submission file.\n\nLet's see if all features are relevant by analysing them.\n## 💡 Missing values","metadata":{}},{"cell_type":"code","source":"train_missing_ratios = train_df.isna().sum() / len(train_df)\ntest_missing_ratios = test_df.isna().sum() / len(test_df)\n\nplt.figure(figsize=(10, 4))\n\nplt.subplot(1, 2, 1)\nplt.bar(train_missing_ratios.index,\n        train_missing_ratios.values,\n        color=['red' if ratio == 1 else 'orange' for ratio in train_missing_ratios.values])\nplt.xlabel('Feature', fontsize=12)\nplt.ylabel('Missing values ratio', fontsize=12)\nplt.title('Missing values in TRAIN SET', fontsize=16)\nplt.xticks(rotation=90)\nplt.legend(handles=[mpatches.Patch(color='orange'),\n                    mpatches.Patch(color='red')], \n           labels=['Partially missing values', 'Completely missing values'])\n\nplt.subplot(1, 2, 2)\nplt.bar(test_missing_ratios.index,\n        test_missing_ratios.values,\n        color=['red' if ratio == 1 else 'orange' for ratio in test_missing_ratios.values])\nplt.title('Missing values in TEST SET', fontsize=16)\nplt.xticks(rotation=90)\n\nplt.tight_layout()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:35.462165Z","iopub.execute_input":"2023-03-24T03:42:35.462712Z","iopub.status.idle":"2023-03-24T03:42:43.409495Z","shell.execute_reply.started":"2023-03-24T03:42:35.462677Z","shell.execute_reply":"2023-03-24T03:42:43.408497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### - Insights\n- There are a lot of missing values in some columns.\n- We will drop the `fullscreen`, `hq` and `music` features at least, and maybe the ones with more than 80% of missing values. To make this decision about keeping or not the columns with high missing ratio, we will have to evaluate the relevance of them.\n- The train and test dataframes have similar ratios of missing values which are good news.\n\n## 💡 Analysis of the features\nLet's analyse all the features in order to understand what they mean.\n### 'Session id' and 'index'\n> The ID of the session the event took place in\n\n> The index of the event for the session","metadata":{}},{"cell_type":"code","source":"train_events_per_session = train_df['session_id'].value_counts()\ntest_events_per_session = test_df['session_id'].value_counts()\n\ntrain_percentiles, test_percentiles = [], []\nxrange = range(0, 100, 10)\nfor q in xrange:\n    train_perc = np.percentile(train_events_per_session, q)\n    test_perc = np.percentile(test_events_per_session, q)\n    train_percentiles.append(train_perc)\n    test_percentiles.append(test_perc)\n    \nplt.figure(figsize=(12, 8))\n\nplt.subplot(2, 2, 1)\nsns.histplot(train_events_per_session.values, kde=True)\nplt.xlabel(\"Number of events per session\", fontsize=12)\nplt.ylabel(\"Count\", fontsize=12)\nplt.title(\"TRAIN SET\", fontsize=16)\nplt.xlim(600, 2000)\n\nplt.subplot(2, 2, 2)\nsns.histplot(test_events_per_session.values, bins=20)\nplt.xlabel(\"Number of events per session\", fontsize=12), plt.ylabel(\"\")\nplt.title(\"TEST SET\", fontsize=16)\nplt.xlim(600, 2000)\n\nplt.subplot(2, 2, 3)\nplt.plot(xrange, train_percentiles)\nplt.axhline(train_events_per_session.median(), color='red', ls='--')\nplt.axhline(train_events_per_session.mean(), color='orange', ls='--')\nplt.xlabel(\"Percentile\", fontsize=12)\nplt.ylabel(\"Number of events per session\", fontsize=12)\nplt.xticks(xrange)\nplt.legend(['Nb of events', 'Median', 'Mean'])\n\nplt.subplot(2, 2, 4)\nplt.plot(xrange, test_percentiles)\nplt.axhline(test_events_per_session.median(), color='red', ls='--')\nplt.axhline(test_events_per_session.mean(), color='orange', ls='--')\nplt.xlabel(\"Percentile\", fontsize=12)\nplt.xticks(xrange)\n\nplt.tight_layout()\nplt.show()\n\ndata = {\"Index\": [\"Nb of sessions\", \"Min nb of events\", \"Max nb of events\"],\n        \"TRAIN\": [str(train_df['session_id'].nunique()),\n                  str(train_events_per_session.min()),\n                  str(train_events_per_session.max())],\n        \"TEST\": [str(test_df['session_id'].nunique()),\n                 str(test_events_per_session.min()),\n                 str(test_events_per_session.max())]}\ndf = pd.DataFrame(data).set_index('Index')\ndf","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:43.411415Z","iopub.execute_input":"2023-03-24T03:42:43.412093Z","iopub.status.idle":"2023-03-24T03:42:46.269522Z","shell.execute_reply.started":"2023-03-24T03:42:43.412056Z","shell.execute_reply":"2023-03-24T03:42:46.268517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- **Train set:**\n - A session typically encompasses anywhere between several hundred to a few thousand individual events.\n - Both the median and the mean are slightly over 1000.\n - The distribution appears to be normal with a slight positive skew.\n - Above the 90th percentile, the values for the number of events are very large. I set a x limit for visualization but the number of events go very far. Should we consider them as outliers?\n- **Test set:**\n - Only 3 sessions are represented in the test set having a fair amount of events.\n - The session with 1501 events is in the tail of the gaussian distribution but still in a reasonable area.\n - Do not forget that the available test set only represents the half of the final one.\n- **Events vs Level:**\n - Currently, we have only analyzed the number of events that occur during the sessions.\n - Keep in mind that a session is divided into question levels, and we aim to predict their success. However, the number of events can vary for each question level. It's possible that the number of actions (clicks) during a question may be related to a correct or incorrect answer. It would be interesting to inspect that later.\n \n### 'Elapsed time' and 'index'\n> How much time has passed (in milliseconds) between the start of the session and when the event was recorded.\n\nThe elapsed time should be correlated with the event index. As more time is spent, it is expected that a higher number of events will occur. This is why I have decided to calculate the mean for each event index and plot it.","metadata":{}},{"cell_type":"code","source":"# Average the elapsed time in seconds for each index\navg_elapsed_time = train_df.groupby('index')['elapsed_time'].mean() / 1000\n\nplt.figure(figsize=(10, 4))\nplt.plot(avg_elapsed_time)\nplt.axvline(2825, color='red', ls='--')\nplt.xlabel(\"Event index\", fontsize=12)\nplt.ylabel(\"Average elapsed time (sec)\", fontsize=12)\nplt.xlim([0, 4000])\nplt.legend(['Average elapsed time', 'Limit before non correlation'])\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:46.271421Z","iopub.execute_input":"2023-03-24T03:42:46.273526Z","iopub.status.idle":"2023-03-24T03:42:47.217788Z","shell.execute_reply.started":"2023-03-24T03:42:46.273468Z","shell.execute_reply":"2023-03-24T03:42:47.216819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- As expected, the average elapsed time increases with the increase in event index, but this trend only holds until around index 2800. After that, the behavior becomes erratic with a peak and then drops and remains constant until much later.\n- This phenomenon can be explained by the fact that as the number of events increases, there are fewer and fewer examples, and the average starts to represent unique cases which are likely outliers. These outliers may include individuals who did \"spam clicks\" to achieve a high number of events in a shorter amount of time or, in the case of the peak, individuals who were inactive for an extended period.\n\nLet's now check the elapsed times for individual cases and see if this trend remains the same.","metadata":{}},{"cell_type":"code","source":"# Get the sessions in the dataframe\nsession_ids = np.array(train_df['session_id'].unique())\n\nplt.figure(figsize=(10, 6))\nfor i in range(16):\n    plt.subplot(4, 4, i + 1)\n    times = train_df[train_df['session_id'] == session_ids[i]]['elapsed_time']\n    plt.plot(times.reset_index(drop=True) / 1000)\nplt.suptitle(\"Individual examples of elapsed time (sec) vs event index\")\nplt.tight_layout()\n#plt.show()   ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:47.219624Z","iopub.execute_input":"2023-03-24T03:42:47.220376Z","iopub.status.idle":"2023-03-24T03:42:49.257529Z","shell.execute_reply.started":"2023-03-24T03:42:47.220336Z","shell.execute_reply":"2023-03-24T03:42:49.255960Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the session ids corresponding to the peak\npeak_session_ids = train_df[train_df['elapsed_time'] > 4e7]['session_id'].unique()\n\nplt.figure(figsize=(10, 6))\nfor i in range(16):\n    plt.subplot(4, 4, i + 1)\n    times = train_df[train_df['session_id'] == peak_session_ids[i]]['elapsed_time']\n    plt.plot(times.reset_index(drop=True) / 1000)\nplt.suptitle(\"Examples corresponding to the peak: elapsed time (sec) vs event index\")\nplt.tight_layout()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:49.262353Z","iopub.execute_input":"2023-03-24T03:42:49.265324Z","iopub.status.idle":"2023-03-24T03:42:51.679734Z","shell.execute_reply.started":"2023-03-24T03:42:49.265276Z","shell.execute_reply":"2023-03-24T03:42:51.678574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"The longest session  of the TRAIN SET lasted about\",\n      int(train_df['elapsed_time'].max() / 8.64e7), \"days.\")\nprint(\"The longest session of the TEST SET lasted about\",\n      int(test_df['elapsed_time'].max() / 1000), \"sec.\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:51.681293Z","iopub.execute_input":"2023-03-24T03:42:51.681910Z","iopub.status.idle":"2023-03-24T03:42:51.716045Z","shell.execute_reply.started":"2023-03-24T03:42:51.681871Z","shell.execute_reply":"2023-03-24T03:42:51.714970Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- Almost all of the examples shown exhibit gaps, indicating brief pauses in activity.\n- Some examples, however, exhibit huge gaps due to extended periods of inactivity.\n- The maximum elapsed time in the test set appears to be more reasonable than the outliers observed in the training set. It may be beneficial to get rid of sessions in which the individual was inactive for an extended period of time.\n\n### 'Event name' and 'name'\n> The name of the event type.\n\n> The event name (e.g. identifies whether a notebook_click is is opening or closing the notebook)\n\nThese 2 columns are quite similar since they describe the action of the user. Let's display there distributions independently and in a pivot table. The data are from the train set.","metadata":{}},{"cell_type":"code","source":"event_names = train_df['event_name'].value_counts()\nnames = train_df['name'].value_counts()\n\nprint(\"Number of event names:\", len(event_names))\nprint(\"Number of names:\", len(names))\n\nplt.figure(figsize=(10, 4))\n\nplt.subplot(1, 2, 1)\nplt.bar(event_names.index, event_names.values, color='orange')\nplt.ylabel(\"Occurence counts\", fontsize=12)\nplt.title(\"Frequency of the 'event names'\", fontsize=16)\nplt.xticks(rotation=70)\n\nplt.subplot(1, 2, 2)\nplt.bar(names.index, names.values, color='brown')\nplt.title(\"Frequency of the 'names'\", fontsize=16)\nplt.xticks(rotation=70)\n\nplt.tight_layout()\nplt.show()\n\n# Pivot table\npivot_table = train_df.pivot_table(index='event_name', columns='name', aggfunc='size')\npivot_table = pivot_table.fillna(0).astype(int)\nplt.figure(figsize=(12, 4))\nsns.heatmap(pivot_table, annot=True, fmt='d', cmap='Greens')\nplt.title(\"Number of occurrences of 'event_name' and 'name'\", fontsize=16)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:51.719843Z","iopub.execute_input":"2023-03-24T03:42:51.720114Z","iopub.status.idle":"2023-03-24T03:42:59.653151Z","shell.execute_reply.started":"2023-03-24T03:42:51.720089Z","shell.execute_reply":"2023-03-24T03:42:59.650573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- The events mainly consist of clicks and occasionally hovers. I am uncertain about the distinction between the various types of clicks.\n- Certain \"names\" only occur with specific \"event_names\". Not all combinations are possible.\n- I have also checked the distribution in the test set, and it is comparable to the distribution in the training set.\n\n### 'Index' and 'level'\n> What level of the game the event occurred in (0 to 22)","metadata":{}},{"cell_type":"code","source":"# Nb of events averaged per question\ngrouped_df = train_df.groupby(['session_id', 'level'])\\\n    ['index'].count().reset_index()\ngrouped_df.columns = ['session_id', 'level', 'index_count']\nmean_counts = grouped_df.groupby('level').mean().drop('session_id', axis=1)\nmean_counts\n\nxrange = range(0, 23)\nplt.figure(figsize=(10, 4))\nplt.plot(mean_counts)\nplt.scatter(xrange, mean_counts, color='black')\nplt.title(\"Average number of events per question level\", fontsize=16)\nplt.xlabel(\"Average count of events\", fontsize=12)\nplt.ylabel(\"Level\", fontsize=12)\nplt.xticks(xrange)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:42:59.654750Z","iopub.execute_input":"2023-03-24T03:42:59.655147Z","iopub.status.idle":"2023-03-24T03:43:01.350455Z","shell.execute_reply.started":"2023-03-24T03:42:59.655108Z","shell.execute_reply":"2023-03-24T03:43:01.349483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot a few individual examples\nplt.figure(figsize=(14, 6))\nfor i in range(12):\n    plt.subplot(4, 3, i + 1)\n    example = grouped_df[\n        grouped_df['session_id'] == session_ids[i]]\\\n        ['index_count'].reset_index(drop=True)\n    plt.plot(example)\n    plt.scatter(range(0, 23), example, color='black')\n    plt.xticks(range(0, 23, 2))\nplt.suptitle(\"Number of events per level: individual session examples\",\n             fontsize=16)\nplt.tight_layout()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:01.352037Z","iopub.execute_input":"2023-03-24T03:43:01.352399Z","iopub.status.idle":"2023-03-24T03:43:02.775965Z","shell.execute_reply.started":"2023-03-24T03:43:01.352362Z","shell.execute_reply":"2023-03-24T03:43:02.774935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- A greater number of events take place during specific periods. For instance, there is a heightened level of event activity observed at level 18.\n- This pattern can also be observed in various individual cases as examples. However, each one remains unique.\n\n### 'Page'\n>  The page number of the event (only for notebook-related events)","metadata":{}},{"cell_type":"code","source":"train_page_number_counts = train_df['page'].value_counts().sort_index()\ntest_page_number_counts = test_df['page'].value_counts().sort_index()\nplt.figure(figsize=(10, 2))\n\nplt.subplot(1, 2, 1)\nplt.bar(range(0, 7), train_page_number_counts, color='gray')\nplt.xlabel(\"Page number\", fontsize=12)\nplt.ylabel(\"Count\", fontsize=12)\nplt.title(\"TRAIN SET\",\n          fontsize=16)\n\nplt.subplot(1, 2, 2)\nplt.bar(range(0, 7), test_page_number_counts, color='gray')\nplt.xlabel(\"Page number\", fontsize=12)\nplt.title(\"TEST SET\",\n          fontsize=16)\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:02.777153Z","iopub.execute_input":"2023-03-24T03:43:02.777871Z","iopub.status.idle":"2023-03-24T03:43:03.142427Z","shell.execute_reply.started":"2023-03-24T03:43:02.777830Z","shell.execute_reply":"2023-03-24T03:43:03.141405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- A lot of values are still missing in this columns (almost 98%) because a page is only indicated when the event is notebook-related.\n- I don't know if this information is relevant or not. Is it worth it to keep it and how should we fill in the missing values? With zeros?\n- The test set has a slightly different distribution from the train set for the page number.\n\n### 'Hover duration'\n> How long (in milliseconds) the hover happened for (only for hover events)","metadata":{}},{"cell_type":"code","source":"train_hover_durations = train_df['hover_duration'].dropna() / 1000\ntest_hover_durations = test_df['hover_duration'].dropna() / 1000\n\nxrange = range(0, 100, 10)\ntrain_percentiles , test_percentiles = [], []\nfor q in xrange:\n    train_perc = np.percentile(train_hover_durations, q)\n    test_perc = np.percentile(test_hover_durations, q)\n    train_percentiles.append(train_perc)\n    test_percentiles.append(test_perc)\n    \nplt.figure(figsize=(12, 4))\n\nplt.subplot(1, 2, 1)\nplt.plot(xrange, train_percentiles)\nplt.axhline(train_hover_durations.median(), color='red', ls='--')\nplt.axhline(train_hover_durations.mean(), color='orange', ls='--')\nplt.xticks(xrange)\nplt.yticks(range(0, 5))\nplt.legend(['Nb of events', 'Median', 'Mean'])\nplt.xlabel(\"Percentile\", fontsize=12)\nplt.ylabel(\"Hover duration in seconds\", fontsize=12)\nplt.title(\"TRAIN SET\", fontsize=16)\n\nplt.subplot(1, 2, 2)\nplt.plot(xrange, test_percentiles)\nplt.axhline(test_hover_durations.median(), color='red', ls='--')\nplt.axhline(test_hover_durations.mean(), color='orange', ls='--')\nplt.xticks(xrange)\nplt.xlabel(\"Percentile\", fontsize=12)\nplt.title(\"TEST SET\", fontsize=16)\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:03.143829Z","iopub.execute_input":"2023-03-24T03:43:03.145800Z","iopub.status.idle":"2023-03-24T03:43:04.116918Z","shell.execute_reply.started":"2023-03-24T03:43:03.145758Z","shell.execute_reply":"2023-03-24T03:43:04.115322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- Similar to the page feature, the hover duration is indicated only for a few rows during an hover event. Typically, the duration lasts anywhere from a few milliseconds to a few seconds, with rare instances lasting more than 4 seconds.\n- In the training set, we can observe that the mean is significantly higher than the median, which is due to the presence of outliers where the user remained on hover for an extended period. This type of examples should be avoided.\n\n### 'Text'\n> The text the player sees during this event","metadata":{}},{"cell_type":"code","source":"train_texts = train_df['text'].dropna().value_counts()\ntest_texts = test_df['text'].dropna().value_counts()\nprint(\"TRAIN | nb of unique texts:\", len(train_texts))\nprint('-' * 60)\nprint(train_texts, '\\n')\nprint(\"TEST | nb of unique texts:\", len(test_texts))\nprint('-' * 60)\nprint(test_texts, '\\n')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:04.118766Z","iopub.execute_input":"2023-03-24T03:43:04.119500Z","iopub.status.idle":"2023-03-24T03:43:05.893828Z","shell.execute_reply.started":"2023-03-24T03:43:04.119457Z","shell.execute_reply":"2023-03-24T03:43:05.892638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- There are many different phrases from the game and some are more common than others.\n- Some text is in ASCII unicode hex format, such as '\\u00f0\\u0178\\u02dc\\u0090'.\n- It may be useful to process these texts in order to generate additional features from them.\n\n### 'Fqid', 'room_fqid' and 'text_fqid'\n> The fully qualified ID of the event\n\n> The fully qualified ID of the room the event took place in\n\n> The fully qualified ID of the text","metadata":{}},{"cell_type":"code","source":"# Count values\ntrain_fqids = train_df['fqid'].value_counts()\ntrain_room_fqids = train_df['room_fqid'].value_counts()\ntrain_text_fqids = train_df['text_fqid'].value_counts()\ntest_fqids = test_df['fqid'].value_counts()\ntest_room_fqids = test_df['room_fqid'].value_counts()\ntest_text_fqids = test_df['text_fqid'].value_counts()\ntrain_fqid_bundle = [train_fqids, train_room_fqids, train_text_fqids]\ntest_fqid_bundle = [test_fqids, test_room_fqids, test_text_fqids]\nfqid_labels = [\"fqid\", \"room_fqid\", \"text_fqid\"]\n\n# Display the unique values\ndef print_fqids(set_name, bundle):\n    for label, value in zip(fqid_labels, bundle):\n        print('-' * 60)\n        print(set_name, label)\n        print('-' * 60)\n        print(value)  \nprint_fqids('TRAIN', train_fqid_bundle)\nprint_fqids('TEST', test_fqid_bundle)\n\n# Number of unique values recap table\ndata = {\"Nb unique values\": fqid_labels,\n        \"TRAIN\": [len(x) for x in train_fqid_bundle],\n        \"TEST\": [len(x) for x in test_fqid_bundle]}\ndf = pd.DataFrame(data).set_index('Nb unique values')\ndf","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:05.895604Z","iopub.execute_input":"2023-03-24T03:43:05.896008Z","iopub.status.idle":"2023-03-24T03:43:09.293613Z","shell.execute_reply.started":"2023-03-24T03:43:05.895968Z","shell.execute_reply":"2023-03-24T03:43:09.292575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- These three features provide information about the player's conversation partner, the player's location in the game during the action, and the text identifier.\n- The 'Fqid' appears to be the identifier of the person or object the player is interacting with, or something more abstract. There are slightly more of these values in the training set.\n- The 'room_fqid' is composed of three words separated by periods. There are no missing values in this column, and there are 19 rooms in the game. Both the training and test sets include all of the rooms. I believe that this feature will be very useful for the model once it has been one-hot encoded. It is also possible to create new features from it by decomposing the string.\n- The 'text_fqid' feature seems to combine the 'room_fqid' and the 'fqid' and add additional information about the text itself. It may be possible to extract the last part of this text to create a new feature.\n\n### 'Level group'\n> Which group of levels - and group of questions - this row belongs to (0-4, 5-12, 13-22)\n\nThis is groups the question levels in 3 ranges. Maybe the difficulty increses along these groups. Nothing else to say about it right now.\n\n### 'Room coordinates' and 'screen coordinates' x and y\n> The coordinates of the click in reference to the in-game room (only for click events)\n\n> The coordinates of the click in reference to the player’s screen (only for click events)\n\nLet's plot the coordinates in order to have a better understanding of what they look like. They might show the regions of interest of the game player.","metadata":{}},{"cell_type":"code","source":"def plot_coordinates(i, set_name):\n    df = train_df if set_name == 'TRAIN' else test_df\n    session_ids = np.array(df['session_id'].unique())\n    one_session = df[df['session_id'] == session_ids[i]]\n    plt.figure(figsize=(14, 4))\n    \n    # Room coordinates\n    coords = one_session[['room_coor_x', 'room_coor_y']].dropna().reset_index(drop=True)\n    x = coords['room_coor_x']\n    y = coords['room_coor_y']\n    plt.subplot(1, 2, 1)\n    plt.plot(x, y, zorder=0, lw=0.5)\n    plt.scatter(x, y, s=5, color='black')\n    plt.scatter(x[0], y[0], s=200, lw=5, color='red', marker='+')\n    plt.scatter(x[-1:], y[-1:], s=200, lw=5, color='orange', marker='+')\n    plt.legend(['Cursor click path', 'Clicks', 'Start', 'End'])\n    plt.gca().set_aspect('equal', adjustable='box')\n    plt.xlabel(\"x\")\n    plt.ylabel(\"y\")\n    plt.title(f\"{set_name} | Room coordinates | Session {i}\")\n    \n    # Screen coordinates\n    coords = one_session[['screen_coor_x', 'screen_coor_y']]\\\n        .dropna().reset_index(drop=True)\n    x = coords['screen_coor_x']\n    y = coords['screen_coor_y']\n    plt.subplot(1, 2, 2)\n    plt.plot(x, y, zorder=0, lw=0.5, color='green')\n    plt.scatter(x, y, s=5, color='black')\n    plt.scatter(x[0], y[0], s=200, lw=5, color='red', marker='+')\n    plt.scatter(x[-1:], y[-1:], s=200, lw=5, color='orange', marker='+')\n    plt.gca().set_aspect('equal', adjustable='box')\n    plt.xlabel(\"x\")\n    plt.ylabel(\"y\")\n    plt.title(f\"{set_name} | Screen coordinates | Session {i}\")\n    \n    plt.tight_layout()\n    plt.show()\n\n# Plot 3 session examples from the train set\nfor i in [27, 3, 99]:\n    plot_coordinates(i, 'TRAIN')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:09.294972Z","iopub.execute_input":"2023-03-24T03:43:09.295443Z","iopub.status.idle":"2023-03-24T03:43:12.104401Z","shell.execute_reply.started":"2023-03-24T03:43:09.295406Z","shell.execute_reply":"2023-03-24T03:43:12.103419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot coordinates of all the sessions of the test set (only 3 sessions)\nfor i in range(3):\n    plot_coordinates(i, 'TEST')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:12.105995Z","iopub.execute_input":"2023-03-24T03:43:12.107094Z","iopub.status.idle":"2023-03-24T03:43:13.672032Z","shell.execute_reply.started":"2023-03-24T03:43:12.107054Z","shell.execute_reply":"2023-03-24T03:43:13.668009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - Insights\n- The clicking patterns across various sessions are alike. There are noticeable clusters of clicks in certain regions.\n- An analysis of the screen coordinates reveals the presence of at least three potential button clusters located at the top-left, bottom-right, and top of the screen. The user repeatedly navigated between these clusters, resulting in a concentration of lines in the diagonal direction.\n- These coordinates are probably a wealth of information that allows us to track the user's actions, making it valuable information for predictions.\n\n## 💡 Labels\nLet's see the distribution of the different classes.\n- 0: incorrect answer to the question\n- 1: correct answer to the question","metadata":{}},{"cell_type":"code","source":"classes_count = labels_df['correct'].value_counts()\nprint(\"Classes count:\\n\", classes_count, \"\\n\")\nprint(\"Ratio:\\n\", classes_count / len(labels_df))\nlabels_df","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-24T03:43:13.673422Z","iopub.execute_input":"2023-03-24T03:43:13.674501Z","iopub.status.idle":"2023-03-24T03:43:13.697245Z","shell.execute_reply.started":"2023-03-24T03:43:13.674463Z","shell.execute_reply":"2023-03-24T03:43:13.694166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_df['question'] = labels_df['session_id'].apply(lambda x: int(x.split('_')[1][1:]))\n\n# Calculate correct ratios\ncorrect_ratios = []\nfor q in range(1, 19):\n    tmp = labels_df[labels_df['question'] == q]['correct']\n    ratio = tmp.sum() / len(tmp) \n    correct_ratios.append(ratio)\n\nxrange = range(1, 19)\nmean = np.mean(correct_ratios)\nplt.figure(figsize=(8, 4))\nplt.bar(x=xrange, height=correct_ratios, color='purple')\nplt.axhline(mean, color='red', ls='--', lw=3)\nplt.xticks(xrange)\nplt.legend([f'Mean: {mean:.3f}'])\nplt.title(\"Correct answers per question\")\nplt.xlabel(\"Question number\")\nplt.ylabel(\"Ratio\")\nplt.show()","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2023-03-24T03:43:13.698759Z","iopub.execute_input":"2023-03-24T03:43:13.699695Z","iopub.status.idle":"2023-03-24T03:43:14.351640Z","shell.execute_reply.started":"2023-03-24T03:43:13.699654Z","shell.execute_reply":"2023-03-24T03:43:14.350534Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### -Insights\n- About 70% of the answers are correct and 30% are incorrect.\n- The dataset is slightly imbalanced.\n- The labels only go up to level 18, even though the questions extend up to level 22 in the training set.","metadata":{}},{"cell_type":"markdown","source":"**Thank you for reading my EDA. If you have any suggestion, please let me know!**","metadata":{}}]}