{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<h1 style=\"text-align: center;\"><font color=\"blue\"> Why are some sessions so long?</h1>\n<h2><center> ","metadata":{}},{"cell_type":"markdown","source":"There are two measures for the length of a session: the number of events and the elapsed time. Here we will consider the **number of events**. It is the number of rows with the same session_id.  [**Part II**](https://www.kaggle.com/code/gehallak/why-are-some-sessions-so-long-part-ii) will be about the elapsed time.","metadata":{}},{"cell_type":"markdown","source":"# 1. Group by sessions\nWe group by sessions and look at the distributions.","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\ntrain = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv')\ntrain['elapsed_time']=train['elapsed_time']/1000.","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-03-25T10:29:05.710321Z","iopub.execute_input":"2023-03-25T10:29:05.710944Z","iopub.status.idle":"2023-03-25T10:31:03.392868Z","shell.execute_reply.started":"2023-03-25T10:29:05.710904Z","shell.execute_reply":"2023-03-25T10:31:03.391658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_session = train.groupby('session_id').agg({'elapsed_time': ['max'], 'session_id': ['count']})\ntrain_session.columns=['max_time','event_nb']","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:31:03.395137Z","iopub.execute_input":"2023-03-25T10:31:03.396768Z","iopub.status.idle":"2023-03-25T10:31:03.988934Z","shell.execute_reply.started":"2023-03-25T10:31:03.396725Z","shell.execute_reply":"2023-03-25T10:31:03.987693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_session.hist('event_nb',bins=50)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:31:03.990700Z","iopub.execute_input":"2023-03-25T10:31:03.991099Z","iopub.status.idle":"2023-03-25T10:31:04.313210Z","shell.execute_reply.started":"2023-03-25T10:31:03.991060Z","shell.execute_reply":"2023-03-25T10:31:04.312165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Most sessions have around 1,000 events. But there are ouliers on the right.","metadata":{}},{"cell_type":"code","source":"train_session['event_nb'].quantile([0, .25, .50, .75 ,0.99,1])","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:37:39.703061Z","iopub.execute_input":"2023-03-25T10:37:39.704217Z","iopub.status.idle":"2023-03-25T10:37:39.717850Z","shell.execute_reply.started":"2023-03-25T10:37:39.704166Z","shell.execute_reply":"2023-03-25T10:37:39.716807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_session=train_session.sort_values('event_nb')\nprint(train_session.head(5))\nprint(train_session.tail(5))","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:31:04.330830Z","iopub.execute_input":"2023-03-25T10:31:04.331227Z","iopub.status.idle":"2023-03-25T10:31:04.344418Z","shell.execute_reply.started":"2023-03-25T10:31:04.331189Z","shell.execute_reply":"2023-03-25T10:31:04.343305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The session with the minimum number of events is 22000610592348800 . it has **591**  events.\n\nThe session with the maximum number of events is 20110310463989760 . it has **19,032**  events.","metadata":{}},{"cell_type":"markdown","source":"# 2. Longest and shortest sessions","metadata":{}},{"cell_type":"markdown","source":"Here we look at both sessions with the most events and least events.","metadata":{}},{"cell_type":"code","source":"SHORTEST_SESSION_ID=22000610592348800\nLONGEST_SESSION_ID=20110310463989760","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:31:04.346157Z","iopub.execute_input":"2023-03-25T10:31:04.346553Z","iopub.status.idle":"2023-03-25T10:31:04.353170Z","shell.execute_reply.started":"2023-03-25T10:31:04.346515Z","shell.execute_reply":"2023-03-25T10:31:04.352141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"longest_session_df=train[train['session_id']==LONGEST_SESSION_ID].reset_index(drop=True)\nshortest_session_df=train[train['session_id']==SHORTEST_SESSION_ID].reset_index(drop=True)\n\nfig, (ax1,ax2) = plt.subplots(1,2, figsize=(10,4))  # 1 row, 2 columns\nlongest_session_df.plot(y='elapsed_time', ax=ax1)\nshortest_session_df.plot(y='elapsed_time', ax=ax2)\nax1.set_title(\"longest session with 19,032 events\")\nax2.set_title(\"shortest session with 591 events\")\nplt.tight_layout() \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:39:59.164535Z","iopub.execute_input":"2023-03-25T10:39:59.165188Z","iopub.status.idle":"2023-03-25T10:40:00.020819Z","shell.execute_reply.started":"2023-03-25T10:39:59.165148Z","shell.execute_reply":"2023-03-25T10:40:00.019775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The elapsed times between the session with the most events and the least events are similar. The player with a lot of events is not spending much more time, he just generates events much faster!! Let's see what kind of events makes the difference.","metadata":{}},{"cell_type":"code","source":"shortest_session_events = shortest_session_df.groupby('event_name').agg({'event_name': ['count']})\nshortest_session_events","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:31:06.056108Z","iopub.execute_input":"2023-03-25T10:31:06.056754Z","iopub.status.idle":"2023-03-25T10:31:06.073870Z","shell.execute_reply.started":"2023-03-25T10:31:06.056715Z","shell.execute_reply":"2023-03-25T10:31:06.072518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"longest_session_events = longest_session_df.groupby('event_name').agg({'event_name': ['count']})\nlongest_session_events","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:31:06.075373Z","iopub.execute_input":"2023-03-25T10:31:06.075850Z","iopub.status.idle":"2023-03-25T10:31:06.092784Z","shell.execute_reply.started":"2023-03-25T10:31:06.075812Z","shell.execute_reply":"2023-03-25T10:31:06.091824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The difference is mostly on **click events**. The player with the most events is apparently clicking frenetically on everything, and generated 19,032 events. In comparison, in the shortest session, the player is very focused, doesn't click randomly and only used 591 events to complete the game. ","metadata":{}},{"cell_type":"markdown","source":"We would like to know if these different behaviours are reflected in the score. (how many questions did they get right). For this we will use **train_labels**.","metadata":{}},{"cell_type":"code","source":"train_labels = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train_labels.csv')\ntrain_labels['session'] = train_labels.session_id.apply(lambda x: int(x.split('_')[0]))\ntrain_labels['q'] = train_labels.session_id.apply(lambda x: int(x.split('_')[-1][1:]))","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:33:23.216098Z","iopub.execute_input":"2023-03-25T10:33:23.219036Z","iopub.status.idle":"2023-03-25T10:33:24.612433Z","shell.execute_reply.started":"2023-03-25T10:33:23.218985Z","shell.execute_reply":"2023-03-25T10:33:24.611350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels_shortest=train_labels[train_labels['session']==SHORTEST_SESSION_ID]\nprint(f'The shortest session score is: {train_labels_shortest.correct.sum()}/18')\ntrain_labels_longest=train_labels[train_labels['session']==LONGEST_SESSION_ID]\nprint(f'The longest session score is: {train_labels_longest.correct.sum()}/18')","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:34:17.414654Z","iopub.execute_input":"2023-03-25T10:34:17.415636Z","iopub.status.idle":"2023-03-25T10:34:17.425070Z","shell.execute_reply.started":"2023-03-25T10:34:17.415595Z","shell.execute_reply":"2023-03-25T10:34:17.423941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see in these two extreme sessions that **the longest session has a bad score while the shortest has a good score**. Is this a general rule ? Let's find out!\n","metadata":{}},{"cell_type":"markdown","source":"# 3. Session Length and score.","metadata":{}},{"cell_type":"code","source":"train_labels_score=train_labels.groupby('session').agg({'correct': ['sum']}).reset_index()\ntrain_labels_score.columns=['session_id','score']\n","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:34:42.361594Z","iopub.execute_input":"2023-03-25T10:34:42.362232Z","iopub.status.idle":"2023-03-25T10:34:42.399880Z","shell.execute_reply.started":"2023-03-25T10:34:42.362192Z","shell.execute_reply":"2023-03-25T10:34:42.398713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_session=train_session.merge(train_labels_score,on='session_id',how='inner')","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:34:58.191255Z","iopub.execute_input":"2023-03-25T10:34:58.192598Z","iopub.status.idle":"2023-03-25T10:34:58.213848Z","shell.execute_reply.started":"2023-03-25T10:34:58.192555Z","shell.execute_reply":"2023-03-25T10:34:58.212761Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_session_score=train_session.groupby('score').agg({'event_nb': ['mean']}).reset_index()\ntrain_session_score.plot.scatter(x='score',y='event_nb')\nplt.xticks(np.arange(1,19))\nplt.xlabel('correct answers')\nplt.ylabel('average number of events')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-25T10:36:47.433979Z","iopub.execute_input":"2023-03-25T10:36:47.434628Z","iopub.status.idle":"2023-03-25T10:36:47.697762Z","shell.execute_reply.started":"2023-03-25T10:36:47.434587Z","shell.execute_reply":"2023-03-25T10:36:47.696447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is a very clear relation between the number of events and the score.\n\nA player clicking frenetically and randomly will generate a lot of event and get a bad score.\n\nA focused player will know exactly where to click, generate less events and get a good score.","metadata":{}},{"cell_type":"markdown","source":"# 4. Conclusion","metadata":{}},{"cell_type":"markdown","source":"Hopefully, this notebook will give you some ideas. Maybe some new features to try or some outlier sessions to remove ?","metadata":{}}]}