{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Predict Student Performance from Game Play","metadata":{}},{"cell_type":"markdown","source":"**Goal** :  predict student performance during game-based learning in real-time.   \\\nThe data contained items related to the game event.    \\\nBefore modeling, I would like to understand the characteristics of data through EDA.\n\n\nThe process is as follows :\n1. Missing values, correlation of columns, and outlier\n2. Visualization of the characteristics of each column\n3. etc.\n","metadata":{}},{"cell_type":"markdown","source":"# Load data","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport gc\nimport plotly.express as px\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots","metadata":{"execution":{"iopub.status.busy":"2023-04-01T09:15:56.203101Z","iopub.execute_input":"2023-04-01T09:15:56.203596Z","iopub.status.idle":"2023-04-01T09:16:00.271003Z","shell.execute_reply.started":"2023-04-01T09:15:56.203552Z","shell.execute_reply":"2023-04-01T09:16:00.269837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train  = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv')\nlabel = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train_labels.csv')\n# test = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/test.csv')\nprint('load finish')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-04-01T08:37:13.169157Z","iopub.execute_input":"2023-04-01T08:37:13.169524Z","iopub.status.idle":"2023-04-01T08:39:07.868848Z","shell.execute_reply.started":"2023-04-01T08:37:13.169493Z","shell.execute_reply":"2023-04-01T08:39:07.867716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head(3)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:40:30.236165Z","iopub.execute_input":"2023-04-01T08:40:30.236554Z","iopub.status.idle":"2023-04-01T08:40:30.266379Z","shell.execute_reply.started":"2023-04-01T08:40:30.236521Z","shell.execute_reply":"2023-04-01T08:40:30.265341Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.shape","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:40:31.412256Z","iopub.execute_input":"2023-04-01T08:40:31.412941Z","iopub.status.idle":"2023-04-01T08:40:31.419894Z","shell.execute_reply.started":"2023-04-01T08:40:31.412905Z","shell.execute_reply":"2023-04-01T08:40:31.418734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you can see in description, there are 20 columns in data. \n\n**Columns**\n* session_id - the ID of the session the event took place in\n* index - the index of the event for the session\n* elapsed_time - how much time has passed (in milliseconds) between the start of the session and when the event was recorded\n* event_name - the name of the event type\n* name - the event name (e.g. identifies whether a notebook_click is is opening or closing the notebook)\n* level - what level of the game the event occurred in (0 to 22)\n* page - the page number of the event (only for notebook-related events)\n* room_coor_x - the coordinates of the click in reference to the in-game room (only for click events)\n* room_coor_y - the coordinates of the click in reference to the in-game room (only for click events)\n* screen_coor_x - the coordinates of the click in reference to the player’s screen (only for click events)\n* screen_coor_y - the coordinates of the click in reference to the player’s screen (only for click events)\n* hover_duration - how long (in milliseconds) the hover happened for (only for hover events)\n* text - the text the player sees during this event\n* fqid - the fully qualified ID of the event\n* room_fqid - the fully qualified ID of the room the event took place in\n* text_fqid - the fully qualified ID of the\n* fullscreen - whether the player is in fullscreen mode\n* hq - whether the game is in high-quality\n* music - whether the game music is on or off\n* level_group - which group of levels - and group of questions - this row belongs to (0-4, 5-12, 13-22)","metadata":{}},{"cell_type":"code","source":"train.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:40:40.073827Z","iopub.execute_input":"2023-04-01T08:40:40.074589Z","iopub.status.idle":"2023-04-01T08:40:46.763613Z","shell.execute_reply.started":"2023-04-01T08:40:40.074550Z","shell.execute_reply":"2023-04-01T08:40:46.762395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* (original data) Since the coordinate-related variables generate information only in click events, 2073272 missing values seem to be caused by not click events \\\n(room_coor_x, room_coor_y, screen_coor_x, screen_coor_y)\n* For similar reasons, hover duraion appears to have produced 24294702 missing values because no information is generated from the click event.\n* For the other four columns, I will find out why there were missing values later...     \\\n(page, test, fqid and text_fqid)\n\n","metadata":{}},{"cell_type":"code","source":"# for numeric variable\ncorr = train[['session_id', 'index', 'elapsed_time','level', 'page',  \n              'hover_duration', 'fullscreen', 'hq', 'music']].corr()\nfig = go.Figure(data= go.Heatmap(z=corr,\n                                 x=corr.index.values,\n                                 y=corr.columns.values,\n                                 colorscale='earth',\n                                 hoverongaps = False\n                                 \n                                 )\n                )\n\nfig.update_layout(margin = dict(t=200,r=200,b=200,l=200),\n    showlegend = False,\n    width = 700, height = 700,\n    autosize = False )\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:40:49.065778Z","iopub.execute_input":"2023-04-01T08:40:49.066507Z","iopub.status.idle":"2023-04-01T08:40:55.215933Z","shell.execute_reply.started":"2023-04-01T08:40:49.066468Z","shell.execute_reply":"2023-04-01T08:40:55.214981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I ignore hovertext when we have missing values in the data by setting the hoverongaps to False.     ","metadata":{}},{"cell_type":"code","source":"high_corr = np.where(corr.abs()>0.5)\nhigh_corr = [[corr.index[x], corr.columns[y]] for x, y in zip(*high_corr) if x!=y and x<y]\nhigh_corr","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:40:55.502661Z","iopub.execute_input":"2023-04-01T08:40:55.502995Z","iopub.status.idle":"2023-04-01T08:40:55.511290Z","shell.execute_reply.started":"2023-04-01T08:40:55.502943Z","shell.execute_reply":"2023-04-01T08:40:55.510251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"index & level and level & page have high correlation.    ","metadata":{}},{"cell_type":"markdown","source":"I'll look for an outlier in every column (next session..)","metadata":{}},{"cell_type":"code","source":"label.head()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:41:02.636037Z","iopub.execute_input":"2023-04-01T08:41:02.636933Z","iopub.status.idle":"2023-04-01T08:41:02.646829Z","shell.execute_reply.started":"2023-04-01T08:41:02.636896Z","shell.execute_reply":"2023-04-01T08:41:02.645470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(label.shape)\nprint('target counts: ', len(label[label.correct==1]))\nprint('# of unique session_id in train_label (= our target) data', label.loc[:,'session_id'].astype(str).str.split('_').str[0].nunique())","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:41:06.100407Z","iopub.execute_input":"2023-04-01T08:41:06.100873Z","iopub.status.idle":"2023-04-01T08:41:07.644563Z","shell.execute_reply.started":"2023-04-01T08:41:06.100831Z","shell.execute_reply":"2023-04-01T08:41:07.643456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label[label.loc[:,'session_id'].astype(str).str.split('_').str[0]==label.loc[:,'session_id'].astype(str).str.split('_').str[0][0]] \n# '20090312431273200'","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:41:10.194931Z","iopub.execute_input":"2023-04-01T08:41:10.195416Z","iopub.status.idle":"2023-04-01T08:41:13.885117Z","shell.execute_reply.started":"2023-04-01T08:41:10.195375Z","shell.execute_reply":"2023-04-01T08:41:13.884036Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label.loc[:,'session_id'].astype(str).str.split('_').str[1].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:41:18.159606Z","iopub.execute_input":"2023-04-01T08:41:18.160004Z","iopub.status.idle":"2023-04-01T08:41:19.059254Z","shell.execute_reply.started":"2023-04-01T08:41:18.159947Z","shell.execute_reply":"2023-04-01T08:41:19.058029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are 18 different categories for each ID (This categories do not match with level or other variables in train data)","metadata":{}},{"cell_type":"code","source":"del train, label\n\nfor i in range(5):\n    gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:41:25.731087Z","iopub.execute_input":"2023-04-01T08:41:25.732025Z","iopub.status.idle":"2023-04-01T08:41:26.371581Z","shell.execute_reply.started":"2023-04-01T08:41:25.731976Z","shell.execute_reply":"2023-04-01T08:41:26.370577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"------------------------------------------------------------------","metadata":{}},{"cell_type":"markdown","source":"# Exploring by Variable","metadata":{}},{"cell_type":"markdown","source":"# **1. session id**  \nnot unique","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['session_id', 'level'])\ntrain.loc[:,'session_id'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:41:54.255886Z","iopub.execute_input":"2023-04-01T08:41:54.256480Z","iopub.status.idle":"2023-04-01T08:43:07.476326Z","shell.execute_reply.started":"2023-04-01T08:41:54.256433Z","shell.execute_reply":"2023-04-01T08:43:07.475286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"19,032 ~ 591 ","metadata":{}},{"cell_type":"code","source":"print('unique # of id :', train.loc[:,'session_id'].nunique())\nprint('min of id :', train.loc[:,'session_id'].min())\nprint('max of id :', train.loc[:,'session_id'].max())","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:47:02.063407Z","iopub.execute_input":"2023-04-01T08:47:02.063774Z","iopub.status.idle":"2023-04-01T08:47:02.249675Z","shell.execute_reply.started":"2023-04-01T08:47:02.063741Z","shell.execute_reply":"2023-04-01T08:47:02.248551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ex_min = train[train.session_id==22000610592348800] # this id has the least number of times\nex_max = train[train.session_id==22100221145014656] # this id has the largest number of times","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:47:03.400526Z","iopub.execute_input":"2023-04-01T08:47:03.400900Z","iopub.status.idle":"2023-04-01T08:47:03.488016Z","shell.execute_reply.started":"2023-04-01T08:47:03.400866Z","shell.execute_reply":"2023-04-01T08:47:03.486828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = make_subplots(rows=1, cols=2, specs=[[{'type':'histogram'}, {'type':'histogram'}]])\n\nfig.add_trace(go.Histogram(x=ex_min['level']),row=1, col=1)\nfig.add_trace(go.Histogram(x=ex_max['level']),row=1, col=2)\n\nfig.update_layout(height=600, width=800, title_text=\"Two example id\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:47:04.574803Z","iopub.execute_input":"2023-04-01T08:47:04.575513Z","iopub.status.idle":"2023-04-01T08:47:04.793841Z","shell.execute_reply.started":"2023-04-01T08:47:04.575475Z","shell.execute_reply":"2023-04-01T08:47:04.792872Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = go.Figure()\n\nfig.add_trace(go.Histogram(x=ex_min['level']))\nfig.add_trace(go.Histogram(x=ex_max['level']))\n\n\nfig.update_layout(\n    \n    barmode='group',\n    bargap=0.15, # gap between bars of adjacent location coordinates.\n    bargroupgap=0.1 # gap between bars of the same location coordinate.\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:47:07.117397Z","iopub.execute_input":"2023-04-01T08:47:07.118329Z","iopub.status.idle":"2023-04-01T08:47:07.129915Z","shell.execute_reply.started":"2023-04-01T08:47:07.118290Z","shell.execute_reply":"2023-04-01T08:47:07.128934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:47:08.504040Z","iopub.execute_input":"2023-04-01T08:47:08.505116Z","iopub.status.idle":"2023-04-01T08:47:09.110006Z","shell.execute_reply.started":"2023-04-01T08:47:08.505072Z","shell.execute_reply":"2023-04-01T08:47:09.109015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"At Level 3, the two graphs show different patterns","metadata":{}},{"cell_type":"markdown","source":"# 2. index\n\nThere are three bifurcation points at which the index drops sharply","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['index', 'level'])\ntrain.loc[:,'index'].describe().round()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:47:11.967783Z","iopub.execute_input":"2023-04-01T08:47:11.968166Z","iopub.status.idle":"2023-04-01T08:47:34.077688Z","shell.execute_reply.started":"2023-04-01T08:47:11.968132Z","shell.execute_reply":"2023-04-01T08:47:34.076607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.histogram(train, x=\"index\", nbins=400)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:48:02.814170Z","iopub.execute_input":"2023-04-01T08:48:02.815132Z","iopub.status.idle":"2023-04-01T08:48:07.148433Z","shell.execute_reply.started":"2023-04-01T08:48:02.815081Z","shell.execute_reply":"2023-04-01T08:48:07.146053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Values were minimized in sections 155 to 159, and 475 to 479","metadata":{}},{"cell_type":"code","source":"train[train.level==0]['index'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:51:55.090206Z","iopub.execute_input":"2023-04-01T08:51:55.090817Z","iopub.status.idle":"2023-04-01T08:51:55.171615Z","shell.execute_reply.started":"2023-04-01T08:51:55.090779Z","shell.execute_reply":"2023-04-01T08:51:55.170660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It seems that the index is not accurately distinguished by the level","metadata":{}},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:52:13.006228Z","iopub.execute_input":"2023-04-01T08:52:13.007153Z","iopub.status.idle":"2023-04-01T08:52:13.632488Z","shell.execute_reply.started":"2023-04-01T08:52:13.007101Z","shell.execute_reply":"2023-04-01T08:52:13.631165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. elapsed_time\nI added columns that changed 'elapsed_time' columns(=milliseconds) to seconds, minutes and hours.","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['elapsed_time'])","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:52:18.862411Z","iopub.execute_input":"2023-04-01T08:52:18.863021Z","iopub.status.idle":"2023-04-01T08:52:40.205380Z","shell.execute_reply.started":"2023-04-01T08:52:18.862978Z","shell.execute_reply":"2023-04-01T08:52:40.204329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.loc[:,'elapsed_time'].describe().round()/1000","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:56:22.097838Z","iopub.execute_input":"2023-04-01T08:56:22.098313Z","iopub.status.idle":"2023-04-01T08:56:22.679175Z","shell.execute_reply.started":"2023-04-01T08:56:22.098276Z","shell.execute_reply":"2023-04-01T08:56:22.677934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#train.loc[:,'elapsed_time_s'] = train.loc[:,'elapsed_time']/1000\n#train.loc[:,'elapsed_time_m'] = train.loc[:,'elapsed_time']/60000\n#train.loc[:,'elapsed_time_h'] = train.loc[:,'elapsed_time']/360000","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:56:24.040811Z","iopub.execute_input":"2023-04-01T08:56:24.042257Z","iopub.status.idle":"2023-04-01T08:56:24.458615Z","shell.execute_reply.started":"2023-04-01T08:56:24.042210Z","shell.execute_reply":"2023-04-01T08:56:24.457497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"(train.loc[:,'elapsed_time'].max()/360000)/24","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:56:26.466466Z","iopub.execute_input":"2023-04-01T08:56:26.467459Z","iopub.status.idle":"2023-04-01T08:56:26.541629Z","shell.execute_reply.started":"2023-04-01T08:56:26.467420Z","shell.execute_reply":"2023-04-01T08:56:26.540647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Maximum of elapsed_time is 230 days.","metadata":{}},{"cell_type":"markdown","source":"# 4-5. event_name / name","metadata":{}},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()\n    \ntrain = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['name', 'event_name', 'index'])","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:56:28.661889Z","iopub.execute_input":"2023-04-01T08:56:28.662948Z","iopub.status.idle":"2023-04-01T08:56:54.490604Z","shell.execute_reply.started":"2023-04-01T08:56:28.662894Z","shell.execute_reply":"2023-04-01T08:56:54.489542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.crosstab(train.name, train.event_name)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:56:54.492346Z","iopub.execute_input":"2023-04-01T08:56:54.492732Z","iopub.status.idle":"2023-04-01T08:57:01.267770Z","shell.execute_reply.started":"2023-04-01T08:56:54.492683Z","shell.execute_reply":"2023-04-01T08:57:01.266725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Maybe i'll drop the name info","metadata":{}},{"cell_type":"code","source":"train['dum'] = 1\nnew = train.groupby(['name', 'event_name','index'])['dum'].sum().reset_index()\nnew[new.name == 'close']","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:58:54.386528Z","iopub.execute_input":"2023-04-01T08:58:54.387422Z","iopub.status.idle":"2023-04-01T08:59:00.181224Z","shell.execute_reply.started":"2023-04-01T08:58:54.387371Z","shell.execute_reply":"2023-04-01T08:59:00.180252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"new[new.dum >15000]","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:59:04.837978Z","iopub.execute_input":"2023-04-01T08:59:04.838362Z","iopub.status.idle":"2023-04-01T08:59:04.852371Z","shell.execute_reply.started":"2023-04-01T08:59:04.838327Z","shell.execute_reply":"2023-04-01T08:59:04.851204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"More data is accumulated at the beginning of the game","metadata":{}},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:59:18.341641Z","iopub.execute_input":"2023-04-01T08:59:18.342294Z","iopub.status.idle":"2023-04-01T08:59:19.112775Z","shell.execute_reply.started":"2023-04-01T08:59:18.342258Z","shell.execute_reply":"2023-04-01T08:59:19.111645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 6. level","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['level', 'event_name'])","metadata":{"execution":{"iopub.status.busy":"2023-04-01T08:59:25.723627Z","iopub.execute_input":"2023-04-01T08:59:25.724716Z","iopub.status.idle":"2023-04-01T08:59:49.864370Z","shell.execute_reply.started":"2023-04-01T08:59:25.724668Z","shell.execute_reply":"2023-04-01T08:59:49.862485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.histogram(train, x=\"level\", color='event_name',  nbins=23)\nfig.update_layout(bargap=0.2)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T09:00:10.149374Z","iopub.execute_input":"2023-04-01T09:00:10.149976Z","iopub.status.idle":"2023-04-01T09:00:20.675728Z","shell.execute_reply.started":"2023-04-01T09:00:10.149922Z","shell.execute_reply":"2023-04-01T09:00:20.674100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"1. checkpoint is only in level 4, 12, 22\n2. In level 22, all events occurs","metadata":{}},{"cell_type":"markdown","source":"# 7. page\n\nnotebook_click","metadata":{}},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()\n    \ntrain = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['level','event_name', 'page', 'elapsed_time'])\ntrain.loc[:,'elapsed_time_m'] = train.loc[:,'elapsed_time']/60000","metadata":{"execution":{"iopub.status.busy":"2023-04-01T09:04:35.291747Z","iopub.execute_input":"2023-04-01T09:04:35.292753Z","iopub.status.idle":"2023-04-01T09:05:01.460772Z","shell.execute_reply.started":"2023-04-01T09:04:35.292715Z","shell.execute_reply":"2023-04-01T09:05:01.459704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.loc[:,'page'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T09:03:13.534821Z","iopub.execute_input":"2023-04-01T09:03:13.535816Z","iopub.status.idle":"2023-04-01T09:03:13.599516Z","shell.execute_reply.started":"2023-04-01T09:03:13.535776Z","shell.execute_reply":"2023-04-01T09:03:13.598443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.pivot_table(train[train.event_name=='notebook_click'],              \n                     index = 'level',    \n                     columns = 'page',   \n                     values = 'elapsed_time_m',     \n                     aggfunc = 'mean') ","metadata":{"execution":{"iopub.status.busy":"2023-04-01T09:05:01.462538Z","iopub.execute_input":"2023-04-01T09:05:01.462901Z","iopub.status.idle":"2023-04-01T09:05:03.548765Z","shell.execute_reply.started":"2023-04-01T09:05:01.462864Z","shell.execute_reply":"2023-04-01T09:05:03.547317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"page 0 : level 1 or higher   \\\npage 1 : level 4 or higher    \\\npage 2 : level 8 or higher    \\\npage 3 : level 10 or higher    \\\npage 4 : level 14 or higher    \\\npage 5 : level 18 or higher    \\\npage 6 : level 20 or higher    \n\n\nThere is no page lev.0 can see.\n","metadata":{}},{"cell_type":"markdown","source":"--------------------------------------------------------------------------------------------------------","metadata":{}},{"cell_type":"markdown","source":"# 8-11. coordinates info (room_coor_x,y & screen_coor_x,y)","metadata":{}},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()\n    \n#train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['index', 'room_coor_x', 'room_coor_y'])","metadata":{"execution":{"iopub.status.busy":"2023-04-01T09:16:37.668471Z","iopub.execute_input":"2023-04-01T09:16:37.669303Z","iopub.status.idle":"2023-04-01T09:17:58.177658Z","shell.execute_reply.started":"2023-04-01T09:16:37.669261Z","shell.execute_reply":"2023-04-01T09:17:58.176523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#print(train[train.loc[:,'event_name']=='checkpoint'].room_coor_x.notnull().sum())\n#print(train[train.loc[:,'event_name']=='map_hover'].room_coor_x.notnull().sum())","metadata":{"execution":{"iopub.status.busy":"2023-03-30T15:46:47.597091Z","iopub.execute_input":"2023-03-30T15:46:47.598029Z","iopub.status.idle":"2023-03-30T15:46:49.373206Z","shell.execute_reply.started":"2023-03-30T15:46:47.597975Z","shell.execute_reply":"2023-03-30T15:46:49.371526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#fig = make_subplots(1,2)\n#fig.add_scatter(\n#        x = train[(train.loc[:,'event_name']=='observation_click')&(train.loc[:,'level']==0)].room_coor_x,\n#        y = train[(train.loc[:,'event_name']=='observation_click')&(train.loc[:,'level']==0)].room_coor_y,\n#              row=1, col=1)\n#fig.add_scatter(\n#        x = train[(train.loc[:,'event_name']=='observation_click')&(train.loc[:,'level']==22)].room_coor_x,\n#        y = train[(train.loc[:,'event_name']=='observation_click')&(train.loc[:,'level']==22)].room_coor_y,\n#              row=1, col=2)\n\n\n#fig.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-30T15:08:03.220990Z","iopub.execute_input":"2023-03-30T15:08:03.221672Z","iopub.status.idle":"2023-03-30T15:08:09.390895Z","shell.execute_reply.started":"2023-03-30T15:08:03.221630Z","shell.execute_reply":"2023-03-30T15:08:09.389998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"인사이트가 잘 안보인다","metadata":{}},{"cell_type":"code","source":"#fig = px.scatter(train[train.level==1], x=\"screen_coor_x\", y=\"screen_coor_y\", animation_frame=\"event_name\", animation_group=\"level\")\n#fig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#  ","metadata":{}},{"cell_type":"markdown","source":"# 12. hover_duration","metadata":{"execution":{"iopub.status.busy":"2023-03-28T14:35:32.682414Z","iopub.execute_input":"2023-03-28T14:35:32.683200Z","iopub.status.idle":"2023-03-28T14:35:32.690538Z","shell.execute_reply.started":"2023-03-28T14:35:32.683158Z","shell.execute_reply":"2023-03-28T14:35:32.688351Z"}}},{"cell_type":"code","source":"# del train\n\nfor i in range(5):\n    gc.collect()\n    \ntrain = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['event_name', 'level', 'elapsed_time'])\ntrain.loc[:,'elapsed_time_m'] = train.loc[:,'elapsed_time']/60000","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.pivot_table(train[train.event_name.isin(['object_hover','map_hover', 'checkpoint'])],              \n                     index = 'level',    \n                     columns = 'event_name',   \n                     values = 'elapsed_time_m',     \n                     aggfunc = 'mean') ","metadata":{"execution":{"iopub.status.busy":"2023-03-30T15:47:44.689633Z","iopub.execute_input":"2023-03-30T15:47:44.690043Z","iopub.status.idle":"2023-03-30T15:47:46.779929Z","shell.execute_reply.started":"2023-03-30T15:47:44.690006Z","shell.execute_reply":"2023-03-30T15:47:46.778775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 13. TEXT","metadata":{}},{"cell_type":"markdown","source":"Some texts need to be decoded.","metadata":{}},{"cell_type":"code","source":"del train\n\nfor i in range(5):\n    gc.collect()\n    \ntrain = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=['text'])\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.text.value_counts().head(20)","metadata":{"execution":{"iopub.status.busy":"2023-03-30T15:47:59.127736Z","iopub.execute_input":"2023-03-30T15:47:59.128692Z","iopub.status.idle":"2023-03-30T15:48:00.068847Z","shell.execute_reply.started":"2023-03-30T15:47:59.128638Z","shell.execute_reply":"2023-03-30T15:48:00.067047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"b\"\\u00f0\\u0178\\u02dc\\u0090 \".decode('unicode_escape')  ","metadata":{"execution":{"iopub.status.busy":"2023-03-30T15:48:10.773538Z","iopub.execute_input":"2023-03-30T15:48:10.774253Z","iopub.status.idle":"2023-03-30T15:48:10.783374Z","shell.execute_reply.started":"2023-03-30T15:48:10.774215Z","shell.execute_reply":"2023-03-30T15:48:10.782025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"ascii는 decoding 해주기","metadata":{}},{"cell_type":"markdown","source":"# 14-16. fqid","metadata":{}},{"cell_type":"markdown","source":"It seems important to make a group with appropriate categories..","metadata":{}},{"cell_type":"code","source":"#del train\n\nfor i in range(5):\n    gc.collect()\n    \n#train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', usecols=[ 'room_fqid'])\n#train.fqid.value_counts()\n#train.loc[:,'place'] = train.room_fqid.astype(str).str.split('\\.').str[1]","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ETC.\n\nfullscreen \nhq \nmusic \nlevel_group","metadata":{}}]}