{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <center>Predict Student Performance from Game Play - 【EDA】","metadata":{}},{"cell_type":"markdown","source":"## 赛题解读\n本次比赛的目的是使用在线教育游戏生成的时间序列数据来判断玩家是否会正确回答问题。\n\n一个会话有18个问题，三个检查点（level 4, level 12, and level 22），每个检查点都有一些问题。\n\n* 数据中的每一行都是一个事件，一个事件由`name`（事件名称）, `event_name`（事件类型）, `fqid`（事件的唯一id）以及 `index`（事件索引）定义\n* 只有笔记本点击事件，才有page\n* 只有点击事件，才有房间和屏幕坐标\n\n## 官方数据介绍\ntrain 训练文件  \n* session_id - 事件发生的会话 ID，每一次游戏对应一个session\n* index - 会话事件的索引\n* elapsed_time - 从会话开始到记录事件之间经过了多长时间（以毫秒为单位）\n* event_name - 事件类型的名称\n* name - 事件名称（例如，标识 notebook_click 是打开还是关闭笔记本）\n* level - 事件发生在游戏的哪个级别（0 到 22）\n* page - 事件的页码（仅适用于与笔记本相关的事件）\n* room_coor_x - 游戏内房间的点击坐标（仅适用于点击事件）\n* room_coor_y - 游戏内房间的点击坐标（仅适用于点击事件）\n* screen_coor_x - 相对于玩家屏幕的点击坐标（仅适用于点击事件）\n* screen_coor_y - 相对于玩家屏幕的点击坐标（仅适用于点击事件）\n* hover_duration - 悬停时间\n* text - 玩家在此事件中看到的文本\n* fqid - 事件的完全限定 ID\n* room_fqid - 发生事件的房间的完全限定 ID\n* text_fqid - 文本的完全限定 ID\n* fullscreen - 是否全屏\n* hq - 是否高清\n* music - 是否播放音乐\n* level_group - 哪组级别 - 和哪组问题 - 此行属于 (0-4, 5-12, 13-22)\n\n## 游戏数据解读\n* **level属性在游戏中并没有明显的体现,建议理解成游戏不同阶段的划分,其中L4,L12,L22是游戏的检查点,会有问题出现**\n* **也就是说,L4,L12,L22是游戏的检查点,玩家需要在L0到L3搜集证据/材料,来回答L4的问题. 在L5到L11搜集问题来回答L12的问题**\n* **所有level中只有L4,L12,L22会涉及到解谜,分别对应的问题是q1-q3,q4-q13,q14-q18**\n* **session应该是用户在浏览器打开游戏窗口时所对应创建的游戏档案,按session分类打开之后能看到每个session大约有1k条左右的数据,每个session中一个level大约有十几到几十条左右数据**","metadata":{}},{"cell_type":"markdown","source":"## 读取数据","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport warnings\nimport gc\n\n# settings\nsns.set(style=\"white\", color_codes=True)\nwarnings.filterwarnings(\"ignore\")\nplt.rcParams['font.sans-serif'] = ['SimHei', 'Microsoft YaHei']\nplt.rcParams['axes.unicode_minus'] = False","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2023-03-13T04:37:10.382358Z","iopub.execute_input":"2023-03-13T04:37:10.382695Z","iopub.status.idle":"2023-03-13T04:37:10.971078Z","shell.execute_reply.started":"2023-03-13T04:37:10.382664Z","shell.execute_reply":"2023-03-13T04:37:10.969698Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 降低内存占用\ndef reduce_mem(df):\n    start_mem = df.memory_usage().sum() / 1024 ** 2\n    for col in df.columns:\n        col_type = df[col].dtypes\n        if col_type != object:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)\n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n    end_mem = df.memory_usage().sum() / 1024 ** 2\n    print('{:.2f} Mb, {:.2f} Mb ({:.2f} %)'.format(start_mem, end_mem, 100 * (start_mem - end_mem) / start_mem))\n    gc.collect()\n    return df","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:39:58.681658Z","iopub.execute_input":"2023-03-13T04:39:58.681974Z","iopub.status.idle":"2023-03-13T04:39:58.696610Z","shell.execute_reply.started":"2023-03-13T04:39:58.681943Z","shell.execute_reply":"2023-03-13T04:39:58.695326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 数据载入\n# 训练集\ntrain_df = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train.csv\")\ntrain_df = reduce_mem(train_df)\ntrain_df.head()\n","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:43:50.355468Z","iopub.execute_input":"2023-03-13T04:43:50.355840Z","iopub.status.idle":"2023-03-13T04:44:17.043823Z","shell.execute_reply.started":"2023-03-13T04:43:50.355805Z","shell.execute_reply":"2023-03-13T04:44:17.042998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 测试集\ntest_df = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/test.csv\")\ntest_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:45:10.130484Z","iopub.execute_input":"2023-03-13T04:45:10.130797Z","iopub.status.idle":"2023-03-13T04:45:10.200928Z","shell.execute_reply.started":"2023-03-13T04:45:10.130769Z","shell.execute_reply":"2023-03-13T04:45:10.200165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 训练集标签\ntargets = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train_labels.csv\")\ntargets.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:45:16.942669Z","iopub.execute_input":"2023-03-13T04:45:16.943023Z","iopub.status.idle":"2023-03-13T04:45:17.190983Z","shell.execute_reply.started":"2023-03-13T04:45:16.942992Z","shell.execute_reply":"2023-03-13T04:45:17.189633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 平均正确率为70.39%","metadata":{}},{"cell_type":"code","source":"targets.describe().T","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:48:26.293506Z","iopub.execute_input":"2023-03-13T04:48:26.293856Z","iopub.status.idle":"2023-03-13T04:48:26.334778Z","shell.execute_reply.started":"2023-03-13T04:48:26.293825Z","shell.execute_reply":"2023-03-13T04:48:26.333339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"标签数据集的session_id格式为`<session_id>_<question #>`，需要拆分session_id和问题编号","metadata":{}},{"cell_type":"code","source":"# 标签拆分\ntargets[\"session\"] = targets[\"session_id\"].apply(lambda x:int(x.split('_')[0]))\ntargets[\"q\"] = targets[\"session_id\"].apply(lambda x:int(x.split('_')[1][1:]))\ntargets.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:49:21.582934Z","iopub.execute_input":"2023-03-13T04:49:21.583323Z","iopub.status.idle":"2023-03-13T04:49:21.905430Z","shell.execute_reply.started":"2023-03-13T04:49:21.583291Z","shell.execute_reply":"2023-03-13T04:49:21.904364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"查看训练集信息，可以得知fullscreen、hq、music三个字段全为空，page、hover_duration、text、text_fqid缺失值较多。","metadata":{}},{"cell_type":"code","source":"train_df.info(verbose=True,null_counts=True)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:52:02.316821Z","iopub.execute_input":"2023-03-13T04:52:02.317234Z","iopub.status.idle":"2023-03-13T04:52:05.715439Z","shell.execute_reply.started":"2023-03-13T04:52:02.317200Z","shell.execute_reply":"2023-03-13T04:52:05.714285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 数据可视化","metadata":{}},{"cell_type":"markdown","source":"### 各个特征的取值个数\n大致分布如下","metadata":{}},{"cell_type":"code","source":"stat = pd.DataFrame([train_df.nunique(), test_df.nunique()]).T.fillna(0)\nstat.columns = ['Number of unique values in train', 'Number of unique values in test']\nstat.style.format(\"{:,.0f}\").background_gradient(cmap='YlGn')","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:55:55.938530Z","iopub.execute_input":"2023-03-13T04:55:55.939660Z","iopub.status.idle":"2023-03-13T04:56:03.203518Z","shell.execute_reply.started":"2023-03-13T04:55:55.939564Z","shell.execute_reply":"2023-03-13T04:56:03.202709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 每道题的平均正确率\n分布大致如下,q1-q3,q4-q13,q14-q18分别是3短,我本来以为分别在3个段内会分别呈现上升趋势(因为3组谜题是类似于完形填空那样来让玩家做出选择的),但是数据图像并没有显示出明显的规律性","metadata":{}},{"cell_type":"code","source":"q_corr = targets.groupby(\"q\")[\"correct\"].mean().mul(100).to_frame().reset_index()\nmean_correct = targets['correct'].mean()*100\npal = sns.color_palette(\"GnBu\", len(q_corr))\nrank = q_corr.sort_values('q')['correct'].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=q_corr,x=\"q\",y=\"correct\",palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0], fmt=\"%.1f%%\")\ng.axhline(mean_correct, color=\"coral\")\ndel mean_correct,pal,rank,g,q_corr","metadata":{"execution":{"iopub.status.busy":"2023-03-13T04:56:17.177115Z","iopub.execute_input":"2023-03-13T04:56:17.178459Z","iopub.status.idle":"2023-03-13T04:56:17.690873Z","shell.execute_reply.started":"2023-03-13T04:56:17.178423Z","shell.execute_reply":"2023-03-13T04:56:17.690131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 答题正确性与session\n我认为此数据分析意义不大,session只是浏览器的一个会话而已,","metadata":{}},{"cell_type":"markdown","source":"#### 正确题数的session数量分布","metadata":{"execution":{"iopub.status.busy":"2023-03-13T05:37:22.864172Z","iopub.execute_input":"2023-03-13T05:37:22.864803Z","iopub.status.idle":"2023-03-13T05:37:22.878041Z","shell.execute_reply.started":"2023-03-13T05:37:22.864769Z","shell.execute_reply":"2023-03-13T05:37:22.876918Z"}}},{"cell_type":"code","source":"correct = targets[targets[\"correct\"]==1]\nsession_correct = correct.groupby(\"session\")[\"correct\"].sum().to_frame().reset_index()\nsession_count = session_correct.groupby(\"correct\")[\"session\"].count().to_frame().reset_index()\npal = sns.color_palette(\"OrRd\", len(session_count))\nrank = session_count.sort_values('session')['correct'].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=session_count,x=\"correct\",y=\"session\",palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0])\ndel pal,g,correct,session_correct,session_count","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:07:33.001596Z","iopub.execute_input":"2023-03-13T06:07:33.001944Z","iopub.status.idle":"2023-03-13T06:07:33.653346Z","shell.execute_reply.started":"2023-03-13T06:07:33.001913Z","shell.execute_reply":"2023-03-13T06:07:33.651777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### 答错题数的session数量分布","metadata":{"execution":{"iopub.status.busy":"2023-03-13T05:21:53.799984Z","iopub.execute_input":"2023-03-13T05:21:53.800362Z","iopub.status.idle":"2023-03-13T05:21:53.806019Z","shell.execute_reply.started":"2023-03-13T05:21:53.800331Z","shell.execute_reply":"2023-03-13T05:21:53.804525Z"}}},{"cell_type":"code","source":"correct = targets[targets[\"correct\"]==0]\nsession_correct = correct.groupby(\"session\")[\"correct\"].count().to_frame().reset_index()\nsession_count = session_correct.groupby(\"correct\")[\"session\"].count().to_frame().reset_index()\npal = sns.color_palette(\"OrRd\", len(session_count))\nrank = session_count.sort_values('session')['correct'].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=session_count,x=\"correct\",y=\"session\",palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0])\ndel pal,g,correct,session_correct,session_count","metadata":{"execution":{"iopub.status.busy":"2023-03-13T05:34:34.251303Z","iopub.execute_input":"2023-03-13T05:34:34.251677Z","iopub.status.idle":"2023-03-13T05:34:34.698996Z","shell.execute_reply.started":"2023-03-13T05:34:34.251646Z","shell.execute_reply":"2023-03-13T05:34:34.698283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 筛选答对题数较少的session进行分析\n**挑选出正确次数为1的两个session来分析,我本来以为正确率较低的session产生的event数据也会较少,没想到也有2k多条**\n**但是正确次数为18的session却只有700条左右的数据,很奇怪**","metadata":{"execution":{"iopub.status.busy":"2023-03-13T05:49:43.785476Z","iopub.execute_input":"2023-03-13T05:49:43.785842Z","iopub.status.idle":"2023-03-13T05:49:43.791056Z","shell.execute_reply.started":"2023-03-13T05:49:43.785809Z","shell.execute_reply":"2023-03-13T05:49:43.789415Z"}}},{"cell_type":"code","source":"correct = targets[targets[\"correct\"]==1]\nsession_correct = correct.groupby(\"session\")[\"correct\"].count().to_frame().reset_index()\nselected_min_session = session_correct[session_correct['correct']==1]\nselected_min_session\n#挑选出正确次数为1的两个session来分析","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:45:59.781182Z","iopub.execute_input":"2023-03-13T06:45:59.781539Z","iopub.status.idle":"2023-03-13T06:45:59.810136Z","shell.execute_reply.started":"2023-03-13T06:45:59.781507Z","shell.execute_reply":"2023-03-13T06:45:59.809013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"selected_max_session = session_correct[session_correct['correct']==18]\nselected_max_session\n#挑选出正确次数为1的两个session来分析","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:46:04.409160Z","iopub.execute_input":"2023-03-13T06:46:04.409549Z","iopub.status.idle":"2023-03-13T06:46:04.425209Z","shell.execute_reply.started":"2023-03-13T06:46:04.409518Z","shell.execute_reply":"2023-03-13T06:46:04.423971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#正确次数为1的session数据\ntrain_df_selected1 = train_df[train_df['session_id'] == 21020008201271976]\nprint(len(train_df_selected1))\nprint(train_df_selected1['level'].unique())","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:46:17.536921Z","iopub.execute_input":"2023-03-13T06:46:17.537662Z","iopub.status.idle":"2023-03-13T06:46:17.555786Z","shell.execute_reply.started":"2023-03-13T06:46:17.537620Z","shell.execute_reply":"2023-03-13T06:46:17.554797Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#正确次数为18的session数据\ntrain_df_selected18 = train_df[train_df['session_id'] == 20100208332760820]\nprint(len(train_df_selected18))\nprint(train_df_selected18['level'].unique())\ndel train_df_selected1, train_df_selected18, selected_max_session, selected_min_session","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:46:24.713273Z","iopub.execute_input":"2023-03-13T06:46:24.713942Z","iopub.status.idle":"2023-03-13T06:46:24.735493Z","shell.execute_reply.started":"2023-03-13T06:46:24.713902Z","shell.execute_reply":"2023-03-13T06:46:24.734394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 各个level_group的平均正确率","metadata":{}},{"cell_type":"code","source":"# 把每个问题对应的level_group加入到训练集标签中\ndef generateLevelgroup(x):\n    if x<=4:\n        return \"0-4\"\n    elif x<=12:\n        return \"5-12\"\n    else:\n        return \"13-22\"\n    \ntargets[\"level_group\"] = targets[\"q\"].apply(lambda x:generateLevelgroup(x))","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:26:54.178768Z","iopub.execute_input":"2023-03-13T06:26:54.179247Z","iopub.status.idle":"2023-03-13T06:26:54.246119Z","shell.execute_reply.started":"2023-03-13T06:26:54.179207Z","shell.execute_reply":"2023-03-13T06:26:54.245332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"level_corr = targets.groupby(\"level_group\")[\"correct\"].mean().mul(100).to_frame().reset_index()\npal = sns.color_palette(\"BuGn\",len(level_corr))\nrank = level_corr.sort_values(\"level_group\")[\"correct\"].argsort()\nplt.figure(figsize=(15,5))\ng = sns.barplot(data=level_corr,x=\"level_group\",y=\"correct\",palette=np.array(pal[::])[rank],)\ng.bar_label(g.containers[0],fmt=\"%.1f%%\")\ndel level_corr,pal,rank,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:26:57.928281Z","iopub.execute_input":"2023-03-13T06:26:57.928691Z","iopub.status.idle":"2023-03-13T06:26:58.132924Z","shell.execute_reply.started":"2023-03-13T06:26:57.928658Z","shell.execute_reply":"2023-03-13T06:26:58.131958Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### event_name 分布情况","metadata":{}},{"cell_type":"code","source":"event_name = pd.DataFrame(train_df[\"event_name\"].value_counts())\npal = sns.color_palette(\"flare\", len(event_name))\nrank = event_name[\"event_name\"].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=event_name,y=event_name.index,x=event_name.event_name,orient='h',palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0],labels=event_name[\"event_name\"])\ndel event_name,pal,rank,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:27:21.988407Z","iopub.execute_input":"2023-03-13T06:27:21.989628Z","iopub.status.idle":"2023-03-13T06:27:23.002202Z","shell.execute_reply.started":"2023-03-13T06:27:21.989558Z","shell.execute_reply":"2023-03-13T06:27:23.001331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### event_name 含义","metadata":{}},{"cell_type":"markdown","source":"参考：\n\n* navigate_click: click on the place we want move protagonist (Jo) to move;\n* observation_click: click on the black object that appears when you take your mouse pointer close to them;\n* notification_click: after navigate click on an object like the notebook, the retirement letter - the click to hide or to continue after the text appears;\n* object_click: click anywhere on the object pop up (only after notification_click)\n* object_hover: the student takes the mouse pointer above an object;\n* map_hover: the student takes the pointer above any place in the map;\n* map_click: click on a place on the map to go there;\n* notebook_click: click on notebook to look for notes made by Jo;\n* checkpoint: the last event of a level-group/chapter in the data.\n","metadata":{}},{"cell_type":"markdown","source":"### name分布情况","metadata":{}},{"cell_type":"code","source":"event_name = pd.DataFrame(train_df[\"name\"].value_counts())\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=event_name,y=event_name.index,x=event_name.name)\ng.bar_label(g.containers[0],labels=event_name[\"name\"])\ndel event_name,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:28:43.147321Z","iopub.execute_input":"2023-03-13T06:28:43.147664Z","iopub.status.idle":"2023-03-13T06:28:44.122536Z","shell.execute_reply.started":"2023-03-13T06:28:43.147633Z","shell.execute_reply":"2023-03-13T06:28:44.121758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### fqid 分布情况","metadata":{}},{"cell_type":"code","source":"event_name = pd.DataFrame(train_df[\"fqid\"].value_counts()[:10])\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=event_name,y=event_name.index,x=event_name.fqid)\ng.bar_label(g.containers[0],labels=event_name[\"fqid\"])\ndel event_name,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:55:41.746508Z","iopub.execute_input":"2023-03-13T06:55:41.746858Z","iopub.status.idle":"2023-03-13T06:55:42.724792Z","shell.execute_reply.started":"2023-03-13T06:55:41.746828Z","shell.execute_reply":"2023-03-13T06:55:42.723870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### session_id中包含的时间信息","metadata":{}},{"cell_type":"markdown","source":"参考：[session_id reverse engineering](https://www.kaggle.com/code/pdnartreb/session-id-reverse-engineering)  \n\n猜想：  \n* 1、2位代表年份，取值为20，21，22\n* 3、4位代表月份，取值从00到11\n* 时间范围从2020年10月到2022年11月\n* 5、6位代表星期，取值从00到06，00代表星期天，01代表星期一\n* 7到15位代表时、分、秒、毫秒","metadata":{}},{"cell_type":"code","source":"session_time = pd.DataFrame(train_df[\"session_id\"].copy()).drop_duplicates().reset_index()\nsession_time[\"year\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[:2])).astype(np.uint8)\nsession_time[\"month\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[2:4]) + 1).astype(np.uint8)\nsession_time[\"weekday\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[4:6])).astype(np.uint8)\nsession_time[\"hour\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[6:8])).astype(np.uint8)\nsession_time[\"minute\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[8:10])).astype(np.uint8)\nsession_time[\"second\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[10:12])).astype(np.uint8)\nsession_time[\"ms\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[12:15])).astype(np.uint16)\nsession_time[\"?\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[15:17])).astype(np.uint8)\nsession_time.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:29:55.462179Z","iopub.execute_input":"2023-03-13T06:29:55.462538Z","iopub.status.idle":"2023-03-13T06:29:55.728529Z","shell.execute_reply.started":"2023-03-13T06:29:55.462507Z","shell.execute_reply":"2023-03-13T06:29:55.727402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 不同时间的session数量分布\nsession中所包含的时间数据应该单纯只能表示用户打开页面或者注册的时间,我认为意义不大","metadata":{}},{"cell_type":"code","source":"session_time[\"year\"] = session_time[\"year\"]+2000\nsession_time[\"date\"] = session_time[\"year\"].astype(str)+'_'+session_time[\"month\"].astype(str)\nplt.figure(figsize=(20, 8))\ng = sns.countplot(x='date', data=session_time)\ng.set_yticklabels(['{}'.format(int(num)) for num in g.get_yticks()])\ng.bar_label(g.containers[0])\ng.xaxis.set_tick_params(rotation=45)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:29:59.078983Z","iopub.execute_input":"2023-03-13T06:29:59.079445Z","iopub.status.idle":"2023-03-13T06:29:59.907535Z","shell.execute_reply.started":"2023-03-13T06:29:59.079410Z","shell.execute_reply":"2023-03-13T06:29:59.906324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15, 5))\ng = sns.countplot(x='weekday', data=session_time)\ng.set_yticklabels(['{}'.format(int(num)) for num in g.get_yticks()])\ng.bar_label(g.containers[0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:56:00.834597Z","iopub.execute_input":"2023-03-13T06:56:00.835641Z","iopub.status.idle":"2023-03-13T06:56:01.040663Z","shell.execute_reply.started":"2023-03-13T06:56:00.835601Z","shell.execute_reply":"2023-03-13T06:56:01.039605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,10))\ng = sns.countplot(x='hour', data=session_time)\ng.set_yticklabels(['{}'.format(int(num)) for num in g.get_yticks()])\ng.bar_label(g.containers[0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:56:10.324886Z","iopub.execute_input":"2023-03-13T06:56:10.325263Z","iopub.status.idle":"2023-03-13T06:56:10.791303Z","shell.execute_reply.started":"2023-03-13T06:56:10.325233Z","shell.execute_reply":"2023-03-13T06:56:10.790331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"玩游戏最多的时间是11月到6月，周一到周五，9点到14点，属于在校时间","metadata":{}},{"cell_type":"markdown","source":"### 文本的长度分布","metadata":{}},{"cell_type":"code","source":"text_stat = pd.DataFrame(train_df['text'].dropna()).reset_index(drop=True)\ntext_stat['length'] = text_stat[\"text\"].apply(lambda x:len(x))\ntext_stat[\"length_group\"] = pd.cut(text_stat['length'],bins=[0,10,20,30,40,50,60,70,80,90],\n       labels=['0-10','10-20','20-30','30-40','40-50','50-60','60-70','70-80','80-90'])\nlength_count = text_stat[\"length_group\"].value_counts().to_frame().reset_index()\npal = sns.color_palette(\"Spectral\",len(length_count))\nrank = length_count.sort_values(\"index\")[\"index\"].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=text_stat,y=length_count['length_group'],x=length_count['index'],palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0])\ndel text_stat,length_count,pal,rank,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T06:56:16.450722Z","iopub.execute_input":"2023-03-13T06:56:16.451419Z","iopub.status.idle":"2023-03-13T06:56:19.024859Z","shell.execute_reply.started":"2023-03-13T06:56:16.451384Z","shell.execute_reply":"2023-03-13T06:56:19.023147Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 流逝时间分析(未完成)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T07:07:43.339310Z","iopub.execute_input":"2023-03-13T07:07:43.339767Z","iopub.status.idle":"2023-03-13T07:07:43.344921Z","shell.execute_reply.started":"2023-03-13T07:07:43.339731Z","shell.execute_reply":"2023-03-13T07:07:43.344094Z"}}},{"cell_type":"markdown","source":"## 异常数据","metadata":{}},{"cell_type":"markdown","source":"参考：[EDA on Game Progress](https://www.kaggle.com/code/abaojiang/eda-on-game-progress)  \n\n* 有些session_id的index不是从0开始\n* session_id和index的组合有重复值\n* 同一个session_id，index小的elapsed_time 大于 index大的elapsed_time\n\n官方解释：index==0表示session开始，并且session不应该有重复的index。如果操作非常快，服务器上的时间戳可能记录不准确。我们可以假设index在elapsed_time不一致的情况下给出了正确的事件顺序。","metadata":{}},{"cell_type":"markdown","source":"### 待分析","metadata":{}},{"cell_type":"markdown","source":"* elapsed_time\n    * 各event_name的elapsed_time\n    * 时间差\n* page\n* 房间坐标和屏幕坐标\n* hover_duration","metadata":{}},{"cell_type":"markdown","source":"#### 参考文章\n* [Kaggle赛题解析：从游戏预测学生成绩](https://mp.weixin.qq.com/s/UaPr4mc0A6S8MwPT5mx0tg)\n* [Student Performance from Game Play - EDA](https://www.kaggle.com/code/demche/student-performance-from-game-play-eda)\n* [Meaning of Each Event Name and EDA](https://www.kaggle.com/code/shashwatraman/meaning-of-each-event-name-and-eda)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}