{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <center>Predict Student Performance from Game Play - 【EDA】","metadata":{}},{"cell_type":"markdown","source":"## 赛题解读","metadata":{}},{"cell_type":"markdown","source":"[Kaggle赛题解析：从游戏预测学生成绩](https://mp.weixin.qq.com/s/UaPr4mc0A6S8MwPT5mx0tg)","metadata":{}},{"cell_type":"markdown","source":"本次比赛使用了Kaggle的时间序列API，测试数据将以不允许访问未来数据的分组形式提供。  \n\n本次比赛的目的是使用在线教育游戏生成的时间序列数据来判断玩家是否会正确回答问题。\n\n一个会话有18个问题，三个检查点（level 4, level 12, and level 22），每个检查点都有一些问题。在每个检查点，选手都可以访问该部分之前的所有测试数据。\n\n* 数据中的每一行都是一个事件，一个事件由`name`（事件名称）, `event_name`（事件类型）, `fqid`（事件的唯一id）以及 `index`（事件索引）定义\n* 只有笔记本点击事件，才有page\n* 只有点击事件，才有房间和屏幕坐标","metadata":{}},{"cell_type":"markdown","source":"train 训练文件  \n* session_id - 事件发生的会话 ID，每一次游戏对应一个session\n* index - 会话事件的索引\n* elapsed_time - 从会话开始到记录事件之间经过了多长时间（以毫秒为单位）\n* event_name - 事件类型的名称\n* name - 事件名称（例如，标识 notebook_click 是打开还是关闭笔记本）\n* level - 事件发生在游戏的哪个级别（0 到 22）\n* page - 事件的页码（仅适用于与笔记本相关的事件）\n* room_coor_x - 游戏内房间的点击坐标（仅适用于点击事件）\n* room_coor_y - 游戏内房间的点击坐标（仅适用于点击事件）\n* screen_coor_x - 相对于玩家屏幕的点击坐标（仅适用于点击事件）\n* screen_coor_y - 相对于玩家屏幕的点击坐标（仅适用于点击事件）\n* hover_duration - 悬停时间\n* text - 玩家在此事件中看到的文本\n* fqid - 事件的完全限定 ID\n* room_fqid - 发生事件的房间的完全限定 ID\n* text_fqid - 文本的完全限定 ID\n* fullscreen - 是否全屏\n* hq - 是否高清\n* music - 是否播放音乐\n* level_group - 哪组级别 - 和哪组问题 - 此行属于 (0-4, 5-12, 13-22)","metadata":{}},{"cell_type":"markdown","source":"## 读取数据","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nimport seaborn as sns\nimport warnings\nimport gc\n\n# settings\nsns.set(style=\"white\", color_codes=True)\nwarnings.filterwarnings(\"ignore\")\nplt.rcParams['font.sans-serif'] = ['SimHei', 'Microsoft YaHei']\nplt.rcParams['axes.unicode_minus'] = False","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2023-03-13T15:52:47.654123Z","iopub.execute_input":"2023-03-13T15:52:47.654603Z","iopub.status.idle":"2023-03-13T15:52:47.662442Z","shell.execute_reply.started":"2023-03-13T15:52:47.654562Z","shell.execute_reply":"2023-03-13T15:52:47.661426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def reduce_mem(df):\n    start_mem = df.memory_usage().sum() / 1024 ** 2\n    for col in df.columns:\n        col_type = df[col].dtypes\n        if col_type != object:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)\n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n    end_mem = df.memory_usage().sum() / 1024 ** 2\n    print('{:.2f} Mb, {:.2f} Mb ({:.2f} %)'.format(start_mem, end_mem, 100 * (start_mem - end_mem) / start_mem))\n    gc.collect()\n    return df","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:45:25.903389Z","iopub.execute_input":"2023-03-13T15:45:25.904518Z","iopub.status.idle":"2023-03-13T15:45:25.917414Z","shell.execute_reply.started":"2023-03-13T15:45:25.904452Z","shell.execute_reply":"2023-03-13T15:45:25.916313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 训练集\ntrain_df = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train.csv\")\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:45:25.918790Z","iopub.execute_input":"2023-03-13T15:45:25.919899Z","iopub.status.idle":"2023-03-13T15:46:31.449382Z","shell.execute_reply.started":"2023-03-13T15:45:25.919859Z","shell.execute_reply":"2023-03-13T15:46:31.447997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = reduce_mem(train_df)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:31.451964Z","iopub.execute_input":"2023-03-13T15:46:31.452419Z","iopub.status.idle":"2023-03-13T15:46:35.342763Z","shell.execute_reply.started":"2023-03-13T15:46:31.452383Z","shell.execute_reply":"2023-03-13T15:46:35.340687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 测试集\ntest_df = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/test.csv\")\ntest_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:35.344134Z","iopub.execute_input":"2023-03-13T15:46:35.344607Z","iopub.status.idle":"2023-03-13T15:46:35.400981Z","shell.execute_reply.started":"2023-03-13T15:46:35.344571Z","shell.execute_reply":"2023-03-13T15:46:35.399905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 训练集标签\ntargets = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train_labels.csv\")\ntargets.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:35.402292Z","iopub.execute_input":"2023-03-13T15:46:35.403151Z","iopub.status.idle":"2023-03-13T15:46:35.643170Z","shell.execute_reply.started":"2023-03-13T15:46:35.403114Z","shell.execute_reply":"2023-03-13T15:46:35.641162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"平均正确率为70.39%","metadata":{}},{"cell_type":"code","source":"targets.describe().T","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:35.644362Z","iopub.execute_input":"2023-03-13T15:46:35.645037Z","iopub.status.idle":"2023-03-13T15:46:35.678155Z","shell.execute_reply.started":"2023-03-13T15:46:35.645001Z","shell.execute_reply":"2023-03-13T15:46:35.676967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"标签数据集的session_id格式为`<session_id>_<question #>`，需要拆分session_id和问题编号","metadata":{}},{"cell_type":"code","source":"# 标签拆分\ntargets[\"session\"] = targets[\"session_id\"].apply(lambda x:int(x.split('_')[0]))\ntargets[\"q\"] = targets[\"session_id\"].apply(lambda x:int(x.split('_')[1][1:]))\ntargets.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:35.679549Z","iopub.execute_input":"2023-03-13T15:46:35.679952Z","iopub.status.idle":"2023-03-13T15:46:36.070923Z","shell.execute_reply.started":"2023-03-13T15:46:35.679914Z","shell.execute_reply":"2023-03-13T15:46:36.069891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"在提交文件样例中，第一列是`<session_id>_<question #>`，第二列是是否正确回答了问题，第三列是`<session_id>_<level_group>`。  \n\n从提交样例中可以看出，q1到q3 在 level 0-4 回答，q4到q13 在 level 5-12 回答，q14到q18 在 level 13-22 回答。","metadata":{}},{"cell_type":"code","source":"sample = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/sample_submission.csv\")\nsample","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-03-13T15:46:36.075722Z","iopub.execute_input":"2023-03-13T15:46:36.076865Z","iopub.status.idle":"2023-03-13T15:46:36.099936Z","shell.execute_reply.started":"2023-03-13T15:46:36.076823Z","shell.execute_reply":"2023-03-13T15:46:36.099011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"查看训练集信息，可以得知fullscreen、hq、music三个字段全为空，page、hover_duration、text、text_fqid缺失值较多。","metadata":{}},{"cell_type":"code","source":"train_df.info(verbose=True,null_counts=True)","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:36.106299Z","iopub.execute_input":"2023-03-13T15:46:36.109775Z","iopub.status.idle":"2023-03-13T15:46:40.735060Z","shell.execute_reply.started":"2023-03-13T15:46:36.109726Z","shell.execute_reply":"2023-03-13T15:46:40.734159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 数据可视化","metadata":{}},{"cell_type":"markdown","source":"参考：[Student Performance from Game Play - EDA](https://www.kaggle.com/code/demche/student-performance-from-game-play-eda)","metadata":{}},{"cell_type":"markdown","source":"### 各个特征的取值个数","metadata":{}},{"cell_type":"code","source":"stat = pd.DataFrame([train_df.nunique(), test_df.nunique()]).T.fillna(0)\nstat.columns = ['Number of unique values in train', 'Number of unique values in test']\nstat.style.format(\"{:,.0f}\").background_gradient(cmap='YlGn')","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:40.736551Z","iopub.execute_input":"2023-03-13T15:46:40.737167Z","iopub.status.idle":"2023-03-13T15:46:49.232907Z","shell.execute_reply.started":"2023-03-13T15:46:40.737134Z","shell.execute_reply":"2023-03-13T15:46:49.231965Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 每道题的平均正确率","metadata":{}},{"cell_type":"code","source":"q_corr = targets.groupby(\"q\")[\"correct\"].mean().mul(100).to_frame().reset_index()\nmean_correct = targets['correct'].mean()*100\npal = sns.color_palette(\"GnBu\", len(q_corr))\nrank = q_corr.sort_values('q')['correct'].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=q_corr,x=\"q\",y=\"correct\",palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0], fmt=\"%.1f%%\")\ng.axhline(mean_correct, color=\"coral\")\ndel mean_correct,pal,rank,g,q_corr","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:49.234516Z","iopub.execute_input":"2023-03-13T15:46:49.235174Z","iopub.status.idle":"2023-03-13T15:46:49.819024Z","shell.execute_reply.started":"2023-03-13T15:46:49.235139Z","shell.execute_reply":"2023-03-13T15:46:49.817954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 答对题数的session数量分布","metadata":{}},{"cell_type":"code","source":"correct = targets[targets[\"correct\"]==1]\nsession_correct = correct.groupby(\"session\")[\"correct\"].sum().to_frame().reset_index()\nsession_count = session_correct.groupby(\"correct\")[\"session\"].count().to_frame().reset_index()\npal = sns.color_palette(\"OrRd\", len(session_count))\nrank = session_count.sort_values('session')['correct'].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=session_count,x=\"correct\",y=\"session\",palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0])\ndel pal,g,correct,session_correct,session_count","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:49.820670Z","iopub.execute_input":"2023-03-13T15:46:49.821307Z","iopub.status.idle":"2023-03-13T15:46:50.380559Z","shell.execute_reply.started":"2023-03-13T15:46:49.821271Z","shell.execute_reply":"2023-03-13T15:46:50.379562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 各个level_group的平均正确率","metadata":{}},{"cell_type":"code","source":"# 把每个问题对应的level_group加入到训练集标签中\ndef generateLevelgroup(x):\n    if x<=4:\n        return \"0-4\"\n    elif x<=12:\n        return \"5-12\"\n    else:\n        return \"13-22\"\n    \ntargets[\"level_group\"] = targets[\"q\"].apply(lambda x:generateLevelgroup(x))","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:50.382109Z","iopub.execute_input":"2023-03-13T15:46:50.382876Z","iopub.status.idle":"2023-03-13T15:46:50.455981Z","shell.execute_reply.started":"2023-03-13T15:46:50.382829Z","shell.execute_reply":"2023-03-13T15:46:50.454796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"level_corr = targets.groupby(\"level_group\")[\"correct\"].mean().mul(100).to_frame().reset_index()\npal = sns.color_palette(\"BuGn\",len(level_corr))\nrank = level_corr.sort_values(\"level_group\")[\"correct\"].argsort()\nplt.figure(figsize=(15,5))\ng = sns.barplot(data=level_corr,x=\"level_group\",y=\"correct\",palette=np.array(pal[::])[rank],)\ng.bar_label(g.containers[0],fmt=\"%.1f%%\")\ndel level_corr,pal,rank,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:50.457671Z","iopub.execute_input":"2023-03-13T15:46:50.458032Z","iopub.status.idle":"2023-03-13T15:46:50.694256Z","shell.execute_reply.started":"2023-03-13T15:46:50.458000Z","shell.execute_reply":"2023-03-13T15:46:50.691885Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### event_name 分布情况","metadata":{}},{"cell_type":"code","source":"event_name = pd.DataFrame(train_df[\"event_name\"].value_counts())\npal = sns.color_palette(\"flare\", len(event_name))\nrank = event_name[\"event_name\"].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=event_name,y=event_name.index,x=event_name.event_name,orient='h',palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0],labels=event_name[\"event_name\"])\ndel event_name,pal,rank,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:50.695443Z","iopub.execute_input":"2023-03-13T15:46:50.695783Z","iopub.status.idle":"2023-03-13T15:46:52.011606Z","shell.execute_reply.started":"2023-03-13T15:46:50.695752Z","shell.execute_reply":"2023-03-13T15:46:52.010570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### event_name 含义","metadata":{}},{"cell_type":"markdown","source":"参考：[Meaning of Each Event Name and EDA](https://www.kaggle.com/code/shashwatraman/meaning-of-each-event-name-and-eda)\n\n* navigate_click: 移动角色到点击的位置,当鼠标移动到某些物体或者人物上，会出现黑色图标，点击黑色图标也属于navigate_click\n* observation_click: \n    * 点击某些黑色图标后会出现的对话框，出现对话框后的点击属于observation_click，observation_click后对话框消失。\n    * observation_click可以在屏幕上的任意位置。\n    * 点击人物也会出现文本，但这些文本出现后的点击并不是observation_click，而是person_click或者cutscene_click。\n    * 如果单击对象会导致两个或多个角色之间的对话，则继续对话的单击是cutscene_clicks\n* notification_click: 对某个物体navigate_click后弹出窗口，窗口底部有文字，文字出现后的点击属于notification_click\n* object_click: 窗口文字消失后，或没有文字的窗口，点击窗口的任意位置都属于object_click\n* object_hover: 当鼠标移动到可点击对象上时，鼠标指针样式会不一样，此时会记录鼠标悬停时间，保存在hover_duration中\n* map_hover: 当鼠标悬停在地图上的对象时，同样会记录悬停时间\n* map_click: 点击地图上的位置属于map_click\n* notebook_click: 点击打开笔记本，或是翻页、关闭等，都属于notebook_click\n* checkpoint: 一个level-group中发生的最后一个事件。玩家回答问题发生在checkpoint之后。\n","metadata":{}},{"cell_type":"markdown","source":"### name分布情况","metadata":{}},{"cell_type":"code","source":"event_name = pd.DataFrame(train_df[\"name\"].value_counts())\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=event_name,y=event_name.index,x=event_name.name)\ng.bar_label(g.containers[0],labels=event_name[\"name\"])\ndel event_name,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:52.012843Z","iopub.execute_input":"2023-03-13T15:46:52.013153Z","iopub.status.idle":"2023-03-13T15:46:53.252922Z","shell.execute_reply.started":"2023-03-13T15:46:52.013122Z","shell.execute_reply":"2023-03-13T15:46:53.251723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### fqid 分布情况","metadata":{}},{"cell_type":"code","source":"event_name = pd.DataFrame(train_df[\"fqid\"].value_counts()[:10])\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=event_name,y=event_name.index,x=event_name.fqid)\ng.bar_label(g.containers[0],labels=event_name[\"fqid\"])\ndel event_name,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:53.254547Z","iopub.execute_input":"2023-03-13T15:46:53.254874Z","iopub.status.idle":"2023-03-13T15:46:54.397228Z","shell.execute_reply.started":"2023-03-13T15:46:53.254837Z","shell.execute_reply":"2023-03-13T15:46:54.396119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### session_id中包含的时间信息","metadata":{}},{"cell_type":"markdown","source":"参考：[session_id reverse engineering](https://www.kaggle.com/code/pdnartreb/session-id-reverse-engineering)  \n\n猜想：  \n* 1、2位代表年份，取值为20，21，22\n* 3、4位代表月份，取值从00到11\n* 时间范围从2020年10月到2022年11月\n* 5、6位代表星期，取值从00到06，00代表星期天，01代表星期一\n* 7到15位代表时、分、秒、毫秒","metadata":{}},{"cell_type":"code","source":"session_time = pd.DataFrame(train_df[\"session_id\"].copy()).drop_duplicates().reset_index()\nsession_time[\"year\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[:2])).astype(np.uint8)\nsession_time[\"month\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[2:4]) + 1).astype(np.uint8)\nsession_time[\"weekday\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[4:6])).astype(np.uint8)\nsession_time[\"hour\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[6:8])).astype(np.uint8)\nsession_time[\"minute\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[8:10])).astype(np.uint8)\nsession_time[\"second\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[10:12])).astype(np.uint8)\nsession_time[\"ms\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[12:15])).astype(np.uint16)\nsession_time[\"?\"] = session_time[\"session_id\"].apply(lambda x: int(str(x)[15:17])).astype(np.uint8)\nsession_time.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:54.398713Z","iopub.execute_input":"2023-03-13T15:46:54.399040Z","iopub.status.idle":"2023-03-13T15:46:54.799309Z","shell.execute_reply.started":"2023-03-13T15:46:54.399008Z","shell.execute_reply":"2023-03-13T15:46:54.798148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 不同时间的session数量分布","metadata":{}},{"cell_type":"code","source":"session_time[\"year\"] = session_time[\"year\"]+2000\nsession_time[\"date\"] = session_time[\"year\"].astype(str)+'_'+session_time[\"month\"].astype(str)\nplt.figure(figsize=(20, 8))\ng = sns.countplot(x='date', data=session_time)\ng.set_yticklabels(['{}'.format(int(num)) for num in g.get_yticks()])\ng.bar_label(g.containers[0])\ng.xaxis.set_tick_params(rotation=45)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:54.800899Z","iopub.execute_input":"2023-03-13T15:46:54.801225Z","iopub.status.idle":"2023-03-13T15:46:55.431195Z","shell.execute_reply.started":"2023-03-13T15:46:54.801192Z","shell.execute_reply":"2023-03-13T15:46:55.427951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15, 5))\ng = sns.countplot(x='weekday', data=session_time)\ng.set_yticklabels(['{}'.format(int(num)) for num in g.get_yticks()])\ng.bar_label(g.containers[0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:55.432850Z","iopub.execute_input":"2023-03-13T15:46:55.433200Z","iopub.status.idle":"2023-03-13T15:46:55.671186Z","shell.execute_reply.started":"2023-03-13T15:46:55.433167Z","shell.execute_reply":"2023-03-13T15:46:55.670248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,10))\ng = sns.countplot(x='hour', data=session_time)\ng.set_yticklabels(['{}'.format(int(num)) for num in g.get_yticks()])\ng.bar_label(g.containers[0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:55.672500Z","iopub.execute_input":"2023-03-13T15:46:55.673038Z","iopub.status.idle":"2023-03-13T15:46:56.262041Z","shell.execute_reply.started":"2023-03-13T15:46:55.673004Z","shell.execute_reply":"2023-03-13T15:46:56.260848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"玩游戏最多的时间是11月到6月，周一到周五，9点到14点，属于在校时间","metadata":{}},{"cell_type":"markdown","source":"### 文本的长度分布","metadata":{}},{"cell_type":"code","source":"text_stat = pd.DataFrame(train_df['text'].dropna()).reset_index(drop=True)\ntext_stat['length'] = text_stat[\"text\"].apply(lambda x:len(x))\ntext_stat[\"length_group\"] = pd.cut(text_stat['length'],bins=[0,10,20,30,40,50,60,70,80,90],\n       labels=['0-10','10-20','20-30','30-40','40-50','50-60','60-70','70-80','80-90'])\nlength_count = text_stat[\"length_group\"].value_counts().to_frame().reset_index()\npal = sns.color_palette(\"Spectral\",len(length_count))\nrank = length_count.sort_values(\"index\")[\"index\"].argsort()\nplt.figure(figsize=(20, 10))\ng = sns.barplot(data=text_stat,y=length_count['length_group'],x=length_count['index'],palette=np.array(pal[::])[rank])\ng.bar_label(g.containers[0])\ndel text_stat,length_count,pal,rank,g","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:46:56.263693Z","iopub.execute_input":"2023-03-13T15:46:56.264369Z","iopub.status.idle":"2023-03-13T15:46:59.459923Z","shell.execute_reply.started":"2023-03-13T15:46:56.264326Z","shell.execute_reply":"2023-03-13T15:46:59.459017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 坐标特征——各个房间的坐标位置分布密度分析","metadata":{}},{"cell_type":"code","source":"rooms = list(train_df['room_fqid'].unique())\nfor room in rooms:\n    data_room = train_df[train_df['room_fqid'] == room]\n    data_room = data_room[['room_coor_x', 'room_coor_y', 'screen_coor_x', 'screen_coor_y']].dropna().reset_index(drop=True)\n    data_room = data_room.apply(pd.to_numeric, errors='coerce')\n    g = sns.pairplot(\n        data=data_room,\n        x_vars=[\"room_coor_x\"],\n        y_vars=[\"room_coor_y\"],\n        kind='hist',\n        height=6\n    )\n    g.fig.suptitle(f\"{room}: game room coordinates\")\n    g.savefig('g0.png', dpi=300)\n    plt.close(g.fig)\n    \n    g = sns.pairplot(\n        data=data_room,\n        x_vars=[\"screen_coor_x\"],\n        y_vars=[\"screen_coor_y\"],\n        kind='hist',\n        height=6\n    )\n    g.fig.suptitle(f\"{room}: player's screen coordinates\")\n    g.savefig('g1.png', dpi=300)\n    plt.close(g.fig)\n    \n    f, ax = plt.subplots(1, 2, figsize=(20, 20))\n    ax[0].imshow(mpimg.imread('g0.png'))\n    ax[1].imshow(mpimg.imread('g1.png'))\n    [axarr.set_axis_off() for axarr in ax.ravel()]\n    plt.tight_layout()\n    plt.show()\n\ndel room, rooms, data_room, g, f, ax","metadata":{"execution":{"iopub.status.busy":"2023-03-13T15:52:54.976558Z","iopub.execute_input":"2023-03-13T15:52:54.977026Z","iopub.status.idle":"2023-03-13T15:54:46.377190Z","shell.execute_reply.started":"2023-03-13T15:52:54.976987Z","shell.execute_reply":"2023-03-13T15:54:46.376014Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 异常数据","metadata":{}},{"cell_type":"markdown","source":"参考：[EDA on Game Progress](https://www.kaggle.com/code/abaojiang/eda-on-game-progress)  \n\n* 有些session_id的index不是从0开始\n* session_id和index的组合有重复值\n* 同一个session_id，index小的elapsed_time 大于 index大的elapsed_time\n* 有些session未包含所有22个level\n\n官方解释：index==0表示session开始，并且session不应该有重复的index。如果操作非常快，服务器上的时间戳可能记录不准确。我们可以假设index在elapsed_time不一致的情况下给出了正确的事件顺序。","metadata":{}}]}