{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":45533,"databundleVersionId":5748852,"sourceType":"competition"}],"dockerImageVersionId":30458,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom matplotlib.colors import ListedColormap\nfrom matplotlib.patches import ConnectionPatch","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-05-11T06:58:14.100192Z","iopub.execute_input":"2023-05-11T06:58:14.101337Z","iopub.status.idle":"2023-05-11T06:58:14.107008Z","shell.execute_reply.started":"2023-05-11T06:58:14.101285Z","shell.execute_reply":"2023-05-11T06:58:14.105924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"days = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']\ndays_abb = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']\ny_ticks = ['Low', 'Medium', 'High']\n\ndef create_perf_graph(perf):\n    fig, ax = plt.subplots(figsize=(7, 3))\n\n    sns.lineplot(x=days, y=perf, ax=ax, marker='o', markersize=5)\n\n    y_label = ax.set_ylabel('Perfomance', x=0, y=0.86)\n    \n    ax.set_yticks([1,2,3])\n    ax.set_yticklabels(y_ticks)\n    \n    ax.set_title('Students Perfomance Throughtout The Week', x=0.19, y=1.05)\n    \n    ax.set_ylim(0.8)\n\n    sns.despine(top=True, right=True)\n    \n    return ax","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T05:58:07.630161Z","iopub.execute_input":"2023-05-11T05:58:07.631271Z","iopub.status.idle":"2023-05-11T05:58:07.641568Z","shell.execute_reply.started":"2023-05-11T05:58:07.631224Z","shell.execute_reply":"2023-05-11T05:58:07.640359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Overview","metadata":{}},{"cell_type":"markdown","source":"The purpose of this Kaggle notebook is to examine how the **day** and **time of day** when students play a game may impact their **final performance**., which is measured by their quiz score. To achieve this, we will use the **reverse engineering** method described in this [notebook](https://www.kaggle.com/code/pdnartreb/session-id-reverse-engineering) and consider partitioning the data by day of the week and time of day. Our primary hypothesis is that the **mean scores of these groups differ**. To test this, we will use an **ANOVA test**, followed by **multiple t-tests with Bonferroni correction**","metadata":{}},{"cell_type":"markdown","source":"# What are the expecter results?","metadata":{}},{"cell_type":"markdown","source":"Before diving into the analysis, I would like to share my opinion on how the data may appear, firstly for the day of the week partition. These assumptions will be illustrated in a graphs.","metadata":{}},{"cell_type":"markdown","source":"#### Variant 1\nThe game is designed for 3-5 grade students. We can assume that they still **may not be as used to studying** as high school students.","metadata":{}},{"cell_type":"code","source":"ax = create_perf_graph([2,2,2,2,2,1,1])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T05:58:08.610614Z","iopub.execute_input":"2023-05-11T05:58:08.611084Z","iopub.status.idle":"2023-05-11T05:58:08.789945Z","shell.execute_reply.started":"2023-05-11T05:58:08.611042Z","shell.execute_reply":"2023-05-11T05:58:08.788667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Therefore, it is possible that the graph will show a **medium amount** of effort and performance by students **on weekdays**, with an even more **relaxed performance on weekends**.","metadata":{}},{"cell_type":"markdown","source":"#### Variant 2\nIn this scenario, we can observe that the weeks begin with a **high level of performance**, which gradually **decreases** until it reaches its lowest point on Friday.","metadata":{}},{"cell_type":"code","source":"ax = create_perf_graph([3,3,2,2,1,2,3])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T05:58:09.189182Z","iopub.execute_input":"2023-05-11T05:58:09.189613Z","iopub.status.idle":"2023-05-11T05:58:09.357106Z","shell.execute_reply.started":"2023-05-11T05:58:09.189571Z","shell.execute_reply":"2023-05-11T05:58:09.355984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"However, **performance improves** again over the weekend as the students take a break.","metadata":{}},{"cell_type":"markdown","source":"#### Variant 3\nAccording to this assumption, the weeks start with a **high level of energy and enthusiasm**, but the **decrease in performance** is more rapid compared to the second assumption.","metadata":{}},{"cell_type":"code","source":"ax = create_perf_graph([3,3,2,1,3,2,3])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T05:58:09.747811Z","iopub.execute_input":"2023-05-11T05:58:09.748256Z","iopub.status.idle":"2023-05-11T05:58:09.924258Z","shell.execute_reply.started":"2023-05-11T05:58:09.748217Z","shell.execute_reply":"2023-05-11T05:58:09.923161Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Interestingly, I assume that students are filled with joy once the weekdays are over, and therefore, they **do their best to complete any remaining tasks** by the end of the week. On Saturday, they take a **short break**, and then their **performance improves** again on Sunday.","metadata":{}},{"cell_type":"markdown","source":"#### Day Time Assumption","metadata":{}},{"cell_type":"markdown","source":"Regarding the time of day, I assume that perfomance follows the **natural biorhythm of humans**. Typically, our productivity level is **highest** in the morning, followed by a **dip** after lunch, a **slight peak** in the late afternoon, and a **decline** in efficiency towards the evening. The graph reflects this concept.\n\n<img  src='https://s3.amazonaws.com/wordpress-production/wp-content/uploads/2017/11/time-of-day.jpg' width=50%>","metadata":{}},{"cell_type":"markdown","source":"# Data Loading","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', \n                 usecols=['session_id'],\n                 dtype = {'session_id':'object'})\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T05:58:10.702658Z","iopub.execute_input":"2023-05-11T05:58:10.703104Z","iopub.status.idle":"2023-05-11T05:58:54.680399Z","shell.execute_reply.started":"2023-05-11T05:58:10.703063Z","shell.execute_reply":"2023-05-11T05:58:54.679277Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train_labels.csv')\n\nlabels[['session_id', 'question']] = labels['session_id'].str.split('_', expand=True)\nlabels = labels.pivot(columns='question', index='session_id', values='correct')\n\nscores = labels.sum(axis=1)\nscores = scores.rename('score')\n\nscores.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T05:58:54.682167Z","iopub.execute_input":"2023-05-11T05:58:54.683186Z","iopub.status.idle":"2023-05-11T05:58:56.820153Z","shell.execute_reply.started":"2023-05-11T05:58:54.683143Z","shell.execute_reply":"2023-05-11T05:58:56.819082Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Extraction","metadata":{}},{"cell_type":"code","source":"sessions = pd.DataFrame(df['session_id'].unique(), columns=['session_id'])\nsessions.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:19.559726Z","iopub.execute_input":"2023-05-11T06:00:19.560894Z","iopub.status.idle":"2023-05-11T06:00:22.394858Z","shell.execute_reply.started":"2023-05-11T06:00:19.560807Z","shell.execute_reply":"2023-05-11T06:00:22.393879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sessions['year'] = sessions['session_id'].str.slice(start=0, stop=2).astype(np.int8)\nsessions['month'] = sessions['session_id'].str.slice(start=2, stop=4).astype(np.int8)\n\nsessions['day'] = sessions['session_id'].str.slice(start=4, stop=6).astype(np.int8)\n\nsessions['hour'] = sessions[\"hour\"] = sessions[\"session_id\"].str.slice(start=6, stop=8).astype(np.uint8)\n\nsessions = sessions.set_index('session_id')\nsessions.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:22.397071Z","iopub.execute_input":"2023-05-11T06:00:22.397814Z","iopub.status.idle":"2023-05-11T06:00:22.491455Z","shell.execute_reply.started":"2023-05-11T06:00:22.397771Z","shell.execute_reply":"2023-05-11T06:00:22.490185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Moving Sunday from the beggining of the week to its end\nt = sessions['day']\nsessions['day'] = sessions['day'].map({0:7}).fillna(t)\nsessions.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:22.493091Z","iopub.execute_input":"2023-05-11T06:00:22.493471Z","iopub.status.idle":"2023-05-11T06:00:22.510947Z","shell.execute_reply.started":"2023-05-11T06:00:22.493434Z","shell.execute_reply":"2023-05-11T06:00:22.509602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"date_score = pd.concat([sessions, scores], axis=1)\ndate_score.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:22.51333Z","iopub.execute_input":"2023-05-11T06:00:22.514012Z","iopub.status.idle":"2023-05-11T06:00:22.553424Z","shell.execute_reply.started":"2023-05-11T06:00:22.51396Z","shell.execute_reply":"2023-05-11T06:00:22.55234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## General Trends EDA","metadata":{}},{"cell_type":"markdown","source":"In this section we explore **general tendencies** for game for present 3 years: 2020, 2021, 2022. In this interval we will consider **game activity** and **mean score** for each month.","metadata":{}},{"cell_type":"code","source":"date_score.year.unique()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:29.30618Z","iopub.execute_input":"2023-05-11T06:00:29.306612Z","iopub.status.idle":"2023-05-11T06:00:29.314698Z","shell.execute_reply.started":"2023-05-11T06:00:29.306573Z","shell.execute_reply":"2023-05-11T06:00:29.313721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Game Activity","metadata":{}},{"cell_type":"code","source":"game_activity = date_score.groupby(['year', 'month']).size()\ngame_activity.head()","metadata":{"_kg_hide-input":false,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2023-05-11T06:00:29.687791Z","iopub.execute_input":"2023-05-11T06:00:29.689306Z","iopub.status.idle":"2023-05-11T06:00:29.701349Z","shell.execute_reply.started":"2023-05-11T06:00:29.689249Z","shell.execute_reply":"2023-05-11T06:00:29.700343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(13, 5))\n\nsns.lineplot(x=range(9), y=game_activity[:9], ax=ax, color='orange')\nsns.lineplot(x=range(8, len(game_activity)), y=game_activity[8:], ax=ax, color='grey')\n\ny_label = ax.set_ylabel('Number Of Sessions', y=0.832)\n\nax.set_xlim(-0.5)\nax.set_ylim(0, 2250)\n\nax.set_xticks([-0.5,2.5,5.5,8.5,11.5,14.5,17.5,20.5,23.5,26.5])\nax.set_xticklabels([])\n\nax.set_xticks([1, 4, 7, 10, 13, 16, 19, 22, 25], minor=True)\nplt.tick_params(which='minor', bottom=False, top=False, left=False, right=False)\nax.set_xticklabels(['2020 Q4','2021 Q1','2021 Q2','2021 Q3', '2021 Q4','2022 Q1','2022 Q2','2022 Q3', '2022 Q4'],\n                   minor=True)\n\nax.set_title('Total Game Activity In Three Years', x=0.122, y=1.085, size=16)\nax.text(x=-2.25, y=2350, s=\"The assumed\", color='grey')\nax.text(x=0.35, y=2350, s='start of data collection', color='orange')\nax.text(x=4.75, y=2350, s='in 2020 may reveal the popularity in beggining', color='grey')\n\nsns.despine(top=True, right=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:00:29.943644Z","iopub.execute_input":"2023-05-11T06:00:29.94474Z","iopub.status.idle":"2023-05-11T06:00:30.296251Z","shell.execute_reply.started":"2023-05-11T06:00:29.944686Z","shell.execute_reply":"2023-05-11T06:00:30.295097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since data collection may not have started precisely at the beginning of first month or ended at the end of last month, the values at the ***tips*** of each month are ***extremely low***.","metadata":{}},{"cell_type":"markdown","source":"## Quiz Perfomance","metadata":{}},{"cell_type":"code","source":"date_score['year_month'] = date_score['year'].astype('str') + '_' + date_score['month'].astype('str')\ndate_score.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:30.534261Z","iopub.execute_input":"2023-05-11T06:00:30.535464Z","iopub.status.idle":"2023-05-11T06:00:30.610883Z","shell.execute_reply.started":"2023-05-11T06:00:30.535412Z","shell.execute_reply":"2023-05-11T06:00:30.609752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(13, 5))\n                   \nsns.pointplot(data=date_score, x='year_month', y='score', \n              ax=ax, \n              errorbar='se',\n              color='grey')\n\ny_label = ax.set_ylabel('Mean Quiz Score', y=0.86)\n\nax.set_xlim(-0.5)\nax.set_ylim(11, 14)\n\nax.set_xticks([-0.5, 2.5,5.5,8.5,11.5,14.5,17.5,20.5,23.5,26.5])\nax.set_xticklabels([])\n\nax.set_xticks([1, 4, 7, 10, 13, 16, 19, 22,25], minor=True)\nplt.tick_params(which='minor', bottom=False, top=False, left=False, right=False)\nax.set_xticklabels(['2020 Q4','2021 Q1','2021 Q2','2021 Q3', '2021 Q4','2022 Q1','2022 Q2','2022 Q3', '2022 Q4'],\n                   minor=True)\n\nax.set_xlabel('')\n\nax.set_title('Quiz Perfomance Activity In Three Years', x=0.16, y=1.085, size=16)\nax.text(x=-2.08, y=14.15, color='grey', s='Over the course of three years, there was only a ')\nax.text(x=7.125, y=14.15, s='slight increase by 0.75 point', color='orange')\nax.text(x=12.6, y=14.15, s='in the quiz score', color='grey')\n\nsns.lineplot(x=[-1,26], y=[11.97,12.76], color='orange', linewidth=3)\n\nsns.despine(top=True, right=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:00:30.692911Z","iopub.execute_input":"2023-05-11T06:00:30.693669Z","iopub.status.idle":"2023-05-11T06:00:31.137049Z","shell.execute_reply.started":"2023-05-11T06:00:30.693625Z","shell.execute_reply":"2023-05-11T06:00:31.135966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As previously noted, the little number of samples at the **tips** of the data results in a **high mean error**.","metadata":{}},{"cell_type":"markdown","source":"## Days Of The Week","metadata":{}},{"cell_type":"markdown","source":"In this section, we examine our main hypothesis, which suggests that **performance varies** *throughout the week and can be attributed to **three assumptions** outlined earlier:\n\n1. Distribution of **activity** among different groups\n2. Verification of ANOVA **requirements**\n3. Implementation of **ANOVA test**\n4. Conducting multiple **t-tests**\n5. **Plotting quiz performance** for each group with standard error.","metadata":{}},{"cell_type":"markdown","source":"## Activity","metadata":{}},{"cell_type":"code","source":"game_activity = date_score.groupby(['day']).size()\ngame_activity.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:34.571434Z","iopub.execute_input":"2023-05-11T06:00:34.572392Z","iopub.status.idle":"2023-05-11T06:00:34.584275Z","shell.execute_reply.started":"2023-05-11T06:00:34.572341Z","shell.execute_reply":"2023-05-11T06:00:34.582975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(7, 5))\n\nsns.lineplot(x=game_activity.index[4:], y=game_activity.iloc[4:],\n             ax=ax, color='grey', marker='o', markerfacecolor='c')\nsns.lineplot(x=game_activity.index[:5], y=game_activity.iloc[:5],\n             ax=ax, color='grey', marker='o', markerfacecolor='orange')\n\nax.set_xlabel('')\n\nax.set_ylabel('Number Of Sessions', y=0.84)\nax.set_xticks(range(1,8))\nax.set_xticklabels(days)\n\nax.set_ylim(1000,4500)\n\nax.set_title('Game Activity Throughtout The Week', x=0.215, y=1.085, size=14)\nax.text(-1.09+1, 4675, 'Work hard on', color='grey')\nax.text(0.1+1, 4675, 'weekday', color='orange')\nax.text(0.88+1, 4675, '- rest on', color='grey')\nax.text(1.65+1, 4675, 'weekends', color='c')\n\nsns.despine(top=True, right=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:00:34.58698Z","iopub.execute_input":"2023-05-11T06:00:34.587471Z","iopub.status.idle":"2023-05-11T06:00:34.856968Z","shell.execute_reply.started":"2023-05-11T06:00:34.587428Z","shell.execute_reply":"2023-05-11T06:00:34.855889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If we measure students' performance based on the **number of sessions** throughout the week, it appears similar to our **second model**, but with a slightly **higher performance** observed in the middle of the week.","metadata":{}},{"cell_type":"markdown","source":"## Hypothesis Testing: Day Of Week","metadata":{}},{"cell_type":"markdown","source":"### ANOVA Reqierements","metadata":{}},{"cell_type":"markdown","source":"1. **Homogeneity of variances** - the variances are equal across all groups.","metadata":{}},{"cell_type":"code","source":"ax = sns.boxplot(y=date_score.day, x=date_score.score, orient='h', color='#ADD8E6')\n\nax.set_title('Variance Homogeneity', x=0.01)\nax.set_ylabel('')\n# ax.set_xticks(range(1,8))\nax.set_yticklabels(days)\nsns.despine(top=True, right=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:00:34.858792Z","iopub.execute_input":"2023-05-11T06:00:34.859435Z","iopub.status.idle":"2023-05-11T06:00:35.151113Z","shell.execute_reply.started":"2023-05-11T06:00:34.859392Z","shell.execute_reply":"2023-05-11T06:00:35.14996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As observed, the variances are relatively similar. What is particularly interesing is the pattern observed in the medians of the scores, indicating moderate performance during weekdays and higher performance during weekends. It does not align with any of our initial assumptions.","metadata":{}},{"cell_type":"markdown","source":"2. **Independence of observations** - each subject must belong to only one group","metadata":{}},{"cell_type":"markdown","source":"3. **Categorical independent** variable - day of the week, and a **continuous dependent** variable - score. It is important to note that while the score is a discrete numerical variable, we can treat it as continuous for the purpose of this analysis.","metadata":{}},{"cell_type":"markdown","source":"4. **Equal group sisez** - group sizes within weekdays (Monday to Friday) and weekends (Saturday and Sunday) are approximately equal. However, there are differences between the group sizes of weekdays and weekends in 4 times!","metadata":{}},{"cell_type":"markdown","source":"Now that we have checked the relevant conditions, although not all of them are fully met, we can proceed with running the **ANOVA test**.","metadata":{}},{"cell_type":"code","source":"from scipy.stats import f_oneway\n\ngroups = []\nfor day in range(1,8):\n    groups.append(date_score.loc[date_score['day'] == day, 'score'].values)\nf_oneway(*groups)","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:39.246716Z","iopub.execute_input":"2023-05-11T06:00:39.247575Z","iopub.status.idle":"2023-05-11T06:00:39.264392Z","shell.execute_reply.started":"2023-05-11T06:00:39.247529Z","shell.execute_reply":"2023-05-11T06:00:39.26337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pairwise comparisons","metadata":{}},{"cell_type":"markdown","source":"The results of the ANOVA test indicate **statistical significance**! However, it only confirms the presence of differences between **at least two groups** and does not specify the exact nature of those differences. \n\nTo determine the specific group differences, we will conduct **multiple t-tests with Bonferroni correction**. This correction is necessary as we will be conducting a total of **21 tests,** and using the standard threshold may lead to obtaining significant results by chance where there are no true differences. **The Bonferroni correction** is a **simple** yet **conservative** approach, where the threshold p-value is divided by the number of tests conducted.","metadata":{}},{"cell_type":"code","source":"# Performing multiple t-tests\nfrom scipy.stats import ttest_ind\n\np_value = 0.05\ncomparisons = 7*6/2\np_value = p_value/comparisons\n\np_matrix = pd.DataFrame(index=days_abb, columns=days_abb, dtype=float)\nf_matrix = pd.DataFrame(index=days_abb, columns=days_abb, dtype=float)\n\nfor i in range(len(groups)):\n    for j in range(i+1, len(groups)):\n        results = ttest_ind(groups[i], groups[j])\n        p_matrix.iloc[i,j] = results[1]\n        f_matrix.iloc[i,j] = results[0]\n            \np_matrix = p_matrix.fillna(0)\np_matrix = p_matrix.transpose() + p_matrix\nnp.fill_diagonal(p_matrix.values, 1)\nsignif_matrix = p_matrix < p_value","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:00:40.05072Z","iopub.execute_input":"2023-05-11T06:00:40.051213Z","iopub.status.idle":"2023-05-11T06:00:40.083597Z","shell.execute_reply.started":"2023-05-11T06:00:40.051169Z","shell.execute_reply":"2023-05-11T06:00:40.082334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cyan = (0, 100/256, 100/256,1)\ngrey = (0.9,0.9,0.9,1)\ncmap = ListedColormap([grey, cyan])\n\nax = sns.heatmap(signif_matrix, cbar=False, cmap=cmap)\n\nplt.yticks(rotation=0)\nax.tick_params(left=False, bottom=False)\n\nax.set_title('Significant Difference', color=cyan, x=0.11)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:00:40.944653Z","iopub.execute_input":"2023-05-11T06:00:40.945578Z","iopub.status.idle":"2023-05-11T06:00:41.168491Z","shell.execute_reply.started":"2023-05-11T06:00:40.945522Z","shell.execute_reply":"2023-05-11T06:00:41.165785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Significant differences in quiz scores were found between certain days of the week. Tuesday differed from Monday, and Friday differed from Wednesday. Additionally, weekends differed from weekdays, but there was no significant difference between Saturday and Sunday.","metadata":{}},{"cell_type":"markdown","source":"# Plotting Relationship","metadata":{}},{"cell_type":"markdown","source":"Let's now explore the mean quiz scores graph for more clarity.","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(7, 5))\n\nsns.pointplot(data=date_score, x='day', y='score', ax=ax, errorbar='se')\n\nax.set_xlabel('')\nax.set_xticks(range(7))\nax.set_xticklabels(days_abb)\n\nax.set_title('Quiz Perfomance Throughout the day', x=0.179, y=1.05)\n\nax.set_ylim(12.4,13.4)\nax.set_ylabel('score',y=0.967)\n\nsns.despine(right=True, top=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:51:12.915191Z","iopub.execute_input":"2023-05-11T06:51:12.915631Z","iopub.status.idle":"2023-05-11T06:51:13.125721Z","shell.execute_reply.started":"2023-05-11T06:51:12.915592Z","shell.execute_reply":"2023-05-11T06:51:13.124606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"When measuring performance based on scores, the observed pattern appears to be a combination of the **first and third variants**. Both variants suggest a **decline** in performance throughout the week and an **increase** in performance during weekends or holidays. However, it should be noted that the **mean error** for weekdays is higher due to **smaller sample sizes** (1,000 versus 4,000 for weekdays).","metadata":{}},{"cell_type":"markdown","source":"As a final note, we could consider **PROJECTING** the data specifically for **weekdays** and **weekends** for further analysis of performance patterns and feature engineering.","metadata":{}},{"cell_type":"markdown","source":"# Day Time","metadata":{}},{"cell_type":"markdown","source":"In this section, we will examine **performance** throughout the day. This chapter will be much shorter, as the ANOVA test will be left as an **exercise for YOU** to perform!","metadata":{}},{"cell_type":"markdown","source":"# Activity","metadata":{}},{"cell_type":"code","source":"game_activity = date_score.groupby('hour').size()\ngame_activity.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:43:31.294177Z","iopub.execute_input":"2023-05-11T06:43:31.2946Z","iopub.status.idle":"2023-05-11T06:43:31.305434Z","shell.execute_reply.started":"2023-05-11T06:43:31.294562Z","shell.execute_reply":"2023-05-11T06:43:31.304387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12,4))\n\nsns.lineplot(x=game_activity.index, y=game_activity, ax=ax, color='grey')\n\nax.set_ylim(0,3000)\nax.set_ylabel('Number Of Sessions',x=0,y=0.79)\n\nax.set_xticks(range(24))\nax.set_xlim(0,23)\n\nax.set_title('Game Activity Throughout the Day', x=0.112, y=1.1, size=14)\nax.text(x=-1.59, y=3170, s=\"Players' activity aligns with the\", color='grey')\nax.text(x=3.78, y=3170, s=\" human biological rhythm\", color='orange')\n\nsns.despine(right=True, top=True)","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:45:36.922739Z","iopub.execute_input":"2023-05-11T06:45:36.923186Z","iopub.status.idle":"2023-05-11T06:45:37.27287Z","shell.execute_reply.started":"2023-05-11T06:45:36.923147Z","shell.execute_reply":"2023-05-11T06:45:37.271354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Projection","metadata":{}},{"cell_type":"markdown","source":"There are a few issues with performing ANOVA on these 24 groups:\n\n1. The **large number of groups** results in a significant number of comparisons (24*23/2=276)\n2. The **night data** is particularly **noisy** due to the limited sample size of night players 🦉\n\nI suggest categorizing our data into broader groups based on time periods: Night (0-5), Morning (6-11), Day (12-17), and Evening (18-23). This division can be visually represented on the graph.","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12,4))\n\nsns.pointplot(data=date_score, x='hour', y='score', \n              errorbar='se',\n              color='grey')\n\nax.set_title('Quiz Perfomance throughout the day', size=14, x=0.128, y=1.1)\nax.text(x=-2.05, y=15.2,s='For simplicity of our analysis we do categorical', color='grey')\nax.text(x=6.52, y=15.2,s='projection', color='black')\nax.set_ylabel('score', y=0.962)\nax.set_ylim(11.5,15)\n\nplt.axvline(x = 5.5, color = 'black', linestyle='--')\nplt.axvline(x = 11.5, color = 'black', linestyle='--')\nplt.axvline(x = 17.5, color = 'black', linestyle='--')\n\nax.text(x=0.5,y=14.5,s='Night')\nax.text(x=6.5,y=14.5,s='Morning')\nax.text(x=12.5,y=14.5,s='Day')\nax.text(18.5,y=14.5,s='Evening')\n\nsns.despine(right=True, top=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T06:45:48.382862Z","iopub.execute_input":"2023-05-11T06:45:48.383286Z","iopub.status.idle":"2023-05-11T06:45:48.797707Z","shell.execute_reply.started":"2023-05-11T06:45:48.383245Z","shell.execute_reply":"2023-05-11T06:45:48.79669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Upon closer examination, it becomes clear that the observed pattern does **not align** with our **initial assumption** regarding the impact of time of day. Additionally, the data from players during the **night** period appears to be quite **messy and inconsistent**.","metadata":{}},{"cell_type":"code","source":"day_time = {}\n\nfor i in range(24):\n    if i < 6:\n        day_time[i] = 0\n    elif i < 12:\n        day_time[i] = 1\n    elif i < 18:\n        day_time[i] = 2\n    else:\n        day_time[i] = 3\n\ndate_score['day_time'] = date_score['hour'].map(day_time)\ndate_score.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-11T06:45:48.800032Z","iopub.execute_input":"2023-05-11T06:45:48.800779Z","iopub.status.idle":"2023-05-11T06:45:48.824303Z","shell.execute_reply.started":"2023-05-11T06:45:48.800735Z","shell.execute_reply":"2023-05-11T06:45:48.82306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To speed up the process of writing this notebook and provide an **opportunity for YOU to practice** the steps involved in applying the **ANOVA test**, I will proceed with directly plotting the data without performing any statistical tests. This way, you can observe the visual representation and undertake the mutlitple comparisons analysis yourself, consolidating your knowledge through hands-on practice.","metadata":{}},{"cell_type":"code","source":"performance = date_score.groupby('day_time')['score'].mean()\nperformance","metadata":{"execution":{"iopub.status.busy":"2023-05-11T07:08:18.581568Z","iopub.execute_input":"2023-05-11T07:08:18.582054Z","iopub.status.idle":"2023-05-11T07:08:18.593322Z","shell.execute_reply.started":"2023-05-11T07:08:18.582009Z","shell.execute_reply":"2023-05-11T07:08:18.592156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(9,4))\n\nsns.pointplot(data=date_score, x='day_time', y='score', \n              errorbar='se',\n              color='grey')\n\nax.set_title('Quiz Perfomance throughout the day', size=14, x=0.172, y=1.1)\nax.text(x=-0.83,y=13.45,s='Statistically significant', color='grey')\nax.text(x=0.1,y=13.45,s='difference', color='orange')\nax.text(x=0.53,y=13.45,s='observed in 2 pairs', color='grey')\n\nax.set_ylabel('score', y=0.96)\nax.set_ylim(12.5,13.4)\n\nax.set_xticklabels(['Night', 'Morning', 'Day', 'Evening'])\nax.set_xlabel('')\n\narrow = ConnectionPatch(xyA=(1, performance[1]),\n                        xyB=(3, performance[3]),\n                        coordsA=\"data\",\n                        coordsB=\"data\",\n                        arrowstyle='<|-|>,head_width=0.4,head_length=0.8',\n                        linewidth=1.5,\n                        edgecolor='orange',\n                        facecolor='orange',\n                        connectionstyle='arc3,rad=.4')\nax.add_artist(arrow)\n                        \narrow = ConnectionPatch(xyA=(2, performance[2]),\n                        xyB=(3, performance[3]),\n                        coordsA=\"data\",\n                        coordsB=\"data\",\n                        arrowstyle='<|-|>,head_width=0.4,head_length=0.8',\n                        linewidth=1.5,\n                        edgecolor='orange',\n                        facecolor='orange',\n                        connectionstyle='arc3,rad=.3')\nax.add_artist(arrow)\n\nsns.despine(right=True, top=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-11T07:17:51.969775Z","iopub.execute_input":"2023-05-11T07:17:51.970229Z","iopub.status.idle":"2023-05-11T07:17:52.26716Z","shell.execute_reply.started":"2023-05-11T07:17:51.970189Z","shell.execute_reply":"2023-05-11T07:17:52.266033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Wind up 👨‍💻","metadata":{}},{"cell_type":"markdown","source":"### So, what do we have in the end?\n\n- We generated **2 features** - day of the week and hour of the day.\n- Applied **ANOVA test** for multiple groups comparison to identify difference in quiz scores among all groups created by our 2 independent variable. It required us to check all the **requirements for ANOVA test**.\n- Performed **multiple t-tests** to further investigate the observed differences. To reduce the risk of false positive results, we employed the **Bonferroni correction**.\n- We discussed how number of groups may be lowered using **categorical projection** to broader groups: weekdays and weekends; morning, afternoon, day and evening.\n- Tried to draw pretty graphs! 😀\n\nI hope you enjoyed this long journey and find something new for you. Thank you for reading my notebook! If you have any questions, suggestions or ideas on this topic - leave them in the comments.","metadata":{}}]}