{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Project: Investigate a Dataset NFL\nby: **Abdallah Mahmoud**, **Ali Aboelnaga**, **Habiba Mourad**, **Omnia Ahmed**, **Reem Tarek**\n\n## Table of Contents\n<ul>\n<li><a href=\"#intro\">1. Introduction</a></li>\n<li><a href=\"#wrangling\">2. Data Wrangling</a></li>\n<li><a href=\"#eda\">3. Exploratory Data Analysis</a></li>\n<li><a href=\"#conclusions\">4. Conclusions</a></li>\n</ul>","metadata":{}},{"cell_type":"markdown","source":"<a id='intro'></a>\n## 1. Introduction\n### Overview\nThe 2022 Big Data Bowl data contains Next Gen Stats player tracking, play, game, player, and PFF scouting data for all 2018-2020 Special Teams plays.\n\n### Questions\n<ul>\n<li><a href=\"#q1\">1. What is the most frequent kickoff type?</a></li>\n<li><a href=\"#q2\">2. How often does the actual kick direction differ from the intended direction?</a></li>\n<li><a href=\"#q3\">3. Relation between hangTime and OperationTime</a></li>\n<li><a href=\"#q4\">4. Number of plays each quarter</a></li>\n<li><a href=\"#q5\">5. Number of plays each down</a></li>\n<li><a href=\"#q6\">6. Minimum, maximum and average play result</a></li>\n<li><a href=\"#q7\">7. Top 10 frequent possession teams</a></li>\n<li><a href=\"#q8\">8. Most frequent specialTeamsPlayType</a></li>\n<li><a href=\"#q9\">9. What is the relation between height and position</a></li>\n<li><a href=\"#q10\">10. What college graduates most special teams players</a></li>\n<li><a href=\"#q11\">11. Which season has most plays? </a></li>\n<li><a href=\"#q12\">12. Number of matches by week</a></li>\n<li><a href=\"#q13\">13. Number of games for every date</a></li>\n<li><a href=\"#q14\">14. Number of games based on Timing</a></li>\n<li><a href=\"#q15\">15. Number of games through years</a></li>\n</ul>","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport statsmodels.api as sm\nfrom statsmodels.graphics.gofplots import qqplot\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport plotly.express as px\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\n\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.150323Z","iopub.execute_input":"2022-01-13T12:05:48.150954Z","iopub.status.idle":"2022-01-13T12:05:48.157926Z","shell.execute_reply.started":"2022-01-13T12:05:48.150905Z","shell.execute_reply":"2022-01-13T12:05:48.156934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='wrangling'></a>\n## 2. Data Wrangling\n\nIn this section of the report, we will load in the data, check for cleanliness, and then trim and clean the dataset for analysis.","metadata":{}},{"cell_type":"markdown","source":"### 2.1 PFFScouting Data","metadata":{}},{"cell_type":"code","source":"pf = pd.read_csv('/kaggle/input/nfl-big-data-bowl-2022/PFFScoutingData.csv')\npf.columns","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.163354Z","iopub.execute_input":"2022-01-13T12:05:48.16373Z","iopub.status.idle":"2022-01-13T12:05:48.224771Z","shell.execute_reply.started":"2022-01-13T12:05:48.163681Z","shell.execute_reply":"2022-01-13T12:05:48.223751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since the data is huge, we will work only on columns that we find most interesting to us and drop the rest.","metadata":{}},{"cell_type":"code","source":"pf = pf[['gameId', 'hangTime', 'operationTime', 'kickType', 'kickDirectionIntended', 'kickDirectionActual']]","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.226426Z","iopub.execute_input":"2022-01-13T12:05:48.226738Z","iopub.status.idle":"2022-01-13T12:05:48.241357Z","shell.execute_reply.started":"2022-01-13T12:05:48.226699Z","shell.execute_reply":"2022-01-13T12:05:48.240539Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's check first for duplicates and drop them","metadata":{}},{"cell_type":"code","source":"pf.duplicated().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.243199Z","iopub.execute_input":"2022-01-13T12:05:48.243912Z","iopub.status.idle":"2022-01-13T12:05:48.261614Z","shell.execute_reply.started":"2022-01-13T12:05:48.243867Z","shell.execute_reply":"2022-01-13T12:05:48.260786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pf.drop_duplicates(inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.262781Z","iopub.execute_input":"2022-01-13T12:05:48.262993Z","iopub.status.idle":"2022-01-13T12:05:48.275802Z","shell.execute_reply.started":"2022-01-13T12:05:48.262969Z","shell.execute_reply":"2022-01-13T12:05:48.27512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, check for `null` values","metadata":{}},{"cell_type":"code","source":"print(\"data shape:\", pf.shape)\npf.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.277135Z","iopub.execute_input":"2022-01-13T12:05:48.277617Z","iopub.status.idle":"2022-01-13T12:05:48.290964Z","shell.execute_reply.started":"2022-01-13T12:05:48.277573Z","shell.execute_reply":"2022-01-13T12:05:48.289918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are lots of missing values\n\nWe have many choices:-\n1. drop columns with too many `NaN`\n2. replace them with 0\n3. fill them with the mean/mode\n4. leave them\n\nsince we have many data we can drop the null values.","metadata":{}},{"cell_type":"code","source":"pf.dropna(axis=0, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.293775Z","iopub.execute_input":"2022-01-13T12:05:48.293993Z","iopub.status.idle":"2022-01-13T12:05:48.304347Z","shell.execute_reply.started":"2022-01-13T12:05:48.293967Z","shell.execute_reply":"2022-01-13T12:05:48.303331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pf.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.305477Z","iopub.execute_input":"2022-01-13T12:05:48.305998Z","iopub.status.idle":"2022-01-13T12:05:48.311419Z","shell.execute_reply.started":"2022-01-13T12:05:48.305965Z","shell.execute_reply":"2022-01-13T12:05:48.310746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pf.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.312636Z","iopub.execute_input":"2022-01-13T12:05:48.313013Z","iopub.status.idle":"2022-01-13T12:05:48.331003Z","shell.execute_reply.started":"2022-01-13T12:05:48.312979Z","shell.execute_reply":"2022-01-13T12:05:48.330266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.2 Plays Data","metadata":{}},{"cell_type":"code","source":"plays = pd.read_csv('/kaggle/input/nfl-big-data-bowl-2022/plays.csv')\nprint(plays.columns)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.332521Z","iopub.execute_input":"2022-01-13T12:05:48.332995Z","iopub.status.idle":"2022-01-13T12:05:48.458356Z","shell.execute_reply.started":"2022-01-13T12:05:48.332962Z","shell.execute_reply":"2022-01-13T12:05:48.457368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"select only target columns","metadata":{}},{"cell_type":"code","source":"plays = plays[['gameId', 'playId', 'quarter', 'down', 'yardsToGo', 'possessionTeam', 'specialTeamsPlayType',\n              'specialTeamsResult', 'playResult']]","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.460084Z","iopub.execute_input":"2022-01-13T12:05:48.460785Z","iopub.status.idle":"2022-01-13T12:05:48.467658Z","shell.execute_reply.started":"2022-01-13T12:05:48.460744Z","shell.execute_reply":"2022-01-13T12:05:48.467011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"check for duplicates","metadata":{}},{"cell_type":"code","source":"plays.duplicated().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.468802Z","iopub.execute_input":"2022-01-13T12:05:48.469375Z","iopub.status.idle":"2022-01-13T12:05:48.487721Z","shell.execute_reply.started":"2022-01-13T12:05:48.469331Z","shell.execute_reply":"2022-01-13T12:05:48.48714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"again, check for nulls","metadata":{}},{"cell_type":"code","source":"plays.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.489192Z","iopub.execute_input":"2022-01-13T12:05:48.489564Z","iopub.status.idle":"2022-01-13T12:05:48.499241Z","shell.execute_reply.started":"2022-01-13T12:05:48.489534Z","shell.execute_reply":"2022-01-13T12:05:48.498666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"we have no `nulls`","metadata":{}},{"cell_type":"code","source":"plays.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.500285Z","iopub.execute_input":"2022-01-13T12:05:48.500649Z","iopub.status.idle":"2022-01-13T12:05:48.507688Z","shell.execute_reply.started":"2022-01-13T12:05:48.500619Z","shell.execute_reply":"2022-01-13T12:05:48.506926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plays.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.508806Z","iopub.execute_input":"2022-01-13T12:05:48.508995Z","iopub.status.idle":"2022-01-13T12:05:48.525015Z","shell.execute_reply.started":"2022-01-13T12:05:48.508971Z","shell.execute_reply":"2022-01-13T12:05:48.524408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.3 Player Data","metadata":{}},{"cell_type":"code","source":"players = pd.read_csv('/kaggle/input/nfl-big-data-bowl-2022/players.csv')\nprint(players.columns)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.526023Z","iopub.execute_input":"2022-01-13T12:05:48.526346Z","iopub.status.idle":"2022-01-13T12:05:48.54719Z","shell.execute_reply.started":"2022-01-13T12:05:48.526309Z","shell.execute_reply":"2022-01-13T12:05:48.546687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players.duplicated().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.548122Z","iopub.execute_input":"2022-01-13T12:05:48.548416Z","iopub.status.idle":"2022-01-13T12:05:48.556157Z","shell.execute_reply.started":"2022-01-13T12:05:48.54839Z","shell.execute_reply":"2022-01-13T12:05:48.555389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.558111Z","iopub.execute_input":"2022-01-13T12:05:48.558328Z","iopub.status.idle":"2022-01-13T12:05:48.56657Z","shell.execute_reply.started":"2022-01-13T12:05:48.558302Z","shell.execute_reply":"2022-01-13T12:05:48.565815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"drop nulls","metadata":{}},{"cell_type":"code","source":"players.dropna(inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.568927Z","iopub.execute_input":"2022-01-13T12:05:48.569153Z","iopub.status.idle":"2022-01-13T12:05:48.576586Z","shell.execute_reply.started":"2022-01-13T12:05:48.56912Z","shell.execute_reply":"2022-01-13T12:05:48.575884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"convert `birthDate` to datetime","metadata":{}},{"cell_type":"code","source":"def func(row):\n    s = row['height'].split('-')\n    if len(s)==1:\n        return int(row['height'])\n    else:\n        return int(row['height'].split('-')[0])*12+int(row['height'].split('-')[1])","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.577677Z","iopub.execute_input":"2022-01-13T12:05:48.578015Z","iopub.status.idle":"2022-01-13T12:05:48.586716Z","shell.execute_reply.started":"2022-01-13T12:05:48.577969Z","shell.execute_reply":"2022-01-13T12:05:48.585669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players['height'] = players.apply(func, axis=1)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.587955Z","iopub.execute_input":"2022-01-13T12:05:48.588688Z","iopub.status.idle":"2022-01-13T12:05:48.64557Z","shell.execute_reply.started":"2022-01-13T12:05:48.588644Z","shell.execute_reply":"2022-01-13T12:05:48.644635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players['birthDate'] = pd.to_datetime(players['birthDate'])","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.646923Z","iopub.execute_input":"2022-01-13T12:05:48.647147Z","iopub.status.idle":"2022-01-13T12:05:48.655919Z","shell.execute_reply.started":"2022-01-13T12:05:48.64712Z","shell.execute_reply":"2022-01-13T12:05:48.655123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.657002Z","iopub.execute_input":"2022-01-13T12:05:48.6575Z","iopub.status.idle":"2022-01-13T12:05:48.669502Z","shell.execute_reply.started":"2022-01-13T12:05:48.657459Z","shell.execute_reply":"2022-01-13T12:05:48.668663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.670627Z","iopub.execute_input":"2022-01-13T12:05:48.670838Z","iopub.status.idle":"2022-01-13T12:05:48.6845Z","shell.execute_reply.started":"2022-01-13T12:05:48.670812Z","shell.execute_reply":"2022-01-13T12:05:48.683458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2.4 Games Data","metadata":{}},{"cell_type":"code","source":"games = pd.read_csv('/kaggle/input/nfl-big-data-bowl-2022/games.csv')\ngames.columns","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.685518Z","iopub.execute_input":"2022-01-13T12:05:48.685821Z","iopub.status.idle":"2022-01-13T12:05:48.700473Z","shell.execute_reply.started":"2022-01-13T12:05:48.685795Z","shell.execute_reply":"2022-01-13T12:05:48.699738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"games.duplicated().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.701389Z","iopub.execute_input":"2022-01-13T12:05:48.701872Z","iopub.status.idle":"2022-01-13T12:05:48.710515Z","shell.execute_reply.started":"2022-01-13T12:05:48.701829Z","shell.execute_reply":"2022-01-13T12:05:48.709491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"games.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.713111Z","iopub.execute_input":"2022-01-13T12:05:48.713363Z","iopub.status.idle":"2022-01-13T12:05:48.721666Z","shell.execute_reply.started":"2022-01-13T12:05:48.713333Z","shell.execute_reply":"2022-01-13T12:05:48.720938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# games['gameDate'] = pd.to_datetime(games['gameDate'])","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.722766Z","iopub.execute_input":"2022-01-13T12:05:48.722977Z","iopub.status.idle":"2022-01-13T12:05:48.728765Z","shell.execute_reply.started":"2022-01-13T12:05:48.722947Z","shell.execute_reply":"2022-01-13T12:05:48.72795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"games.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.729864Z","iopub.execute_input":"2022-01-13T12:05:48.730582Z","iopub.status.idle":"2022-01-13T12:05:48.737815Z","shell.execute_reply.started":"2022-01-13T12:05:48.73051Z","shell.execute_reply":"2022-01-13T12:05:48.737113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"games.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.73872Z","iopub.execute_input":"2022-01-13T12:05:48.739302Z","iopub.status.idle":"2022-01-13T12:05:48.753116Z","shell.execute_reply.started":"2022-01-13T12:05:48.739265Z","shell.execute_reply":"2022-01-13T12:05:48.752428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='eda'></a>\n## 3. Exploratory Data Analysis","metadata":{}},{"cell_type":"markdown","source":"<a id='q1'></a>\n### 3.1 What is the most frequent kickoff type?","metadata":{}},{"cell_type":"code","source":"pf['kickType'].unique()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.754045Z","iopub.execute_input":"2022-01-13T12:05:48.75422Z","iopub.status.idle":"2022-01-13T12:05:48.759472Z","shell.execute_reply.started":"2022-01-13T12:05:48.754198Z","shell.execute_reply":"2022-01-13T12:05:48.758952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pf['kickType'] = pf['kickType'].map({'N':'standard punt style', 'A':'Aussie-style punts', 'R':'Rugby style punt'})","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.760508Z","iopub.execute_input":"2022-01-13T12:05:48.760915Z","iopub.status.idle":"2022-01-13T12:05:48.76939Z","shell.execute_reply.started":"2022-01-13T12:05:48.760884Z","shell.execute_reply":"2022-01-13T12:05:48.768759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pf['kickType'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.770493Z","iopub.execute_input":"2022-01-13T12:05:48.770728Z","iopub.status.idle":"2022-01-13T12:05:48.782909Z","shell.execute_reply.started":"2022-01-13T12:05:48.770697Z","shell.execute_reply":"2022-01-13T12:05:48.782277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(pf['kickType'].value_counts(), title='Most frequent kick types', labels={'index':'style', 'value':'count'})","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:48.783749Z","iopub.execute_input":"2022-01-13T12:05:48.784463Z","iopub.status.idle":"2022-01-13T12:05:49.702007Z","shell.execute_reply.started":"2022-01-13T12:05:48.784414Z","shell.execute_reply":"2022-01-13T12:05:49.70108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**The most frequent kick type is the `standerd`, followed by the `Aussie`**","metadata":{}},{"cell_type":"markdown","source":"<a id='q2'></a>\n### 3.2 How often does the actual kick direction differ from the intended direction?","metadata":{}},{"cell_type":"code","source":"pf[pf['kickDirectionActual']!=pf['kickDirectionIntended']].shape[0]","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:49.703536Z","iopub.execute_input":"2022-01-13T12:05:49.704012Z","iopub.status.idle":"2022-01-13T12:05:49.711967Z","shell.execute_reply.started":"2022-01-13T12:05:49.703966Z","shell.execute_reply":"2022-01-13T12:05:49.711171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Actual_Intend_dir = pd.pivot_table(data=pf, index=['kickDirectionIntended'], columns='kickDirectionActual', values='gameId', aggfunc='count').fillna(0)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:49.71402Z","iopub.execute_input":"2022-01-13T12:05:49.714336Z","iopub.status.idle":"2022-01-13T12:05:49.740012Z","shell.execute_reply.started":"2022-01-13T12:05:49.714298Z","shell.execute_reply":"2022-01-13T12:05:49.739379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(7,5))\nsns.heatmap(Actual_Intend_dir, cmap='Blues', annot=True);","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:49.744843Z","iopub.execute_input":"2022-01-13T12:05:49.745263Z","iopub.status.idle":"2022-01-13T12:05:50.020841Z","shell.execute_reply.started":"2022-01-13T12:05:49.745223Z","shell.execute_reply":"2022-01-13T12:05:50.019994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**There are 191 cases where the player misses the inteded direction.**<br>\n<br>most frequent:-\n- 90 times the intended direction was `L` (left), but the actual was `C` (center)\n- 71 times the intended direction was `R` (right), but the actual was `C` (center)","metadata":{}},{"cell_type":"markdown","source":"<a id='q3'></a>\n### 3.3 Relation between hangTime and OperationTime","metadata":{}},{"cell_type":"code","source":"px.scatter(data_frame=pf, x='operationTime', y='hangTime', marginal_x='histogram', marginal_y='histogram')","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.022503Z","iopub.execute_input":"2022-01-13T12:05:50.023027Z","iopub.status.idle":"2022-01-13T12:05:50.200042Z","shell.execute_reply.started":"2022-01-13T12:05:50.022984Z","shell.execute_reply":"2022-01-13T12:05:50.199146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Cheack for normailty with QQ-Plot","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\n\nx1 = plt.subplot(1, 2, 1)\nq1 = qqplot(pf['hangTime'], line='s',ax = x1)\n\nx2 = plt.subplot(1, 2, 2)\nq2 = qqplot(pf['operationTime'], line='s',ax = x2)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.201295Z","iopub.execute_input":"2022-01-13T12:05:50.202085Z","iopub.status.idle":"2022-01-13T12:05:50.581127Z","shell.execute_reply.started":"2022-01-13T12:05:50.20204Z","shell.execute_reply":"2022-01-13T12:05:50.580291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**There is no observed relation between `hangTime` and `OperationTime`**","metadata":{}},{"cell_type":"code","source":"plays.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.582682Z","iopub.execute_input":"2022-01-13T12:05:50.583488Z","iopub.status.idle":"2022-01-13T12:05:50.594893Z","shell.execute_reply.started":"2022-01-13T12:05:50.58342Z","shell.execute_reply":"2022-01-13T12:05:50.59435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q4'></a>\n### 3.4 Number of plays each quarter","metadata":{}},{"cell_type":"code","source":"px.bar(plays['quarter'].value_counts(), width=700, height=400, title='Number of plays each quarter', labels={'index':'quarter', 'value':'count'})","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.595855Z","iopub.execute_input":"2022-01-13T12:05:50.596137Z","iopub.status.idle":"2022-01-13T12:05:50.807034Z","shell.execute_reply.started":"2022-01-13T12:05:50.596111Z","shell.execute_reply":"2022-01-13T12:05:50.806469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q5'></a>\n### 3.5 Number of plays each down","metadata":{}},{"cell_type":"code","source":"px.bar(plays['down'].value_counts(), width=700, height=400, title='Number of plays each down', labels={'index':'down', 'value':'count'})","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.808027Z","iopub.execute_input":"2022-01-13T12:05:50.808347Z","iopub.status.idle":"2022-01-13T12:05:50.864706Z","shell.execute_reply.started":"2022-01-13T12:05:50.808321Z","shell.execute_reply":"2022-01-13T12:05:50.863713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q6'></a>\n### 3.6 Minimum, maximum, and average play result","metadata":{}},{"cell_type":"code","source":"plays['playResult'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.86598Z","iopub.execute_input":"2022-01-13T12:05:50.866225Z","iopub.status.idle":"2022-01-13T12:05:50.877295Z","shell.execute_reply.started":"2022-01-13T12:05:50.866199Z","shell.execute_reply":"2022-01-13T12:05:50.876586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.histogram(plays['playResult'], width=1000, height=500)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.878494Z","iopub.execute_input":"2022-01-13T12:05:50.87917Z","iopub.status.idle":"2022-01-13T12:05:50.996954Z","shell.execute_reply.started":"2022-01-13T12:05:50.879121Z","shell.execute_reply":"2022-01-13T12:05:50.996234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12,7))\nsns.violinplot(plays['playResult']);","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:50.998134Z","iopub.execute_input":"2022-01-13T12:05:50.998719Z","iopub.status.idle":"2022-01-13T12:05:51.289148Z","shell.execute_reply.started":"2022-01-13T12:05:50.998681Z","shell.execute_reply":"2022-01-13T12:05:51.288381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"    - minimum result is -72\n    - maximum result is 82\n    - average result is 27\n<br>We can see that the most frequent results are **0**, and **40**","metadata":{}},{"cell_type":"markdown","source":"<a id='q7'></a>\n### 3.7 Top 10 frequent possession teams","metadata":{}},{"cell_type":"code","source":"top_10 = plays['possessionTeam'].value_counts()[:10]\ntop_10","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:51.290508Z","iopub.execute_input":"2022-01-13T12:05:51.290875Z","iopub.status.idle":"2022-01-13T12:05:51.299334Z","shell.execute_reply.started":"2022-01-13T12:05:51.29083Z","shell.execute_reply":"2022-01-13T12:05:51.298512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(top_10, width=1000, height=500, orientation='h', title='Number of plays each possessionTeam', labels={'index':'possessionTeam', 'value':'count'})","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:51.300757Z","iopub.execute_input":"2022-01-13T12:05:51.30106Z","iopub.status.idle":"2022-01-13T12:05:51.360805Z","shell.execute_reply.started":"2022-01-13T12:05:51.301021Z","shell.execute_reply":"2022-01-13T12:05:51.360032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q8'></a>\n### 3.8 Most frequent specialTeamsPlayType","metadata":{}},{"cell_type":"code","source":"t = plays['specialTeamsPlayType'].value_counts()\nt","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:51.361996Z","iopub.execute_input":"2022-01-13T12:05:51.362853Z","iopub.status.idle":"2022-01-13T12:05:51.371297Z","shell.execute_reply.started":"2022-01-13T12:05:51.362811Z","shell.execute_reply":"2022-01-13T12:05:51.370714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.pie(names=t.index, values=t, title='Special teams play type')","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:51.372258Z","iopub.execute_input":"2022-01-13T12:05:51.372478Z","iopub.status.idle":"2022-01-13T12:05:51.42977Z","shell.execute_reply.started":"2022-01-13T12:05:51.372431Z","shell.execute_reply":"2022-01-13T12:05:51.429026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**The most frequent specialTeamsPlayType is Kickoff**","metadata":{}},{"cell_type":"code","source":"players.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:51.430811Z","iopub.execute_input":"2022-01-13T12:05:51.431268Z","iopub.status.idle":"2022-01-13T12:05:51.442798Z","shell.execute_reply.started":"2022-01-13T12:05:51.431239Z","shell.execute_reply":"2022-01-13T12:05:51.44212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q9'></a>\n### 3.9 What is the relation between `Position`, `Height`","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\nsns.lineplot(data=players, x='Position', y='height');","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:51.443645Z","iopub.execute_input":"2022-01-13T12:05:51.444278Z","iopub.status.idle":"2022-01-13T12:05:52.222047Z","shell.execute_reply.started":"2022-01-13T12:05:51.444246Z","shell.execute_reply":"2022-01-13T12:05:52.221196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\nsns.lineplot(data=players, x='Position', y='weight');","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:52.223148Z","iopub.execute_input":"2022-01-13T12:05:52.223349Z","iopub.status.idle":"2022-01-13T12:05:52.992184Z","shell.execute_reply.started":"2022-01-13T12:05:52.223324Z","shell.execute_reply":"2022-01-13T12:05:52.991613Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q10'></a>\n### 3.10 What college graduates most special teams players?","metadata":{}},{"cell_type":"code","source":"c = players['collegeName'].value_counts()[:10]\n\nplt.figure(figsize=(12,7))\nplt.barh(c.index, c);","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:52.993372Z","iopub.execute_input":"2022-01-13T12:05:52.993774Z","iopub.status.idle":"2022-01-13T12:05:53.225455Z","shell.execute_reply.started":"2022-01-13T12:05:52.993743Z","shell.execute_reply":"2022-01-13T12:05:53.22469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q11'></a>\n### 3.10 Who is the tallest and shortest players?","metadata":{}},{"cell_type":"code","source":"players['height'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.226518Z","iopub.execute_input":"2022-01-13T12:05:53.226732Z","iopub.status.idle":"2022-01-13T12:05:53.236301Z","shell.execute_reply.started":"2022-01-13T12:05:53.226706Z","shell.execute_reply":"2022-01-13T12:05:53.23541Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players.query('height==66')","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.238075Z","iopub.execute_input":"2022-01-13T12:05:53.238709Z","iopub.status.idle":"2022-01-13T12:05:53.257106Z","shell.execute_reply.started":"2022-01-13T12:05:53.238666Z","shell.execute_reply":"2022-01-13T12:05:53.256336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players.query('height==81')","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.258402Z","iopub.execute_input":"2022-01-13T12:05:53.25865Z","iopub.status.idle":"2022-01-13T12:05:53.274239Z","shell.execute_reply.started":"2022-01-13T12:05:53.258623Z","shell.execute_reply":"2022-01-13T12:05:53.273357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q11'></a>\n### 3.11 Which season has most plays?","metadata":{}},{"cell_type":"code","source":"print(\"Unique NFL seasons and their counts :\")\ng_season = games.pivot_table(index = ['season'], aggfunc = 'size') \ng_season = g_season.reset_index()\ng_season.columns= [\"Seasons\", \"Counts\"]\ng_season = g_season.sort_values(\"Counts\", ascending = False)\nprint(g_season)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.275298Z","iopub.execute_input":"2022-01-13T12:05:53.275526Z","iopub.status.idle":"2022-01-13T12:05:53.285464Z","shell.execute_reply.started":"2022-01-13T12:05:53.275498Z","shell.execute_reply":"2022-01-13T12:05:53.284886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(g_season, x=\"Seasons\", y=\"Counts\", title=\"NFL Seasons\", color=\"Seasons\")","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.286588Z","iopub.execute_input":"2022-01-13T12:05:53.287073Z","iopub.status.idle":"2022-01-13T12:05:53.3565Z","shell.execute_reply.started":"2022-01-13T12:05:53.287039Z","shell.execute_reply":"2022-01-13T12:05:53.355935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q12'></a>\n### 3.12 Number of matches by week","metadata":{}},{"cell_type":"code","source":"print(\"Unique NFL weeks and their counts :\")\ng_week = games.pivot_table(index = ['week'], aggfunc = 'size') \ng_week = g_week.reset_index()\ng_week.columns= [\"Weeks\", \"Counts\"]\ng_week = g_week.sort_values(\"Counts\", ascending = False)\nprint(g_week)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.357523Z","iopub.execute_input":"2022-01-13T12:05:53.357884Z","iopub.status.idle":"2022-01-13T12:05:53.367115Z","shell.execute_reply.started":"2022-01-13T12:05:53.35782Z","shell.execute_reply":"2022-01-13T12:05:53.366614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(g_week, x=\"Weeks\", y=\"Counts\", title=\"NFL Weeks\", color=\"Weeks\")","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.368057Z","iopub.execute_input":"2022-01-13T12:05:53.368538Z","iopub.status.idle":"2022-01-13T12:05:53.427389Z","shell.execute_reply.started":"2022-01-13T12:05:53.368508Z","shell.execute_reply":"2022-01-13T12:05:53.426459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp=games['week']\nMatches_on_weekly_basis=pd.DataFrame()\nMatches_on_weekly_basis['week']=temp.value_counts().index\nMatches_on_weekly_basis['Count']=temp.value_counts().values\nMatches_on_weekly_basis=Matches_on_weekly_basis.sort_values(by='week')\nfig = px.line(Matches_on_weekly_basis, x=\"week\", y=\"Count\", title='Matches on Weekly Basis')\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.428623Z","iopub.execute_input":"2022-01-13T12:05:53.429025Z","iopub.status.idle":"2022-01-13T12:05:53.49885Z","shell.execute_reply.started":"2022-01-13T12:05:53.428992Z","shell.execute_reply":"2022-01-13T12:05:53.498049Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q13'></a>\n### 3.13 Number of games for every date","metadata":{}},{"cell_type":"code","source":"print(\"Unique NFL dates and their counts :\")\ng_date = games.pivot_table(index = ['gameDate'], aggfunc = 'size') \ng_date = g_date.reset_index()\ng_date.columns= [\"Date\", \"Counts\"]\ng_date = g_date.sort_values(\"Counts\", ascending = False)\nprint(g_date)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.500054Z","iopub.execute_input":"2022-01-13T12:05:53.500279Z","iopub.status.idle":"2022-01-13T12:05:53.511482Z","shell.execute_reply.started":"2022-01-13T12:05:53.500251Z","shell.execute_reply":"2022-01-13T12:05:53.510653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.bar(g_date, x=\"Date\", y=\"Counts\", title='Number of games for every date', color=\"Counts\")\nplt.figure(figsize=(9,7))\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.512757Z","iopub.execute_input":"2022-01-13T12:05:53.513113Z","iopub.status.idle":"2022-01-13T12:05:53.57553Z","shell.execute_reply.started":"2022-01-13T12:05:53.513072Z","shell.execute_reply":"2022-01-13T12:05:53.574658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = games['gameDate'].value_counts().reset_index()\ntemp.columns = ['date', 'games']\ntemp = temp.sort_values('games')\n\nfig = px.line(temp, x='date',y=\"games\",  title='Number of games for every date')\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.577178Z","iopub.execute_input":"2022-01-13T12:05:53.5775Z","iopub.status.idle":"2022-01-13T12:05:53.633907Z","shell.execute_reply.started":"2022-01-13T12:05:53.57746Z","shell.execute_reply":"2022-01-13T12:05:53.63316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q14'></a>\n### 3.14 Number of games based on Timing\n","metadata":{}},{"cell_type":"code","source":"print(\"Unique NFL timings and their counts :\")\ng_time = games.pivot_table(index = ['gameTimeEastern'], aggfunc = 'size') \ng_time = g_time.reset_index()\ng_time.columns= [\"Time\", \"Counts\"]\ng_time = g_time.sort_values(\"Counts\", ascending = False)\nprint(g_time)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.63489Z","iopub.execute_input":"2022-01-13T12:05:53.635085Z","iopub.status.idle":"2022-01-13T12:05:53.644726Z","shell.execute_reply.started":"2022-01-13T12:05:53.635062Z","shell.execute_reply":"2022-01-13T12:05:53.644219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(g_time, x=\"Time\", y=\"Counts\", title=\"Number of games based on Timing\", color=\"Counts\")","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.645533Z","iopub.execute_input":"2022-01-13T12:05:53.646097Z","iopub.status.idle":"2022-01-13T12:05:53.703768Z","shell.execute_reply.started":"2022-01-13T12:05:53.646065Z","shell.execute_reply":"2022-01-13T12:05:53.70289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp = games['gameTimeEastern'].value_counts().reset_index()\ntemp.columns = ['time', 'games']\ntemp = temp.sort_values('games')\nplt.figure(figsize=(7,5))\nfig = px.line(temp, x='time',y=\"games\",  title='Number of games based on Timing')\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.704879Z","iopub.execute_input":"2022-01-13T12:05:53.70516Z","iopub.status.idle":"2022-01-13T12:05:53.762691Z","shell.execute_reply.started":"2022-01-13T12:05:53.70513Z","shell.execute_reply":"2022-01-13T12:05:53.762117Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='q15'></a>\n### 3.15 Number of games through years\n","metadata":{}},{"cell_type":"code","source":"games['gameYear'] = pd.DatetimeIndex(games['gameDate']).year\nprint(games[\"gameYear\"])","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.763751Z","iopub.execute_input":"2022-01-13T12:05:53.763963Z","iopub.status.idle":"2022-01-13T12:05:53.772157Z","shell.execute_reply.started":"2022-01-13T12:05:53.763938Z","shell.execute_reply":"2022-01-13T12:05:53.771253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"number of games through years :\")\ng_year = games.pivot_table(index = ['gameYear'], aggfunc = 'size') \ng_year = g_year.reset_index()\ng_year.columns= [\"Year\", \"Counts\"]\ng_year = g_year.sort_values(\"Counts\", ascending = False)\nprint(g_year)","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.773234Z","iopub.execute_input":"2022-01-13T12:05:53.773432Z","iopub.status.idle":"2022-01-13T12:05:53.786229Z","shell.execute_reply.started":"2022-01-13T12:05:53.773409Z","shell.execute_reply":"2022-01-13T12:05:53.785484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(g_year, x=\"Year\", y=\"Counts\", title=\"number of games through years \", color=\"Year\")","metadata":{"execution":{"iopub.status.busy":"2022-01-13T12:05:53.787581Z","iopub.execute_input":"2022-01-13T12:05:53.788005Z","iopub.status.idle":"2022-01-13T12:05:53.846178Z","shell.execute_reply.started":"2022-01-13T12:05:53.787967Z","shell.execute_reply":"2022-01-13T12:05:53.845651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='conclusions'></a>\n## 4. Conclusions","metadata":{}},{"cell_type":"markdown","source":"1. The most frequent kick type is the standerd, followed by the Aussie.\n2. There are 191 cases where the player misses the inteded direction,90 times the intended direction was L (left), but the actual was C (center).\n3. The hangTime and OperationTime are normally distributed, but there is no relation bewteen them.\n4. Most playes happen in the second quarter, while only few are in the 5th quarter.\n5. Most plays heppen in down 0, folowed by 4, and only few happpen in down 1,2,3.\n6. the minimum result was -72, maximum was 82, average was 27 and we can see that the most frequent results are **0**, and **40**.\n7. The most frequent teams are `BAL` and `NO`.\n8. Most frequent specialTeamsPlayType is kickoff.\n9. `T` and `OT` tends to be the tallest players.\n10. Alabama graduates most special teams players.\n11. Number of plays in each season is almost the same.\n12. Number of plays per weeks varu from 41 to 48.\n13. Most number of matches happend on 01/03/2021, and least happened on 11/04/2019.\n14. Most plays starts at 13:00.\n15. 2020 has the most number of plays.","metadata":{}}]}