{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# NFL Big Data Bowl 2022\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Problem Statement:**\n\n* Before National Football League (NFL) coaches celebrate a big W, they strategize ways to improve field position and score points. \n* Both of these objectives receive significant contributions from special teams plays, which consist of punts, kickoffs, field goals and extra points. \n* These play types take on important roles in a game’s final score—so much so that coaches say they're a third of the game. \n* Yet special teams remain an understudied part of American football, with an opportunity for data science to offer better ways to understand its impact.\n\n**Note:** In this competition, we need to quantify what happens on special teams plays. We might have to create a new special teams metric, quantify team or individual strategies, rank players, or even something we haven’t considered.\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Dataset Involves:**\n\nThe 2022 Big Data Bowl data contains Next Gen Stats player tracking, play, game, player, and PFF scouting data for all 2018-2020 Special Teams plays. \n\n* **Game data:** The games.csv contains the teams playing in each game. The key variable is **gameId**.\n\n* **Play data:** The plays.csv file contains play-level information from each game. The key variables are **gameId** and **playId**.\n\n* **Player data:** The players.csv file contains player-level information from players that participated in any of the tracking data files. The key variable is **nflId**.\n\n* **Tracking data:** Files tracking[season].csv contain player tracking data from season [season]. The key variables are **gameId**, **playId**, and **nflId**.\n\n* **PFF Scouting data:** The PFFScoutingData.csv file contains play-level scouting information for each game. The key variables are **gameId** and **playId**.\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Importing Libraries:**\n\n* To get started we will use Python for data pre-processing and data analysis.\n\n* Import python libraries as necessary to get started for data load and later import other libraries as needed\n\n---\n","metadata":{}},{"cell_type":"code","source":"# Importing package numpys (For Numerical Python)\nimport numpy as np \n# Importing for data analysis\nimport pandas as pd \n# module finds all the pathnames matching a specified pattern\nimport glob \n# module provides a portable way of using operating system dependent functionality\nimport os \n # Importing pyplot interface using matplotlib\nimport matplotlib.pyplot as plt \n# Importing seaborn library for interactive visualization\nimport seaborn as sns \n# Importing WordCloud for text data visualization\nfrom wordcloud import WordCloud\n# Importing matplotlib for plots\nimport matplotlib\n#Importing datetime for using datetime\nfrom datetime import datetime","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:07.164904Z","iopub.execute_input":"2021-11-12T12:46:07.165562Z","iopub.status.idle":"2021-11-12T12:46:08.180929Z","shell.execute_reply.started":"2021-11-12T12:46:07.165443Z","shell.execute_reply":"2021-11-12T12:46:08.179984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Definition/Description of Game data\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Data Definition:**\n* **Game data:** The games.csv contains the teams playing in each game. The key variable is **gameId**.","metadata":{}},{"cell_type":"code","source":"# Loading dataset games.csv\ngame_data = pd.read_csv('../input/nfl-big-data-bowl-2022/games.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.182433Z","iopub.execute_input":"2021-11-12T12:46:08.182664Z","iopub.status.idle":"2021-11-12T12:46:08.200748Z","shell.execute_reply.started":"2021-11-12T12:46:08.182637Z","shell.execute_reply":"2021-11-12T12:46:08.199992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the structure of game dataset?**\n\n---","metadata":{}},{"cell_type":"markdown","source":"| No. | Feature Name | Description of the feature |\n| :-- | :--| :--| \n|01| **gameId**   | Game identifier, unique (numeric) |\n|02| **season** | Season of game                 |\n|03| **week**   | Week of game                 |\n|04| **gameDate**   | Game Date (time, mm/dd/yyyy)  |\n|05| **gameTimeEastern**   | Start time of game (time, HH:MM:SS, EST)|\n|06| **homeTeamAbbr**   | Home team three-letter code (text)  |\n|07| **visitorTeamAbbr**   | Visiting team three-letter code (text)|","metadata":{}},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of game dataset is:', game_data.shape)\n\n# print summary of dataframe\ngame_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.201954Z","iopub.execute_input":"2021-11-12T12:46:08.202856Z","iopub.status.idle":"2021-11-12T12:46:08.228345Z","shell.execute_reply.started":"2021-11-12T12:46:08.202812Z","shell.execute_reply":"2021-11-12T12:46:08.227494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**game dataset information:**\n\n* There are 764 data points (rows) and 7 feature (column) in game dataset.\n* There are three numerical column and four columns are of categorical or object type.\n* There are no missing values (non-null count is same) for all seven columns.\n","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for game dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\ngame_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.230725Z","iopub.execute_input":"2021-11-12T12:46:08.232859Z","iopub.status.idle":"2021-11-12T12:46:08.252033Z","shell.execute_reply.started":"2021-11-12T12:46:08.232813Z","shell.execute_reply":"2021-11-12T12:46:08.251161Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for game dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for both object and numeric type\ngame_data.describe(include='all').round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.253740Z","iopub.execute_input":"2021-11-12T12:46:08.254299Z","iopub.status.idle":"2021-11-12T12:46:08.290712Z","shell.execute_reply.started":"2021-11-12T12:46:08.254256Z","shell.execute_reply":"2021-11-12T12:46:08.289730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**game dataset data description:**\n\n* There are no missing value for **gameId** as total count is 764.\n* Mean value is same as median value for **season** and **week** which appears that distribution is normal.\n* There are 33 unique **homeTeamAbbr** for which data points are available.\n * PHI **homeTeamAbbr** has the highest frequency of occurrence.\n* There are 33 unique **visitorTeamAbbr** for which data points are available.\n * ATL **visitorTeamAbbr** has the highest frequency of occurrence.\n* There are 16 unique **gameTimeEastern** for which data points are available.\n * 13:00:00 **gameTimeEastern** has the highest frequency of occurrence.\n* There are 151 unique **gameDate** for which data points are available.\n * 01/03/2021 **gameDate** has the highest frequency of occurrence.\n\n---","metadata":{}},{"cell_type":"markdown","source":"# Data Analysis/EDA of Game data\n\n---","metadata":{}},{"cell_type":"code","source":"# convert gameDate from object type to datetime type\ngame_data['gameDate']= pd.to_datetime(game_data['gameDate'])\n\n# group by date without index\ngame_data_by_date = game_data.groupby(by = 'gameDate', as_index = False).agg('max')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.291936Z","iopub.execute_input":"2021-11-12T12:46:08.292151Z","iopub.status.idle":"2021-11-12T12:46:08.355491Z","shell.execute_reply.started":"2021-11-12T12:46:08.292125Z","shell.execute_reply":"2021-11-12T12:46:08.354700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend for game dataset with respect to Game Date for Home Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# trend for game dataset with respect to homeTeamAbbr\nfigure = plt.figure(figsize = [15, 7])\nsns.lineplot(x = 'gameDate', y = 'homeTeamAbbr', hue= 'season', data = game_data_by_date, color = '#D96552')\n\nplt.xlabel('Game Date', size = 14)\nplt.ylabel('Home Team', size = 14)\nplt.title('Game Trend', size = 16)\nplt.grid(b = True, axis = 'y')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.356740Z","iopub.execute_input":"2021-11-12T12:46:08.357589Z","iopub.status.idle":"2021-11-12T12:46:08.851937Z","shell.execute_reply.started":"2021-11-12T12:46:08.357541Z","shell.execute_reply":"2021-11-12T12:46:08.851295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Yearly Game trend for Home Team shows variations for each season and 2020 season datapoints seems on higher side.\n\nNote: The National Football League (NFL) is a professional American football league consisting of 32 teams, divided equally between the National Football Conference (NFC) and the American Football Conference (AFC).","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for game dataset with respect to Game Date for Visitor Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# trend for game dataset with respect to visitorTeamAbbr\nfigure = plt.figure(figsize = [15, 7])\nsns.lineplot(x = 'gameDate', y = 'visitorTeamAbbr', hue= 'season', data = game_data_by_date, color = '#D96552')\n\nplt.xlabel('Game Date', size = 14)\nplt.ylabel('Visitor Team', size = 14)\nplt.title('Game Trend', size = 16)\nplt.grid(b = True, axis = 'y')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:08.852843Z","iopub.execute_input":"2021-11-12T12:46:08.853351Z","iopub.status.idle":"2021-11-12T12:46:09.306547Z","shell.execute_reply.started":"2021-11-12T12:46:08.853322Z","shell.execute_reply":"2021-11-12T12:46:09.305954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Yearly Game trend for Visitor Team shows variations for each season and 2020 season datapoints seems on higher side.\n\nNote: The National Football League (NFL) is a professional American football league consisting of 32 teams, divided equally between the National Football Conference (NFC) and the American Football Conference (AFC).","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for game dataset with respect to Game Week for Home Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# trend for game dataset with respect to homeTeamAbbr\nfigure = plt.figure(figsize = [15, 7])\nsns.lineplot(x = 'week', y = 'homeTeamAbbr', hue= 'season', data = game_data_by_date, color = '#D96552')\n\nplt.xlabel('Game Week', size = 14)\nplt.ylabel('Home Team', size = 14)\nplt.title('Game Trend', size = 16)\nplt.grid(b = True, axis = 'y')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:09.307589Z","iopub.execute_input":"2021-11-12T12:46:09.307927Z","iopub.status.idle":"2021-11-12T12:46:11.170422Z","shell.execute_reply.started":"2021-11-12T12:46:09.307899Z","shell.execute_reply":"2021-11-12T12:46:11.169642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Weekly Game trend for Home Team shows variations for each season and variation for 2020 season datapoints seems on higher side.\n\nNote: The NFL season format consists of a three-week preseason, an 18-week regular season (each team plays 17 games),and a 14-team single-elimination playoff culminating in the Super Bowl, the league's championship game.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for game dataset with respect to Game Week for Visitor Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# trend for game dataset with respect to visitorTeamAbbr\nfigure = plt.figure(figsize = [15, 7])\nsns.lineplot(x = 'week', y = 'visitorTeamAbbr', hue= 'season', data = game_data_by_date, color = '#D96552')\n\nplt.xlabel('Game Week', size = 14)\nplt.ylabel('Visitor Team', size = 14)\nplt.title('Game Trend', size = 16)\nplt.grid(b = True, axis = 'y')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:11.173193Z","iopub.execute_input":"2021-11-12T12:46:11.173431Z","iopub.status.idle":"2021-11-12T12:46:12.908109Z","shell.execute_reply.started":"2021-11-12T12:46:11.173403Z","shell.execute_reply":"2021-11-12T12:46:12.907201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Weekly Game trend for Visitor Team shows variations for each season and variation for 2020 season datapoints seems on higher side.\n\nNote: The NFL season format consists of a three-week preseason, an 18-week regular season (each team plays 17 games),and a 14-team single-elimination playoff culminating in the Super Bowl, the league's championship game.","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"---\n# Data Definition/Description of Play data\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Data Definition:**\n* **Play data:** The plays.csv file contains play-level information from each game. The key variables are **gameId** and **playId**.\n","metadata":{}},{"cell_type":"code","source":"# Loading dataset plays.csv\nplay_data = pd.read_csv('../input/nfl-big-data-bowl-2022/plays.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:12.909340Z","iopub.execute_input":"2021-11-12T12:46:12.909567Z","iopub.status.idle":"2021-11-12T12:46:13.035959Z","shell.execute_reply.started":"2021-11-12T12:46:12.909540Z","shell.execute_reply":"2021-11-12T12:46:13.035087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the structure of play dataset?**\n\n---","metadata":{}},{"cell_type":"markdown","source":"| No. | Feature Name | Description of the feature |\n| :-- | :--| :--| \n|01| **gameId**   | Game identifier, unique (numeric) |\n|02| **playId** | Play identifier, not unique across games (numeric) |\n|03| **playDescription**   | Description of play (text)              |\n|04| **quarter**   | Game quarter (numeric)  |\n|05| **down**   | Down (numeric)|\n|06| **yardsToGo**   | Distance needed for a first down (numeric)  |\n|07| **possessionTeam**   | Team punting, placekicking or kicking off the ball (text)|\n|08| **specialTeamsPlayType**   | Formation of play: Extra Point, Field Goal, Kickoff or Punt (text)             |\n|09| **specialTeamsPlayResult**   | Special Teams outcome of play dependent on play type: \n|||Blocked Kick Attempt, Blocked Punt, Downed, Fair Catch, Kick Attempt Good, Kick Attempt No Good, Kickoff Team Recovery, Muffed, Non-Special Teams Result, Out of Bounds, Return or Touchback (text) |\n|10| **kickerId**   | nflId of placekicker, punter or kickoff specialist on play (numeric)|\n|11| **returnerId**   | nflId(s) of returner(s) on play if there was a special teams return. Multiple returners on a play are separated by a ; (text)  |\n|12| **kickBlockerId**   | nflId of blocker of kick on play if there was a blocked field goal or blocked punt (numeric)|\n|13| **yardlineSide**   | 3-letter team code corresponding to line-of-scrimmage (text)              |\n|14| **yardlineNumber**   | Yard line at line-of-scrimmage (numeric)  |\n|15| **gameClock**   | Time on clock of play (MM:SS)|\n|16| **penaltyCodes**   |  NFL categorization of the penalties that occurred on the play. Multiple penalties on a play are separated by a ; (text)  |\n|17| **penaltyJerseyNumber**   | Jersey number and team code of the player committing each penalty. Multiple penalties on a play are separated by a ; (text)|\n|18| **penaltyYards**   | yards gained by possessionTeam by penalty (numeric)|\n|19| **preSnapHomeScore**   | Home score prior to the play (numeric)  |\n|20| **preSnapVisitorScore**   | Visiting team score prior to the play (numeric)|\n|21| **passResult**   | Scrimmage outcome of the play if specialTeamsPlayResult is \"Non-Special Teams Result\" \n|||(C: Complete pass, I: Incomplete pass, S: Quarterback sack, IN: Intercepted pass, R: Scramble, ' ': Designed Rush, text)  |\n|22| **kickLength**   | Kick length in air of kickoff, field goal or punt (numeric)|\n|23| **kickReturnYardage**   | Yards gained by return team if there was a return on a kickoff or punt (numeric)|\n|24| **playResult**   | Net yards gained by the kicking team, including penalty yardage (numeric)|\n|25| **absoluteYardlineNumber**   | Location of ball downfield in tracking data coordinates (numeric)|","metadata":{}},{"cell_type":"markdown","source":"\n","metadata":{}},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of play dataset is:', play_data.shape)\n\n# print summary of dataframe\nplay_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:13.037388Z","iopub.execute_input":"2021-11-12T12:46:13.037921Z","iopub.status.idle":"2021-11-12T12:46:13.071971Z","shell.execute_reply.started":"2021-11-12T12:46:13.037886Z","shell.execute_reply":"2021-11-12T12:46:13.070906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**play dataset information:**\n\n* There are 19979 data points (rows) and 25 feature (column) in play dataset.\n* There are fifteen numerical column and ten columns are of categorical or object type.\n* There are missing values (non-null count is not same as 19979) for columns\n  * kickerId, returnerId, kickBlockerId,\n  * yardlineside, penaltyCodes, penaltyJerseyNumbers, penaltyYards,\n  * passResult, kickLength and kickReturnYardage.\n","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for play dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\nplay_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:13.073546Z","iopub.execute_input":"2021-11-12T12:46:13.074093Z","iopub.status.idle":"2021-11-12T12:46:13.106282Z","shell.execute_reply.started":"2021-11-12T12:46:13.074058Z","shell.execute_reply":"2021-11-12T12:46:13.105379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for play dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for numeric type\nplay_data.describe().round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:13.107833Z","iopub.execute_input":"2021-11-12T12:46:13.108140Z","iopub.status.idle":"2021-11-12T12:46:13.179928Z","shell.execute_reply.started":"2021-11-12T12:46:13.108106Z","shell.execute_reply":"2021-11-12T12:46:13.179062Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**plays dataset data description:**\n\n*  There are no missing value for **gameId** and **playId** as they have total count of 19979 data points.\n*  Mean value for **quarter** is closer to median value which appears that distribution is normal.\n*  Mean value for **down** is higher than median value which appears that distribution is positively skewed.\n*  Mean value for **yardsToGo** is higher than median value which appears that distribution is positively skewed.\n * **yardsToGo** distribution does appear to have outliers.\n* **kickerId** has missing value as total count is not 19979\n* **kickBlockerId** has many missing values as total count is showing as 100\n*  Mean value for **yardlineNumber** is less than median value which appears that distribution is negatively skewed.\n*  Mean value for **penaltyYards** is less than median value which appears that distribution is negatively skewed.\n*  Mean value for **preSnapHomeScore** is higher than median value which appears that distribution is positively skewed.\n*  Mean value for **preSnapVisitorScore** is higher than median value which appears that distribution is positively skewed.\n*  Mean value for **kickLength** is closer to median value which appears that distribution is normal.\n * **kickLength** has missing values as total count is not 19979\n*  Mean value for **kickReturnYardage** is closer to median value which appears that distribution is normal.\n * **kickReturnYardage** has missing values as total count is not 19979\n*  Mean value for **absoluteYardlineNumber** is closer to median value which appears that distribution is normal.","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for object type\nplay_data.describe(include=[object])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:13.181051Z","iopub.execute_input":"2021-11-12T12:46:13.181492Z","iopub.status.idle":"2021-11-12T12:46:13.259059Z","shell.execute_reply.started":"2021-11-12T12:46:13.181462Z","shell.execute_reply":"2021-11-12T12:46:13.258033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**play dataset data description:**\n\n* There are 12355 unique **playDescription** for which data points are available.\n * W.Lutz kicks 65 yards from NO 35 to end zone **playDescription** has the highest frequency of occurrence.\n* There are 33 unique **possessionTeam** for which data points are available.\n * BAL **possessionTeam** has the highest frequency of occurrence.\n* There are 4 unique **specialTeamsPlayType** for which data points are available.\n * Kickoff **specialTeamsPlayType** has the highest frequency of occurrence.\n* There are 12 unique **specialTeamsResult** for which data points are available.\n * Kick Attempt Good **specialTeamsResult** has the highest frequency of occurrence.\n* **returnerId** has missing value as total count is not 19979\n* There are 33 unique **yardlineSide** for which data points are available.\n * NYJ **yardlineSide** has the highest frequency of occurrence.\n *  **yardlineSide** has missing value as total count is not 19979\n* There are 900 unique **gameClock** for which data points are available.\n * 15:00 **gameClock** has the highest frequency of occurrence.\n* There are 71 unique **penaltyCodes** for which data points are available.\n * OH **penaltyCodes** has the highest frequency of occurrence.\n * **penaltyCodes** has missing values as total count is not 19979\n* There are 700 unique **penaltyJerseyNumbers** for which data points are available.\n * BAL 41 **penaltyJerseyNumbers** has the highest frequency of occurrence.\n * **penaltyJerseyNumbers** has missing values as total count is not 19979\n* There are 4 unique **passResult** for which data points are available.\n * C **passResult** has the highest frequency of occurrence.\n * **passResult** has missing values as total count is not 19979\n---","metadata":{}},{"cell_type":"markdown","source":"---\n# Data Analysis/EDA of Play data\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Possession Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of possessionTeam\nplay_data['possessionTeam'].value_counts().plot(kind='pie', explode=[0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(20,15), shadow=True, startangle=135, legend=False, cmap='rainbow')\nplt.suptitle(t = 'Percentage distribution count for Possession Team', y = 1.05, size = 30)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = play_data['possessionTeam'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:13.260185Z","iopub.execute_input":"2021-11-12T12:46:13.260399Z","iopub.status.idle":"2021-11-12T12:46:14.224369Z","shell.execute_reply.started":"2021-11-12T12:46:13.260373Z","shell.execute_reply":"2021-11-12T12:46:14.219837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows almost equal distribution for possession team - Team punting, placekicking or kicking off the ball.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution of Penalty Yards with respect to Possession Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of penaltyYards with respect to possessionTeam\nsns.displot(data=play_data, x=\"penaltyYards\", hue='possessionTeam', height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:14.225684Z","iopub.execute_input":"2021-11-12T12:46:14.226006Z","iopub.status.idle":"2021-11-12T12:46:16.754296Z","shell.execute_reply.started":"2021-11-12T12:46:14.225961Z","shell.execute_reply":"2021-11-12T12:46:16.753489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows majority of possession team having gained yards with 10 penalty yards.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of specialTeamsPlayType\nplay_data['specialTeamsPlayType'].value_counts().plot(kind='pie', explode=[0.1,0.1,0.1,0.1], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(10,5), shadow=True, startangle=135, legend=False, cmap='winter')\nplt.suptitle(t = 'Percentage distribution count for Special Teams Play Type', y = 1.05, size = 20)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = play_data['specialTeamsPlayType'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:16.755583Z","iopub.execute_input":"2021-11-12T12:46:16.755821Z","iopub.status.idle":"2021-11-12T12:46:16.939832Z","shell.execute_reply.started":"2021-11-12T12:46:16.755793Z","shell.execute_reply":"2021-11-12T12:46:16.939005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows that Kickoff and Punt forms majority formation of play for Special Teams Play Type in this datapoint.\n\nNote: Special teams is the team that takes care of Kickoff, Punt and Field Goal attempts.\n\n* A Kickoff is a type of free kick where the ball is placed on a tee (or held) at the kicking team's 35-yard line.\n\n* A Punt is type of kick by dropping the ball from the hands and kicking the ball before it reaches the ground.\n\n* An Extra Point called the Point After Touchdown (PAT) conversion or the offense magnificently kicks the ball through the goal post to earn one point. Two extra points can also be scored by running or throwing the ball into the end zone similar to a touchdown.\n\n* A Field Goal can be recorded by the team if the ball is place kicked, drop kicked or free kicked in between the goal posts in the opponent’s end zone.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Kick Length with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickLength with respect to specialTeamsPlayType\nsns.displot(data=play_data, x='kickLength', hue='specialTeamsPlayType',height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:16.941216Z","iopub.execute_input":"2021-11-12T12:46:16.941583Z","iopub.status.idle":"2021-11-12T12:46:17.918668Z","shell.execute_reply.started":"2021-11-12T12:46:16.941543Z","shell.execute_reply":"2021-11-12T12:46:17.917986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Kick Return Yardage with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickReturnYardage with respect to specialTeamsPlayType\nsns.displot(data=play_data, x='kickReturnYardage', hue='specialTeamsPlayType',height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:17.919725Z","iopub.execute_input":"2021-11-12T12:46:17.920337Z","iopub.status.idle":"2021-11-12T12:46:19.191549Z","shell.execute_reply.started":"2021-11-12T12:46:17.920303Z","shell.execute_reply":"2021-11-12T12:46:19.190724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Play Result with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of playResult with respect to specialTeamsPlayType\nsns.displot(data=play_data, x='playResult', hue='specialTeamsPlayType',height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:19.193117Z","iopub.execute_input":"2021-11-12T12:46:19.193541Z","iopub.status.idle":"2021-11-12T12:46:20.457918Z","shell.execute_reply.started":"2021-11-12T12:46:19.193497Z","shell.execute_reply":"2021-11-12T12:46:20.456865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Pass Result?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of passResult\nplay_data['passResult'].value_counts().plot(kind='pie', explode=[0.1,0.1,0.1,0.1], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(10,5), shadow=True, startangle=135, legend=False, cmap='autumn')\nplt.suptitle(t = 'Percentage distribution count for Pass Result', y = 1.05, size = 20)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = play_data['passResult'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:20.459670Z","iopub.execute_input":"2021-11-12T12:46:20.459975Z","iopub.status.idle":"2021-11-12T12:46:20.639550Z","shell.execute_reply.started":"2021-11-12T12:46:20.459936Z","shell.execute_reply":"2021-11-12T12:46:20.638702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows that C- Complete Pass and I - Incomplete Pass are majority scrimmage outcome of play for Pass Result for Non-Special Teams Result.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Yard Line Number with respect to Pass Result?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of yardlineNumber with respect to passResult\nsns.displot(data=play_data, x='yardlineNumber', hue='passResult',kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:20.641326Z","iopub.execute_input":"2021-11-12T12:46:20.641694Z","iopub.status.idle":"2021-11-12T12:46:21.143190Z","shell.execute_reply.started":"2021-11-12T12:46:20.641645Z","shell.execute_reply":"2021-11-12T12:46:21.142355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I: Incomplete pass, C: Complete pass, IN: Intercepted pass, S: Quarterback sack\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Pre Snap Home Score with respect to Pass Result?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of preSnapHomeScore with respect to passResult\nsns.displot(data=play_data, x='preSnapHomeScore', hue='passResult',kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:21.144368Z","iopub.execute_input":"2021-11-12T12:46:21.144577Z","iopub.status.idle":"2021-11-12T12:46:21.638026Z","shell.execute_reply.started":"2021-11-12T12:46:21.144551Z","shell.execute_reply":"2021-11-12T12:46:21.637183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I: Incomplete pass, C: Complete pass, IN: Intercepted pass, S: Quarterback sack","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Pre Snap Visitor Score with respect to Pass Result?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of preSnapVisitorScore with respect to passResult\nsns.displot(data=play_data, x='preSnapVisitorScore', hue='passResult',kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:21.639213Z","iopub.execute_input":"2021-11-12T12:46:21.639435Z","iopub.status.idle":"2021-11-12T12:46:22.148489Z","shell.execute_reply.started":"2021-11-12T12:46:21.639409Z","shell.execute_reply":"2021-11-12T12:46:22.147522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I: Incomplete pass, C: Complete pass, IN: Intercepted pass, S: Quarterback sack","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Special Teams Result?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of specialTeamsResult\nplay_data['specialTeamsResult'].value_counts().plot(kind='pie', explode=[0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.3,0.4,0.5,0.6], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(20,15), shadow=True, startangle=135, legend=False, cmap='rainbow')\nplt.suptitle(t = 'Percentage distribution count for Special Teams Result', y = 1.05, size = 30)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = play_data['specialTeamsResult'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:22.149585Z","iopub.execute_input":"2021-11-12T12:46:22.149812Z","iopub.status.idle":"2021-11-12T12:46:22.603125Z","shell.execute_reply.started":"2021-11-12T12:46:22.149787Z","shell.execute_reply":"2021-11-12T12:46:22.602256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows that Kick Attempt Good, Return and Touchback are majority outcome of play for Speical Teams Result.","metadata":{}},{"cell_type":"markdown","source":"---\n# Data Definition/Description of Player data\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Data Definition:**\n* **Player data:** The players.csv file contains player-level information from players that participated in any of the tracking data files. The key variable is **nflId**.\n","metadata":{}},{"cell_type":"code","source":"# Loading dataset players.csv\nplayer_data = pd.read_csv('../input/nfl-big-data-bowl-2022/players.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:22.604286Z","iopub.execute_input":"2021-11-12T12:46:22.604505Z","iopub.status.idle":"2021-11-12T12:46:22.625213Z","shell.execute_reply.started":"2021-11-12T12:46:22.604478Z","shell.execute_reply":"2021-11-12T12:46:22.624440Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the structure of player dataset?**\n\n---","metadata":{}},{"cell_type":"markdown","source":"| No. | Feature Name | Description of the feature |\n| :-- | :--| :--| \n|01| **nflId**   | Player identification number, unique across players (numeric)   |\n|02| **Height**   | Player height (text)  |\n|03| **Weight**   | Player weight (numeric)|\n|04| **birthDate**   | Date of birth (YYYY-MM-DD)  |\n|05| **collegeName**   | Player college (text)|\n|06| **Position**   | Player position (text) |\n|07| **displayName**   | Player name (text) |","metadata":{}},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of player dataset is:', player_data.shape)\n\n# print summary of dataframe\nplayer_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:22.630413Z","iopub.execute_input":"2021-11-12T12:46:22.630658Z","iopub.status.idle":"2021-11-12T12:46:22.800342Z","shell.execute_reply.started":"2021-11-12T12:46:22.630630Z","shell.execute_reply":"2021-11-12T12:46:22.799330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**player dataset information:**\n\n* There are 2732 data points (rows) and 7 feature (column) in player dataset.\n* There are two numerical column and five columns are of categorical or object type.\n* There are missing values (non-null count is not same as 1732) for columns\n  * birthDate, collegeName","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for player dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\nplayer_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:22.801746Z","iopub.execute_input":"2021-11-12T12:46:22.802800Z","iopub.status.idle":"2021-11-12T12:46:22.822651Z","shell.execute_reply.started":"2021-11-12T12:46:22.802751Z","shell.execute_reply":"2021-11-12T12:46:22.821507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for player dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for both object and numeric type\nplayer_data.describe(include='all').round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:22.824100Z","iopub.execute_input":"2021-11-12T12:46:22.824977Z","iopub.status.idle":"2021-11-12T12:46:22.862290Z","shell.execute_reply.started":"2021-11-12T12:46:22.824940Z","shell.execute_reply":"2021-11-12T12:46:22.861385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**player dataset data description:**\n\n*  There are no missing value for **nflId** as they have total count of 2732 data points.\n* There are 30 unique **height** for which data points are available.\n * 6-3 **height** has the highest frequency of occurrence.\n*  Mean value for **weight** is closer to median value which appears that distribution is normal.\n* There are 2035 unique **birthDate** for which data points are available.\n * 1997-02-14 **birthDate** has the highest frequency of occurrence.\n * There are missing value for **birthDate**\n* There are 322 unique **collegName** for which data points are available.\n * Alabama **collegeName** has the highest frequency of occurrence.\n * There are missing value for **collegeName**\n* There are 26 unique **Position** for which data points are available.\n * WR **Position** has the highest frequency of occurrence.\n* There are 2718 unique **displayName** for which data points are available.\n * Chris Jones **displayName** has the highest frequency of occurrence.","metadata":{}},{"cell_type":"markdown","source":"# Data Analysis/EDA of Player data\n\n---","metadata":{}},{"cell_type":"code","source":"# convert gameDate from object type to datetime type\nplayer_data['birthDate']= pd.to_datetime(player_data['birthDate'])\n# group by birthdate without index\nplayer_data_by_birthdate = player_data.groupby(by = 'birthDate', as_index = False).agg('max')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:22.863522Z","iopub.execute_input":"2021-11-12T12:46:22.863768Z","iopub.status.idle":"2021-11-12T12:46:23.525998Z","shell.execute_reply.started":"2021-11-12T12:46:22.863741Z","shell.execute_reply":"2021-11-12T12:46:23.525080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the distribution count for Birth Date?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of birthDate\nsns.displot(data=player_data, x='birthDate', kde=True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:23.527079Z","iopub.execute_input":"2021-11-12T12:46:23.527314Z","iopub.status.idle":"2021-11-12T12:46:23.964231Z","shell.execute_reply.started":"2021-11-12T12:46:23.527285Z","shell.execute_reply":"2021-11-12T12:46:23.963610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above distribution for Birth Date shows that majority of player are born around mid-90's.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for player dataset with respect to Birth Date and Height?**\n\n---","metadata":{}},{"cell_type":"code","source":"# trend for player with respect to height\nfigure = plt.figure(figsize = [15, 7])\nsns.lineplot(x = 'birthDate', y = 'height', data = player_data_by_birthdate, color = 'b')\n\nplt.xlabel('Birth Date', size = 14)\nplt.ylabel('Height', size = 14)\nplt.title('Player Trend', size = 16)\nplt.grid(b = True, axis = 'y')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:23.965274Z","iopub.execute_input":"2021-11-12T12:46:23.966033Z","iopub.status.idle":"2021-11-12T12:46:24.385975Z","shell.execute_reply.started":"2021-11-12T12:46:23.965986Z","shell.execute_reply":"2021-11-12T12:46:24.384947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Trend for Birth Date with respect to Height shows that there is good distribution of height across various age group.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for player dataset with respect to Birth Date and Weight?**\n\n---","metadata":{}},{"cell_type":"code","source":"# trend for player with respect to weight\nfigure = plt.figure(figsize = [15, 7])\nsns.lineplot(x = 'birthDate', y = 'weight', data = player_data_by_birthdate, color = 'g')\n\nplt.xlabel('Birth Date', size = 14)\nplt.ylabel('Weight', size = 14)\nplt.title('Player Trend', size = 16)\nplt.grid(b = True, axis = 'y')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:24.387077Z","iopub.execute_input":"2021-11-12T12:46:24.387286Z","iopub.status.idle":"2021-11-12T12:46:24.748025Z","shell.execute_reply.started":"2021-11-12T12:46:24.387260Z","shell.execute_reply":"2021-11-12T12:46:24.747231Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Trend for Birth Date with respect to Weight shows that there is good distribution of weight across various age group.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Position?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of Position\nplayer_data['Position'].value_counts().plot(kind='pie', explode=[0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.2,0.3,0.4,0.5], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(25,20), shadow=True, startangle=135, legend=False, cmap='winter')\nplt.suptitle(t = 'Percentage distribution count for Position', y = 1.05, size = 35)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = player_data['Position'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:24.748867Z","iopub.execute_input":"2021-11-12T12:46:24.749085Z","iopub.status.idle":"2021-11-12T12:46:25.591124Z","shell.execute_reply.started":"2021-11-12T12:46:24.749058Z","shell.execute_reply":"2021-11-12T12:46:25.590343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows WR and CB as majority of player position which exists in this datapoints.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the text distribution view for Display Name?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get unique Display Name\ndisplay_name= player_data['displayName'].unique()\n# convert numpy array to string\ndisplay_name_str = \",\".join(display_name)\n\n# create WordCloud with converted string\nwordcloud = WordCloud(width = 1000, height = 500, random_state=1, background_color='white', collocations=True).generate(display_name_str)\nplt.figure(figsize=(15, 15))\nplt.imshow(wordcloud) \nplt.axis(\"off\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:25.592573Z","iopub.execute_input":"2021-11-12T12:46:25.592881Z","iopub.status.idle":"2021-11-12T12:46:27.262025Z","shell.execute_reply.started":"2021-11-12T12:46:25.592845Z","shell.execute_reply":"2021-11-12T12:46:27.261254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Majority of names starting with J such as Johnson, Jordan, Jones, Justin etc. dominates this datapoints.","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"---\n# Data Definition/Description of Tracking data\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Data Definition:**\n* **Tracking data:** Files tracking[season].csv contain player tracking data from season [season]. The key variables are **gameId**, **playId**, and **nflId**.\n\n","metadata":{}},{"cell_type":"code","source":"# Loading dataset tracking2018.csv\ntracking2018_data = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2018.csv')\n\n# Loading dataset tracking2019.csv\ntracking2019_data = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2019.csv')\n\n# Loading dataset tracking2020.csv\ntracking2020_data = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2020.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:46:27.263326Z","iopub.execute_input":"2021-11-12T12:46:27.263761Z","iopub.status.idle":"2021-11-12T12:48:28.958281Z","shell.execute_reply.started":"2021-11-12T12:46:27.263723Z","shell.execute_reply":"2021-11-12T12:48:28.957557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the structure of tracking dataset?**\n\n---","metadata":{}},{"cell_type":"markdown","source":"| No. | Feature Name | Description of the feature |\n| :-- | :--| :--| \n|01| **time**   | Time stamp of play (time, yyyy-mm-dd, hh:mm:ss) |\n|02| **x** | Player position along the long axis of the field, 0 - 120 yards.(numeric) |\n|03| **y**   | Player position along the short axis of the field, 0 - 53.3 yards.(numeric)  |\n|04| **s**   | Speed in yards/second (numeric)  |\n|05| **a**   | Speed in yards/second^2 (numeric)|\n|06| **dis**   | Distance traveled from prior time point, in yards (numeric) |\n|07| **o**   | Player orientation (deg), 0 - 360 degrees (numeric)|\n|08| **dir**   | Angle of player motion (deg), 0 - 360 degrees (numeric)|\n|09| **event**   | Tagged play details, including moment of ball snap, pass release, pass catch, tackle, etc (text) |\n|10| **nflId**   | Player identification number, unique across players (numeric) |\n|11| **displayName**   | Player name (text) |\n|12| **jerseyNumber**   | Jersey number of player (numeric) |\n|13| **position**   | Player position group (text) |\n|14| **team**   | Team (away or home) of corresponding player (text) |\n|15| **frameId**   | Frame identifier for each play, starting at 1 (numeric) |\n|16| **gameId**   | Game identifier, unique (numeric) |\n|17| **playId**   | Play identifier, not unique across games (numeric) |\n|18| **playDirection**   | Direction that the offense is moving (left or right)|","metadata":{}},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of tracking 2018 season dataset is:', tracking2018_data.shape)\n\n# print summary of dataframe\ntracking2018_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:28.959285Z","iopub.execute_input":"2021-11-12T12:48:28.959628Z","iopub.status.idle":"2021-11-12T12:48:28.972696Z","shell.execute_reply.started":"2021-11-12T12:48:28.959585Z","shell.execute_reply":"2021-11-12T12:48:28.971798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of tracking 2019 season dataset is:', tracking2019_data.shape)\n\n# print summary of dataframe\ntracking2019_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:28.974153Z","iopub.execute_input":"2021-11-12T12:48:28.974583Z","iopub.status.idle":"2021-11-12T12:48:28.987255Z","shell.execute_reply.started":"2021-11-12T12:48:28.974535Z","shell.execute_reply":"2021-11-12T12:48:28.986362Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of tracking 2020 season dataset is:', tracking2020_data.shape)\n\n# print summary of dataframe\ntracking2020_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:28.988715Z","iopub.execute_input":"2021-11-12T12:48:28.989036Z","iopub.status.idle":"2021-11-12T12:48:29.004115Z","shell.execute_reply.started":"2021-11-12T12:48:28.988961Z","shell.execute_reply":"2021-11-12T12:48:29.003018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for tracking 2018 season dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\ntracking2018_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:29.005666Z","iopub.execute_input":"2021-11-12T12:48:29.005991Z","iopub.status.idle":"2021-11-12T12:48:29.037483Z","shell.execute_reply.started":"2021-11-12T12:48:29.005954Z","shell.execute_reply":"2021-11-12T12:48:29.036557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for tracking 2019 season dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\ntracking2019_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:29.038637Z","iopub.execute_input":"2021-11-12T12:48:29.038929Z","iopub.status.idle":"2021-11-12T12:48:29.066772Z","shell.execute_reply.started":"2021-11-12T12:48:29.038900Z","shell.execute_reply":"2021-11-12T12:48:29.066155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for tracking 2020 season dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\ntracking2020_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:29.067912Z","iopub.execute_input":"2021-11-12T12:48:29.068250Z","iopub.status.idle":"2021-11-12T12:48:29.103291Z","shell.execute_reply.started":"2021-11-12T12:48:29.068213Z","shell.execute_reply":"2021-11-12T12:48:29.102704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for tracking 2018 season dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for both object and numeric type\ntracking2018_data.describe(include='all').round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:29.104489Z","iopub.execute_input":"2021-11-12T12:48:29.104740Z","iopub.status.idle":"2021-11-12T12:48:53.094355Z","shell.execute_reply.started":"2021-11-12T12:48:29.104713Z","shell.execute_reply":"2021-11-12T12:48:53.093476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for tracking 2019 season dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for both object and numeric type\ntracking2019_data.describe(include='all').round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:48:53.095489Z","iopub.execute_input":"2021-11-12T12:48:53.095723Z","iopub.status.idle":"2021-11-12T12:49:15.875688Z","shell.execute_reply.started":"2021-11-12T12:48:53.095696Z","shell.execute_reply":"2021-11-12T12:49:15.874529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for tracking 2020 season dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for both object and numeric type\ntracking2020_data.describe(include='all').round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:49:15.877016Z","iopub.execute_input":"2021-11-12T12:49:15.877270Z","iopub.status.idle":"2021-11-12T12:49:38.268474Z","shell.execute_reply.started":"2021-11-12T12:49:15.877239Z","shell.execute_reply":"2021-11-12T12:49:38.267895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** Using sample of tracking2020, tracking 2019 and tracking 2018 season dataset respectively for EDA","metadata":{}},{"cell_type":"code","source":"# create sample of tracking dataset for EDA (For faster execution and to avoid out of memory issue for notebook)\ntracking_data_sample = pd.concat([tracking2020_data.sample(n=100000, random_state=1),tracking2019_data.sample(n=100000, random_state=1),tracking2018_data.sample(n=100000, random_state=1)])","metadata":{"execution":{"iopub.status.busy":"2021-11-12T12:49:38.269659Z","iopub.execute_input":"2021-11-12T12:49:38.270023Z","iopub.status.idle":"2021-11-12T12:49:40.685398Z","shell.execute_reply.started":"2021-11-12T12:49:38.269978Z","shell.execute_reply":"2021-11-12T12:49:40.684598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#reset index after appending all three dataset\ntracking_data_sample.reset_index(inplace=True)\n# get quick summary of new dataset sample\ntracking_data_sample.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-12T12:49:40.686474Z","iopub.execute_input":"2021-11-12T12:49:40.686702Z","iopub.status.idle":"2021-11-12T12:49:40.712560Z","shell.execute_reply.started":"2021-11-12T12:49:40.686677Z","shell.execute_reply":"2021-11-12T12:49:40.711703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Data Analysis/EDA of Tracking data\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of team\ntracking_data_sample['team'].value_counts().plot(kind='pie', explode=[0.1,0.1,0.1], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(10,5), shadow=True, startangle=135, legend=False, cmap='plasma')\nplt.suptitle(t = 'Percentage distribution count for Team', y = 1.05, size = 20)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = tracking_data_sample['team'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:49:40.713789Z","iopub.execute_input":"2021-11-12T12:49:40.714077Z","iopub.status.idle":"2021-11-12T12:49:40.962466Z","shell.execute_reply.started":"2021-11-12T12:49:40.714048Z","shell.execute_reply":"2021-11-12T12:49:40.961536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows that there is equal distribution of datapoints for home and away team for a player.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the percentage distribution count for Play Direction?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of playDirection\ntracking_data_sample['playDirection'].value_counts().plot(kind='pie', explode=[0.1,0.1], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(10,5), shadow=True, startangle=135, legend=False, cmap='cividis')\nplt.suptitle(t = 'Percentage distribution count for Play Direction', y = 1.05, size = 20)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = tracking_data_sample['playDirection'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:49:40.963871Z","iopub.execute_input":"2021-11-12T12:49:40.964150Z","iopub.status.idle":"2021-11-12T12:49:41.204337Z","shell.execute_reply.started":"2021-11-12T12:49:40.964122Z","shell.execute_reply":"2021-11-12T12:49:41.203570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above plot shows that there is equal distribution of datapoints for play direction offense is moving (left or right).","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Player Position with respect to Position, Play Direction and Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution plot for player position- Long and Short axis\nsns.relplot(data=tracking_data_sample, x=\"x\", y=\"y\", col=\"playDirection\", hue=\"team\", style=\"position\", kind=\"scatter\", palette='dark')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:49:41.205521Z","iopub.execute_input":"2021-11-12T12:49:41.205846Z","iopub.status.idle":"2021-11-12T12:49:56.436389Z","shell.execute_reply.started":"2021-11-12T12:49:41.205813Z","shell.execute_reply":"2021-11-12T12:49:56.435533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above scatter plot shows that there is slight variation between play direction for Home and Away team with respect to Player position - Long and Short Axis for various player position.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Player Speed and Acceleration with respect to Position, Play Direction and Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"# distribution plot for player speed and acceleration\nsns.relplot(data=tracking_data_sample, x=\"s\", y=\"a\", col=\"playDirection\", hue=\"team\", style=\"position\", kind=\"scatter\", palette='bright')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:49:56.437593Z","iopub.execute_input":"2021-11-12T12:49:56.437853Z","iopub.status.idle":"2021-11-12T12:50:11.469330Z","shell.execute_reply.started":"2021-11-12T12:49:56.437826Z","shell.execute_reply":"2021-11-12T12:50:11.468459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above scatter plot shows that there is slight variation between play direction for Home and Away team with respect to Player Speed and Acceleration for various player position.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Player Orientation and Motion with respect to Position, Play Direction and Team?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution plot for orientation\nsns.relplot(data=tracking_data_sample, x=\"o\", y=\"dir\", col=\"playDirection\", hue=\"team\", style=\"position\", kind=\"scatter\", palette='rocket')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:11.470437Z","iopub.execute_input":"2021-11-12T12:50:11.470677Z","iopub.status.idle":"2021-11-12T12:50:26.684530Z","shell.execute_reply.started":"2021-11-12T12:50:11.470649Z","shell.execute_reply":"2021-11-12T12:50:26.683698Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above scatter plot shows that there is not much variation between play direction for Home and Away team with respect to Player Orientation and Angle of Player Motion for various player position.","metadata":{}},{"cell_type":"markdown","source":"---\n# Data Definition/Description of PFF Scouting data\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Data Definition:**\n* **PFF Scouting data:** The PFFScoutingData.csv file contains play-level scouting information for each game. The key variables are **gameId** and **playId**.\n","metadata":{}},{"cell_type":"code","source":"# Loading dataset PFFScoutingData.csv\nPFFScouting_data = pd.read_csv('../input/nfl-big-data-bowl-2022/PFFScoutingData.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:26.685785Z","iopub.execute_input":"2021-11-12T12:50:26.685990Z","iopub.status.idle":"2021-11-12T12:50:26.770251Z","shell.execute_reply.started":"2021-11-12T12:50:26.685964Z","shell.execute_reply":"2021-11-12T12:50:26.769544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the structure of PFFScouting dataset?**\n\n---","metadata":{}},{"cell_type":"markdown","source":"| No. | Feature Name | Description of the feature |\n| :-- | :--| :--| \n|01| **gameId**   | Game identifier, unique (numeric) |\n|02| **playId**   | Play identifier, not unique across games (numeric) |\n|03| **snapDetail**   | On Punts, whether the snap was on target and if not, provides detail (H: High, L: Low, <: Left, >: Right, OK: Accurate Snap, text)  |\n|04| **snapTime**   | Timing from snap to kick on punt plays in seconds: (numeric)  |\n|05| **operationTime**   | Timing from snap to kick on punt plays in seconds: (numeric)  |\n|06| **hangTime**   |  Hangtime of player's punt or kickoff attempt in seconds. Timing is taken from impact with foot to impact with the ground or a player. (numeric)|\n|07| **kickType**   | Kickoff or Punt Type (text)|\n|||Possible values for kickoff plays:|\n|||D: Deep - your normal deep kick with decent hang time\n|||F: Flat - different than a Squib in that it will have some hang time and no roll but has a lower trajectory and hang time than a Deep kick off\n|||K: Free Kick - Kick after a safety\n|||O: Obvious Onside - score and situation dictates the need to regain possession. Also the hands team is on for the returning team\n|||P: Pooch kick - high for hangtime but not a lot of distance - usually targeting an upman\n|||Q: Squib - low-line drive kick that bounces or rolls considerably, with virtually no hang time\n|||S: Surprise Onside - accounting for score and situation an onsides kick that the returning team doesn’t expect. Hands teams probably aren't on the field\n|||B: Deep Direct OOB - Kickoff that is aimed deep (regular kickoff) that goes OOB directly (doesn't bounce)\n||| Possible values for punt plays:\n|||N: Normal - standard punt style\n|||R: Rugby style punt\n|||A: Nose down or Aussie-style punts\n|08| **kickDirectionIntended**   | Intended kick direction from the kicking team's perspective - based on how coverage unit sets up and other factors (L: Left, R: Right, C: Center, text).|\n|09| **kickDirectionActual**   | Actual kick direction from the kicking team's perspective (L: Left, R: Right, C: Center, text).|\n|10| **returnDirectionIntended**   | The return direction the punt return or kick off return unit is set up for from the return team's perspective (L: Left, R: Right, C: Center, text). |\n|11| **returnDirectionActual**   | Actual return direction from the return team's perspective (L: Left, R: Right, C: Center, text). |\n|12| **missedTacklers**   | Jersey number and team code of player(s) charged with a missed tackle on the play. It will be reasonable to assume that he should have brought down the ball carrier and failed to do so. This situation does not have to entail contact, but it most frequently does. Missed tackles on a QB by a pass rusher are also included here. Multiple missed tacklers on a play are separated by a ; (text). |\n|13| **assistTacklers**   | Jersey number and team code of player(s) assisting on the tackle. Multiple assist tacklers on a play are separated by a ; (text). |\n|14| **tacklers**   | Jersey number and team code of player making the tackle (text). |\n|15| **kickoffReturnFormation**   | 3 digit code indicating the number of players in the Front Wall, Mid Wall and Back Wall (text). |\n|16| **gunners**   | Jersey number and team code of player(s) lined up as gunner on punt unit. Multiple gunners on a play are separated by a ; (text). |\n|17| **puntRushers**   | Jersey number and team code of player(s) on the punt return unit with \"Punt Rush\" role for actively trying to block the punt. Does not include players crossing the line of scrimmage to engage in punt coverage players in a \"Hold Up\" role. Multiple punt rushers on a play are separated by a ; (text).|\n|18| **specialTeamsSafeties**   | Jersey number and team code for player(s) with \"Safety\" roles on kickoff coverage and field goal/extra point block units - and those not actively advancing towards the line of scrimmage on the punt return unit. Multiple special teams safeties on a play are separated by a ; (text).|\n|19| **vises**   | Jersey number and team code for player(s) with a \"Vise\" role on the punt return unit. Multiple vises on a play are separated by a ; (text).|\n|20| **kickContactType**   | Detail on how a punt was fielded, or what happened when it wasn't fielded (text).|\n|||Possible values:|\n|||BB: Bounced Backwards\n|||BC: Bobbled Catch from Air\n|||BF: Bounced Forwards\n|||BOG: Bobbled on Ground\n|||CC: Clean Catch from Air\n|||CFFG: Clean Field From Ground\n|||DEZ: Direct to Endzone\n|||ICC: Incidental Coverage Team Contact\n|||KTB: Kick Team Knocked Back\n|||KTC: Kick Team Catch\n|||KTF: Kick Team Knocked Forward\n|||MBC: Muffed by Contact with Non-Designated Returner\n|||MBDR: Muffed by Designated Returner\n|||OOB: Directly Out Of Bounds","metadata":{}},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of PFFScouting Data dataset is:', PFFScouting_data.shape)\n\n# print summary of dataframe\nPFFScouting_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:26.771823Z","iopub.execute_input":"2021-11-12T12:50:26.772121Z","iopub.status.idle":"2021-11-12T12:50:26.809401Z","shell.execute_reply.started":"2021-11-12T12:50:26.772081Z","shell.execute_reply":"2021-11-12T12:50:26.808378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**PFFScouting dataset information:**\n\n* There are 19979 data points (rows) and 20 feature (column) in PFFScouting dataset.\n* There are five numerical column and fifteen columns are of categorical or object type.\n* There are missing values (non-null count is not same as 19979) for all columns except **gameId** and **playId**","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for PFFScouting dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\nPFFScouting_data.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:26.810230Z","iopub.execute_input":"2021-11-12T12:50:26.810438Z","iopub.status.idle":"2021-11-12T12:50:26.837772Z","shell.execute_reply.started":"2021-11-12T12:50:26.810406Z","shell.execute_reply":"2021-11-12T12:50:26.836821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for PFFScouting dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for numeric type\nPFFScouting_data.describe().round(1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:26.839167Z","iopub.execute_input":"2021-11-12T12:50:26.839525Z","iopub.status.idle":"2021-11-12T12:50:27.133434Z","shell.execute_reply.started":"2021-11-12T12:50:26.839453Z","shell.execute_reply":"2021-11-12T12:50:27.132707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**PFFScouting dataset data description:**\n\n*  There are no missing value for **gameId** and **playId** as they have total count of 19979 data points.\n*  Mean value for **snapTime** is same as median value which appears that distribution is normal.\n * There are missing values for **snapTime**\n*  Mean value for **operationTime** is same as median value which appears that distribution is normal.\n * There are missing values for **operationTime**\n*  Mean value for **hangTime** is same as median value which appears that distribution is normal.\n * There are missing values for **hangTime**","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for object type\nPFFScouting_data.describe(include=[object])","metadata":{"execution":{"iopub.status.busy":"2021-11-12T12:50:27.134445Z","iopub.execute_input":"2021-11-12T12:50:27.135151Z","iopub.status.idle":"2021-11-12T12:50:27.244500Z","shell.execute_reply.started":"2021-11-12T12:50:27.135119Z","shell.execute_reply":"2021-11-12T12:50:27.243584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**PFFScouting dataset data description:**\n\n* There are 5 unique **snapDetail** for which data points are available.\n * OK **snapDetail** has the highest frequency of occurrence.\n * There are missing value for **snapDetail**\n* There are 11 unique **kickType** for which data points are available.\n * D **kickType** has the highest frequency of occurrence.\n * There are missing value for **kickType**\n* There are 3 unique **kickDirectionIntended** for which data points are available.\n * C **kickDirectionIntended** has the highest frequency of occurrence.\n * There are missing value for **kickDirectionIntended**\n* There are 3 unique **kickDirectionActual** for which data points are available.\n * C **kickDirectionActual** has the highest frequency of occurrence.\n * There are missing value for **kickDirectionActual**\n* There are 3 unique **returnDirectionIntended** for which data points are available.\n * C **returnDirectionIntended** has the highest frequency of occurrence.\n * There are missing value for **returnDirectionIntended**\n* There are 3 unique **returnDirectionActual** for which data points are available.\n * C **returnDirectionActual** has the highest frequency of occurrence.\n * There are missing value for **returnDirectionActual**\n* There are 860 unique **missedTackler** for which data points are available.\n * BAL 10 **missedTackler** has the highest frequency of occurrence.\n * There are missing value for **missedTackler**\n* There are 576 unique **assistTackler** for which data points are available.\n * LAC 48 **assistTackler** has the highest frequency of occurrence.\n * There are missing value for **assistTackler**\n* There are 1081 unique **tackler** for which data points are available.\n * ARI 47 **tackler** has the highest frequency of occurrence.\n * There are missing value for **tackler**\n* There are 16 unique **kickoffReturnFormation** for which data points are available.\n * 8-0-2 **kickoffReturnFormation** has the highest frequency of occurrence.\n * There are missing value for **kickoffReturnFormation**\n* There are 926 unique **gunners** for which data points are available.\n * SEA28;SEA23 **gunners** has the highest frequency of occurrence.\n * There are missing value for **gunners**\n* There are 2089 unique **puntRushers** for which data points are available.\n * PIT 45 **puntRushers** has the highest frequency of occurrence.\n * There are missing value for **puntRushers**\n* There are 3107 unique **specialTeamsSafeties** for which data points are available.\n * HOU 33;HOU 41 **specialTeamsSafeties** has the highest frequency of occurrence.\n * There are missing value for **specialTeamsSafeties**\n* There are 1779 unique **vises** for which data points are available.\n * LA 31;LA 25 **vises** has the highest frequency of occurrence.\n * There are missing value for **vises**\n* There are 14 unique **kickContactType** for which data points are available.\n * CC **kickContactType** has the highest frequency of occurrence.\n * There are missing value for **kickContactType**\n \n ---","metadata":{}},{"cell_type":"markdown","source":"# Data Analysis/EDA of PFF Scouting data\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution count for Snap Detail?**\n\n---","metadata":{}},{"cell_type":"code","source":"# plot percentage distribution count of snapDetail\nPFFScouting_data['snapDetail'].value_counts().plot(kind='pie', explode=[0.1,0.3,0.5,0.7,0.9], fontsize=14, autopct='%3.1f%%', \n                                               figsize=(10,5), shadow=True, startangle=135, legend=False, cmap='winter')\nplt.suptitle(t = 'Percentage distribution count for Snap Detail', y = 1.05, size = 20)\nplt.ylabel('')\nplt.axis('equal')\nplt.legend(labels = PFFScouting_data['snapDetail'].value_counts().index, loc ='lower left', frameon = True)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:27.246103Z","iopub.execute_input":"2021-11-12T12:50:27.246374Z","iopub.status.idle":"2021-11-12T12:50:27.461956Z","shell.execute_reply.started":"2021-11-12T12:50:27.246345Z","shell.execute_reply":"2021-11-12T12:50:27.461050Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"H: High, L: Low, <: Left, >: Right, OK: Accurate Snap\n\nAbove plot shows that majority datapoints for Snap Detail is Accurate Snap.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Snap Time with respect to Snap Detail?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of snapTime with respect to snapDetail\nsns.displot(data=PFFScouting_data, x='snapTime', hue='snapDetail',kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:27.463196Z","iopub.execute_input":"2021-11-12T12:50:27.463424Z","iopub.status.idle":"2021-11-12T12:50:27.987121Z","shell.execute_reply.started":"2021-11-12T12:50:27.463394Z","shell.execute_reply":"2021-11-12T12:50:27.986293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"H: High, L: Low, <: Left, >: Right, OK: Accurate Snap\n\nAbove density plot shows that majority of snap was on target with timing between 0.6 to 1.0 seconds for snap to kick on punt plays.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Operation Time with respect to Snap Detail?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of operationTime with respect to snapDetail\nsns.displot(data=PFFScouting_data, x='operationTime', hue='snapDetail',kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:27.988206Z","iopub.execute_input":"2021-11-12T12:50:27.988429Z","iopub.status.idle":"2021-11-12T12:50:28.536634Z","shell.execute_reply.started":"2021-11-12T12:50:27.988402Z","shell.execute_reply":"2021-11-12T12:50:28.535826Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"H: High, L: Low, <: Left, >: Right, OK: Accurate Snap\n\nAbove density plot shows that majority of snap was on target with timing between 1.5 to 2.5 seconds for snap to kick on punt plays.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Hang Time with respect to Snap Detail?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of hangTime with respect to snapDetail\nsns.displot(data=PFFScouting_data, x='hangTime', hue='snapDetail',kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:28.537818Z","iopub.execute_input":"2021-11-12T12:50:28.538037Z","iopub.status.idle":"2021-11-12T12:50:29.079994Z","shell.execute_reply.started":"2021-11-12T12:50:28.538011Z","shell.execute_reply":"2021-11-12T12:50:29.079126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"H: High, L: Low, <: Left, >: Right, OK: Accurate Snap\n\nAbove density plot shows that majority of player's kickoff or punt attempt was on target with timing between 3 to 6 seconds for impact with foot to impact with ground.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution count for kickType?**\n\n---","metadata":{}},{"cell_type":"code","source":"#check distribution and frequency of kickType\nPFFScouting_data['kickType'].value_counts()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:29.081423Z","iopub.execute_input":"2021-11-12T12:50:29.082304Z","iopub.status.idle":"2021-11-12T12:50:29.092525Z","shell.execute_reply.started":"2021-11-12T12:50:29.082263Z","shell.execute_reply":"2021-11-12T12:50:29.091800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| | | |\n| :-- | :--| :--| \n|| **kickType**   | Kickoff or Punt Type (text)|\n|||**Possible values for kickoff plays:**|\n|||D: Deep - your normal deep kick with decent hang time\n|||F: Flat - different than a Squib in that it will have some hang time and no roll but has a lower trajectory and hang time than a Deep kick off\n|||K: Free Kick - Kick after a safety\n|||O: Obvious Onside - score and situation dictates the need to regain possession. Also the hands team is on for the returning team\n|||P: Pooch kick - high for hangtime but not a lot of distance - usually targeting an upman\n|||Q: Squib - low-line drive kick that bounces or rolls considerably, with virtually no hang time\n|||S: Surprise Onside - accounting for score and situation an onsides kick that the returning team doesn’t expect. Hands teams probably aren't on the field\n|||B: Deep Direct OOB - Kickoff that is aimed deep (regular kickoff) that goes OOB directly (doesn't bounce)\n||| **Possible values for punt plays:**\n|||N: Normal - standard punt style\n|||R: Rugby style punt\n|||A: Nose down or Aussie-style punts","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Kick Type with respect to Kick Direction Intended?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickType with respect to kickDirectionIntended\nsns.displot(data=PFFScouting_data, x='kickType', hue='kickDirectionIntended',height=8, aspect=.8)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-12T12:50:29.093763Z","iopub.execute_input":"2021-11-12T12:50:29.094365Z","iopub.status.idle":"2021-11-12T12:50:29.745297Z","shell.execute_reply.started":"2021-11-12T12:50:29.094321Z","shell.execute_reply":"2021-11-12T12:50:29.744595Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above distribution plot shows that majority of Deep kick and Normal Punt had Kick Direction Intended as Center and Left.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Kick Type with respect to Kick Direction Actual?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickType with respect to kickDirectionActual\nsns.displot(data=PFFScouting_data, x='kickType', hue='kickDirectionActual',height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:29.746773Z","iopub.execute_input":"2021-11-12T12:50:29.747256Z","iopub.status.idle":"2021-11-12T12:50:30.384967Z","shell.execute_reply.started":"2021-11-12T12:50:29.747214Z","shell.execute_reply":"2021-11-12T12:50:30.384175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above distribution plot shows that majority of Deep kick and Normal Punt had Kick Direction Actual as Center and Left.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Kick Type with respect to Return Direction Intended?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickType with respect to returnDirectionIntended\nsns.displot(data=PFFScouting_data, x='kickType', hue='returnDirectionIntended',height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:30.386665Z","iopub.execute_input":"2021-11-12T12:50:30.387014Z","iopub.status.idle":"2021-11-12T12:50:31.008553Z","shell.execute_reply.started":"2021-11-12T12:50:30.386974Z","shell.execute_reply":"2021-11-12T12:50:31.007683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above distribution plot shows that majority of Deep kick and Normal Punt had Return Direction Intended as Center and Right.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is distribution for Kick Type with respect to Return Direction Actual?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickType with respect to returnDirectionActual\nsns.displot(data=PFFScouting_data, x='kickType', hue='returnDirectionActual',height=8, aspect=.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:31.010205Z","iopub.execute_input":"2021-11-12T12:50:31.010507Z","iopub.status.idle":"2021-11-12T12:50:31.638432Z","shell.execute_reply.started":"2021-11-12T12:50:31.010464Z","shell.execute_reply":"2021-11-12T12:50:31.637603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above distribution plot shows that majority of Deep kick and Normal Punt had Return Direction Actual as Center and Left.","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"---\n# Data Definition/Description of NFL 2022 data\n\n---","metadata":{}},{"cell_type":"markdown","source":"Note: Merge various dataset to arrive at combined sample dataset for EDA","metadata":{}},{"cell_type":"markdown","source":"---\n**Merge game and play dataset - game_play_data**\n\n---","metadata":{}},{"cell_type":"code","source":"# Merge of game and play dataset using key as gameId\ngame_play_data = pd.merge(game_data, play_data, how=\"inner\", on=[\"gameId\"])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:31.639933Z","iopub.execute_input":"2021-11-12T12:50:31.640245Z","iopub.status.idle":"2021-11-12T12:50:31.676605Z","shell.execute_reply.started":"2021-11-12T12:50:31.640204Z","shell.execute_reply":"2021-11-12T12:50:31.675723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of game_play_data dataset is:', game_play_data.shape)\n\n# print summary of dataframe\ngame_play_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:31.678168Z","iopub.execute_input":"2021-11-12T12:50:31.678411Z","iopub.status.idle":"2021-11-12T12:50:31.722487Z","shell.execute_reply.started":"2021-11-12T12:50:31.678382Z","shell.execute_reply":"2021-11-12T12:50:31.721726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print first 5 rows of dataframe\ngame_play_data.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:31.723786Z","iopub.execute_input":"2021-11-12T12:50:31.723999Z","iopub.status.idle":"2021-11-12T12:50:31.752167Z","shell.execute_reply.started":"2021-11-12T12:50:31.723973Z","shell.execute_reply":"2021-11-12T12:50:31.751196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Merge player and tracking2020_sample dataset- player_tracking_data**\n\n---","metadata":{}},{"cell_type":"code","source":"# Merge of player and tracking2020_data_sample season dataset using key as nflId\nplayer_tracking_data = pd.merge(player_data,tracking_data_sample, how=\"inner\", on=[\"nflId\"])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:31.754486Z","iopub.execute_input":"2021-11-12T12:50:31.755011Z","iopub.status.idle":"2021-11-12T12:50:32.348541Z","shell.execute_reply.started":"2021-11-12T12:50:31.754969Z","shell.execute_reply":"2021-11-12T12:50:32.347701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of player_tracking_data dataset is:', player_tracking_data.shape)\n\n# print summary of dataframe\nplayer_tracking_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:32.349878Z","iopub.execute_input":"2021-11-12T12:50:32.350179Z","iopub.status.idle":"2021-11-12T12:50:32.705639Z","shell.execute_reply.started":"2021-11-12T12:50:32.350142Z","shell.execute_reply":"2021-11-12T12:50:32.704771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print first 5 rows of dataframe\nplayer_tracking_data.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:32.706660Z","iopub.execute_input":"2021-11-12T12:50:32.706867Z","iopub.status.idle":"2021-11-12T12:50:32.732772Z","shell.execute_reply.started":"2021-11-12T12:50:32.706843Z","shell.execute_reply":"2021-11-12T12:50:32.732163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Merge game_play_data and PFFScouting dataset- game_play_scouting_data**\n\n---","metadata":{}},{"cell_type":"code","source":"# Merge of game_play_data and PFFScouting_data dataset using key as gameId and playId\ngame_play_scouting_data = pd.merge(game_play_data, PFFScouting_data, how=\"inner\", left_on=[\"gameId\",\"playId\"], right_on=[\"gameId\",\"playId\"])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:32.733674Z","iopub.execute_input":"2021-11-12T12:50:32.734404Z","iopub.status.idle":"2021-11-12T12:50:32.771287Z","shell.execute_reply.started":"2021-11-12T12:50:32.734359Z","shell.execute_reply":"2021-11-12T12:50:32.770355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of game_play_scouting_data dataset is:', game_play_scouting_data.shape)\n\n# print summary of dataframe\ngame_play_scouting_data.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:32.774792Z","iopub.execute_input":"2021-11-12T12:50:32.775058Z","iopub.status.idle":"2021-11-12T12:50:32.840572Z","shell.execute_reply.started":"2021-11-12T12:50:32.775026Z","shell.execute_reply":"2021-11-12T12:50:32.839743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print first 5 rows of dataframe\ngame_play_scouting_data.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:32.841886Z","iopub.execute_input":"2021-11-12T12:50:32.842539Z","iopub.status.idle":"2021-11-12T12:50:32.867571Z","shell.execute_reply.started":"2021-11-12T12:50:32.842490Z","shell.execute_reply":"2021-11-12T12:50:32.866796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Merge game_play_scouting and player_tracking dataset- nfl2022_data_sample**\n\n---","metadata":{}},{"cell_type":"code","source":"# Merge of game_play_scouting_data and player_tracking_data dataset using key as gameId and playId\nnfl2022_data_sample = pd.merge(game_play_scouting_data, player_tracking_data, how=\"inner\", left_on=[\"gameId\",\"playId\"], right_on=[\"gameId\",\"playId\"])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:32.868647Z","iopub.execute_input":"2021-11-12T12:50:32.868883Z","iopub.status.idle":"2021-11-12T12:50:33.608788Z","shell.execute_reply.started":"2021-11-12T12:50:32.868855Z","shell.execute_reply":"2021-11-12T12:50:33.607912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of nfl2022_data_sample dataset is:', nfl2022_data_sample.shape)\n\n# print summary of dataframe\nnfl2022_data_sample.info()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:33.610012Z","iopub.execute_input":"2021-11-12T12:50:33.610246Z","iopub.status.idle":"2021-11-12T12:50:34.624285Z","shell.execute_reply.started":"2021-11-12T12:50:33.610218Z","shell.execute_reply":"2021-11-12T12:50:34.623400Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print first 5 rows of dataframe\nnfl2022_data_sample.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:34.625269Z","iopub.execute_input":"2021-11-12T12:50:34.625469Z","iopub.status.idle":"2021-11-12T12:50:34.654133Z","shell.execute_reply.started":"2021-11-12T12:50:34.625445Z","shell.execute_reply":"2021-11-12T12:50:34.652575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Data Analysis/EDA of NFL 2022 data\n\n---","metadata":{}},{"cell_type":"markdown","source":"Review data type of different variable in nfl2022_data_sample for EDA","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What are the numeric float variables in nfl2022_data_sample?**\n\n---","metadata":{}},{"cell_type":"code","source":"# Get list of numeric float variables\ns = (nfl2022_data_sample.dtypes == 'float64')\nnumeric_float_cols = list(s[s].index)\n\nprint(\"Numerical float variables:\")\nprint(numeric_float_cols)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:34.655487Z","iopub.execute_input":"2021-11-12T12:50:34.655751Z","iopub.status.idle":"2021-11-12T12:50:34.667203Z","shell.execute_reply.started":"2021-11-12T12:50:34.655721Z","shell.execute_reply":"2021-11-12T12:50:34.663074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What are the numeric int variables in nfl2022_data_sample?**\n\n---","metadata":{}},{"cell_type":"code","source":"# Get list of numeric int variables\ns = (nfl2022_data_sample.dtypes == 'int64')\nnumeric_int_cols = list(s[s].index)\n\nprint(\"Numerical int variables:\")\nprint(numeric_int_cols)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:34.668411Z","iopub.execute_input":"2021-11-12T12:50:34.669304Z","iopub.status.idle":"2021-11-12T12:50:34.683825Z","shell.execute_reply.started":"2021-11-12T12:50:34.669261Z","shell.execute_reply":"2021-11-12T12:50:34.682762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What are the object variables in nfl2022_data_sample?**\n\n---","metadata":{}},{"cell_type":"code","source":"# Get list of object variables\ns = (nfl2022_data_sample.dtypes == 'object')\nobject_cols = list(s[s].index)\n\nprint(\"Object variables:\")\nprint(object_cols)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:34.692033Z","iopub.execute_input":"2021-11-12T12:50:34.692293Z","iopub.status.idle":"2021-11-12T12:50:34.699314Z","shell.execute_reply.started":"2021-11-12T12:50:34.692263Z","shell.execute_reply":"2021-11-12T12:50:34.698415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What are the datetime variables in nfl2022_data_sample?**\n\n---","metadata":{}},{"cell_type":"code","source":"# Get list of datetime variables\ns = (nfl2022_data_sample.dtypes == 'datetime64[ns]')\ndatetime_cols = list(s[s].index)\n\nprint(\"datetime variables:\")\nprint(datetime_cols)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:34.700940Z","iopub.execute_input":"2021-11-12T12:50:34.701902Z","iopub.status.idle":"2021-11-12T12:50:34.723027Z","shell.execute_reply.started":"2021-11-12T12:50:34.701863Z","shell.execute_reply":"2021-11-12T12:50:34.722057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the correlation for key numeric variables in nfl2022_data_sample?**\n\n---","metadata":{}},{"cell_type":"code","source":"#checking correlation between key numeric variables via heatmap\ncorr = nfl2022_data_sample[['penaltyYards', 'kickLength', 'kickReturnYardage', 'snapTime', 'operationTime', 'hangTime', 'x', 'y', 's', 'a', 'dis', 'o', 'dir']].corr(method='pearson')\nplt.figure(figsize=(25,25))\nsns.heatmap(corr,vmax=.8,linewidth=.01, square = True, annot = True,cmap='YlGnBu',linecolor ='black')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:34.724550Z","iopub.execute_input":"2021-11-12T12:50:34.725538Z","iopub.status.idle":"2021-11-12T12:50:36.354747Z","shell.execute_reply.started":"2021-11-12T12:50:34.725496Z","shell.execute_reply":"2021-11-12T12:50:36.353861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above heatmap plot for correlation shows that there is not much strong positive or negative relationship between various numeric variables but there is strong positive correlation between Speed(s) and Distance(dis).\n\nNote: s is Player speed in yards/second and dis is distance traveled from prior time in yards.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution and relationship for key numerical variables in nfl2022_data_sample?**\n\n---","metadata":{}},{"cell_type":"code","source":"#plot pairwise relationship for key numeric variables\nsns.pairplot(data=nfl2022_data_sample[['penaltyYards', 'kickLength', 'kickReturnYardage', 'snapTime', 'operationTime', 'hangTime', 'x', 'y', 's', 'a', 'dis', 'o', 'dir']],palette='rainbow',diag_kind='kde')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:50:36.355930Z","iopub.execute_input":"2021-11-12T12:50:36.356162Z","iopub.status.idle":"2021-11-12T12:52:10.689932Z","shell.execute_reply.started":"2021-11-12T12:50:36.356134Z","shell.execute_reply":"2021-11-12T12:52:10.688953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above pair plot shows there is good distribution of data points between numeric variable.\nNote that Player Orientation(o) and Angle of Player Motion(dir) has different distribution curve probably owing to value of data points in degrees.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Kick Type and Kick Length with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickType, kickLength with respect to specialTeamsPlayType\nsns.catplot(data=nfl2022_data_sample, x=\"kickType\", y=\"kickLength\", hue=\"specialTeamsPlayType\", height=8, aspect=0.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:52:10.691383Z","iopub.execute_input":"2021-11-12T12:52:10.692148Z","iopub.status.idle":"2021-11-12T12:52:22.277121Z","shell.execute_reply.started":"2021-11-12T12:52:10.692110Z","shell.execute_reply":"2021-11-12T12:52:22.276316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"D: Deep, N: Normal - standard punt style, A: Nose down or Aussie-style punts, Q: Squib, P: Pooch kick, F: Flat, O: Obvious Onside, K: Free Kick, R: Rugby style punt, S: Surprise Onside, B: Deep Direct OOB","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Kick Contact Type and Kick Length with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of kickContactType, kickLength with respect to specialTeamsPlayType\nsns.catplot(data=nfl2022_data_sample, x=\"kickContactType\", y=\"kickLength\", hue=\"specialTeamsPlayType\", height=8, aspect=0.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:52:22.278547Z","iopub.execute_input":"2021-11-12T12:52:22.279447Z","iopub.status.idle":"2021-11-12T12:52:31.590150Z","shell.execute_reply.started":"2021-11-12T12:52:22.279410Z","shell.execute_reply":"2021-11-12T12:52:31.589632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CC: Clean Catch from Air, BF: Bounced Forwards, BB: Bounced Backwards, MBC: Muffed by Contact with Non-Designated Returner, CFFG: Clean Field From Ground, MBDR: Muffed by Designated Returner, OOB: Directly Out Of Bounds, KTF: Kick Team Knocked Forward, BC: Bobbled Catch from Air, KTB: Kick Team Knocked Back, BOG: Bobbled on Ground, DEZ: Direct to Endzone, KTC: Kick Team Catch, ICC: Incidental Coverage Team Contact \n\n\n\n\n\n\n\n\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Snap Detail and Snap Time with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of snapDetail, snapTime with respect to specialTeamsPlayType\nsns.catplot(data=nfl2022_data_sample, x=\"snapDetail\", y=\"snapTime\", hue=\"specialTeamsPlayType\", height=8, aspect=0.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:52:31.591002Z","iopub.execute_input":"2021-11-12T12:52:31.591461Z","iopub.status.idle":"2021-11-12T12:52:39.287272Z","shell.execute_reply.started":"2021-11-12T12:52:31.591431Z","shell.execute_reply":"2021-11-12T12:52:39.286422Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"OK: Accurate Snap, <: Left, H: High, >: Right, L: Low","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Snap Detail and Operation Time with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of snapDetail, operationTime with respect to specialTeamsPlayType\nsns.catplot(data=nfl2022_data_sample, x=\"snapDetail\", y=\"operationTime\", hue=\"specialTeamsPlayType\", height=8, aspect=0.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:52:39.288401Z","iopub.execute_input":"2021-11-12T12:52:39.288633Z","iopub.status.idle":"2021-11-12T12:52:46.807360Z","shell.execute_reply.started":"2021-11-12T12:52:39.288594Z","shell.execute_reply":"2021-11-12T12:52:46.806738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"OK: Accurate Snap, <: Left, H: High, >: Right, L: Low","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Snap Detail and Hang Time with respect to Special Teams Play Type?**\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution of snapDetail, hangTime with respect to specialTeamsPlayType\nsns.catplot(data=nfl2022_data_sample, x=\"snapDetail\", y=\"hangTime\", hue=\"specialTeamsPlayType\", height=8, aspect=0.8)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:52:46.808398Z","iopub.execute_input":"2021-11-12T12:52:46.809110Z","iopub.status.idle":"2021-11-12T12:52:54.311007Z","shell.execute_reply.started":"2021-11-12T12:52:46.809065Z","shell.execute_reply":"2021-11-12T12:52:54.310461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"OK: Accurate Snap, <: Left, H: High, >: Right, L: Low","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"---\n# Summary\n---\n\n**EDA Involves:**\n* Providing insights on **Special Teams plays** and associated metrics.\n\n---","metadata":{}},{"cell_type":"code","source":"#distribution count for specialTeamsPlayType with respect to team and season\nsns.catplot(data=nfl2022_data_sample, x=\"specialTeamsPlayType\", col=\"team\", hue=\"season\", kind=\"count\", height=8, aspect=0.8, palette=\"rainbow\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:52:54.312009Z","iopub.execute_input":"2021-11-12T12:52:54.312639Z","iopub.status.idle":"2021-11-12T12:53:00.120247Z","shell.execute_reply.started":"2021-11-12T12:52:54.312590Z","shell.execute_reply":"2021-11-12T12:53:00.119416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Number of Kickoff appears higher for 2020 season, Number of Punt seems higher for 2018 season, Number of Field Goal is almost similar for all the season and Number of Extra Point seems higher for 2020 season as per datapoints from sample data.","metadata":{}},{"cell_type":"markdown","source":"---\n**Highlights:**\n\n---\n**Q:What is the distribution of Special Teams Result and Special Teams Play Type with respect to Player Position along the Long axis of field?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get specialTeamsResult and specialTeamsPlayType sorted by x\nposition_longaxis = nfl2022_data_sample.groupby(by = ['specialTeamsPlayType','specialTeamsResult'], as_index = False)['x'].agg('mean').sort_values(by ='x', ascending = False)\n# set plot figure size\nfigure = plt.figure(figsize = [15, 10])\n# plot comparison of specialTeamsResult\nsns.barplot(x = 'x', y ='specialTeamsResult', hue='specialTeamsPlayType',data = position_longaxis[0:12])\n# set plot label\nplt.xlabel(xlabel = 'Player Position - Long Axis, 0-120 yards')\nplt.ylabel(ylabel = 'Special Teams Result')\nplt.grid(b = True, axis = 'x')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T14:38:52.357225Z","iopub.execute_input":"2021-11-12T14:38:52.357540Z","iopub.status.idle":"2021-11-12T14:38:53.184417Z","shell.execute_reply.started":"2021-11-12T14:38:52.357502Z","shell.execute_reply":"2021-11-12T14:38:53.183531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q:What is the distribution of Special Teams Result and Special Teams Play Type with respect to Player Position along the Short axis of field?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get specialTeamsResult and specialTeamsPlayType sorted by y\nposition_shortaxis = nfl2022_data_sample.groupby(by = ['specialTeamsPlayType','specialTeamsResult'], as_index = False)['y'].agg('mean').sort_values(by ='y', ascending = False)\n# set plot figure size\nfigure = plt.figure(figsize = [15, 10])\n# plot comparison of specialTeamsResult\nsns.barplot(x = 'y', y ='specialTeamsResult', hue='specialTeamsPlayType',data = position_shortaxis[0:12])\n# set plot label\nplt.xlabel(xlabel = 'Player Position - Short Axis, 0-53.3 yards')\nplt.ylabel(ylabel = 'Special Teams Result')\nplt.grid(b = True, axis = 'x')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T14:44:42.884091Z","iopub.execute_input":"2021-11-12T14:44:42.884751Z","iopub.status.idle":"2021-11-12T14:44:43.475697Z","shell.execute_reply.started":"2021-11-12T14:44:42.884713Z","shell.execute_reply":"2021-11-12T14:44:43.475056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q:What is the distribution of Special Teams Result and Special Teams Play Type with respect to Player Speed?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get specialTeamsResult and specialTeamsPlayType sorted by s\nspeed = nfl2022_data_sample.groupby(by = ['specialTeamsPlayType','specialTeamsResult'], as_index = False)['s'].agg('mean').sort_values(by ='s', ascending = False)\n# set plot figure size\nfigure = plt.figure(figsize = [15, 10])\n# plot comparison of specialTeamsResult\nsns.barplot(data = speed[0:12],x = 's', y ='specialTeamsResult', hue='specialTeamsPlayType',palette='winter')\n# set plot label\nplt.xlabel(xlabel = 'Speed in yards/second')\nplt.ylabel(ylabel = 'Special Teams Result')\nplt.grid(b = True, axis = 'x')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T14:45:35.875020Z","iopub.execute_input":"2021-11-12T14:45:35.875709Z","iopub.status.idle":"2021-11-12T14:45:36.295277Z","shell.execute_reply.started":"2021-11-12T14:45:35.875663Z","shell.execute_reply":"2021-11-12T14:45:36.294267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q:What is the distribution of Special Teams Result and Special Teams Play Type with respect to Player Acceleration?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get specialTeamsResult and specialTeamsPlayType sorted by a\nacceleration = nfl2022_data_sample.groupby(by = ['specialTeamsPlayType','specialTeamsResult'], as_index = False)['a'].agg('mean').sort_values(by ='a', ascending = False)\n# set plot figure size\nfigure = plt.figure(figsize = [15, 10])\n# plot comparison of specialTeamsResult\nsns.barplot(data = acceleration[0:12], x = 'a', y ='specialTeamsResult',hue='specialTeamsPlayType',palette='winter')\n# set plot label\nplt.xlabel(xlabel = 'Speed in yards/second^2')\nplt.ylabel(ylabel = 'Special Teams Result')\nplt.grid(b = True, axis = 'x')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T14:46:43.615605Z","iopub.execute_input":"2021-11-12T14:46:43.615947Z","iopub.status.idle":"2021-11-12T14:46:44.105269Z","shell.execute_reply.started":"2021-11-12T14:46:43.615914Z","shell.execute_reply":"2021-11-12T14:46:44.104416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q:What is the distribution of Special Teams Result and Special Teams Play Type with respect to Player Orientation?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get specialTeamsResult and specialTeamsPlayType sorted by o\norientation = nfl2022_data_sample.groupby(by = ['specialTeamsPlayType','specialTeamsResult'], as_index = False)['o'].agg('mean').sort_values(by ='o', ascending = False)\n# set plot figure size\nfigure = plt.figure(figsize = [15, 10])\n# plot comparison of specialTeamsResult\nsns.barplot(data = orientation[0:12],x = 'o', y ='specialTeamsResult',hue='specialTeamsPlayType',palette='autumn')\n# set plot label\nplt.xlabel(xlabel = 'Player Orientation (deg), 0-360 degrees')\nplt.ylabel(ylabel = 'Special Teams Result')\nplt.grid(b = True, axis = 'x')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T14:47:31.394433Z","iopub.execute_input":"2021-11-12T14:47:31.395036Z","iopub.status.idle":"2021-11-12T14:47:31.949771Z","shell.execute_reply.started":"2021-11-12T14:47:31.395001Z","shell.execute_reply":"2021-11-12T14:47:31.948729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q:What is the distribution of Special Teams Result and Special Teams Play Type with respect to angle of Player motion?**\n\n---","metadata":{}},{"cell_type":"code","source":"# get specialTeamsResult and specialTeamsPlayType sorted by dir\ndirection = nfl2022_data_sample.groupby(by = ['specialTeamsPlayType','specialTeamsResult'], as_index = False)['dir'].agg('mean').sort_values(by ='dir', ascending = False)\n# set plot figure size\nfigure = plt.figure(figsize = [15, 10])\n# plot comparison of specialTeamsResult\nsns.barplot(data = direction[0:12],x = 'dir', y ='specialTeamsResult',hue='specialTeamsPlayType',palette='autumn')\n# set plot label\nplt.xlabel(xlabel = 'Angle of Player Motion (deg), 0-360 degrees')\nplt.ylabel(ylabel = 'Special Teams Result')\nplt.grid(b = True, axis = 'x')\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T14:48:12.645156Z","iopub.execute_input":"2021-11-12T14:48:12.645442Z","iopub.status.idle":"2021-11-12T14:48:13.301514Z","shell.execute_reply.started":"2021-11-12T14:48:12.645411Z","shell.execute_reply":"2021-11-12T14:48:13.300742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Thank you and Happy Learning.**\n\n---","metadata":{}},{"cell_type":"code","source":"thank_you_str=\"Thanks,Happy Learning,Collaboration,Thankyou,Keep Learning\"\n# create WordCloud with converted string\nwordcloud = WordCloud(width = 1000, height = 500, random_state=1, background_color='white', collocations=True).generate(thank_you_str)\nplt.figure(figsize=(20, 20))\nplt.imshow(wordcloud) \nplt.axis(\"off\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-12T12:53:02.482638Z","iopub.execute_input":"2021-11-12T12:53:02.483409Z","iopub.status.idle":"2021-11-12T12:53:03.050567Z","shell.execute_reply.started":"2021-11-12T12:53:02.483365Z","shell.execute_reply":"2021-11-12T12:53:03.049688Z"},"trusted":true},"execution_count":null,"outputs":[]}]}