{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

NFL Big Data Bowl Basic EDA for beginner

\n\n
","metadata":{}},{"cell_type":"markdown","source":"### PLEASE UPVOTE if you like this notebook. It will keep me motivated to update my notebook.","metadata":{}},{"cell_type":"markdown","source":"#### What are you trying to do in this notebook ?\n\nIn this notebook I will predict how many yards a team will gain on a rushing play in an NFL regular season game. I will loop through a series of rushing plays; for each play. I'll use the information that is provided in this notebook and it will predict how many yards the team will gain on the play as a cumulative probability distribution. Once I'll make that prediction, I can move on to the next rushing play.\n\n#### Why are you trying it ?\n\nThis competition is different from most Kaggle Competitions in that:\n\n* In this kaggle notebook I may not other data sources, GPU, or internet access.\n* This is a two-stage competition. In Stage One I can edit my Notebooks and improve my model, where Public Leaderboard scores are based on my predictions on rushing play.\n* The purpose of this competition is to control the flow of information that is to ensure the future data to make predictions for the current rushing play.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\n\nplt.style.use('fivethirtyeight')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2021-12-01T08:20:45.055232Z","iopub.execute_input":"2021-12-01T08:20:45.055851Z","iopub.status.idle":"2021-12-01T08:20:46.053013Z","shell.execute_reply.started":"2021-12-01T08:20:45.055814Z","shell.execute_reply":"2021-12-01T08:20:46.052106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

1. Game Data

","metadata":{}},{"cell_type":"markdown","source":"### **Game data:** The games.csv contains the teams playing in each game. The key variable is gameId.\n\n* **gameId:** Game identifier, unique (numeric)\n\n* **gameDate:** Game Date (time, mm/dd/yyyy)\n\n* **gameTimeEastern:** Start time of game (time, HH:MM:SS, EST)\n\n* **homeTeamAbbr:** Home team three-letter code (text)\n\n* **visitorTeamAbbr:** Visiting team three-letter code (text)\n\n* **week:** Week of game (numeric)","metadata":{}},{"cell_type":"code","source":"games = pd.read_csv('../input/nfl-big-data-bowl-2022/games.csv')\ngames","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","execution":{"iopub.status.busy":"2021-12-01T08:20:46.054786Z","iopub.execute_input":"2021-12-01T08:20:46.055109Z","iopub.status.idle":"2021-12-01T08:20:46.098942Z","shell.execute_reply.started":"2021-12-01T08:20:46.055059Z","shell.execute_reply":"2021-12-01T08:20:46.098089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Function for Downcast","metadata":{}},{"cell_type":"markdown","source":"Downcast is a great skill to compress data size which helps to save memory.","metadata":{}},{"cell_type":"code","source":"def downcast(df, verbose=True):\n start_mem = df.memory_usage().sum() / 1024**2\n for col in df.columns:\n dtype_name = df[col].dtype.name\n if dtype_name == 'object':\n pass\n elif dtype_name == 'bool':\n df[col] = df[col].astype('int8')\n elif dtype_name.startswith('int') or (df[col].round() == df[col]).all():\n df[col] = pd.to_numeric(df[col], downcast='integer')\n else:\n df[col] = pd.to_numeric(df[col], downcast='float')\n end_mem = df.memory_usage().sum() / 1024**2\n if verbose:\n print('{:.1f}% Compressed'.format(100 * (start_mem - end_mem) / start_mem))\n \n return df","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.100553Z","iopub.execute_input":"2021-12-01T08:20:46.100862Z","iopub.status.idle":"2021-12-01T08:20:46.113052Z","shell.execute_reply.started":"2021-12-01T08:20:46.100830Z","shell.execute_reply":"2021-12-01T08:20:46.112209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"games = downcast(games)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.114596Z","iopub.execute_input":"2021-12-01T08:20:46.114944Z","iopub.status.idle":"2021-12-01T08:20:46.134574Z","shell.execute_reply.started":"2021-12-01T08:20:46.114914Z","shell.execute_reply":"2021-12-01T08:20:46.133229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Function for making feature summary ","metadata":{}},{"cell_type":"code","source":"def resumetable(df):\n print(f'Shape : {df.shape}')\n summary = pd.DataFrame(df.dtypes, columns=['Data Type'])\n summary = summary.reset_index()\n summary = summary.rename(columns={'index': 'Feature'})\n summary['Num of null'] = df.isnull().sum().values\n summary['Num of unique'] = df.nunique().values\n summary['First value'] = df.loc[0].values\n summary['Second value'] = df.loc[1].values\n summary['Third value'] = df.loc[2].values\n return summary\n\nresumetable(games)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.139197Z","iopub.execute_input":"2021-12-01T08:20:46.139522Z","iopub.status.idle":"2021-12-01T08:20:46.173368Z","shell.execute_reply.started":"2021-12-01T08:20:46.139491Z","shell.execute_reply":"2021-12-01T08:20:46.172522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Function for writing percent at the top of the bar graph","metadata":{}},{"cell_type":"code","source":"def write_percent(ax, total_size):\n '''Traverse the figure object and display the ratio at the top of the bar graph.'''\n for patch in ax.patches:\n height = patch.get_height() # Figure height (number of data)\n width = patch.get_width() # Figure width\n left_coord = patch.get_x() # The x-axis position on the left edge of the figure\n percent = height/total_size*100 # percent\n \n # Type text in the (x, y) coordinates\n ax.text(x=left_coord + width/2.0, # x-axis position\n y=height + total_size*0.001, # y-axis position\n s=f'{percent:1.1f}%', # Text\n ha='center') # in the middle","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.176402Z","iopub.execute_input":"2021-12-01T08:20:46.176730Z","iopub.status.idle":"2021-12-01T08:20:46.184527Z","shell.execute_reply.started":"2021-12-01T08:20:46.176698Z","shell.execute_reply":"2021-12-01T08:20:46.183426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Make derivative features (month, day, hour)","metadata":{}},{"cell_type":"code","source":"games['month'] = games['gameDate'].apply(lambda x: int(x.split('/')[0]))\ngames['day'] = games['gameDate'].apply(lambda x: int(x.split('/')[1]))\ngames['hour'] = games['gameTimeEastern'].apply(lambda x: int(x.split(':')[0]))","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.185897Z","iopub.execute_input":"2021-12-01T08:20:46.186327Z","iopub.status.idle":"2021-12-01T08:20:46.206551Z","shell.execute_reply.started":"2021-12-01T08:20:46.186284Z","shell.execute_reply":"2021-12-01T08:20:46.205330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Visualization","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=15) # Set font size\nplt.figure(figsize=(7, 6)) # Set figure size\n\nax = sns.countplot(x='season', data=games)\nwrite_percent(ax, len(games)) \nax.set_title('Number of games for season');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.208060Z","iopub.execute_input":"2021-12-01T08:20:46.208887Z","iopub.status.idle":"2021-12-01T08:20:46.396317Z","shell.execute_reply.started":"2021-12-01T08:20:46.208835Z","shell.execute_reply":"2021-12-01T08:20:46.395164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### As the years go by, the number of games increases","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=15)\nplt.figure(figsize=(8, 6))\n\nax = sns.countplot(x='month', data=games)\nwrite_percent(ax, len(games))\nax.set_title('Number of games for month');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.398148Z","iopub.execute_input":"2021-12-01T08:20:46.398589Z","iopub.status.idle":"2021-12-01T08:20:46.552162Z","shell.execute_reply.started":"2021-12-01T08:20:46.398542Z","shell.execute_reply":"2021-12-01T08:20:46.551125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### The game was held from September to January. There are especially many games in December, and they are rarely held in January","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(15, 7))\n\nax = sns.countplot(x='day', data=games)\nwrite_percent(ax, len(games))\nax.set_title('Number of games for day');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.553949Z","iopub.execute_input":"2021-12-01T08:20:46.554415Z","iopub.status.idle":"2021-12-01T08:20:46.962872Z","shell.execute_reply.started":"2021-12-01T08:20:46.554367Z","shell.execute_reply":"2021-12-01T08:20:46.961834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(15, 7))\n\nax = sns.countplot(x='gameTimeEastern', data=games)\nwrite_percent(ax, len(games))\nax.set_title('Number of games for gameTimeEastern');\nax.tick_params('x', labelrotation=30) # rotate 30 degree of x label","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:46.964754Z","iopub.execute_input":"2021-12-01T08:20:46.965306Z","iopub.status.idle":"2021-12-01T08:20:47.239483Z","shell.execute_reply.started":"2021-12-01T08:20:46.965255Z","shell.execute_reply":"2021-12-01T08:20:47.238464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(15, 7))\n\nax = sns.countplot(x='hour', data=games)\nwrite_percent(ax, len(games))\nax.set_title('Number of games for hour');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.240868Z","iopub.execute_input":"2021-12-01T08:20:47.241264Z","iopub.status.idle":"2021-12-01T08:20:47.545245Z","shell.execute_reply.started":"2021-12-01T08:20:47.241226Z","shell.execute_reply":"2021-12-01T08:20:47.544229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### The most games were held at 1, 4, and 8","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(15, 7))\n\nax = sns.countplot(x='week', data=games)\nwrite_percent(ax, len(games))\nax.set_title('Number of games for week');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.546756Z","iopub.execute_input":"2021-12-01T08:20:47.547061Z","iopub.status.idle":"2021-12-01T08:20:47.804472Z","shell.execute_reply.started":"2021-12-01T08:20:47.547032Z","shell.execute_reply":"2021-12-01T08:20:47.803546Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

2. Player Data

","metadata":{}},{"cell_type":"markdown","source":"### **Player data:** The players.csv file contains player-level information from players that participated in any of the tracking data files. The key variable is nflId\n\n* **nflId:** Player identification number, unique across players (numeric)\n\n* **height:** Player height (text)\n\n* **weight:** Player weight (numeric)\n\n* **birthDate:** Date of birth (YYYY-MM-DD)\n\n* **collegeName:** Player college (text)\n\n* **position:** Player position (text)\n\n* **displayName:** Player name (text)","metadata":{}},{"cell_type":"code","source":"players = pd.read_csv('../input/nfl-big-data-bowl-2022/players.csv')\nplayers","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.805812Z","iopub.execute_input":"2021-12-01T08:20:47.806151Z","iopub.status.idle":"2021-12-01T08:20:47.846227Z","shell.execute_reply.started":"2021-12-01T08:20:47.806118Z","shell.execute_reply":"2021-12-01T08:20:47.845431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players = downcast(players)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.847395Z","iopub.execute_input":"2021-12-01T08:20:47.847861Z","iopub.status.idle":"2021-12-01T08:20:47.857146Z","shell.execute_reply.started":"2021-12-01T08:20:47.847817Z","shell.execute_reply":"2021-12-01T08:20:47.856164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"resumetable(players)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.858824Z","iopub.execute_input":"2021-12-01T08:20:47.859423Z","iopub.status.idle":"2021-12-01T08:20:47.887452Z","shell.execute_reply.started":"2021-12-01T08:20:47.859379Z","shell.execute_reply":"2021-12-01T08:20:47.886436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Convert all heights to feet","metadata":{}},{"cell_type":"code","source":"check = players['height'].str.split('-', expand=True)\n\ncheck.columns = ['first', 'second']\n\ncheck.loc[(check['second'].notnull()), 'first'] = check[check['second'].notnull()]['first'].astype(np.int16) * 12 + check[check['second'].notnull()]['second'].astype(np.int16)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.888934Z","iopub.execute_input":"2021-12-01T08:20:47.889285Z","iopub.status.idle":"2021-12-01T08:20:47.932242Z","shell.execute_reply.started":"2021-12-01T08:20:47.889250Z","shell.execute_reply":"2021-12-01T08:20:47.931313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players['height'] = check['first']\nplayers['height'] = players['height'].astype(np.float32)\nplayers['height'] /= 12\n\nplayers","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.933343Z","iopub.execute_input":"2021-12-01T08:20:47.933773Z","iopub.status.idle":"2021-12-01T08:20:47.955824Z","shell.execute_reply.started":"2021-12-01T08:20:47.933741Z","shell.execute_reply":"2021-12-01T08:20:47.954744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=15) \nplt.figure(figsize=(10, 6))\n\nax = sns.distplot(players['height'], bins=12)\nax.set_title('Height Distribution');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:47.957476Z","iopub.execute_input":"2021-12-01T08:20:47.958033Z","iopub.status.idle":"2021-12-01T08:20:48.157418Z","shell.execute_reply.started":"2021-12-01T08:20:47.957996Z","shell.execute_reply":"2021-12-01T08:20:48.156582Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=15) \nplt.figure(figsize=(10, 6))\n\nax = sns.distplot(players['weight'])\nax.set_title('Weight Distribution');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:48.160584Z","iopub.execute_input":"2021-12-01T08:20:48.161004Z","iopub.status.idle":"2021-12-01T08:20:48.370934Z","shell.execute_reply.started":"2021-12-01T08:20:48.160968Z","shell.execute_reply":"2021-12-01T08:20:48.370097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"top_players_colleage = players['collegeName'].value_counts()[:20].reset_index()\ntop_players_colleage.columns = ['collageName', 'numberOfPlayers']","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:48.372459Z","iopub.execute_input":"2021-12-01T08:20:48.372766Z","iopub.status.idle":"2021-12-01T08:20:48.381798Z","shell.execute_reply.started":"2021-12-01T08:20:48.372735Z","shell.execute_reply":"2021-12-01T08:20:48.380590Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=10) \nplt.figure(figsize=(15, 12))\n\nax = sns.barplot(x='numberOfPlayers', y='collageName', data=top_players_colleage)\nax.set_title('Number of players for collegeName');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:48.383231Z","iopub.execute_input":"2021-12-01T08:20:48.383875Z","iopub.status.idle":"2021-12-01T08:20:48.694023Z","shell.execute_reply.started":"2021-12-01T08:20:48.383826Z","shell.execute_reply":"2021-12-01T08:20:48.693205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Create birth year feature","metadata":{}},{"cell_type":"code","source":"players['birthYear'] = 0","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:48.695494Z","iopub.execute_input":"2021-12-01T08:20:48.696138Z","iopub.status.idle":"2021-12-01T08:20:48.702190Z","shell.execute_reply.started":"2021-12-01T08:20:48.696087Z","shell.execute_reply":"2021-12-01T08:20:48.701350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are NA values in birthDate so that we should drop them","metadata":{}},{"cell_type":"code","source":"players.dropna(subset=['birthDate'], inplace=True)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:48.703327Z","iopub.execute_input":"2021-12-01T08:20:48.703627Z","iopub.status.idle":"2021-12-01T08:20:48.716599Z","shell.execute_reply.started":"2021-12-01T08:20:48.703599Z","shell.execute_reply":"2021-12-01T08:20:48.715559Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Extract birth year","metadata":{}},{"cell_type":"code","source":"for idx, row in players.iterrows():\n if len(row['birthDate'].split('/')) == 3: # ex) 05/17/1994 \n players.loc[idx, 'birthYear'] = row['birthDate'].split('/')[2]\n \n elif len(row['birthDate'].split('-')) == 3: # ex) 1995-05-05\n players.loc[idx, 'birthYear'] = row['birthDate'].split('-')[0]","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:48.718004Z","iopub.execute_input":"2021-12-01T08:20:48.718376Z","iopub.status.idle":"2021-12-01T08:20:49.849890Z","shell.execute_reply.started":"2021-12-01T08:20:48.718343Z","shell.execute_reply":"2021-12-01T08:20:49.849031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=15) \nplt.figure(figsize=(10, 5))\n\nax = sns.distplot(players['birthYear'], bins=25)\nax.set_title('Players birth year Distribution');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:49.851402Z","iopub.execute_input":"2021-12-01T08:20:49.851835Z","iopub.status.idle":"2021-12-01T08:20:50.080166Z","shell.execute_reply.started":"2021-12-01T08:20:49.851790Z","shell.execute_reply":"2021-12-01T08:20:50.079022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Those born in 1995 are the most common","metadata":{}},{"cell_type":"code","source":"players['birthYear'].min(), players['birthYear'].max()","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.082141Z","iopub.execute_input":"2021-12-01T08:20:50.082617Z","iopub.status.idle":"2021-12-01T08:20:50.092456Z","shell.execute_reply.started":"2021-12-01T08:20:50.082570Z","shell.execute_reply":"2021-12-01T08:20:50.091335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### The oldest player was born in 1972, and the youngest player was born in 1999","metadata":{}},{"cell_type":"markdown","source":"\n

3. Play Data

","metadata":{}},{"cell_type":"markdown","source":"### **Play data:** The plays.csv file contains play-level information from each game. The key variables are gameId and playId\n- gameId: Game identifier, unique (numeric)\n- playId: Play identifier, not unique across games (numeric)\n- playDescription: Description of play (text)\n- quarter: Game quarter (numeric)\n- down: Down (numeric)\n- yardsToGo: Distance needed for a first down (numeric)\n- possessionTeam: Team punting, placekicking or kicking off the ball (text)\n- specialTeamsPlayType: Formation of play: Extra Point, Field Goal, Kickoff or Punt (text)\n- specialTeamsPlayResult: Special Teams outcome of play dependent on play type: Blocked Kick Attempt, Blocked Punt, Downed, Fair Catch, Kick Attempt Good, Kick Attempt No Good, Kickoff Team Recovery, Muffed, Non-Special Teams Result, Out of Bounds, Return or Touchback (text)\n- kickerId: nflId of placekicker, punter or kickoff specialist on play (numeric)\n- returnerId: nflId(s) of returner(s) on play if there was a special teams return. Multiple returners on a play are separated by a ; (text)\n- kickBlockerId: nflId of blocker of kick on play if there was a blocked field goal or blocked punt (numeric)\n- yardlineSide: 3-letter team code corresponding to line-of-scrimmage (text)\n- yardlineNumber: Yard line at line-of-scrimmage (numeric) \n- gameClock: Time on clock of play (MM:SS)\n- penaltyCodes: NFL categorization of the penalties that occurred on the play. Multiple penalties on a play are separated by a ; (text)\n- penaltyJerseyNumber: Jersey number and team code of the player committing each penalty. Multiple penalties on a play are separated by a ; (text)\n- penaltyYards: yards gained by possessionTeam by penalty (numeric)\n- preSnapHomeScore: Home score prior to the play (numeric)\n- preSnapVisitorScore: Visiting team score prior to the play (numeric)\n- passResult: Scrimmage outcome of the play if specialTeamsPlayResult is \"Non-Special Teams Result\" (C: Complete pass, I: Incomplete pass, S: Quarterback sack, IN: Intercepted pass, R: Scramble, ' ': Designed Rush, text)\n- kickLength: Kick length in air of kickoff, field goal or punt (numeric)\n- kickReturnYardage: Yards gained by return team if there was a return on a kickoff or punt (numeric)\n- playResult: Net yards gained by the kicking team, including penalty yardage (numeric)\n- absoluteYardlineNumber: Location of ball downfield in tracking data coordinates (numeric)","metadata":{}},{"cell_type":"code","source":"plays = pd.read_csv('../input/nfl-big-data-bowl-2022/plays.csv')\n\nplays","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.094139Z","iopub.execute_input":"2021-12-01T08:20:50.094615Z","iopub.status.idle":"2021-12-01T08:20:50.311148Z","shell.execute_reply.started":"2021-12-01T08:20:50.094569Z","shell.execute_reply":"2021-12-01T08:20:50.309892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plays = downcast(plays)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.312730Z","iopub.execute_input":"2021-12-01T08:20:50.313176Z","iopub.status.idle":"2021-12-01T08:20:50.341167Z","shell.execute_reply.started":"2021-12-01T08:20:50.313130Z","shell.execute_reply":"2021-12-01T08:20:50.340295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"resumetable(plays)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.342192Z","iopub.execute_input":"2021-12-01T08:20:50.342616Z","iopub.status.idle":"2021-12-01T08:20:50.429495Z","shell.execute_reply.started":"2021-12-01T08:20:50.342585Z","shell.execute_reply":"2021-12-01T08:20:50.428572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### There are lots of null values in `returnerld`, `kickBlockerId`, `penaltyCodes`, `penaltyJerseyNumbers`, `penaltyYards`, `passResult`, `kickReturnYardage` features","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(12, 6))\n\nax = sns.countplot(x='quarter', data=plays)\nwrite_percent(ax, len(plays))\nax.set_title('Number of plays of every quarter');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.432151Z","iopub.execute_input":"2021-12-01T08:20:50.432612Z","iopub.status.idle":"2021-12-01T08:20:50.608963Z","shell.execute_reply.started":"2021-12-01T08:20:50.432563Z","shell.execute_reply":"2021-12-01T08:20:50.607997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(12, 6))\n\nax = sns.countplot(x='down', data=plays)\nwrite_percent(ax, len(plays))\nax.set_title('Number of plays of every down');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.615422Z","iopub.execute_input":"2021-12-01T08:20:50.615773Z","iopub.status.idle":"2021-12-01T08:20:50.777493Z","shell.execute_reply.started":"2021-12-01T08:20:50.615740Z","shell.execute_reply":"2021-12-01T08:20:50.776320Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=12) \nplt.figure(figsize=(12, 6))\n\nax = sns.countplot(x='yardsToGo', data=plays)\nax.set_title('Number of plays for every yards to go category');","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:50.779106Z","iopub.execute_input":"2021-12-01T08:20:50.779444Z","iopub.status.idle":"2021-12-01T08:20:51.142934Z","shell.execute_reply.started":"2021-12-01T08:20:50.779411Z","shell.execute_reply":"2021-12-01T08:20:51.142054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mpl.rc('font', size=15) \nplt.figure(figsize=(10, 5))\n\nax = sns.distplot(plays['playResult'], bins=25);\nax.set_title('playResult Distribution'); ","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:51.144472Z","iopub.execute_input":"2021-12-01T08:20:51.144796Z","iopub.status.idle":"2021-12-01T08:20:51.371111Z","shell.execute_reply.started":"2021-12-01T08:20:51.144765Z","shell.execute_reply":"2021-12-01T08:20:51.369884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"playResult: Net yards gained by the offense, including penalty yardage (numeric)","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=15) \nplt.figure(figsize=(10, 5))\n\nax = sns.distplot(plays['preSnapHomeScore'], bins=12);\nax.set_title('preSnapHomeScore Distribution'); ","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:51.372773Z","iopub.execute_input":"2021-12-01T08:20:51.373249Z","iopub.status.idle":"2021-12-01T08:20:51.688291Z","shell.execute_reply.started":"2021-12-01T08:20:51.373201Z","shell.execute_reply":"2021-12-01T08:20:51.687173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"preSnapHomeScore: Home score prior to the play (numeric)","metadata":{}},{"cell_type":"code","source":"mpl.rc('font', size=15) \nplt.figure(figsize=(10, 5))\n\nax = sns.distplot(plays['preSnapVisitorScore'], bins=12);\nax.set_title('preSnapVisitorScore Distribution'); ","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:51.689522Z","iopub.execute_input":"2021-12-01T08:20:51.689855Z","iopub.status.idle":"2021-12-01T08:20:51.889448Z","shell.execute_reply.started":"2021-12-01T08:20:51.689824Z","shell.execute_reply":"2021-12-01T08:20:51.888422Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"preSnapVisitorScore: Visiting team score prior to the play (numeric)","metadata":{}},{"cell_type":"markdown","source":"\n

4. Tracking Data

","metadata":{}},{"cell_type":"code","source":"tracking2018 = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2018.csv')\ntracking2018.head()","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:20:51.890500Z","iopub.execute_input":"2021-12-01T08:20:51.890787Z","iopub.status.idle":"2021-12-01T08:21:39.404964Z","shell.execute_reply.started":"2021-12-01T08:20:51.890758Z","shell.execute_reply":"2021-12-01T08:21:39.403971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tracking2018 = downcast(tracking2018)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:39.406287Z","iopub.execute_input":"2021-12-01T08:21:39.406619Z","iopub.status.idle":"2021-12-01T08:21:43.222301Z","shell.execute_reply.started":"2021-12-01T08:21:39.406587Z","shell.execute_reply":"2021-12-01T08:21:43.220977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This function is taken from the post created by ROB MULLA. See the post [here](https://www.kaggle.com/robikscube/nfl-big-data-bowl-2022-twitch-stream-eda). Thank you ROB MULLA :)","metadata":{}},{"cell_type":"markdown","source":"#### 2018123000 and playId == 36","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12, 8))\ntracking2018.query('gameId == 2018123000 and playId == 36').groupby('team') \\\n .plot(x='x', y='y', ax=ax, style='.')\nplt.legend().remove();","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:43.223833Z","iopub.execute_input":"2021-12-01T08:21:43.224272Z","iopub.status.idle":"2021-12-01T08:21:43.979520Z","shell.execute_reply.started":"2021-12-01T08:21:43.224236Z","shell.execute_reply":"2021-12-01T08:21:43.978723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### gameId == 2018091001 and playId == 4033","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12, 8))\ntracking2018.query('gameId == 2018091001 and playId == 4033').groupby('team') \\\n .plot(x='x', y='y', ax=ax, style='.')\nplt.legend().remove();","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:43.980594Z","iopub.execute_input":"2021-12-01T08:21:43.981010Z","iopub.status.idle":"2021-12-01T08:21:44.295838Z","shell.execute_reply.started":"2021-12-01T08:21:43.980980Z","shell.execute_reply":"2021-12-01T08:21:44.294604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### gameId == 2018091609 and position == \"CB\"","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12, 8))\ntracking2018.query('gameId == 2018091609 and position == \"CB\"').groupby('team') \\\n .plot(x='x', y='y', ax=ax, style='.')\nplt.legend().remove();","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:44.297595Z","iopub.execute_input":"2021-12-01T08:21:44.298055Z","iopub.status.idle":"2021-12-01T08:21:45.004474Z","shell.execute_reply.started":"2021-12-01T08:21:44.298005Z","shell.execute_reply":"2021-12-01T08:21:45.003517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### gameId == 2018091609 and position == \"LB\"","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12, 8))\ntracking2018.query('gameId == 2018091609 and position == \"LB\"').groupby('team') \\\n .plot(x='x', y='y', ax=ax, style='.')\nplt.legend().remove();","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:45.005823Z","iopub.execute_input":"2021-12-01T08:21:45.006186Z","iopub.status.idle":"2021-12-01T08:21:45.736832Z","shell.execute_reply.started":"2021-12-01T08:21:45.006153Z","shell.execute_reply":"2021-12-01T08:21:45.735822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### gameId == 2018091609 and position == \"RB\"","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12, 8))\ntracking2018.query('gameId == 2018091609 and position == \"RB\"').groupby('team') \\\n .plot(x='x', y='y', ax=ax, style='.')\nplt.legend().remove();","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:45.738244Z","iopub.execute_input":"2021-12-01T08:21:45.738557Z","iopub.status.idle":"2021-12-01T08:21:46.473646Z","shell.execute_reply.started":"2021-12-01T08:21:45.738526Z","shell.execute_reply":"2021-12-01T08:21:46.472514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Function to create football field\n","metadata":{}},{"cell_type":"markdown","source":"This function is taken from the post created by JARON_MICHAL. See the post [here](https://www.kaggle.com/jaronmichal/tracking-data-visualization)","metadata":{}},{"cell_type":"code","source":"import matplotlib.patches as patches\nfrom matplotlib.patches import Arc\nfrom matplotlib import pyplot as plt\nimport matplotlib.patches as mpatches\n\n# Change size of the figure\nplt.rcParams['figure.figsize'] = [12, 8]\ndef drawPitch(width, height, color=\"w\"):\n fig = plt.figure()\n ax = plt.axes(xlim=(-10, width + 30), ylim=(-15, height + 5))\n plt.axis('off')\n\n # Grass around pitch\n rect = patches.Rectangle((-10, -5), width + 40, height + 10, linewidth=1, facecolor='#3f995b', capstyle='round')\n ax.add_patch(rect)\n ###################\n\n # Pitch boundaries\n rect = plt.Rectangle((0, 0), width + 20, height, ec=color, fc=\"None\", lw=2)\n ax.add_patch(rect)\n ###################\n\n # vertical lines - every 5 yards\n for i in range(21):\n plt.plot([10 + 5 * i, 10 + 5 * i], [0, height], c=\"w\", lw=2)\n ###################\n \n # distance markers - every 10 yards\n for yards in range(10, width, 10):\n yards_text = yards if yards <= width / 2 else width - yards\n # top markers\n plt.text(10 + yards - 2, height - 7.5, yards_text, size=15, c=\"w\", weight=\"bold\")\n # botoom markers\n plt.text(10 + yards - 2, 7.5, yards_text, size=15, c=\"w\", weight=\"bold\", rotation=180)\n ###################\n\n # yards markers - every yard\n # bottom markers\n for x in range(20):\n for j in range(1, 5):\n plt.plot([10 + x * 5 + j, 10 + x * 5 + j], [1, 3], color=\"w\", lw=2)\n\n # top markers\n for x in range(20):\n for j in range(1, 5):\n plt.plot([10 + x * 5 + j, 10 + x * 5 + j], [height - 1, height - 3], color=\"w\", lw=2)\n\n # middle bottom markers\n y = (height - 18.5) / 2\n for x in range(20):\n for j in range(1, 5):\n plt.plot([10 + x * 5 + j, 10 + x * 5 + j], [y, y + 2], color=\"w\", lw=2)\n\n # middle top markers\n for x in range(20):\n for j in range(1, 5):\n plt.plot([10 + x * 5 + j, 10 + x * 5 + j], [height - y, height - y - 2], color=\"w\", lw=2)\n ###################\n\n # draw home end zone\n plt.text(2.5, (height - 15) / 2, \"HOME\", size=30, c=\"w\", weight=\"bold\", rotation=90)\n rect = plt.Rectangle((0, 0), 10, height, ec=color, fc=\"#0064dc\", lw=2)\n ax.add_patch(rect)\n\n # draw away end zone \n plt.text(111, (height - 15) / 2, \"AWAY\", size=30, c=\"w\", weight=\"bold\", rotation=-90)\n rect = plt.Rectangle((width + 10, 0), 10, height, ec=color, fc=\"#c80014\", lw=2)\n ax.add_patch(rect)\n ###################\n \n # draw extra spot point\n # left\n y = (height - 3) / 2\n plt.plot([10 + 2, 10 + 2], [y, y + 3], c=\"w\", lw=2)\n \n # right\n plt.plot([width + 10 - 2, width + 10 - 2], [y, y + 3], c=\"w\", lw=2)\n ###################\n \n # draw goalpost\n goal_width = 6 # yards\n y = (height - goal_width) / 2\n # left\n plt.plot([0, 0], [y, y + goal_width], \"-\", c=\"y\", lw=10, ms=20)\n # right\n plt.plot([width + 20, width + 20], [y, y + goal_width], \"-\", c=\"y\", lw=10, ms=20)\n \n return fig, ax","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-12-01T08:21:46.476313Z","iopub.execute_input":"2021-12-01T08:21:46.477009Z","iopub.status.idle":"2021-12-01T08:21:46.512303Z","shell.execute_reply.started":"2021-12-01T08:21:46.476956Z","shell.execute_reply":"2021-12-01T08:21:46.511500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" fig, ax = drawPitch(100, 53.3)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:46.513695Z","iopub.execute_input":"2021-12-01T08:21:46.514337Z","iopub.status.idle":"2021-12-01T08:21:47.171019Z","shell.execute_reply.started":"2021-12-01T08:21:46.514287Z","shell.execute_reply":"2021-12-01T08:21:47.170202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Function to create animation","metadata":{}},{"cell_type":"code","source":"games_ids = {}\ngames_tracking2018 = tracking2018.groupby(by=[\"gameId\"])\nfor game, data in games_tracking2018:\n games_ids[game] = list(set(data.playId.tolist()))","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:47.172049Z","iopub.execute_input":"2021-12-01T08:21:47.172470Z","iopub.status.idle":"2021-12-01T08:21:49.578917Z","shell.execute_reply.started":"2021-12-01T08:21:47.172438Z","shell.execute_reply":"2021-12-01T08:21:49.577864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def extract_one_game(game_id, play_id, df):\n game = df[(df.gameId == game_id) & (df.playId == play_id)]\n home = {}\n away = {}\n balls = []\n \n players = game.sort_values(['frameId'], ascending=True).groupby('nflId')\n for id, dx in players:\n jerseyNumber = int(dx.jerseyNumber.iloc[0])\n if dx.team.iloc[0] == \"home\":\n home[jerseyNumber] = list(zip(dx.x.tolist(), dx.y.tolist()))\n elif dx.team.iloc[0] == \"away\":\n away[jerseyNumber] = list(zip(dx.x.tolist(), dx.y.tolist()))\n\n\n ball_df = game.sort_values(['frameId'], ascending=True) \n ball_df = ball_df[ball_df.team == \"football\"]\n balls = list(zip(ball_df.x.tolist(), ball_df.y.tolist()))\n return home, away, balls","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-12-01T08:21:49.580434Z","iopub.execute_input":"2021-12-01T08:21:49.580769Z","iopub.status.idle":"2021-12-01T08:21:49.592733Z","shell.execute_reply.started":"2021-12-01T08:21:49.580736Z","shell.execute_reply":"2021-12-01T08:21:49.591320Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from matplotlib import animation\nfrom IPython.display import HTML\ndef animate_one_play(game_id, play_id, df):\n fig, ax = drawPitch(100, 53.3)\n \n home, away, balls = extract_one_game(game_id, play_id, df)\n\n team_left, = ax.plot([], [], 'o', markersize=20, markerfacecolor=\"r\", markeredgewidth=2, markeredgecolor=\"white\", zorder=7)\n team_right, = ax.plot([], [], 'o', markersize=20, markerfacecolor=\"b\", markeredgewidth=2, markeredgecolor=\"white\", zorder=7)\n ball, = ax.plot([], [], 'o', markersize=10, markerfacecolor=\"black\", markeredgewidth=2, markeredgecolor=\"white\", zorder=7)\n drawings = [team_left, team_right, ball]\n\n def init():\n team_left.set_data([], [])\n team_right.set_data([], [])\n ball.set_data([], [])\n return drawings\n\n def draw_teams(i):\n X = []\n Y = []\n for k, v in home.items():\n x, y = v[i]\n X.append(x)\n Y.append(y)\n team_left.set_data(X, Y)\n \n X = []\n Y = []\n for k, v in away.items():\n x, y = v[i]\n X.append(x)\n Y.append(y)\n team_right.set_data(X, Y)\n\n def animate(i):\n draw_teams(i)\n \n x, y = balls[i]\n ball.set_data([x, y])\n return drawings\n \n # !May take a while!\n anim = animation.FuncAnimation(fig, animate, init_func=init,\n frames=len(balls), interval=100, blit=True)\n\n return HTML(anim.to_html5_video())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-12-01T08:21:49.594344Z","iopub.execute_input":"2021-12-01T08:21:49.594681Z","iopub.status.idle":"2021-12-01T08:21:49.621421Z","shell.execute_reply.started":"2021-12-01T08:21:49.594650Z","shell.execute_reply":"2021-12-01T08:21:49.620149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"animate_one_play(2018123000, 36, tracking2018)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T08:21:49.623275Z","iopub.execute_input":"2021-12-01T08:21:49.623763Z","iopub.status.idle":"2021-12-01T08:22:02.828776Z","shell.execute_reply.started":"2021-12-01T08:21:49.623713Z","shell.execute_reply":"2021-12-01T08:22:02.827658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Are the proposed findings actionable?\n#### Is this a way of looking at tracking data that is novel?\n\nMy notebook is to generate actionable, practical, and novel insights from player tracking data that corresponds to special teams play. \n\nThere are several points but are not limited to:\n\n* Create a new special teams metric. \n* Quantify special teams strategy. \n* Rank special teams players. \nSubmissions that examine one idea more thoroughly are preferred versus those that examine several ideas somewhat thoroughly.\n\n\n#### Is this project creative?\n\nWell, this competition has both creative and analytical properties.","metadata":{}}]}