{"cells":[{"metadata":{"trusted":true,"_uuid":"12f519d145f90ccdfcc3f90029734dc7c09b2b68"},"cell_type":"markdown","source":"# NFL Punt Analytics Competition - Starter EDA\n\n![punt](https://nbcprofootballtalk.files.wordpress.com/2018/12/gettyimages-1067771316-e1543962247286.jpg?w=560&h=316&crop=1)\n\nIn this competition we are tasked with analzying punt plays for player safety and proposing rule changes."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pylab as plt\nimport os\nplt.style.use('ggplot')\n\n# For interactive plots\nfrom plotly import offline\nimport plotly.graph_objs as go\n\n\npd.set_option('max.columns', None)\noffline.init_notebook_mode()\nconfig = dict(showLink=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c1bde655644a2af68eab1accb1472c8e2a653ba9"},"cell_type":"markdown","source":"## Data Overview\n- **Game Data**: Game level data that specifies the type of season (pre, reg, post), week, and hosting city and team. Each game is uniquely identified across all seasons\nusing `GameKey`.\n- **Play Information**: Play level data that describes the type of play, possession team, score and a brief narrative of each play. Plays are uniquely identified using a\nits `PlayID` along with the corresponding `GameKey`. `PlayIDs` are not unique.\n- **Player Punt Data**: Player level data that specifies the traditional football position for each player. Each player is identified using his `GSISID`.\n- **Play Player Role Data**: Play and player level data that specifies a punt specific player role. This dataset will specify each player that played in each play. A\nplayer’s role in a play is uniquely defined by the `Gamekey` `PlayID` and `GSISID`.\n- **Video Review**: Injury level data that provides a detailed description of the concussion-producing event. Video Review data are only available in cases in which the\ninjury play can be identified. Each video review case can be identified using a combination of `GameKey`, `PlayID`, and `GSISID`. A brief narrative of the play events is\nprovided.\n- **NGS**: Next Gen Stats – player level data that describes the movement of each player during a play. NGS data is processed by BIOCORE to produce relevant speed and\ndirection data. The NGS data is identified using `GameKey`, `PlayID`, and `GSISID`. Player data for each play is provided as a function of time (Time) for the duration of\nthe play.\n\n![](https://storage.googleapis.com/kaggle-media/competitions/NFL%20player%20safety%20analytics/key_variables.jpg)"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"## Lets explore the smaller datasets"},{"metadata":{"trusted":true,"_uuid":"3c7259c74843ca0d1365a1e2a9503d74cc1861bf"},"cell_type":"code","source":"# Read the input data\nppd = pd.read_csv('../input/player_punt_data.csv')\ngd = pd.read_csv('../input/game_data.csv')\npprd = pd.read_csv('../input/play_player_role_data.csv')\nvr = pd.read_csv('../input/video_review.csv')\nvfi = pd.read_csv('../input/video_footage-injury.csv')\npi = pd.read_csv('../input/play_information.csv')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"230c424324a5fadee72db07e33556eed74b644c4"},"cell_type":"markdown","source":"# Game Data `gd` for 666 games\n\n* Season_Year: NFL Season (YYYY)\n* Season_Type: Preseason (Pre), Regular season (Reg), Post season (Post)\n* GameKey: Numeric game identifier, unique across seasons (####)\n* Game_Date: Date of game (MM/DD/YYYY)\n* Week: Week number, defined within Season_Type (##)\n* Game_Day: Day of the week (text)\n* Game_Site: Hosting city (text)\n* Start_Time: Time at start of game (24:00:00)\n* Home_Team: Long format of hosting team (text)\n* HomeTeamCode Team abbreviation (ABC)\n* Visit_Team: Long format of visiting team (text)\n* VisitTeamCode: Team abbreviation (ABC)\n* Stadium: Stadium Name (text)\n* StadiumType: Indoor, outdoor or closure type - semi categorical (text)\n* Turf: Natural or type of synthetic turf (text)\n* GameWeather: Description of indoor weather, same as \"OutdoorWeather\" for open stadiums (text)\n* Temperature: Stadium temperature (Degrees F)\n* OutdoorWeather: Description of outdoor weather (free-form text)"},{"metadata":{"trusted":true,"_uuid":"954e184e103b3f20581f39e2d037396e354c8a65"},"cell_type":"code","source":"gd.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d98251f4718fd088c3b20c34de6043597e87b8b4"},"cell_type":"code","source":"gd.plot(kind='scatter', x='Week', y='Temperature', figsize=(15, 5), title='NFL Game Data Week vs. Temperature')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7055c6df86ef32644ba2bb2165983e033e2699eb"},"cell_type":"code","source":"gd['count'] = 1\ngd.groupby('Turf') \\\n    .count()[['count']] \\\n    .sort_values('count', ascending=False) \\\n    .plot(kind='bar', figsize=(15, 5), rot=85, title='Count of Games by Turf Type')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bd95de4c2c51f1e67f21fe1aa6777fec688c55cb"},"cell_type":"code","source":"gd.groupby('Start_Time') \\\n    .count()[['count']] \\\n    .plot(kind='bar', figsize=(15, 5), rot=85, title='Count of Games by Start Time', color='g')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5f0cc4f71aef8ad028b9169beb74d8c169af898c"},"cell_type":"markdown","source":"# Video Review Data (*concussion-producing event*)\n- 37 Video Review plays\n- 6 Have Friendly Fire, 28 don't, and 1 it's unclear "},{"metadata":{"trusted":true,"_uuid":"1ea0de47b9c69fe62f20c4379ea262b4b58a3c74"},"cell_type":"code","source":"vr.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dc2b2f901bba953cc596a46699f6767a27ab55af"},"cell_type":"code","source":"vr.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8125534feee825c6944a45548a73d7c176a0956f"},"cell_type":"code","source":"vr['count'] = 1\nvr.groupby('Player_Activity_Derived') \\\n    .count()[['count']] \\\n    .sort_values('count', ascending=False) \\\n    .plot(kind='barh', figsize=(15, 5), title='Count of Player Activity Derived')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0baccb8e0f1e75e6b56b75ef5245cd96dbaf7c89"},"cell_type":"code","source":"vr['count'] = 1\nvr.groupby('Primary_Partner_Activity_Derived') \\\n    .count()[['count']] \\\n    .sort_values('count', ascending=False) \\\n    .plot(kind='barh', figsize=(15, 5), title='Count of Primary Partner Activity Derived', color='g')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"64636d8428890b56bdaa5d96bcf8bf32b20def30"},"cell_type":"code","source":"vr['count'] = 1\nvr.groupby('Primary_Impact_Type') \\\n    .count()[['count']] \\\n    .sort_values('count', ascending=False) \\\n    .plot(kind='barh', figsize=(15, 5), title='Count of Primary Impact Type', color='b')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5a9cd7de91d9cf8d01e75d21d5fad8580199aa8b"},"cell_type":"markdown","source":"# Play information Dataset\n- 6681 Plays with information\n- Team names and Score\n- Position on the field\n- Time of play\n- All plays are punts"},{"metadata":{"trusted":true,"_uuid":"325e0165d0655173bf79c8a9a3c2d05c9b27057b"},"cell_type":"code","source":"pi.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"310a90b79fcc53eae99ae00b4ba1c0e3faf66ba3"},"cell_type":"code","source":"pi['count'] = 1\npi.groupby('Poss_Team').count()[['count']] \\\n    .sort_values('count', ascending=False) \\\n    .plot(kind='bar', figsize=(15, 5), title='Count of punts per team', color='k')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f13edec2dd6c5432f018ced9b8f62de34a2a67a0"},"cell_type":"code","source":"# They are all punts!!! :D\npi['Play_Type'].unique()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ba8e317c22da59e7bfab29b8789a1af1b37a9e2e"},"cell_type":"markdown","source":"# Using @crawford 's starter code\n## Using provided starter code from here: https://www.kaggle.com/crawford/nfl-punt-analytics-starter-kernel"},{"metadata":{"trusted":true,"_uuid":"938add918858997ffe3d43870cdb87d113b53875","_kg_hide-input":true},"cell_type":"code","source":"# Loading and plotting functions\n\ndef load_plays_for_game(GameKey):\n    \"\"\"\n    Returns a dataframe of play data for a given game (GameKey)\n    \"\"\"\n    play_information = pd.read_csv('../input/play_information.csv')\n    play_information = play_information[play_information['GameKey'] == GameKey]\n    return play_information\n\n\ndef load_game_and_ngs(ngs_file=None, GameKey=None):\n    \"\"\"\n    Returns a dataframe of player movements (NGS data) for a given game\n    \"\"\"\n    if ngs_file is None:\n        print(\"Specifiy an NGS file.\")\n        return None\n    if GameKey is None:\n        print('Specify a GameKey')\n        return None\n    # Merge play data with NGS data    \n    plays = load_plays_for_game(GameKey)\n    ngs = pd.read_csv(ngs_file, low_memory=False)\n    merged = pd.merge(ngs, plays, how=\"inner\", on=[\"GameKey\", \"PlayID\", \"Season_Year\"])\n    return merged\n\n\ndef plot_play(game_df, PlayID, player1=None, player2=None, custom_layout=False):\n    \"\"\"\n    Plots player movements on the field for a given game, play, and two players\n    \"\"\"\n    game_df = game_df[game_df.PlayID==PlayID]\n    \n    GameKey=str(pd.unique(game_df.GameKey)[0])\n    HomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\n    VisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\n    YardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n    \n    traces=[]   \n    if (player1 is not None) & (player2 is not None):\n        game_df = game_df[ (game_df['GSISID']==player1) | (game_df['GSISID']==player2)]\n        for player in pd.unique(game_df.GSISID):\n            player = int(player)\n            trace = go.Scatter(\n                x = game_df[game_df.GSISID==player].x,\n                y = game_df[game_df.GSISID==player].y,\n                name='GSISID '+str(player),\n                mode='markers'\n            )\n            traces.append(trace)\n    else:\n        print(\"Specify GSISIDs for player1 and player2\")\n        return None\n    \n    if custom_layout is not True:\n        layout = load_layout()\n        layout['title'] =  HomeTeam + \\\n        ' vs. ' + VisitingTeam + \\\n        '<br>Possession: ' + \\\n        YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\n    data = traces\n    fig = dict(data=data, layout=layout)\n    play_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\n    print(\"\\n\\n\\t\",play_description)\n    offline.iplot(fig, config=config)\n    \ndef load_layout():\n    \"\"\"\n    Returns a dict for a Football themed Plot.ly layout \n    \"\"\"\n    layout = dict(\n        title = \"Player Activity\",\n        plot_bgcolor='darkseagreen',\n        showlegend=True,\n        xaxis=dict(\n            autorange=False,\n            range=[0, 120],\n            showgrid=False,\n            zeroline=False,\n            showline=True,\n            linecolor='black',\n            linewidth=1,\n            mirror=True,\n            ticks='',\n            tickmode='array',\n            tickvals=[10,20, 30, 40, 50, 60, 70, 80, 90, 100, 110],\n            ticktext=['Goal', 10, 20, 30, 40, 50, 40, 30, 20, 10, 'Goal'],\n            showticklabels=True\n        ),\n        yaxis=dict(\n            title='',\n            autorange=False,\n            range=[-3.3,56.3],\n            showgrid=False,\n            zeroline=False,\n            showline=True,\n            linecolor='black',\n            linewidth=1,\n            mirror=True,\n            ticks='',\n            showticklabels=False\n        ),\n        shapes=[\n            dict(\n                type='line',\n                layer='below',\n                x0=0,\n                y0=0,\n                x1=120,\n                y1=0,\n                line=dict(\n                    color='white',\n                    width=2\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=0,\n                y0=53.3,\n                x1=120,\n                y1=53.3,\n                line=dict(\n                    color='white',\n                    width=2\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=10,\n                y0=0,\n                x1=10,\n                y1=53.3,\n                line=dict(\n                    color='white',\n                    width=10\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=20,\n                y0=0,\n                x1=20,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=30,\n                y0=0,\n                x1=30,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=40,\n                y0=0,\n                x1=40,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=50,\n                y0=0,\n                x1=50,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=60,\n                y0=0,\n                x1=60,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),dict(\n                type='line',\n                layer='below',\n                x0=70,\n                y0=0,\n                x1=70,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),dict(\n                type='line',\n                layer='below',\n                x0=80,\n                y0=0,\n                x1=80,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=90,\n                y0=0,\n                x1=90,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),dict(\n                type='line',\n                layer='below',\n                x0=100,\n                y0=0,\n                x1=100,\n                y1=53.3,\n                line=dict(\n                    color='white'\n                )\n            ),\n            dict(\n                type='line',\n                layer='below',\n                x0=110,\n                y0=0,\n                x1=110,\n                y1=53.3,\n                line=dict(\n                    color='white',\n                    width=10\n                )\n            )\n        ]\n    )\n    return layout\n\nlayout = load_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9e58a52ac3a3547fdfa5996de51ceab2f34df957"},"cell_type":"code","source":"# Load the movements of players in GameKey 280. \ngame280 = load_game_and_ngs('../input/NGS-2016-reg-wk13-17.csv',GameKey=280)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e430ef5bf62220dafafaadbb2820fa487f260dc7"},"cell_type":"code","source":"# Plot a single play, with two players\nplot_play(game_df=game280, PlayID=2918, player1=32864, player2=32725)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1f53eb13148254a39cfd8ad17b1b841e5724dfa9"},"cell_type":"markdown","source":"## Try modifying to see all player in a play"},{"metadata":{"trusted":true,"_uuid":"8c72d9b6acf8087092b21585b14e17a792cb2d9a"},"cell_type":"code","source":"def plot_play_all_players(game_df, PlayID, custom_layout=False):\n    \"\"\"\n    Plots player movements on the field for a given game, play, and two players\n    \"\"\"\n    game_df = game_df[game_df.PlayID==PlayID]\n    \n    GameKey=str(pd.unique(game_df.GameKey)[0])\n    HomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\n    VisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\n    player1 = game_df[(game_df.PlayID==PlayID)]['GSISID'].values[0]\n    YardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n    \n    traces=[]   \n    for player in pd.unique(game_df.GSISID):\n        player = int(player)\n        trace = go.Scatter(\n            x = game_df[game_df.GSISID==player].x,\n            y = game_df[game_df.GSISID==player].y,\n            name='GSISID '+str(player),\n            mode='markers'\n        )\n        traces.append(trace)\n    if custom_layout is not True:\n        layout = load_layout()\n        layout['title'] =  HomeTeam + \\\n        ' vs. ' + VisitingTeam + \\\n        '<br>Possession: ' + \\\n        YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\n    data = traces\n    fig = dict(data=data, layout=layout)\n    play_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\n    print(\"\\n\\n\\t\",play_description)\n    offline.iplot(fig, config=config)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c756f99865213e3ba0714e21e8a086b6e4859369"},"cell_type":"code","source":"plot_play_all_players(game_df=game280, PlayID=2918)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4ed7e07b029fc11d3e19b4b65cba843b05177d51"},"cell_type":"markdown","source":"A few notes after plotting all players:\n- It looks like anyone to stepped on the field during the time of the play is picked up. \n- This could just be players who were close on the sideline, or players who are coming on for the next play?\n- I think our main concern would be with the players most likely to take a high impact or the players injured."},{"metadata":{"_uuid":"773c1894c609ff0d1410c23eb93da9db689a20fc"},"cell_type":"markdown","source":"## Add player position to plots"},{"metadata":{"trusted":true,"_uuid":"87bd1494bb7279e50b7fa28f1a7b4e85d3040d76"},"cell_type":"code","source":"pprd.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"657f31f634d0790f5dbe86da1f28080c25be64d3"},"cell_type":"code","source":"\n# inputs\ncustom_layout = False\ngame_df=game280\nPlayID=2918\n\n\n# Function code\ngame_df = game_df[game_df.PlayID==PlayID]\n\nGameKey=str(pd.unique(game_df.GameKey)[0])\nHomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\nVisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\nplayer1 = game_df[(game_df.PlayID==PlayID)]['GSISID'].values[0]\nYardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n\ntraces=[]   \nfor player in pd.unique(game_df.GSISID):\n    player = int(player)\n    trace = go.Scatter(\n        x = game_df[game_df.GSISID==player].x,\n        y = game_df[game_df.GSISID==player].y,\n        name='GSISID '+str(player),\n        mode='markers'\n    )\n    traces.append(trace)\nif custom_layout is not True:\n    layout = load_layout()\n    layout['title'] =  HomeTeam + \\\n    ' vs. ' + VisitingTeam + \\\n    '<br>Possession: ' + \\\n    YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\ndata = traces\nfig = dict(data=data, layout=layout)\nplay_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\nprint(\"\\n\\n\\t\",play_description)\noffline.iplot(fig, config=config)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f12f4853f8f00a0326bab8ae9c01fde417fc9f4"},"cell_type":"markdown","source":"## Player Role Data"},{"metadata":{"trusted":true,"_uuid":"557aab849f734a4be5feb7f760aca70cb5f0c986"},"cell_type":"code","source":"pprd.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1db739272f2324eef9f168a79eab3b66a4cd791b"},"cell_type":"code","source":"pprd.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c1ab0643a5a94cdee4f7eb1488213e61cf5a1999"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}