{"cells":[{"metadata":{"_uuid":"e44f6342b75f4e27f0c9795de0f81caca5dac6e1"},"cell_type":"markdown","source":"# NFL Punt Safety Analysis\nCarlin Eng\n\n**Head on over to Youtube for a narrated version of this analysis: https://youtu.be/EHAPOUm1IG4**  \n**Presentation slides are in [this Google Doc](https://docs.google.com/presentation/d/1GKqfaTn7BzZydY3__NUnrmFJmYPXIdO506Vg-FnUWCg/edit?usp=sharing)**\n\nI am proposing the following rule changes:  \n\n1. Amend Rule 9, Article 2, to allow ineligible receivers on the line of scrimmage to advance more than 1 yard beyond the line of scrimmage prior to the kick.\n2. Require the punter to set up at least 15 yards behind the line of scrimmage when receiving the snap.\n3. Change pre-season rules to eliminate punt returns. All punts during the preseason should be fair caught, downed, or out of bounds.\n\nThe kernel below shows the analysis that led me to these proposals."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# kaggle/python docker image: https://github.com/kaggle/docker-python\nimport os\nimport datetime\n\nimport numpy as np\nimport pandas as pd\nimport holoviews as hv\nhv.extension('bokeh')\n\nfrom scipy import stats","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bce6d06c31a23bd9e7182a4daca540efac8e76db"},"cell_type":"markdown","source":"For data manipulation and exploratory analysis, I prefer to use SQL.\n\nI've loaded the data provided into a local Postgresql instance, and run the following queries to create the \"plays_categorized\" dataset:"},{"metadata":{"trusted":true,"_uuid":"642f7cda46b71e51d3248900093cda1ee2b58a56"},"cell_type":"markdown","source":"```sql\nCREATE TABLE nfl_punt.role_types (\n\trole varchar(32)\n\t, role_type varchar(32)\n);\n\n-- Manually map each of the roles to either 'return' or 'coverage' unit\nINSERT INTO nfl_punt.role_types (role, role_type) VALUES\n\t('PR', 'return')\n\t, ('PDL1', 'return')\n\t, ('PDR1', 'return')\n\t, ('PRG', 'coverage')\n\t, ('P', 'coverage')\n\t, ('PLG', 'coverage')\n\t, ('PRT', 'coverage')\n\t, ('PLS', 'coverage')\n\t, ('PLT', 'coverage')\n\t, ('PLW', 'coverage')\n\t, ('PDR2', 'return')\n\t, ('PRW', 'coverage')\n\t, ('PDL2', 'return')\n\t, ('GL', 'coverage')\n\t, ('GR', 'coverage')\n\t, ('PDL3', 'return')\n\t, ('PDR3', 'return')\n\t, ('VL', 'return')\n\t, ('VR', 'return')\n\n\t, ('PPR', 'coverage')\n\t, ('PLL', 'return')\n\t, ('PLR', 'return')\n\t, ('VRo', 'return')\n\t, ('VRi', 'return')\n\t, ('VLi', 'return')\n\t, ('VLo', 'return')\n\t, ('PDL4', 'return')\n\t, ('PDR4', 'return')\n\t, ('PLM', 'return')\n\t, ('PLR1', 'return')\n\t, ('PLR2', 'return')\n\t, ('PLL2', 'return')\n\t, ('PLL1', 'return')\n\t, ('PFB', 'return')\n\t, ('PDL5', 'return')\n\t, ('PDR5', 'return')\n\t, ('GRo', 'coverage')\n\t, ('GRi', 'coverage')\n\t, ('GLi', 'coverage')\n\t, ('GLo', 'coverage')\n\n\t, ('PDL6', 'return')\n\t, ('PLR3', 'return')\n\t, ('PLL3', 'return')\n\t, ('PPLi', 'coverage')\n\t, ('PPLo', 'coverage')\n\t, ('PC', 'coverage')\n\t, ('PDR6', 'return')\n\t, ('PPRo', 'coverage')\n\t, ('PPRi', 'coverage')\n\t, ('PLM1', 'return')\n;\n\n-- Create a table containing NGS data for only injuries\nCREATE TABLE ngs_injuries AS\nSELECT\n\ta.*\nFROM ngs a\nJOIN video_review b\nON a.season_year = b.season_year\nAND a.game_key = b.game_key\nAND a.play_id = b.play_id\n;\n\n-- Get relevant columns all in a single table related to injury plays\nDROP TABLE nfl_punt.injury_wide;\nCREATE TABLE nfl_punt.injury_wide AS\nSELECT\n\ta.season_year\n\t, a.game_key\n\t, a.play_id\n\t, a.gsisid\n\t, pi.season_type\n\t, pi.yard_line\n\t, pi.poss_team\n\t, a.player_activity_derived\n\t, a.turnover_related\n\t, a.primary_impact_type\n\t, a.primary_partner_gsisid\n\t, a.primary_partner_activity_derived\n\t, a.friendly_fire\n\t, role_1.role AS player_role\n\t, rt1.role_type AS player_role_type\n\t, role_2.role AS primary_partner_role\n\t, footage.preview_link\n\nFROM video_review a\nLEFT JOIN player_role role_1 \n\tON a.season_year = role_1.season_year\n\tAND a.game_key = role_1.game_key\n\tAND a.play_id = role_1.play_id\n\tAND a.gsisid = role_1.gsisid\nLEFT JOIN player_role role_2\n\tON a.season_year = role_2.season_year\n\tAND a.game_key = role_2.game_key\n\tAND a.play_id = role_2.play_id\n\tAND a.primary_partner_gsisid = role_2.gsisid\nLEFT JOIN nfl_punt.role_types rt1\n\tON role_1.role = rt1.role\nLEFT JOIN nfl_punt.video_footage_injury footage\n\tON a.season_year = footage.season\n\tAND a.game_key = footage.game_key\n\tAND a.play_id = footage.play_id\nLEFT JOIN nfl_punt.play_information pi\n\tON a.season_year = pi.season_year\n\tAND a.game_key = pi.game_key\n\tAND a.play_id = pi.play_id\n;\n\n-- Create a table with NGS data for injured/partner players during plays w/ injuries\nDROP TABLE ngs_injured_players;\nCREATE TABLE ngs_injured_players AS\nSELECT\n\t*\n\t, DATE_PART('seconds', time - play_start) AS time_since_start\nFROM (\n\tSELECT\n\t\ta.*\n\t\t, 'injured' AS role\n\t\t, MIN(a.time) OVER (PARTITION BY a.season_year, a.game_key, a.play_id, a.gsisid) AS play_start\n\tFROM ngs_injuries a\n\tJOIN injury_wide b\n\tON a.season_year = b.season_year\n\tAND a.game_key = b.game_key\n\tAND a.play_id = b.play_id\n\tAND a.gsisid = b.gsisid\n\tUNION ALL\n\tSELECT\n\t\tc.*\n\t\t, 'partner' AS role\n\t\t, MIN(c.time) OVER (PARTITION BY c.season_year, c.game_key, c.play_id, c.gsisid) AS play_start\n\tFROM ngs_injuries c\n\tJOIN injury_wide d\n\tON c.season_year = d.season_year\n\tAND c.game_key = d.game_key\n\tAND c.play_id = d.play_id\n\tAND c.gsisid = d.primary_partner_gsisid\n) z\n;\n\nDROP TABLE nfl_punt.punter_names;\nCREATE TABLE nfl_punt.punter_names AS\nSELECT \n\tseason_year\n\t, poss_team\n\t, punter_gsisid\n\t, TRIM(both ' ' FROM t[array_length(t, 1)]) AS player_name\n\t, COUNT(*) AS nrows\nFROM (\n\n\tSELECT\n\t\t*\n\t\t, regexp_split_to_array(first_desc, '\\)') AS t\n\tFROM (\n\t\tSELECT \n\t\t\tpi.season_year\n\t\t\t, pi.season_type\n\t\t\t, pi.game_key\n\t\t\t, pi.play_id\n\t\t\t, pi.poss_team\n\t\t\t, pr.gsisid  AS punter_gsisid\n\t\t\t, SPLIT_PART(play_description, 'punt', 1) AS first_desc\n\t\tFROM nfl_punt.play_information pi\n\t\tJOIN (SELECT * FROM nfl_punt.player_role WHERE role = 'P' ) pr\n\t\tON pi.season_year = pr.season_year\n\t\tand pi.game_key = pr.game_key\n\t\tand pi.play_id = pr.play_id\n\t\tWHERE play_description NOT ILIKE '%PENALTY%'\n\t) a\n) b\n\nWHERE TRIM(both ' ' FROM t[array_length(t, 1)]) != '.'\nAND TRIM(both ' ' FROM t[array_length(t, 1)]) NOT ILIKE '%fake punt%'\nGROUP BY 1,2,3,4\nHAVING COUNT(*) > 1\n;\n-- Looks like some mis-categorized data, need to do some cleanup. R.Leone is mistakenly listed as punter 31876.\n-- DELETE FROM nfl_punt.punter_names WHERE punter_gsisid = 31876 AND player_name = 'R.Leone';\n\n-- Want to create a punter stats table\n-- In each season:\n-- n_punts, n_injuries\n\n-- Get NGS data just for punters\nDROP TABLE nfl_punt.punter_ngs;\nCREATE TABLE nfl_punt.punter_ngs AS\nSELECT\n\tngs.*\n\t, names.player_name\nFROM nfl_punt.ngs ngs\nLEFT JOIN nfl_punt.player_role role\n\tON ngs.season_year = role.season_year\n\tAND ngs.game_key = role.game_key\n\tAND ngs.play_id = role.play_id\n\tAND ngs.gsisid = role.gsisid\nJOIN (SELECT player_name, MAX(punter_gsisid) AS punter_gsisid FROM nfl_punt.punter_names GROUP BY 1) names\nON ngs.gsisid = names.punter_gsisid\nWHERE role.role = 'P'\n;\n\nCREATE INDEX ON nfl_punt.punter_ngs (season_year, game_key, play_id, gsisid);\n\n-- Summarize data for each punt, including times when events occur\nDROP TABLE nfl_punt.punts_summarized;\nCREATE TABLE nfl_punt.punts_summarized AS\nSELECT\n\ta.season_year\n\t, a.game_key\n\t, c.season_type\n\t, a.play_id\n\t, a.player_name\n\t, a.gsisid\n\t, CASE WHEN b.gsisid IS NOT NULL THEN 1 ELSE 0 END AS injury\n\t, MAX(CASE WHEN event = 'line_set' THEN time ELSE NULL END) AS line_set_time\n\t, MAX(CASE WHEN event = 'ball_snap' THEN time ELSE NULL END) AS snap_time\n\t, MAX(CASE WHEN event = 'punt' THEN time ELSE NULL END) AS punt_time\n\t, MAX(CASE WHEN event = 'punt_received' THEN time ELSE NULL END) AS receive_time\n\t, MAX(CASE WHEN event = 'tackle' THEN time ELSE NULL END) AS tackle_time\n\t, MAX(CASE WHEN event = 'first_contact' THEN time ELSE NULL END) AS first_contact_time\n\t, MAX(CASE WHEN event = 'punt_downed' THEN time ELSE NULL END) AS punt_downed_time\n\t, MAX(CASE WHEN event = 'fair_catch' THEN time ELSE NULL END) AS fair_catch_time\n\t, MAX(CASE WHEN event = 'out_of_bounds' THEN time ELSE NULL END) AS out_of_bounds_time\n\t, MAX(CASE WHEN event = 'play_submit' THEN time ELSE NULL END) AS submit_time\n\nFROM nfl_punt.punter_ngs a\nLEFT JOIN nfl_punt.injury_wide b\n\tON a.season_year = b.season_year\n\tAND a.game_key = b.game_key\n\tAND a.play_id = b.play_id\nLEFT JOIN nfl_punt.game_data c \n\tON a.season_year = c.season_year\n\tAND a.game_key = c.game_key\nGROUP BY 1,2,3,4,5,6,7\n;\n\n-- For each punt determine where the punter was relative to line of scrimmage when ball is punted\nCREATE TABLE nfl_punt.punter_location AS\nSELECT\n\tngs.season_year\n\t, ngs.game_key\n\t, ngs.play_id\n\t, ngs.time AS snap_time\n\t, pc.season_type\n\t, MAX(CASE WHEN role.role = 'P' THEN x ELSE NULL END) AS p_x\n\t, MAX(CASE WHEN role.role = 'P' THEN y ELSE NULL END) AS p_y\n\t, MAX(CASE WHEN role.role = 'PLS' THEN x ELSE NULL END) AS ls_x\n\t, MAX(CASE WHEN role.role = 'PLS' THEN x ELSE NULL END) AS ls_y\n\nFROM nfl_punt.ngs ngs\nLEFT JOIN nfl_punt.player_role role\n\tON ngs.season_year = role.season_year\n\tAND ngs.game_key = role.game_key\n\tAND ngs.play_id = role.play_id\n\tAND ngs.gsisid = role.gsisid\nLEFT JOIN nfl_punt.play_information pi\n\tON ngs.season_year = pi.season_year\n\tAND ngs.game_key = pi.game_key\n\tAND ngs.play_id = pi.play_id\nWHERE role.role IN ('P', 'PLS')\nAND ngs.event = 'ball_snap'\nGROUP BY 1,2,3,4,5,6,7,8,9\n;\n\n-- For each punt, determine the position of the punter at the time when the ball was received\n-- event = 'punt_received', 'fair_catch', 'punt_downed', 'touchback'\nDROP TABLE nfl_punt.returner_location;\nCREATE TABLE nfl_punt.returner_location AS\nSELECT\n\tngs.season_year\n\t, c.season_type\n\t, ngs.game_key\n\t, ngs.play_id\n\t, ngs.gsisid AS returner_gsisid\n\t, d.gsisid AS punter_gsisid\n\t, ngs.time\n\t, ngs.event\n\t, ngs.x\n\t, ngs.y\n\t, ngs.dis\n\t, ngs.o\n\t, ngs.dir\n\t, CASE WHEN b.gsisid IS NOT NULL THEN 1 ELSE 0 END AS injury\n\nFROM nfl_punt.ngs ngs\nJOIN (SELECT * FROM nfl_punt.player_role WHERE role = 'PR') r\n\tON ngs.season_year = r.season_year\n\tAND ngs.game_key = r.game_key\n\tAND ngs.play_id = r.play_id\n\tAND ngs.gsisid = r.gsisid\nLEFT JOIN nfl_punt.injury_wide b\n\tON ngs.season_year = b.season_year\n\tAND ngs.game_key = b.game_key\n\tAND ngs.play_id = b.play_id\nLEFT JOIN nfl_punt.game_data c \n\tON ngs.season_year = c.season_year\n\tAND ngs.game_key = c.game_key\nLEFT JOIN nfl_punt.punts_summarized d\n\tON ngs.season_year = d.season_year\n\tAND ngs.game_key = d.game_key\n\tAND ngs.play_id = d.play_id\nWHERE event IN ('punt_received', 'fair_catch', 'punt_downed', 'touchback')\n;\n\n\n-- Categorize each play using the description field\n-- Include a lot of other relevant columns calculated above\nDROP TABLE nfl_punt.plays_categorized;\nCREATE TABLE nfl_punt.plays_categorized AS\nSELECT\n\n\ta.season_year\n\t, a.season_type\n\t, a.game_key\n\t, a.game_date\n\t, a.week\n\t, a.play_id\n\t, a.game_clock\n\t, a.yard_line\n\t, a.quarter\n\t, a.play_type\n\t, a.poss_team\n\t, a.home_team_visit_team\n\t, a.score_home_visiting\n\t, a.punt_outcome\n\t, c.gsisid AS punter_gsisid\n\t, a.name_match[1] AS punter_name\n\t, a.yards_punted[1]::integer AS yards_punted\n\t, CASE WHEN b.play_id IS NOT NULL THEN 1 ELSE 0 END AS injury\n\t, b.gsisid AS injured_gsisid\n\t, d.player_numbers AS injured_numbers\n\t, b.primary_impact_type\n\t, b.player_activity_derived\n\t, b.primary_partner_gsisid\n\t, b.primary_partner_activity_derived\n\t, b.friendly_fire\n\t, b.player_role\n\t, b.player_role_type\n\t, b.primary_partner_role\n\t, b.preview_link\n\n\t, e.line_set_time\n\t, e.snap_time\n\t, e.punt_time\n\t, e.receive_time\n\t, e.tackle_time\n\t, e.first_contact_time\n\t, e.punt_downed_time\n\t, e.fair_catch_time\n\t, e.out_of_bounds_time\n\t, e.submit_time\n\n\t, f.p_x AS punter_x\n\t, f.p_y AS punter_y\n\t, f.ls_y AS longsnap_x\n\t, f.ls_y AS longsnap_y\n\nFROM (\n\tSELECT\n\t\t*\n\t\t, CASE WHEN play_description ILIKE '%Touchback%' THEN 'touchback'\n\t\t\tWHEN play_description ILIKE '%punts%' AND play_description ILIKE '%out of bounds%' THEN 'out_of_bounds'\n\t\t\tWHEN play_description ILIKE '%fair catch%' THEN 'fair_catch'\n\t\t\tWHEN play_description ILIKE '%downed%' THEN 'downed'\n\t\t\tWHEN play_description ILIKE '%MUFFS catch%' THEN 'muffs_catch'\n\t\t\tWHEN play_description ILIKE '%Direct snap%' THEN 'direct_snap'\n\t\t\tWHEN play_description ILIKE '%punts%' AND play_description ILIKE '%for __ yards%' THEN 'catch_and_run'\n\t\t\tWHEN play_description ILIKE '%punts%' AND play_description ILIKE '%for _ yard%' THEN 'catch_and_run'\n\t\t\tWHEN play_description ILIKE '%punts%' AND play_description ILIKE '%for no gain%' THEN 'catch_and_run'\n\n\t\t\tWHEN play_description ILIKE '%false start%' THEN 'false_start'\n\t\t\tWHEN play_description ILIKE '%PENALTY%' THEN 'penalty'\n\t\t\tWHEN play_description ILIKE '%pass incomplete%' THEN 'pass_incomplete'\n\t\t\tWHEN play_description ILIKE '%pass%' THEN 'pass'\n\t\t\tWHEN play_description ILIKE '%fake%' THEN 'fake_punt'\n\t\t\tWHEN play_description ILIKE '%BLOCKED%' THEN 'blocked'\n\t\t\tWHEN play_description ILIKE '%FUMBLES%' THEN 'fumbled'\n\t\t\tWHEN play_description ILIKE '%safety%' THEN 'safety'\n\t\t\tWHEN play_description ILIKE '%up the middle%' OR play_description ILIKE '%left end%' OR play_description ILIKE '%right%' THEN 'fake_punt'\n\n\t\t\tELSE play_description\n\t\t\tEND AS punt_outcome\n\t\t, REGEXP_MATCHES(SPLIT_PART(play_description, 'punts', 1), '([\\w]\\.[\\w'']*)') AS name_match\n\t\t, REGEXP_MATCHES(play_description, 'punts ([\\w]*) yards') AS yards_punted\n\tFROM nfl_punt.play_information\n) a\n\nLEFT JOIN nfl_punt.injury_wide b\n\tON a.season_year = b.season_year\n\tAND a.game_key = b.game_key\n\tAND a.play_id = b.play_id\nLEFT JOIN nfl_punt.player_role c\n\tON a.season_year = c.season_year\n\tAND a.game_key = c.game_key\n\tAND a.play_id = c.play_id\nLEFT JOIN (SELECT gsisid, STRING_AGG(number, ',') AS player_numbers FROM nfl_punt.player_punt_data GROUP BY 1) d\n\tON b.gsisid = d.gsisid\nLEFT JOIN nfl_punt.punts_summarized e\n\tON a.season_year = e.season_year\n\tAND a.game_key = e.game_key\n\tAND a.play_id = e.play_id\nLEFT JOIN nfl_punt.punter_location f\n\tON a.season_year = f.season_year\n\tAND a.game_key = f.game_key\n\tAND a.play_id = f.play_id\nWHERE LENGTH( a.name_match[1] ) > 2\nAND c.role = 'P'\n;\n\nCREATE INDEX ON nfl_punt.plays_categorized(season_year, game_key, play_id);\n\n-- Summarize stats for each punter by year, and overall\nDROP TABLE nfl_punt.punter_stats_by_year;\nCREATE TABLE nfl_punt.punter_stats_by_year AS\nSELECT\n\tngs.season_year\n\t, ngs.gsisid\n\t, ngs.player_name\n\t, COUNT(DISTINCT ngs.play_id) AS n_punts\n\t, COUNT(DISTINCT inj.play_id) AS n_injuries\n\t, COUNT(DISTINCT CASE WHEN pc.punt_outcome = 'catch_and_run' THEN ngs.play_id ELSE NULL END) AS num_catch_and_run\n\t, COUNT(DISTINCT CASE WHEN pc.punt_outcome = 'fair_catch' THEN ngs.play_id ELSE NULL END) AS num_fair_catch\n\t, COUNT(DISTINCT CASE WHEN pc.punt_outcome = 'out_of_bounds' THEN ngs.play_id ELSE NULL END) AS num_out_of_bounds\n\t, COUNT(DISTINCT CASE WHEN pc.punt_outcome = 'downed' THEN ngs.play_id ELSE NULL END) AS num_downed\n\t, COUNT(DISTINCT CASE WHEN pc.punt_outcome NOT IN ('catch_and_run', 'fair_catch', 'out_of_bounds', 'downed') THEN ngs.play_id ELSE NULL END) AS num_other_outcome\n\t, COUNT(DISTINCT CASE WHEN pc.player_role_type = 'coverage' THEN ngs.play_id ELSE NULL END) AS num_coverage_injured\n\t, COUNT(DISTINCT CASE WHEN pc.player_role_type = 'return' THEN ngs.play_id ELSE NULL END) AS num_return_injured\n\nFROM nfl_punt.punter_ngs ngs\nLEFT JOIN injury_wide inj\n\tON ngs.season_year = inj.season_year\n\tAND ngs.game_key = inj.game_key\n\tAND ngs.play_id = inj.play_id\nLEFT JOIN nfl_punt.plays_categorized pc\n\tON ngs.season_year = pc.season_year\n\tAND ngs.game_key = pc.game_key\n\tAND ngs.play_id = pc.play_id\nGROUP BY 1,2,3\n;\n\nDROP TABLE nfl_punt.punter_stats;\nCREATE TABLE nfl_punt.punter_stats AS\nSELECT\n\tgsisid\n\t, player_name\n\t, SUM(n_punts) AS total_punts\n\t, SUM(n_injuries) AS total_injuries\n\t, SUM(n_injuries)::float / SUM(n_punts) AS inj_rate\n\t, SUM(num_catch_and_run) AS num_catch_and_run\n\t, SUM(num_fair_catch) AS num_fair_catch\n\t, SUM(num_out_of_bounds) AS num_out_of_bounds\n\t, SUM(num_downed) AS num_downed\n\t, SUM(num_other_outcome) AS num_other_outcome\n\t, SUM(num_coverage_injured) AS num_coverage_injured\n\t, SUM(num_return_injured) AS num_return_injured\nFROM nfl_punt.punter_stats_by_year\nGROUP BY 1,2\n;\n```"},{"metadata":{"trusted":true,"_uuid":"8c48e4027b35f6ec83de4febb71e831709fb5003"},"cell_type":"code","source":"# This dataset was created by loading data into a local Postgres server, and running the queries above.\nplays = pd.read_csv(\"../input/punt-plays-categorized/plays_categorized.csv\", sep='|')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3fa5bbb560ea6db98e09f9be6257e2ab8a2586ed"},"cell_type":"markdown","source":"Create some filters to help with our analysis:"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"where_kern = plays.punter_name == 'B.Kern'\nnot_kern = plays.punter_name != 'B.Kern'\nwhere_hekker = plays.punter_name == 'J.Hekker'\ndid_punt = plays.punt_time.notnull()\nis_injury = plays.injury == 1\nnot_injury = plays.injury == 0","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fb47cf6b4c3267a68350e374fceeea9fa88e3712"},"cell_type":"markdown","source":"Of all the punts in the list, how many injuries were there?"},{"metadata":{"trusted":true,"_uuid":"e6371dfceb8e1151e592c192fd34b43e37c42d33"},"cell_type":"code","source":"plays[['injury', 'game_key']].groupby('injury').count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"94f884df1365c301e9a668c5f2346c0b20f67f84"},"cell_type":"markdown","source":"What were the players doing when they suffered a concussion?\nMost players were either tackling (13/37), being blocked (10/27) or blocking (8/27).\nTackled players only made up 6/37 of the injured players."},{"metadata":{"trusted":true,"_uuid":"d69aa69102c618dd57fb2973260844cca54ecfb0"},"cell_type":"code","source":"plays[is_injury][['player_activity_derived', 'game_key']].groupby('player_activity_derived').count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b5ab52a6156d7679ac447d67ca8ec4e1d4655132"},"cell_type":"markdown","source":"Let's take a look at injured players by role:\n\"role_type\" was manually mapped from punt position to one of either coverage or return."},{"metadata":{"trusted":true,"_uuid":"4d78f3179a7a0e9e4ac9c2ce49d81b135db5df2c"},"cell_type":"code","source":"plays[is_injury][['player_role_type', 'game_key']].groupby('player_role_type').count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"88b3215e22dada56f128ccd4ad058eca2c56ece8"},"cell_type":"markdown","source":"27 of the 37 concussions were suffered by those on the coverage unit!\n\nThis is somewhat counter-intuitive. I would have expected the \"tackled\" player to suffer concussions most often. I suspect this is related to player's general awareness of their situation. A punt returner is expecting to get tackled, and can often brace themselves for impact. On the other hand, the coverage unit is usually fixated on the returner, or the football, and has higher potential to be blindsided by a blocker."},{"metadata":{"_uuid":"7e5d0c65e4c3f6267323b66ec4e5f01f5cb648c4"},"cell_type":"markdown","source":"Of the 37 injuries, how do they break down in terms of the outcome of the punt? E.g., if the punt resulted in a fair catch, vs if the returner cought the ball and tried to return it. These were categorized using the \"description\" field. The logic is in the attached SQL."},{"metadata":{"trusted":true,"_uuid":"c2d3ca23c0b1c78dcbaac6db171cd1e5924cc873"},"cell_type":"code","source":"plays[is_injury][['punt_outcome', 'game_key']].groupby('punt_outcome').count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"413554c8d305f1401940cef72d2ec57fd01b6608"},"cell_type":"markdown","source":"Using concussion data that the NFL published in via it's [Play Smart Play Safe](https://www.playsmartplaysafe.com/newsroom/reports/2017-injury-data/), site: \n\n* In 2016, there were\n  * 45 preseason in-game concussions\n  * 172 regular+postseason in-game concussions\n* In 2017, there were\n  * 46 preseason in-game concussions\n  * 189 regular+postseason in-game concussions\n* 91 total pre-season\n* 361 total regular+postseason"},{"metadata":{"trusted":true,"_uuid":"72186413fcd2d3e1cc30c973eb279f19fee9731b"},"cell_type":"code","source":"plays[is_injury][['season_type', 'game_key']].groupby('season_type').count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ade7cd9cbbd5b18fa703172a22811480f77f0592"},"cell_type":"markdown","source":"Preseason is about 19.3% of all plays in a given season, but accounts for 32.4% of concussions!"},{"metadata":{"trusted":true,"_uuid":"9e1a36cc580c4d9ee07223631644a8d702a3ddb5"},"cell_type":"code","source":"preseason_games = 64.0\nregular_season_games = 256\npostseason_games = 11\nprint(preseason_games / (preseason_games + regular_season_games + postseason_games))\nprint(12/(12+25.0))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6902c4b1ca1344b36d92d85c49bcff50c6fd871d"},"cell_type":"markdown","source":"12 preseason punt concussions, 25 regular season punt concussions\n91 - 12 = 79 preseason non-punt concussions, 361 - 25 = 336 regular season non-punt concussions\n\nWe can run a [chi squared test](https://en.wikipedia.org/wiki/Chi-squared_test) to see if there is an interaction effect between punts and concussions, i.e., are preseason punts just as risky as regular season punts?\n\n\"Expected\" values for each cell in the 2x2 season-type vs. play-type table are computed by taking (row total) * (column total / grand total)"},{"metadata":{"trusted":true,"_uuid":"f3981ed40502c42177cc045969f94c6188326c97"},"cell_type":"code","source":"pre_punt = 12\nreg_punt = 25\npre_nonpunt = 79\nreg_nonpunt = 336\ngrand_total = pre_punt+reg_punt+pre_nonpunt+reg_nonpunt\npre_total = pre_punt+pre_nonpunt\nreg_total = reg_punt+reg_nonpunt\npunt_total = pre_punt+reg_punt\nnonpunt_total = pre_nonpunt+reg_nonpunt\n\nobserved = [12, 79, 25, 336]\nexpected = [(punt_total * pre_total/float(grand_total)), \n            (nonpunt_total * pre_total/ float(grand_total)),\n            (punt_total * reg_total/float(grand_total)),\n            (nonpunt_total * reg_total/float(grand_total))]\nstats.chisquare(f_obs=observed, f_exp=expected, ddof=2)\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5efb7eafcbc22beb0d23c323ce637bf3f266a94e"},"cell_type":"markdown","source":"P-value is 0.052 - doesn't cross the magic 0.05 p-value threshold, but definitely worth collecting more data."},{"metadata":{"_uuid":"a51df0757671825b63c3b1a502fa5bbb03abafe1"},"cell_type":"markdown","source":"Punter names were computed using the play_description field in the SQL above.\nHere I ask the question, \"Do some punters' punts cause more injuries than others?\""},{"metadata":{"trusted":true,"_uuid":"af3aad1d843c0c3f0abb9291f3b62d4f4f5b29cd"},"cell_type":"code","source":"plays[['punter_name', 'injury']].groupby('punter_name').agg({'injury': {'num_injuries': np.sum, 'total_punts': np.size, 'injury_rate': np.mean}}).sort_values( [('injury', 'num_injuries')], ascending=False ).head(25)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3fb1efc420dadd015d299356fd52ecab0c84a75a"},"cell_type":"markdown","source":"It looks like some punters are more prone to injury than others.\nLet's compare two Pro Bowlers, Brett Kern (2017, 2018), with 204 punts and 5 players injured during his plays\nWith Johnny Hekker, (2013, 2015-2017), with 209 punts, and 0 players injured during his plays"},{"metadata":{"trusted":true,"_uuid":"236f863ae97228b2727fd5c3829159955a64cc0d"},"cell_type":"code","source":"punts_kern = plays[ did_punt & where_kern ]\npunts_hekker = plays[ did_punt & where_hekker ]\npunts_not_kern = plays[ did_punt & not_kern ]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"86be77fc61138638e9183f4f82836ec088134b3f"},"cell_type":"markdown","source":"How do Brett Kern and Johnny Hekker compare in terms of set-up distance from line of scrimmage?\nTo tell this, we compute the X location of the punter and the X location of the snapper at the time of the snap"},{"metadata":{"trusted":true,"_uuid":"9df834a91d62539828c8c96b9bc8430134c4d418"},"cell_type":"code","source":"kern_snap_punt_distance = abs(punts_kern.punter_x - punts_kern.longsnap_x)\nhekker_snap_punt_distance = abs(punts_hekker.punter_x - punts_hekker.longsnap_x)\nnot_kern_snap_punt_distance = abs(punts_not_kern.punter_x - punts_not_kern.longsnap_x)\nkern_distance_density = [hv.Distribution(kern_snap_punt_distance, label = 'Kern')]\nhekker_distance_density = [hv.Distribution(hekker_snap_punt_distance, label = 'Hekker')]\nnot_kern_distance_density = [hv.Distribution(not_kern_snap_punt_distance, label = 'Not Kern')]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0b45bcde10ec69c99123056fc66b6740b68aff5c"},"cell_type":"code","source":"%%opts Overlay [width=600 legend_position='right']\noverlay = hv.Overlay(kern_distance_density + hekker_distance_density)\noverlay","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9df849ab5365b1deea2abf31a241637de5edb6e5"},"cell_type":"markdown","source":"On a typical play, Brett Kern sets up about 13.3 yards away from the longsnapper\nHekker is 13.6 yards away, 0.3 yards farther."},{"metadata":{"trusted":true,"_uuid":"f67e866462561404ff853df1490253764a6639fb"},"cell_type":"code","source":"print('Brett Kern mean distance from long snapper: {0}'.format(kern_snap_punt_distance.mean()))\nprint('Johnny Hekker mean distance from long snapper: {0}'.format(hekker_snap_punt_distance.mean()))\nprint('Difference: {0}'.format(hekker_snap_punt_distance.mean() - kern_snap_punt_distance.mean()))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"140c195cd1488d1c57962d51d2a635853e30e015"},"cell_type":"markdown","source":"Using a two-sample t-test, this result is statistically significant:"},{"metadata":{"trusted":true,"_uuid":"1dc599477e70bfd65bca8c699d5b7ab260e33e88"},"cell_type":"code","source":"stats.ttest_ind(kern_snap_punt_distance.dropna(), hekker_snap_punt_distance.dropna(), equal_var=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec45225b1e62a0f83557b5ae2ca08d66b76f5a03"},"cell_type":"markdown","source":"How does Kern compare to the rest of punters in general?"},{"metadata":{"trusted":true,"_uuid":"4062c91b156ea49464dd4cecaaad957984860a96"},"cell_type":"code","source":"%%opts Overlay [width=600 legend_position='right']\noverlay = hv.Overlay(kern_distance_density + not_kern_distance_density)\noverlay","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c655048166f3575c0a877aa33b08c654f0002de"},"cell_type":"markdown","source":"For the punting population in general, Kern is again, 0.28 yards closer to the punter than average and again, this result is statistically significant."},{"metadata":{"trusted":true,"_uuid":"3861c8d9a0f488db87e5345af8529030dc8b5c3e"},"cell_type":"code","source":"print('Brett Kern mean distance from long snapper: {0}'.format(kern_snap_punt_distance.mean()))\nprint('Rest of punters mean distance from long snapper: {0}'.format(not_kern_snap_punt_distance.mean()))\nprint('Difference: {0}'.format(not_kern_snap_punt_distance.mean() - kern_snap_punt_distance.mean()))\nprint(stats.ttest_ind(kern_snap_punt_distance.dropna(), not_kern_snap_punt_distance.dropna(), equal_var=False))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ebe7b3ba3bb9503a19d17701234812b2b0902605"},"cell_type":"markdown","source":"How about time from snap to punt? How do Kern and Hekker compare, and how does Kern compare to the rest of punters as a group?"},{"metadata":{"trusted":true,"_uuid":"e0b6a87747544e5122e9ecb31d290c4793aa6f28"},"cell_type":"code","source":"kern_snap_punt = (pd.to_datetime(plays[ did_punt & where_kern ].punt_time) - pd.to_datetime(plays[ did_punt & where_kern ].snap_time)).dt.total_seconds()\nhekker_snap_punt = (pd.to_datetime(plays[ did_punt & where_hekker ].punt_time) - pd.to_datetime(plays[ did_punt & where_hekker ].snap_time)).dt.total_seconds()\nnot_kern_snap_punt = (pd.to_datetime(plays[ did_punt & not_kern ].punt_time) - pd.to_datetime(plays[ did_punt & not_kern ].snap_time)).dt.total_seconds()\n\nkern_snap_punt_time = [hv.Distribution(kern_snap_punt, label = 'Kern')]\nhekker_snap_punt_time = [hv.Distribution(hekker_snap_punt, label = 'Hekker')]\nnot_kern_snap_punt_time = [hv.Distribution(not_kern_snap_punt, label = 'Not Kern')]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"55e007a530dee5aacc2b31992510e0337003c5c2"},"cell_type":"markdown","source":"Kern typically gets his punts off in 1.98 seconds. Hekker in 2.12 seconds. Difference again is statistically significant:"},{"metadata":{"trusted":true,"_uuid":"e64d912b94d6e24cf25bebe91096e18bd1a80cdf"},"cell_type":"code","source":"print('Brett Kern time from snap to punt: {0}'.format(kern_snap_punt.mean()))\nprint('Johnny Hekker time from snap to punt: {0}'.format(hekker_snap_punt.mean()))\nprint('Difference in time from snap to punt: {0}'.format(hekker_snap_punt.mean() - kern_snap_punt.mean()))\n\nprint(stats.ttest_ind(kern_snap_punt.dropna(), hekker_snap_punt, equal_var=False))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c950321e1f1cb775bf277c97943834f0339adabd"},"cell_type":"code","source":"%%opts Overlay [width=600 legend_position='right']\noverlay = hv.Overlay(kern_snap_punt_time + hekker_snap_punt_time)\noverlay","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"467928b92693c299a955936c011b4b5ba4ae7101"},"cell_type":"markdown","source":"Again, comparing Kern to all other punters in the league, Kern is about 0.11 seconds faster than all the other punters"},{"metadata":{"trusted":true,"_uuid":"3a291a5cb633dd88eafab751cae2585b5a3810b2"},"cell_type":"code","source":"print('All punters but Kern: {0}'.format(not_kern_snap_punt.mean()))\nprint('Difference in time from snap to punt: {0}'.format(kern_snap_punt.mean() - not_kern_snap_punt.mean()))\nprint(stats.ttest_ind(kern_snap_punt.dropna(), not_kern_snap_punt.dropna(), equal_var=False))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"37416676176d12982ef06f8941b18c34f6193c87"},"cell_type":"code","source":"%%opts Overlay [width=600 legend_position='right']\noverlay = hv.Overlay(kern_snap_punt_time + not_kern_snap_punt_time)\noverlay","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"84baa9d44a0b66dbc4e13506e70b0bc75946e570"},"cell_type":"markdown","source":"Rather than compare punters, let's compare injury vs non-injury punts. As before, we'll look at distance from longsnapper to punter, and time from snap to punt"},{"metadata":{"trusted":true,"_uuid":"4ba000db174c2bfdca68929e3019a9bb95a84b12"},"cell_type":"code","source":"punts_injury = plays[is_injury & did_punt]\npunts_non_injury = plays[not_injury & did_punt]\n\n# Distance\ninjury_snap_punt_distance = abs(punts_injury.punter_x - punts_injury.longsnap_x)\nnon_injury_snap_punt_distance = abs(punts_non_injury.punter_x - punts_non_injury.longsnap_x)\ninjury_distance_density = [hv.Distribution(injury_snap_punt_distance, label = 'injury')]\nnon_injury_distance_density = [hv.Distribution(non_injury_snap_punt_distance, label = 'not injury')]\n\n# Time\ninjury_snap_punt_time = (pd.to_datetime(punts_injury.punt_time) - pd.to_datetime(punts_injury.snap_time)).dt.total_seconds()\nnon_injury_snap_punt_time = (pd.to_datetime(punts_non_injury.punt_time) - pd.to_datetime(punts_non_injury.snap_time)).dt.total_seconds()\ninjury_time_density = [hv.Distribution(injury_snap_punt_time, label = 'injury')]\nnon_injury_time_density = [hv.Distribution(non_injury_snap_punt_time, label = 'not injury')]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7f4b19c37238c168ede464063515f710eae4c10e"},"cell_type":"markdown","source":"Comparing injury punts vs non-injury punts. Distance from long-snapper to punter:"},{"metadata":{"trusted":true,"_uuid":"7817521e5c92eef514f5d4c5415a8e4ed5792e02"},"cell_type":"code","source":"%%opts Overlay [width=600 legend_position='right']\n\noverlay = hv.Overlay(injury_distance_density + non_injury_distance_density)\noverlay","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"23b2f9aeedf5cc874c094c95db6a1c082713bcfb"},"cell_type":"markdown","source":"In this case, it doesn't look like distance from snap to punt changes much from injury vs non-injury:"},{"metadata":{"trusted":true,"_uuid":"63b041162da5e66f6ab5cd5213009f07b51533b6"},"cell_type":"code","source":"stats.ttest_ind(injury_snap_punt_distance.dropna(), non_injury_snap_punt_distance.dropna(), equal_var=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4e29e0573a433341a676bac5d75023b1f345683c"},"cell_type":"markdown","source":"Comparing injury punts vs non-injury punts. Time from snap to punt:"},{"metadata":{"trusted":true,"_uuid":"ef0b31d9a7d70a6b48d632b66e151a7d92e7aa24"},"cell_type":"code","source":"%%opts Overlay [width=600 legend_position='right']\n\noverlay = hv.Overlay(injury_time_density + non_injury_time_density)\noverlay","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f9206bed8c9d7dcce1644cc966eb137254575b53"},"cell_type":"markdown","source":"Time from snap to punt, injury vs. non-injury. Statistically significant at p=0.05"},{"metadata":{"trusted":true,"_uuid":"d2a0f01b4b2c0bfb6a9ea83ccb9598fabfb6d7f0"},"cell_type":"code","source":"print('Injury punts, time from snap to punt: {0} seconds'.format(injury_snap_punt_time.mean()))\nprint('Non-injury punts, time from snap to punt: {0} seconds'.format(non_injury_snap_punt_time.mean()))\nprint('Non-injury punts were {0} seconds slower from snap to punt'.format(non_injury_snap_punt_time.mean() - injury_snap_punt_time.mean()))\nprint(stats.ttest_ind(injury_snap_punt_time.dropna(), non_injury_snap_punt_time.dropna(), equal_var=False))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1d61bfb91cb2b0424e481b715bd4a065af98d8e1"},"cell_type":"markdown","source":"Are time from snap to punt, and distance from longsnapper to punter related? We can run a regression, and compare the relationship:"},{"metadata":{"trusted":true,"_uuid":"32ade5e601753be90bc7e87d0f9754bba85e24de"},"cell_type":"code","source":"valid_punts = plays[did_punt & plays.punter_x.notnull() & plays.longsnap_y.notnull() & plays.punt_time.notnull() & plays.snap_time.notnull()]\nsnap_punt_time = (pd.to_datetime(valid_punts.punt_time) - pd.to_datetime(valid_punts.snap_time)).dt.total_seconds()\nsnap_punt_distance =  abs(valid_punts.punter_x - valid_punts.longsnap_x)\nprint(stats.linregress(x=snap_punt_distance, y=snap_punt_time))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d8fa45937d5b8c7803d1d49b03f913fa4bd623f4"},"cell_type":"markdown","source":"These two variables are definitely correlated!, rvalue=0.24, and p_value <<< 0.05\nLet's plot to see what it looks like.\n\nThis means if we want to increase time from snap to punt, we should be able to achieve this \nby increasing distance from snap to punt."},{"metadata":{"trusted":true,"_uuid":"8090705f199f52d6c44269a28cf60b723a56809b","scrolled":true},"cell_type":"code","source":"%%opts Scatter [width=600]\n\nslope, intercept, r_value, p_value, std_err = stats.linregress(x=snap_punt_distance, y=snap_punt_time)\nfitted_x = np.linspace(7, 16, 200)\nfitted_y = fitted_x * slope + intercept\n\nobserved = [hv.Scatter(list(zip(snap_punt_distance, snap_punt_time)), label = 'Observed')]\nfitted = [hv.Scatter(list(zip(fitted_x,fitted_y)), label = 'Fitted')]\noverlay = hv.Overlay(observed + fitted)\noverlay\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b55b5aa2c58afdd8c01a2521370921e81e2379ad"},"cell_type":"markdown","source":"Using the analysis presented above, I propose the following rule changes:\n1. Amend Rule 9, Article 2, to allow ineligible receivers on the line of scrimmage to advance more than 1 yard beyond the line of scrimmage prior to the kick.\n  * The goal is to increase the likelihood that the coverage unit makes it downfield to the returner before the returner has a chance to start a punt return for yardage, thus increasing the chance of a fair catch or downed ball, which are less likely to cause injuries.\n2. Require the punter to set up at least 15 yards behind the line of scrimmage when receiving the snap.\n  * The goal is to slow down the punter, and again, give the coverage unit a better chance at forcing a fair catch or downed ball.\n3. Change pre-season rules to eliminate punt returns. All punts during the preseason should be fair caught, downed, or out of bounds.\n  * Because pre-season looks potentially more dangerous, we could take steps to address this part specifically."}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}