{"cells":[{"metadata":{"_uuid":"f8497c111d41c83e8eed457fd2399cec38eddb74"},"cell_type":"markdown","source":"# NFL Punt Rule Suggestion - Enforce Maximum Number of Jammers\n### *Sameer Vinayak*\n[Github](https://github.com/sameervinayak31),\n[LinkedIn](https://www.linkedin.com/in/sameervinayak/)\n\n\n\nIn this notebook, I explored the NFL's question: Is there a rule change that can be implemented on punt plays that can make the plays safer, and in particular reduce the incidence of concussions? \n\nOutlined step by step below, my research led me to analyzing the formations deployed on these plays, in particular by the return team. In the end, my conclusion and proposed rule change are as follows:\n\nPunts, like kickoffs, are innately dangerous plays, consisting of players running towards each other at high speeds. The NFL has acknowledged this as fact on kickoffs, and has made [appropriate rule changes](http://www.espn.com/blog/nflnation/post/_/id/287320/the-nfl-rule-tweaks-saving-kickoff-from-extinction) that keep the integrity of the play intact while reducing the chance of injury. Now, they are looking to do the same for punts. \n\nOnly 8% of the plays in which concussions occured were fair catch plays, which makes sense - fewer injuries occur when a play is shortened. The research below suggests that fair catches are far more likely to occur when return teams deploy 8-man \"boxes\" rather than 6-man boxes. This is a result of the players not in the box - jammers. When 6-man boxes are utilized, two jammers are assigned to block each punt gunner, resulting in an increased likelihood of a well-blocked field in which the returner can operate. 8-man boxes, on the other hand, generally allow the gunners to close is on the returner more easily, leading to fair catches.\n\nMandating teams to use 8-man boxes should not drastically impact the upside of punt plays - it will instead primarily serve as a way to nudge returners to call for a fair catch. While this will reduce the number of meaningful returns, it is a fair trade-off, and one the NFL has exercised in the past on other plays, with the massive benefit of player safety. From a public relations standpoint, the rule change is subtle enough that  I suspect that the implications of the change will not be immediately clear to most fans (in contrast, for example, to recent quarterback safety rules which require penalties to be called, impacting the game in a more obvious way and thus invoking a response from fans), reducing the potential backlash faced from fans who are particularly fond of refraining from deviating from the game's rich history.\n\n### **Rule Proposal**: On any punt play, the return team must have no more than an equal amount of players as the punting team lined up on or outside of the yardage numbers on the field. See diagram [here](https://imgur.com/HCWi0cs). In line with the current rules, the following note applies:\n\"*The restriction does not apply if a team does not present a standard punt formation (an equal number of\n        players on either side of the snapper in a tight formation, and one player split out wide to either side), or if, after the\n        offensive team has assumed a set position, there is a shift, or a player goes in motion*\" [(Rule 9, Section 1)](https://operations.nfl.com/media/3277/2018-nfl-rulebook_final-version.pdf)."},{"metadata":{"_uuid":"43ff703f24db49a1f53b69baaed167b53fbae65a"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"2782426584fae42e7147ee06a6bb3dad82a2a757"},"cell_type":"markdown","source":"## Research & Code Walkthrough"},{"metadata":{"_uuid":"13ce1bcb3c522800ff975ddbc671def44b2761a6"},"cell_type":"markdown","source":"After importing and some simple joining of the different datasets, I imported the NextGenStats files. The NGS dataset is relatively large, \nand manually defining the expected data types before importing helps to limit the amount of space used and\noptimize for speed (adapted from a kernel posted by [Kevin Mader](https://www.kaggle.com/kmader/convert-to-feather-for-use-in-other-kernels)). The play-level contents of the files have been combined and stored in three CSV files for quicker i/o. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport seaborn as sns\nimport gc\nimport math\nimport datetime\nfrom datetime import timedelta\nimport matplotlib.pyplot as plt\nimport random\nimport tqdm\nimport scipy\nfrom statsmodels.stats.proportion import proportions_ztest\nfrom timeit import timeit\nfrom scipy.stats.stats import pearsonr \nimport re\n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"concussion_how = pd.read_csv('../input/NFL-Punt-Analytics-Competition/video_review.csv')\nplayer_punt_role = pd.read_csv('../input/NFL-Punt-Analytics-Competition/play_player_role_data.csv')\nplayer_role = pd.read_csv('../input/NFL-Punt-Analytics-Competition/player_punt_data.csv')\nplayer_role.drop_duplicates(['GSISID'],keep='first',inplace=True)\nvideo = pd.read_csv('../input/NFL-Punt-Analytics-Competition/video_footage-injury.csv')\nvideo_non = pd.read_csv('../input/NFL-Punt-Analytics-Competition/video_footage-control.csv')\nplay_info = pd.read_csv('../input/NFL-Punt-Analytics-Competition/play_information.csv')\n\ntqdm.tqdm.pandas()\npd.set_option('display.max_columns', 500)\n\nconcussion_plays_temp = pd.merge(concussion_how,player_punt_role[['PlayID','GSISID','GameKey','Role']],how='left',on=['PlayID','GSISID','GameKey'])\nconcussion_plays_temp2 = pd.merge(concussion_plays_temp, player_role[['GSISID','Position']],how='left',on='GSISID')\nconcussion_plays = concussion_plays_temp2.rename(columns={'Role':'Player Role','Position':'Player Position'})\n\nplayer_punt_role['GSISID'] = player_punt_role['GSISID'].astype(str)\nconcussion_plays =  pd.merge(concussion_plays,player_punt_role[['PlayID','GSISID','GameKey','Role']],how='left',left_on = ['PlayID','Primary_Partner_GSISID','GameKey'],right_on=['PlayID','GSISID','GameKey'])\nconcussion_plays = concussion_plays.rename(columns={'Role':'Partner Role'})\nnon_subset = pd.read_csv('../input/nflpuntfilesfinal/non_subset_factors (3).csv')\nconcussion_plays = pd.read_csv('../input/nflpuntfilesfinal/concussion_plays_factors (3).csv')\nnon_subset['Men In Box'] = non_subset['Men In Box'].mask(non_subset['Men In Box'] > 11, np.nan)\nnon_concussion_all = pd.read_csv('../input/non-con-all/non_concussion_plays_all (1).csv')\nnon_concussion_all['Men In Box'] = non_concussion_all['Men In Box'].mask(non_concussion_all['Men In Box'] > 11, np.nan)\nnon_concussion_all['Men In Box'] = non_concussion_all['Men In Box'].mask(non_concussion_all['Men In Box'] < 4, np.nan)\nnon_concussion_all['key'] = non_concussion_all['GSISID'].astype(str) + '-'+ non_concussion_all['GameKey'].astype(str) + '-'+ non_concussion_all['PlayID'].astype(str)\nnon_concussion_all = non_concussion_all[~non_concussion_all['key'].isin(concussion_plays['key'])]\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"aafb8acadee32666cf96c9b03d26c20925bac3e2","_kg_hide-output":true,"_kg_hide-input":false},"cell_type":"code","source":"def import_ngs():\n    dtypes = {'Season_Year': 'int16',\n             'GameKey': 'int64',\n             'PlayID': 'int16',\n             'GSISID': 'float32',\n             'Time': 'str',\n             'x': 'float32',\n             'y': 'float32',\n             'dis': 'float32',\n             'o': 'float32',\n             'dir': 'float32',\n             'Event': 'str'}\n    \n    col_names = list(dtypes.keys())\n    \n    # import all ngs files\n    \n    ngs_files = ['NGS-2016-pre.csv',\n                 'NGS-2016-reg-wk1-6.csv',\n                 'NGS-2016-reg-wk7-12.csv',\n                 'NGS-2016-reg-wk13-17.csv',\n                 'NGS-2016-post.csv',\n                 'NGS-2017-pre.csv',\n                 'NGS-2017-reg-wk1-6.csv',\n                 'NGS-2017-reg-wk7-12.csv',\n                 'NGS-2017-reg-wk13-17.csv',\n                 'NGS-2017-post.csv']\n    \n    df_list = []\n    \n    for i in ngs_files:\n        df = pd.read_csv('../input/NFL-Punt-Analytics-Competition/'+i, usecols=col_names,dtype=dtypes)\n        df_list.append(df)\n        print(i)\n        \n    ngs = pd.concat(df_list)\n    \n    # Delete the dataframe list to release memory\n    del df_list\n    gc.collect()\n    \n    # Convert Time to datetime\n    ngs['Time'] = pd.to_datetime(ngs['Time'], format='%Y-%m-%d %H:%M:%S')\n    \n    # See what we have loaded\n    \n    ngs['Season_Year'] = ngs['Season_Year'].astype('category').cat.codes\n    ngs = ngs[~ngs['GSISID'].isna()]\n    ngs['GSISID'] = ngs['GSISID'].astype('int32')\n    ngs.reset_index(drop=True,inplace=True)\n    \n    #ngs.to_feather('ngs_fet.feather')\n    return ngs\n\n#ngs = import_ngs()\n#print('Done.')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"51784e50e99909b59073650dec073aff807ef48b"},"cell_type":"markdown","source":"After much poking around with players' velocities on an individual and play level, I decided to \nbroaded the scope to focus instead on what causes a player to reach faster speeds on a play - their position.\n\nSimilar to how on defense teams enter nickel or dime packages that feature faster players to serve specific purposes,\nthe return team on punt plays varies its personnel and alignment based on its objectives for the play. Note that a key difference is\nthat on defense, the personnel and formation are more likely to be dictated to an extent by the offense's personnel and formation. \nMeanwhile, on punt plays, it appears that the return team often doesn't have to react to the punting team's formation - this remains\nlargely static. I've provided some confirmation for this below in the notebook.\n\nIn order to learn more about the formations that return teams deployed, I used the NGS dataset to estimate the number of \nplayers lined up 'in the box' - that is, directly opposing the offensive line. I defined this as being within 10 yards\nof the long snapper at the time of the snap. I also pulled in the position list for both teams to get a sense for which punt roles were being featured and how\nthat impacted the plays and their safety."},{"metadata":{"trusted":true,"_uuid":"e5b5f5628f5d95f18bbe667c57b07df103cc52ce"},"cell_type":"code","source":"def men_in_box(row):\n    \"\"\"\n    This function calculates the number of players on the return team considered lined up 'in the box' \n    at the time of the snap. I used being within 10 yards of the long snapper as a proxy for being \n    considered lined up in the box. \n    \n    There was one anomalous case that resulted in 20 players being considered 'in the box', so I\n    filtered that out manually. Other than that, the results fit within reason and were confirmed visually \n    where possible.   \n    \"\"\"\n    gsisid = row['GSISID']\n    play = row['PlayID']\n    game = row['GameKey']\n    \n    return_team_pos = [\n 'PDL1','PDL2','PDL3','PDL4','PDL5','PDL6',\n 'PDM','PDR1','PDR2','PDR3','PDR4','PDR5',\n 'PDR6', 'PLL','PLL1','PLL2','PLL3','PLM',\n 'PLM1','PLR','PLR1','PLR2','PLR3','PR',\n 'VL', 'VLi','VLo','VR','VRi','VRo', 'PFB','PLS']\n    \n    #get all players on return team\n    play_tracking = ngs[(ngs['GameKey'] == game) & (ngs['PlayID'] == play)]\n    play_tracking['GSISID'] = play_tracking['GSISID'].astype(str)\n    play_tracking = pd.merge(play_tracking,player_punt_role[['PlayID','GSISID','GameKey','Role']],how='left',on=['PlayID','GSISID','GameKey'])\n    return_team_tracking = play_tracking[play_tracking['Role'].isin(return_team_pos)]\n    return_team_starting_pos = return_team_tracking[return_team_tracking['Event'] == 'ball_snap']\n    if len(return_team_starting_pos[return_team_starting_pos['Role'] == 'PLS']) == 0:\n        return np.nan\n    \n    #locate the long snapper & determine in/out status\n    x_ls = return_team_starting_pos[return_team_starting_pos['Role'] == 'PLS']['x'].values[0]\n    y_ls = return_team_starting_pos[return_team_starting_pos['Role'] == 'PLS']['y'].values[0]\n    men_in_box = 0\n    for gsid in return_team_starting_pos[return_team_starting_pos['Role'] != 'PLS']['GSISID']:\n        player = return_team_starting_pos[return_team_starting_pos['GSISID'] == gsid]\n        x_in = (player['x'].values[0] > x_ls - 10.0) & (player['x'].values[0] < x_ls + 10.0)\n        y_in = (player['y'].values[0] > y_ls - 10.0) & (player['y'].values[0] < y_ls + 10.0) \n        if x_in and y_in:\n            men_in_box += 1\n            \n    #discard any errors\n    if (men_in_box > 11) or (men_in_box < 4):\n        return np.nan\n    return men_in_box\n\n\ndef get_kick_list(key):\n    \"\"\"\n    Takes in the key (created by combining the GameKey and PlayID separated by a hyphen)\n    and outputs the list of roles used by the kicking team on that play.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    return_team_pos = [\n 'PDL1','PDL2','PDL3','PDL4','PDL5','PDL6',\n 'PDM','PDR1','PDR2','PDR3','PDR4','PDR5',\n 'PDR6', 'PLL','PLL1','PLL2','PLL3','PLM',\n 'PLM1','PLR','PLR1','PLR2','PLR3','PR',\n 'VL', 'VLi','VLo','VR','VRi','VRo', 'PFB']\n    \n    all_players = ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))]\n    all_players['GSISID'] = all_players['GSISID'].astype(str)\n    all_players = pd.merge(all_players,player_punt_role[['PlayID','GSISID','GameKey','Role']],how='left',on=['PlayID','GSISID','GameKey'])\n    all_roles = set(all_players['Role'])\n    kick_roles = []\n    for role in all_roles:\n        if role not in return_team_pos:\n            kick_roles.append(role)\n    kick_roles = [x for x in kick_roles if type(x) == str]\n    return kick_roles\n\n\ndef get_ret_list(key):\n    \"\"\"\n    Takes in the play key (created by combining the GameKey and PlayID separated by a hyphen)\n    and outputs the list of roles used by the return team on that play.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    return_team_pos = [\n 'PDL1','PDL2','PDL3','PDL4','PDL5','PDL6',\n 'PDM','PDR1','PDR2','PDR3','PDR4','PDR5',\n 'PDR6', 'PLL','PLL1','PLL2','PLL3','PLM',\n 'PLM1','PLR','PLR1','PLR2','PLR3','PR',\n 'VL', 'VLi','VLo','VR','VRi','VRo', 'PFB']\n    \n    all_players = ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))]\n    all_players['GSISID'] = all_players['GSISID'].astype(str)\n    all_players = pd.merge(all_players,player_punt_role[['PlayID','GSISID','GameKey','Role']],how='left',on=['PlayID','GSISID','GameKey'])\n    all_roles = set(all_players['Role'])\n    ret_roles = []\n    for role in all_roles:\n        if role in return_team_pos:\n            ret_roles.append(role)\n    ret_roles = [x for x in ret_roles if type(x) == str]\n    return ret_roles","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c9e95558f7b24996ea074edfbcbcd9368353998"},"cell_type":"markdown","source":"The chart below show that there are three box sizes (6, 7, and 8) that are predominantly used by return teams. While for all punt plays in which concussions didn't occur the 8-man box was the most common, the same wasn't true for plays that led to concussions. The second chart shows that that distribution leans more towards 6-man boxes."},{"metadata":{"_kg_hide-input":true,"trusted":true,"scrolled":false,"_uuid":"ae9971f7386c72fe53f09723902578fbb316c384"},"cell_type":"code","source":"boxes = ['6','7','8','Other']\nsix = round(100*len(non_concussion_all[non_concussion_all['Men In Box']==6])/len(non_concussion_all['Men In Box'].dropna()),2)\nseven = round(100*len(non_concussion_all[non_concussion_all['Men In Box']==7])/len(non_concussion_all['Men In Box'].dropna()),2)\neight = round(100*len(non_concussion_all[non_concussion_all['Men In Box']==8])/len(non_concussion_all['Men In Box'].dropna()),2)\nother = 100 - six - seven - eight\nfreq = [six,seven,eight,other]\ngraph_df = pd.DataFrame({'Box Size':boxes,'Frequency':freq})\n\nsns.set(style=\"whitegrid\")\nplt.style.use('fivethirtyeight');\nclrs = ['dodgerblue','dodgerblue','indianred','dodgerblue']\nf, ax = plt.subplots(figsize=(10, 10))\nsns.barplot(x='Box Size',y='Frequency',data=graph_df,palette = clrs)\nplt.xticks(size=20);\nplt.yticks(np.arange(0,45,5),size=12)\nplt.ylabel('Frequency (%)');\nplt.title('Frequency of Box Sizes, All Plays');","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"b6099470b55ebdad61260b928ef4390caf501e6c"},"cell_type":"code","source":"\nboxes = ['6','7','8']\nsix = round(100*len(concussion_plays[concussion_plays['Men In Box']==6])/len(concussion_plays['Men In Box'].dropna()),2)\nseven = round(100*len(concussion_plays[concussion_plays['Men In Box']==7])/len(concussion_plays['Men In Box'].dropna()),2)\neight = round(100*len(concussion_plays[concussion_plays['Men In Box']==8])/len(concussion_plays['Men In Box'].dropna()),2)\nfreq = [six,seven,eight]\ngraph_df = pd.DataFrame({'Box Size':boxes,'Frequency':freq})\n\nsns.set(style=\"whitegrid\")\nplt.style.use('fivethirtyeight');\n\n\nf, ax = plt.subplots(figsize=(10, 10))\nclrs = ['indianred','dodgerblue','dodgerblue']\nsns.barplot(x='Box Size',y='Frequency',data=graph_df,palette = clrs)\nplt.xticks(size=20);\nplt.yticks(np.arange(0,45,5),size=12)\nplt.ylabel('Frequency (%)');\nplt.title('Frequency of Box Sizes, Concussion Plays');","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dde164d192fa637024642de98866bd89695b7150"},"cell_type":"markdown","source":"The highly disparate proportions here (40.54% of plays using 6-man boxes for plays during which a concussion occured versus 21.39% for all others) is interesting. Before looking into how the personnel used for these plays varied, I confirmed using a Two Proportions Z Test that the percentages are significantly different given the sample sizes."},{"metadata":{"trusted":true,"_uuid":"acd66f732b7b227bc2b1d1109c70d1c5d37a8c6e","_kg_hide-input":false},"cell_type":"code","source":"def test_significance(concussion_plays, non_concussion_plays):\n    count = concussion_plays['Men In Box'].value_counts()[6]\n    nobs = len(concussion_plays)\n    value = (non_concussion_plays['Men In Box'].value_counts()/len(non_concussion_plays))[6]\n    stat, pval = proportions_ztest(count, nobs, value)\n    if pval < .05:\n        result = \"\"\"\n        Given that the population proportion of instances in which there were six men in the \\n\\\n        box on punts was {pop}, there is only a {pval} probability that the {pct} proportion \\n\\\n        seen in the plays in which concussions occured was drawn from the population \\n\\\n        distribution. Thus, at a 95% confidence level, we reject the null that the sample \\n\\\n        and population means are equal.\n        \"\"\".format(pop= round(value,6), pval = round(pval,6), pct = round(count/nobs,6))\n    else:\n        result =    \"\"\"\n        Given that the population proportion of instances in which there were six men in \\n\\ \n        the box on punts was {pop},there is a {pval} probability that the {pct} proportion \\n\\\n        seen in the plays in which concussions occured was drawn from the population distribution.\\n\\ \n        Thus, we cannot reject the null that the sample and population means are equal.\n        \"\"\".format(pop= round(value,6), pval = round(pval,6), pct = round(count/nobs,6))\n    print(result)\n    return pval\n\npval = test_significance(concussion_plays,non_concussion_all)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"556d9f6bf24edbfaa90084b97fdf9d1700314287"},"cell_type":"markdown","source":"This finding suggests that there may be some signal in the formations used by the return team. As mentioned earlier, the variation in formation and personnel is mainly a return-team-specific feature. Instead of plotting the specific formation on each play and trying to learn the different positions using clustering or other methods that may have been overkill, I used the position labels that were already given. This is an oversimplification, as not all players in the box necessarily run towards the punter when the ball is snapped. However, it serves as a useful model to represent the general formation and play call by the return team.\n\nBelow, we can see that the number of punt team gunners stays the same regardless of box size, but the number of jammers/corners doubles when moving from a 6-man box to an 8-man box. **This is the key feature of box sizes - whether or not the punting team's gunners are being double-teamed**. \n\nMoving forward, the comparisons being made will be between 6- and 8-man boxes. 7-man boxes, as one might expect, are simply a compromise between 6 and 8 in all aspects that were analyzed - for example, the average number of jammers for 7 man box plays was 2.9 (one gunner was double teamed instead of two), roughly halfway between the averages for 6- and 8-man boxes. All pro-8-man box arguments made in this proposal can be made more mild by substituting in 7-man boxes instead. "},{"metadata":{"trusted":true,"_uuid":"12004b2509acee77f9bf69d81c124507e51e609f","_kg_hide-input":true},"cell_type":"code","source":"six_gun = non_subset.groupby('Men In Box').mean()['count_gunners'][6]\neight_gun = non_subset.groupby('Men In Box').mean()['count_gunners'][8]\n\nsix_jam = non_subset.groupby('Men In Box').mean()['count_corners'][6]\neight_jam = non_subset.groupby('Men In Box').mean()['count_corners'][8]\n\nprint(\"Number of gunners on 6 man box plays: \", round(six_gun,1))\nprint(\"Number of gunners on 8 man box plays: \", round(eight_gun,1))\nprint(\" \")\nprint(\"Number of jammers on 6 man box plays: \", round(six_jam,1))\nprint(\"Number of jammers on 8 man box plays: \", round(eight_jam,1))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cdd4b0d4e1a14b91bddf8e85d06c63cff8e824ac"},"cell_type":"markdown","source":"Teams often utilize 6-man boxes (double-teaming both gunners) in order to allow their return man more room to operate. 8-man boxes, on the other hand, theoretically pressure the punter into making a rushed punt (or result in a blocked punt), but allow the gunners to close in on the punter with more ease. In order to move forward with the proposed rule change of mandating that gunners are only single teamed at most, it's important to analyze the different results that come of different box sizes to see the impact that the rule change may cause.\n\nThe data (shown in the output below) suggests a few things about how the box size impacts punt plays:\n* Regardless of the box size, return yardage on plays in which the ball was fielded was approximately 9 yards.\n* It follows popular knowledge about head injuries that higher-speed collisions are more likely to cause injury. The average speed of the players involved in the punts slightly but monotonically decreases as box size increases.\n* The time that it took from the snap to the execution of the punt held steady at about 2 seconds regardless of box size. This does not imply that there is no benefit to rushing the punter with more players, as certain types of punts may require more time to execute, but it does suggest that there would not be a drastic impact on punters' ability to perform.\n* Punt hang time remained steady regardless of box size at ~4.4 seconds.\n* The percentage of plays that resulted in a fumble or blocked punt appeared unimpacted by the box size. While touchdowns were scored at a higher rate on 6-man box plays (1% vs .2% on 8-man box plays), the rarity of the event doesn't lend itself well to any sort of definitive claim.\n* **36% of 8 man box plays resulted in a fair catch, while only 14% of 6 man box plays did.**\n* Only 8% of plays during which a concussion occured were fair catches. The Two Proportions Z Test in the following cell confirms the disparity.\n\nCoupling the last two points together, a story starts to form: As one would expect, concussions occur signficantly less often on plays in which a fair catch has been called, and the use of 8-man boxes positively influences the rate of fair catches due to the inability of jammers to prevent gunners from breaking free from single coverage.  "},{"metadata":{"trusted":true,"_uuid":"48a7d4b52a5257bbee667fcb8c50028d0d04fa73","_kg_hide-input":true},"cell_type":"code","source":"fair_six = non_concussion_all.groupby('Men In Box').mean()['Fair Catch'][6]\nfair_eight = non_concussion_all.groupby('Men In Box').mean()['Fair Catch'][8]\nret_six = non_subset.groupby('Men In Box').mean()['yards_returned'][6]\nret_eight = non_subset.groupby('Men In Box').mean()['yards_returned'][8]\nprint(\"Yards per return, 6-man box: \",ret_six)\nprint(\"Yards per return, 8-man box: \",ret_eight)\nprint(\" \")\nprint(\" \")\nprint(pd.DataFrame(non_subset.groupby('Men In Box').mean()['Average Play Speed']))\nprint(\" \")\nprint(\" \")\ntime_six = non_subset.groupby('Men In Box').mean()['time_to_punt'][6]\ntime_eight = non_subset.groupby('Men In Box').mean()['time_to_punt'][8]\nprint(\"Average snap-to-punt time, 6-man box: \", time_six)\nprint(\"Average snap-to-punt time, 8-man box: \", time_eight)\nprint(\" \")\nprint(\" \")\nhang_six = non_concussion_all.groupby('Men In Box').mean()['Hang Time'][6]\nhang_eight = non_concussion_all.groupby('Men In Box').mean()['Hang Time'][8]\nprint(\"Average punt hang-time, 6-man box: \",hang_six)\nprint(\"Average punt hang-time, 8-man box: \",hang_eight)\n\nprint(\" \")\nprint(\" \")\nprint(\"Percent of plays resulting in a fair catch, 6-man box: \",round(100* fair_six, 2))\nprint(\"Percent of plays resulting in a fair catch, 8-man box: \",round(100 * fair_eight, 2))\nnon_subset.groupby('Men In Box').mean()['time_to_punt'][8]\nprint(\" \")\nprint(\" \")\nfair_c = len(concussion_plays[concussion_plays['Fair Catch'] == True])/len(concussion_plays)\nprint(\"Percent of plays resulting in a fair catch + concussion: \",round(fair_c * 100,2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b136e903e9cf1f0c60a7a2d412ba801625ef86c5"},"cell_type":"code","source":"def test_significance_fair_catch(concussion_plays, non_concussion_plays):\n    count = len(concussion_plays[concussion_plays['Fair Catch'] == True])\n    nobs = len(concussion_plays)\n    value = len(non_concussion_plays[non_concussion_plays['Fair Catch'] == True])/len(non_concussion_plays)\n    stat, pval = proportions_ztest(count, nobs, value)\n    if pval < .05:\n        result = \"\"\"\n        Given that the population proportion of instances in which a fair catch \\n\\\n        occured on punts was {pop},there is only a {pval} probability that the \\n\\\n        {pct} proportion seen in the plays in which concussions occured was drawn \\n\\\n        from the population distribution. Thus, we reject the null that the sample \\n\\\n        and population means are equal.\n        \"\"\".format(pop= round(value,6), pval = round(pval,6), pct = round(count/nobs,6))\n    else:\n        result =    \"\"\"\n        Given that the population proportion of instances in which a fair catch occured on punts was {pop},\n        there is a {pval} probability that the {pct} proportion seen in the plays in which concussions occured \n        was drawn from the population distribution. Thus, we cannot reject the null that the sample and population means are equal.\n        \"\"\".format(pop= round(value,6), pval = round(pval,6), pct = round(count/nobs,6))\n    print(result)\n    return pval\n\npval_fc = test_significance_fair_catch(concussion_plays,non_concussion_all)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3d90e7c9fbfd46660b75f852e114cf9e1e4d1af8"},"cell_type":"markdown","source":"The research presented in this notebook should highlight the impact that gunners have on how punt plays develop. When they are given the ability to release off the line, work their way down the field, and arrive at the returner just before the ball does, returners are forced to call a fair catch. There is skill involved in this, and this rule change would help to highlight that skill. \n\nIt is almost certainly true that mandating single coverage on gunners will result in fewer fielded punts, but the tradeoff is easy to accept given the payout. And as some of the research showed, punt returns will still have their place in impacting games - players will simply choose their spots more carefully.\n\n-----------"},{"metadata":{"_uuid":"dde3994927186cbae8b8a19fc2e0c7528fbef9d0"},"cell_type":"markdown","source":"Below are the functions used to compute different aspects of the research that has been outlined above, followed by the commented out execution of these functions, hidden."},{"metadata":{"trusted":true,"_uuid":"9c581b4389a78b97a8f24b9efc14c76e7c7a1c57","_kg_hide-input":false},"cell_type":"code","source":"def get_player_speeds(df):\n    \"\"\"\n    Takes in a player's NGS records for a single play and\n    returns his average speed after converting to miles per hour.\n    \"\"\"\n    dis = df['dis']\n    mph = dis * 20.455\n    return np.mean(mph)\n\n\ndef get_video_link(game, play):\n    \"\"\"\n    Function used to retrieve video link given game and play.\n    \"\"\"\n    return video[(video['gamekey'] == game) & (video['playid'] == play)]['PREVIEW LINK (5000K)'].values[0]\n\n\ndef check_fair_catch(key):\n    \"\"\"\n    Checks whether or not a given play resulted in a fair catch.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    events = set(ngs[(ngs['PlayID'] == int(playID)) & (ngs['GameKey'] == int(gameID))]['Event'])\n    if 'fair_catch' in events:\n        return True\n    else:\n        return False\n    \ndef play_speed_avg(key):\n    \"\"\"\n    Calculates the average speed for all players that participated\n    in a given play.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    play = ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))][['dis','GSISID']]\n    all_play_speeds = []\n    all_averages = []\n    for gsid in set(play['GSISID']):\n        player_play = play[play['GSISID'] == gsid]\n        player_avg = get_player_speeds(player_play)\n        all_averages.append(player_avg)\n    avg = np.mean(all_averages)\n    return avg\n\n\ndef play_speed_sd(key):\n    \"\"\"\n    Calculates the standard deviation of the  speed for \n    all players that participated in a given play.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    play = ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))][['dis','GSISID']]\n    all_play_speeds = []\n    all_averages = []\n    for gsid in set(play['GSISID']):\n        player_play = play[play['GSISID'] == gsid]\n        player_avg = get_player_speeds(player_play)\n        all_averages.append(player_avg)\n    sd = np.std(all_averages)\n    return sd\n\n\ndef play_speed_max(key):\n    \"\"\"\n    Returns the max speed of the fastest player on a given play.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    play = ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))][['dis','GSISID']]\n    all_play_speeds = []\n    all_averages = []\n    for gsid in set(play['GSISID']):\n        player_play = play[play['GSISID'] == gsid]\n        player_avg = get_player_speeds(player_play)\n        all_averages.append(player_avg)\n    mx = max(all_averages)\n    return mx\n\n\ndef get_hang_time(row):\n    \"\"\"\n    Returns the hang time of the punt on a given play.\n    \"\"\"\n    playID = row['PlayID']\n    GameKey = row['GameKey']\n    gsid = row['GSISID']\n    ngs_play = ngs[(ngs['PlayID'] == playID) & (ngs['GameKey'] == GameKey)]\n    ngs_play = ngs_play[ngs_play['GSISID'] == gsid]\n    punt = ngs_play[ngs_play['Event'] == 'punt']['Time']\n    end_events = ['fair_catch','punt_downed','punt_land','punt_muffed','punt_received']\n    end_hang = ngs_play[ngs_play['Event'].isin(end_events)]\n    if len(end_hang) == 0:\n        return np.nan\n    elif len(end_hang) > 1:\n        end_hang = end_hang[end_hang['Time'] == min(end_hang['Time'])]\n        \n    try:\n        hang_time = end_hang['Time'].values[0] - punt.values[0]\n    except:\n        return np.nan\n    hang_time = int(hang_time)/1000000000\n    if hang_time > 6:\n        return np.nan\n    elif hang_time < 1.5:\n        return np.nan\n    return hang_time\n\n\ndef plot_player_speed(ngs, play,GSISID):\n    \"\"\"\n    Function used to plot a player's velocity throughout a play.\n    \"\"\"\n    player_tracking = ngs[(ngs['PlayID'] == play) & (ngs['GSISID'] == GSISID)].sort_values('Time')\n    player_tracking['Speed'] = player_tracking.apply(lambda x:get_speed(x,player_tracking),axis=1)\n    ax = sns.lineplot(x=range(len(player_tracking)),y='Speed',data=player_tracking, )\n    ax.set(xlabel='Time Elapsed', ylabel='MPH');\n    plt.show()\n    print(get_video_link(player_tracking['GameKey'].iloc[0], play))\n    print('Position: ' + player_role[player_role['GSISID'] == GSISID]['Position'].values[0])\n    print('Number: ' + player_role[player_role['GSISID'] == GSISID]['Number'].values[0])\n    print('Punt Role: ' + concussion_plays[(concussion_plays['GSISID'] == GSISID) & (concussion_plays['PlayID'] == play)]['Player Role'].values[0])\n    print('Activity: ' + concussion_plays[(concussion_plays['GSISID'] == GSISID) & (concussion_plays['PlayID'] == play)]['Player_Activity_Derived'].values[0])\n    \n\ndef count_gunners(roles):\n    count = 0\n    for role in roles:\n        if role[0] == 'G':\n            count += 1\n    return count\n\ndef count_corners(roles):\n    count = 0\n    for role in roles:\n        if role[0] == 'V':\n            count += 1\n    return count\n\n    \ndef yards_returned(key):\n    \"\"\"\n    Parses the number of yards a returner gained from the play description using regex.\n    Any plays involving a penalty or fumble was thrown out.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    play_record = play_info[(play_info['GameKey']==int(gameID)) & (play_info['PlayID']==int(playID))]\n    try:\n        des = play_record['PlayDescription'].values[0]\n    except IndexError:\n        des = play_record['PlayDescription']\n    play_events = set(ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))]['Event'])\n    discard = ['fair_catch','field_goal_attempt','field_goal_play','fumble','out_of_bounds','penalty_flag','punt_blocked','touchback']\n    if any(i in play_events for i in discard):\n        return np.nan\n    elif 'punts' not in des:\n        return np.nan\n    elif \"PENALTY\" in des:\n        return np.nan\n    \n    regex_num = re.compile(r\"[+-]?\\d+(?:\\.\\d+)?\")\n    nums = regex_num.findall(des.split('Center')[1])\n    if len(nums) != 2:\n        return np.nan\n    return int(nums[1])\n\ndef start_position(key):\n    \"\"\"\n    Returns the starting position of a punt. Instead of \n    indicating side of field, wrote all yardage\n    out of 100. 99 yards = 99 yards away from scoring a touchdown.\n    \"\"\"\n    try:\n        keysplit = key.split('-')\n        gameID = keysplit[1]\n        playID = keysplit[2]\n        play_record = play_info[(play_info['GameKey']==int(gameID)) & (play_info['PlayID']==int(playID))]\n        try:\n            poss = play_record['Poss_Team'].values[0]\n            start = play_record['YardLine'].values[0]\n        except IndexError:\n            poss = play_record['Poss_Team']\n            start = play_record['YardLine']\n        play_events = set(ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))])\n        discard = ['fair_catch','field_goal_attempt','field_goal_play','fumble','out_of_bounds','penalty_flag','punt_blocked','touchback']\n        if any(i in play_events for i in discard):\n            return np.nan\n        start_yard = start.split(' ')[1]\n        start_team = start.split(' ')[0]\n        if start_team == poss:\n            yard_line = (50 - int(start_yard)) + 50\n        else:\n            yard_line = int(start_yard)\n        return yard_line\n    except:\n        return np.nan\n    \ndef snap_to_punt(key):\n    \"\"\"\n    Time taken from the ball snap to the punt execution.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    play = ngs[(ngs['GameKey'] == int(gameID)) & (ngs['PlayID'] == int(playID))][['Event','Time']]\n    if 'punt' not in set(play['Event']):\n        return np.nan\n    start = min(play[play['Event']=='ball_snap']['Time'])\n    punt = min(play[play['Event']=='punt']['Time'])\n    time_to_punt = punt - start\n    return time_to_punt.total_seconds()\n\n\ndef check_fumble(key):\n    \"\"\"\n    Returns True if a fumble occured on a play,\n    False otherwise.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    events = set(ngs[(ngs['PlayID'] == int(playID)) & (ngs['GameKey'] == int(gameID))]['Event'])\n    if 'fumble' in events:\n        return True\n    else:\n        return False\n    \n    \ndef check_td(key):\n    \"\"\"\n    Returns True if a touchdown was scored on a play,\n    False otherwise.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    events = set(ngs[(ngs['PlayID'] == int(playID)) & (ngs['GameKey'] == int(gameID))]['Event'])\n    if 'touchdown' in events:\n        return True\n    else:\n        return False\n    \ndef check_block(key):\n    \"\"\"\n    Returns True if a punt was blocked,\n    False otherwise.\n    \"\"\"\n    keysplit = key.split('-')\n    gameID = keysplit[1]\n    playID = keysplit[2]\n    events = set(ngs[(ngs['PlayID'] == int(playID)) & (ngs['GameKey'] == int(gameID))]['Event'])\n    if 'punt_blocked' in events:\n        return True\n    else:\n        return False\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d50153720c5803662fdcccd867069d2e9c02248f"},"cell_type":"code","source":"## calculated columns\n\"\"\"\nconcussion_plays['Fair Catch'] = concussion_plays.progress_apply(lambda x:check_fair_catch(x), axis = 1)\nconcussion_plays['Average Play Speed'] = concussion_plays['key'].progress_apply(lambda x:play_speed_avg(x))\nconcussion_plays['Std Play Speed'] = concussion_plays['key'].progress_apply(lambda x:play_speed_sd(x))\nconcussion_plays['Max Play Speed'] = concussion_plays['key'].progress_apply(lambda x:play_speed_max(x))\nconcussion_plays['Hang Time'] = concussion_plays.progress_apply(lambda x:get_hang_time(x), axis = 1)\nconcussion_plays['Men In Box'] = concussion_plays.progress_apply(lambda x:men_in_box(x), axis = 1)\n\nnon_subset['Average Play Speed'] = non_subset['key'].progress_apply(lambda x:play_speed_avg(x))\nnon_subset['Std Play Speed'] = non_subset['key'].progress_apply(lambda x:play_speed_sd(x))\nnon_subset['Max Play Speed'] = non_subset['key'].progress_apply(lambda x:play_speed_max(x))\n\n#add kick roles\nconcussion_plays['kick_roles'] = concussion_plays['key'].progress_apply(lambda x:get_kick_list(x))\nnon_subset['kick_roles'] = non_subset['key'].progress_apply(lambda x:get_kick_list(x))\n#non_subset_pos = non_subset[non_subset['kick_roles'].str.len() == 11]\n\n#add ret roles\nconcussion_plays['ret_roles'] = concussion_plays['key'].progress_apply(lambda x:get_ret_list(x))\nnon_subset['ret_roles'] = non_subset['key'].progress_apply(lambda x:get_ret_list(x))\n\n#count gunners and corners\nconcussion_plays['count_gunners'] = concussion_plays['kick_roles'].progress_apply(lambda x:count_gunners(x))\nnon_subset['count_gunners'] = non_subset['kick_roles'].progress_apply(lambda x:count_gunners(x))\nconcussion_plays['count_corners'] = concussion_plays['ret_roles'].progress_apply(lambda x:count_corners(x))\nnon_subset['count_corners'] = non_subset['ret_roles'].progress_apply(lambda x:count_corners(x))\n\nnon_subset_pos_temp = non_subset[non_subset['kick_roles'].str.len() == 11]\nnon_subset_pos = non_subset_pos_temp[non_subset_pos_temp['ret_roles'].str.len() == 11]\nnon_subset['yards returned'] = non_subset['key'].progress_apply(lambda x:yards_returned(x))\n\n\nnon_subset['start_position'] = non_subset['key'].progress_apply(lambda x:start_position(x))\nconcussion_plays['start_position'] = concussion_plays['key'].progress_apply(lambda x:start_position(x))\n\nconcussion_plays['time_to_punt'] = concussion_plays['key'].progress_apply(lambda x:snap_to_punt(x))\nnon_subset['time_to_punt'] = non_subset['key'].progress_apply(lambda x:snap_to_punt(x))\nnon_subset['yards returned'] = non_subset['key'].progress_apply(lambda x:yards_returned(x))\n\nnon_subset['touchback'] = non_subset['key'].progress_apply(lambda x:check_touchback(x))\nconcussion_plays['touchback'] = concussion_plays['key'].progress_apply(lambda x:check_touchback(x))\n\nconcussion_plays.to_csv('concussion_plays_factors.csv')\nnon_subset.to_csv('non_subset_factors.csv')\nnon_subset['yards_returned'] = non_subset['key'].progress_apply(lambda x:yards_returned(x))\n\n#concussion_plays['time_to_punt'] = concussion_plays['key'].progress_apply(lambda x:snap_to_punt(x))\nnon_subset['time_to_punt'] = non_subset['key'].progress_apply(lambda x:snap_to_punt(x))\n\nnon_subset['fumble'] = non_subset['key'].progress_apply(lambda x:check_fumble(x))\nnon_subset['touchdown'] = non_subset['key'].progress_apply(lambda x:check_td(x))\n\nnon_subset['blocked'] = non_subset['key'].progress_apply(lambda x:check_block(x))\n\"\"\"\nconcussion_plays.to_csv('concussion_plays_factors.csv')\nnon_subset.to_csv('non_subset_factors.csv')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}