{"cells":[{"metadata":{"_uuid":"ceac190cca5e413f5956b2ddc8e553323c4a9bca"},"cell_type":"markdown","source":"# NFL PUNT ANALYTICS COMPETITION - Exploratory Data Analysis\n\n**Author:** Michael Cho\n\n**Collaborators:** Alejandro Avalos Mar, Eli Levin, Austin Haygood\n\n**Purpose:** The purpose of this analysis is to explore the data provided by the NFL on play information, injury information, and player role information. This analysis will determine areas where the rules can be changed to improve the safety (and fun) of punt plays.\n\n**Summary of Analysis:** A large percent of plays result in a Fair Catch. Fair Catch plays result in a low percentage of injuries. Returns contitute the largest portion of injuries. Injuries appear to occur more with the kicking team."},{"metadata":{"_uuid":"36a17857f26790d3fc53bbd708d41e83041d260f"},"cell_type":"markdown","source":"## Analysis Part 1 - Loading Data and Preprocessing Functions"},{"metadata":{"trusted":true,"_uuid":"98980e4a9d158f9a03a51eb391ede7a56ebce869"},"cell_type":"code","source":"# Import Libraries\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom nltk import word_tokenize\nfrom nltk.util import skipgrams\n\n# Load relevant data for Exploratory Analysis\n\n# Play Information\npunt_play_info = pd.read_csv('../input/play_information.csv')\n\n# Injury Plays\ninjury_plays = pd.read_csv('../input/video_review.csv')\n\n# Player Role Data\nrole_player_data = pd.read_csv('../input/play_player_role_data.csv')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1870405a1b90667441fe8b0d4074ea5a7da2d0c0"},"cell_type":"markdown","source":"The functions below are used throughout the analysis:\n   \n1. **play_type_col** Determines if a play is a given play type using key words in the \"Play Description\" column in play_information.csv\n2. **punt_return** Determines if a punt play results in a return by the return team using key words in the \"Play Description\" column in play_information.csv\n3. **metrics_by_role** Creates DataFrame that shows the count and percent of total injury plays for a given dimension (ex: player role, type of contact, activity)\n4. **metrics_by_play_type** Creates DataFrame that shows a count of the number of plays of a given play type and percent of total plays for that play type\n5. **create_plot_from_table** Using a DataFrame generated from 3 or 4, create a bar chart visualization"},{"metadata":{"trusted":true,"_uuid":"3e421fe9da09cd7778f48d14fc3c0845ee220062"},"cell_type":"code","source":"def play_type_col(s, key_words):\n    token = [x.lower() for x in word_tokenize(s)]\n    if all(x in token for x in key_words):\n        return 'Y'\n    else:\n        return 'N'\n\n\ndef punt_return(s):\n    token_punt = [x.lower() for x in word_tokenize(s)]\n    for triple in list(skipgrams(token_punt, 3, 0)):\n        if triple[0] == 'for':\n            try:\n                yards = int(triple[1])\n                if triple[2] == 'yards':\n                    return 'Y'\n            except:\n                continue\n    return 'N'\n\n\ndef metrics_by_role(df, col, output_cols=None):\n    tot = len(df)\n    output_list = []\n    for i in df[col].unique():\n        num = len(df[df[col]==i])\n        pct_tot_injury = 100*num/tot\n        output_list.append([i, num, pct_tot_injury])\n    \n    if output_cols is None:\n        output_df = pd.DataFrame(output_list, columns=['Role', 'Number of Injuries',\n                                                      'Percent Total Injuries'])\n        output_df = output_df[output_df['Number of Injuries'] != 0]\n        return output_df.sort_values(by=['Percent Total Injuries'], ascending=False)\n\n\ndef metrics_by_play_type(df, cols):\n    output_list = []\n    for col in cols:\n        name_of_play = ' '.join(col.split('_'))\n        col_df = df[df[col]=='Y']\n        col_len = len(col_df)\n        pct_total = 100*col_len/len(df)\n        output_list.append([name_of_play, col_len, pct_total])\n        \n    output_df = pd.DataFrame(output_list, columns=['play type', 'number of plays',\n                                                  'percent of total plays'])\n    \n    output_df = output_df[output_df['number of plays'] != 0]\n    \n    return output_df.sort_values(by=['percent of total plays'], ascending=False)\n    \n    \ndef create_plot_from_table(df, x_col, y_col, y_label, title, axis_angle=0):\n    x_cats = list(df[x_col])\n    y_pos = np.arange(len(x_cats))\n    y_vals = np.array(df[y_col])\n    plt.bar(x_cats, y_vals, align='center', alpha=1)\n    plt.xticks(y_pos, x_cats, rotation=axis_angle)\n    plt.ylabel(y_label)\n    plt.title(title)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d46acd15418640438e04aac3c4c49698bae95610"},"cell_type":"markdown","source":"## Analysis Part 2 - Adding Features"},{"metadata":{"_uuid":"f796fae945c40696ca87835f0d1947aa8aec285c"},"cell_type":"markdown","source":"Create Y/N Column in the ```punt_play_info``` DataFrame for each of the following play types: \n\n1. Fair Catch\n2. Out of Bounds\n3. Downed\n4. Touchback\n5. No Play\n6. Muff\n7. Fumble\n8. Touchdown\n9. Blocked Punt\n10. Penalty\n11. Fake Punt\n12. Punt that is returned"},{"metadata":{"trusted":true,"_uuid":"3eda43aa588839f33551f4dd50aa631f3b68bfa7"},"cell_type":"code","source":"punt_play_info['fair_catch'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['fair', 'catch']), axis=1)\n\npunt_play_info['out_of_bounds'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['out', 'of', 'bounds']), axis=1)\n\npunt_play_info['downed'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['downed', 'by']), axis=1)\n\npunt_play_info['touchback'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['touchback']), axis=1)\n\npunt_play_info['no_play'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['-', 'no', 'play']), axis=1)\n\npunt_play_info['muff'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['muffs']), axis=1)\n\npunt_play_info['fumble'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['fumbles']), axis=1)\n\npunt_play_info['touchdown'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['touchdown']), axis=1)\n\npunt_play_info['block'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['blocked']), axis=1)\n\npunt_play_info['penalty'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['penalty']), axis=1)\n\npunt_play_info['fake_punt'] = punt_play_info.apply(\n    lambda x: play_type_col(x['PlayDescription'], ['fake', 'punt']), axis=1)\n\npunt_play_info['return'] = punt_play_info['PlayDescription'].apply(punt_return)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9963876a0299982da209d9c0c2f8d6613f50b8b7"},"cell_type":"markdown","source":"## Analysis Part 3 - Joining Tables"},{"metadata":{"_uuid":"aa804ba5f9a41e9a380c71db9a1553d1dbf5a867"},"cell_type":"markdown","source":"Now that the features have been added to ```punt_play_info```, we can join that DataFrame to the ```injury_plays``` DataFrame (both as left and right joins). In this way, we can look at injuries by play type. We can also join ```injury_plays``` with ```role_player_data``` so we can look at injury information by player role."},{"metadata":{"trusted":true,"_uuid":"f58800035d7220726d75e2fb99bd2fcac3920856"},"cell_type":"code","source":"punts_join_injuries = punt_play_info.merge(\n    injury_plays, left_on=['PlayID', 'Season_Year', 'GameKey'],\n    right_on=['PlayID', 'Season_Year', 'GameKey'], how='left')\n\ninjury_role_player_join = injury_plays.merge(\n    role_player_data, left_on=['Season_Year', 'GameKey', 'PlayID', 'GSISID'],\n    right_on=['Season_Year', 'GameKey', 'PlayID', 'GSISID'], how='left')\n\npunts_join_injuries_only = punt_play_info.merge(\n    injury_plays, left_on=['PlayID', 'Season_Year', 'GameKey'],\n    right_on=['PlayID', 'Season_Year', 'GameKey'], how='right')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4bb77999f83b465e4a5bee7775f709b20b2cdfe4"},"cell_type":"markdown","source":"## Analysis Part 4 - Injuries by Play Type"},{"metadata":{"_uuid":"6efce17f81cbcecb62a5384821c6a4f4d9bc8604"},"cell_type":"markdown","source":"Now that we have done the preprocessing and joining, we can start to take a look at where and how injuries are occuring. First though, lets look at what types of plays occur during punts in general:"},{"metadata":{"trusted":true,"_uuid":"0881a673dcf9f172cb498f212a04422eb67493d0"},"cell_type":"code","source":"metrics_by_play_type(punt_play_info, ['fair_catch', 'out_of_bounds', 'downed', \n        'touchback', 'no_play', 'muff', 'fumble', 'block', 'fake_punt', 'return'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7561ee22eafeec789e44861e315cb8e1a7ccedd3"},"cell_type":"code","source":"table = metrics_by_play_type(punt_play_info, ['fair_catch', 'downed', 'muff', 'fumble', 'return'])\ncreate_plot_from_table(table, 'play type', 'number of plays', 'Count of Injuries', 'Number of Injuries by Play Type')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cb5f1fd8d32adb1930aa3e844a4d153fa766c6d5"},"cell_type":"markdown","source":"Returns and Fair catches constitute about **37.6 percent** and **24.9 percent** of punt plays, respectively from 2016-2017, with Downed, Out of Bounds, and Touchbacks accounting for the next **28.7 percent** of plays. \n\nNext, let's look at the number of injuries by play types:"},{"metadata":{"trusted":true,"_uuid":"1936bc2acb5a477348a38a90698c22bc4f790740"},"cell_type":"code","source":"table = metrics_by_play_type(punts_join_injuries_only, ['fair_catch', 'out_of_bounds', 'downed', \n        'touchback', 'muff', 'fumble', 'block', 'fake_punt', 'return'])\ncreate_plot_from_table(table, 'play type', 'number of plays', 'Count of Injuries', 'Number of Injuries by Play Type')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"86df2766f4717be96c3226f48389bcef91e5620d"},"cell_type":"markdown","source":"As evidenced by the bar chart above, return plays cause a dispropotionate amount of the number of injuries. In contrast, Fair Catch plays results in a much lower number of injuries. In terms of ratios, we see that returning a punt results in nearly 10 times higher likelihood of getting a concussion when compared with a fair catch:"},{"metadata":{"trusted":true,"_uuid":"dd595dec8539e7d7f5eecd1d23da2082e668d8eb"},"cell_type":"code","source":"print('Percent of Return Plays that result in injury: {} percent, ({}/{} plays)'.format(100*29/2510, 29, 2510))\nprint('Percent of Fair Catch Plays that result in injury: {} percent, ({}/{} plays)'.format(100*2/1663, 2, 1663))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ab3625a4e16fe5a8fa8f08793abcaeda62392072"},"cell_type":"markdown","source":"From this analysis, we can conclude that a rule that improves safety would **encourage more fair catch plays**"},{"metadata":{"_uuid":"0b3b158ac4748e23e54a52da4dacecdff30734d4"},"cell_type":"markdown","source":"## Analysis Part 5 - Injuries by Role"},{"metadata":{"_uuid":"98089c093047fede08d8b13c8a84fd6a3359e8f7"},"cell_type":"markdown","source":"Now, let's look at injuries by player information. Here we can see a count of injuries by the role of player during the punt play:"},{"metadata":{"trusted":true,"_uuid":"443f91914fbef18625f127db12bd71162d0d4f9b"},"cell_type":"code","source":"injuries_by_role = metrics_by_role(injury_role_player_join, 'Role')\ninjuries_by_role","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"194b48c522d7aec847606ef604740833e76e4724"},"cell_type":"markdown","source":"We can see from this table that with the exception of the returner (PR) a majority of the other injuries are with the **kicking** team. We can see this fact even more clearly in the bar chart below:"},{"metadata":{"trusted":true,"_uuid":"ad0ac479ef24fe1f332f900af12e68c97f69ba2c"},"cell_type":"code","source":"num_kicking_team_injuries = 0\nnum_return_team_injuries = 0\nfor row in range(len(injuries_by_role)):\n    role = injuries_by_role['Role'].iloc[row]\n    if role in ['PLW', 'PRG', 'GL', 'PLG', 'PRW', 'PRT', 'PLT', 'PLS', 'GR', 'P', 'PPR']:\n        num_kicking_team_injuries += injuries_by_role['Number of Injuries'].iloc[row]\n    elif role in ['PR', 'VR', 'PFB', 'PDR1', 'PDL2', 'PLL']:\n        num_return_team_injuries += injuries_by_role['Number of Injuries'].iloc[row]\n\ntable = pd.DataFrame([['Kicking Team', num_kicking_team_injuries, num_kicking_team_injuries*100/37], \n         ['Returning Team', num_return_team_injuries, num_return_team_injuries*100/37]], \n                     columns=['Team', 'Number of Injuries', 'Percent of Total Injuries'])\n\ncreate_plot_from_table(table, 'Team', 'Number of Injuries', 'Count of Injuries', 'Number of Injuries by Team')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"002dc604ef6e63efbce3871f895bad56bf3aee6a"},"cell_type":"markdown","source":"Next, we can look at the type of contact that caused the injuries. Here, it looks like the count is mostly split between **helmet-to-body** and **helmet-to-helmet** contact"},{"metadata":{"trusted":true,"_uuid":"2a61c326d4a1e24d45789619e199f8c59bcdda41"},"cell_type":"code","source":"table = metrics_by_role(injury_role_player_join, 'Primary_Impact_Type')\ncreate_plot_from_table(table, 'Role', 'Number of Injuries', 'Count of Injuries', \n                       'Number of Injuries by Contact', axis_angle=45)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"76149e8c35599870f3b3940e68ca76bc54396c81"},"cell_type":"markdown","source":"Looking at injuries by activity, the number of injuries looks equal between **blocked / blocking** and **tackled / tackling** players"},{"metadata":{"trusted":true,"_uuid":"de76f701cd5c24725aef6348a1938df3954ea8ea"},"cell_type":"code","source":"table = metrics_by_role(injury_role_player_join, 'Primary_Partner_Activity_Derived')\ncreate_plot_from_table(table, 'Role', 'Number of Injuries', 'Count of Injuries', \n                       'Number of Injuries by Activity', axis_angle=45)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"600bd321727fa3c6b0ffbbb6533b4d28894ac55d"},"cell_type":"markdown","source":"## Analysis Part 6 - Speed of Engaging Players"},{"metadata":{"trusted":true,"_uuid":"53711c7c10802604cbe931393705266ce4bb25c2"},"cell_type":"code","source":"def convert_to_mph(dis_vector, converter):\n    mph_vector = dis_vector * converter\n    return mph_vector\n\ndef get_speed(ng_data, playId, gameKey, player, partner,use_loaded_table = False):\n    \n    if use_loaded_table==False:\n        ng_data = pd.read_csv(ng_data)\n    else:\n        #ng_data is the table not the file location\n        pass\n    ng_data['mph'] = convert_to_mph(ng_data['dis'], 20.455)\n    player_data = ng_data.loc[(ng_data.GameKey == gameKey) & (ng_data.PlayID == playId) \n                               & (ng_data.GSISID == player)].sort_values('Time')\n    partner_data = ng_data.loc[(ng_data.GameKey == gameKey) & (ng_data.PlayID == playId) \n                              & (ng_data.GSISID == partner)].sort_values('Time')\n    player_grouped = player_data.groupby(['GameKey','PlayID','GSISID'], \n                               as_index = False)['mph'].agg({'max_mph': max,\n                                                             'avg_mph': np.mean\n                                                            })\n    player_grouped['involvement'] = 'player_injured'\n    partner_grouped = partner_data.groupby(['GameKey','PlayID','GSISID'], \n                               as_index = False)['mph'].agg({'max_mph': max,\n                                                             'avg_mph': np.mean\n                                                            })\n    partner_grouped['involvement'] = 'primary_partner'\n    return pd.concat([player_grouped, partner_grouped], axis = 0)[['involvement',\n                                                                   'max_mph',\n                                                                   'avg_mph']].reset_index(drop=True)\n\n#Run an example\nget_speed('../input/NGS-2016-pre.csv', 3129, 5, 31057, 32482)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"538d81fc34f76ecea2852cd318bd1f8dfd56c5c5"},"cell_type":"code","source":"injury_plays.columns = [col.lower() for col in injury_plays.columns]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d44f88617e4dfaac1216a16e3b0c40c87e03ce60"},"cell_type":"code","source":"for _file in ['../input/NGS-2016-pre.csv',\n              '../input/NGS-2016-reg-wk1-6.csv',\n              '../input/NGS-2016-reg-wk7-12.csv',\n              '../input/NGS-2016-reg-wk13-17.csv',\n              '../input/NGS-2016-post.csv',\n              '../input/NGS-2017-pre.csv',\n              '../input/NGS-2017-reg-wk1-6.csv',\n              '../input/NGS-2017-reg-wk7-12.csv',\n              '../input/NGS-2017-reg-wk13-17.csv',\n              '../input/NGS-2017-post.csv']:\n    ng_data = pd.read_csv(_file, low_memory=False)\n    for idx, row in injury_plays.iterrows():\n        try:\n            a = get_speed(ng_data, row.playid, row.gamekey, row.gsisid, int(row.primary_partner_gsisid),use_loaded_table=True)\n            speeds = a.values.flatten().flatten()[[1,2,4,5]]\n            injury_plays.at[idx,'player_max_mph'] = speeds[0]\n            injury_plays.at[idx,'player_avg_mph'] = speeds[1]\n            injury_plays.at[idx,'primary_partner_max_mph'] = speeds[2]\n            injury_plays.at[idx,'primary_partner_avg_mph'] = speeds[3]\n        except:\n            continue","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"92accd75466d90616105967a1cad50b366d3fdea"},"cell_type":"code","source":"speed_results = \\\ninjury_plays.groupby(['player_activity_derived',\n            'primary_partner_activity_derived',\n            'friendly_fire']).\\\n                agg({'playid':'count',\n                     'player_max_mph':'mean',\n                     'primary_partner_max_mph':'mean',\n                     'player_avg_mph':'mean',\n                     'primary_partner_avg_mph':'mean'}).round(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"376f54850652c2352cb4f6cdbcf4141ee5ee194c"},"cell_type":"code","source":"speed_results","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e05d5dae3f5cfd2d61215e013e4902e98085cfe1"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}