{"cells":[{"metadata":{"trusted":true,"_uuid":"c5f822db3014687b94ac443d63df4ae2a7bf405f"},"cell_type":"markdown","source":"**Team**\n\nMatthew Sherwood\n\nJill Clements\n\nRyan Klobus\n\nNOTE: The core of our analysis of the below cells is in our PowerPoint presentation. Please refer to the presentation to see our proposed rule changes and how we use the below data analysis to support our findings.\n"},{"metadata":{"_uuid":"a0afe876464ccd85f9e16076082bdba547aedd37"},"cell_type":"markdown","source":"Starting points: Preserving game integrity\n\nThere are many possible rule changes that could be implemented to prevent concussions on punt plays. However, from our first-hand domain knowledge of the sport, we know just how important it is to preserve this great game. Hence, we only want to look at certain features of the game that we could realistically change.\n\nHere are the potential features of punts we investigated:\n\n* Defensive and offensive formations\n* Player-specific paths and involvement\n* High velocity collisions\n\nFrom these investigations, we propose two rules:\n* Sherwood Rule 1: Only one defender may line-up against one gunner. If two defenders from the punt return team are lined up against a single gunner at the time of the punt team being “set” on the ball will result in a five-yard penalty against the punt return team.\n* Sherwood Rule #2: Any defensemen, punt return player, who travels five yards past the line of scrimmage (i.e. trying to block the punt) is not permitted to then run back over the line of scrimmage to interact with players downfield. This rule will result in a fifteen-yard penalty against the return team implemented on the following play. \n","attachments":{}},{"metadata":{"_uuid":"0c51b23eff0eb95d2ab7e94ca63e90531e84a0b9"},"cell_type":"markdown","source":"**Data Overview**<a></a>\n\nBefore testing out any of our own hypotheses as to why concussions may be occurring during punt plays, we first wanted to look at how the data is broken down. This will enable us to see any new patterns we did not previously think of and can then do further analysis."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"_kg_hide-input":false,"_kg_hide-output":false},"cell_type":"code","source":"import os\nimport gc\nimport time\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import pyplot as plt\nfrom matplotlib.animation import FuncAnimation\n\ndata_path = '../input/NFL-Punt-Analytics-Competition'\nimages_path = '../input/images'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a5d994745131eebbf8333ec2ced971cea35791cd"},"cell_type":"code","source":"# load data\nplayer_role_data = pd.read_csv(data_path+'/play_player_role_data.csv', index_col=['GameKey', 'PlayID'])\nplay_information_data = pd.read_csv(data_path+'/play_information.csv', index_col=['GameKey', 'PlayID'])\nconcussion_data = pd.read_csv(data_path+'/video_review.csv', index_col=['GameKey', 'PlayID'])\nconcussion_data = concussion_data.replace({'Primary_Partner_GSISID': {'Unclear': np.nan}})\nconcussion_data.GSISID = concussion_data.GSISID.astype(float)\nconcussion_data.Primary_Partner_GSISID = concussion_data.Primary_Partner_GSISID.astype(float)\n\n# get the punt-specific roles of both players involved in the concussion\nconcussion_data = concussion_data.merge(player_role_data[['Role', 'GSISID']], left_on=['GameKey', 'PlayID', 'GSISID'], right_on=['GameKey', 'PlayID', 'GSISID'])\nconcussion_data = concussion_data.rename(columns={'Role': 'P1'})\nconcussion_data = concussion_data.merge(player_role_data[['Role', 'GSISID']], left_on=['GameKey', 'PlayID', 'Primary_Partner_GSISID'], right_on=['GameKey', 'PlayID', 'GSISID'])\nconcussion_data = concussion_data.rename(columns={'Role': 'P2'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5bc668a5e0a85685a802ed84e1f365384b8e8d3d"},"cell_type":"code","source":"# Offense and defensive positions\noffense = ['PLS', 'P', 'GL', 'PLW', 'PRW', 'PRT', 'GR', 'PLT', 'PLG', 'PRG', 'PC', 'PPR', 'PPL',\n           'PPLi', 'GLo', 'GRi', 'GRo', 'PPLo', 'GLi', 'PPRi', 'PPRo', 'PLM1']\ndefense = ['PDR2', 'PLR', 'PLR2', 'PDR4', 'PLL', 'PDL5', 'PDL4', 'VLi', 'PR', 'VLo', 'PLL2', 'PDL3',\n           'PLL1', 'VR', 'VRo', 'VRi', 'PDL1', 'PDR1', 'PDL2', 'PDR3', 'VL', 'PLM', 'PFB', \n           'PDR5', 'PLR1', 'PDL6', 'PLL3', 'PLR3', 'PDR6', 'PDM']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"51269532c618e6a7a518996cae86b5af99a401ca"},"cell_type":"code","source":"# load ngs data for concussions\n# files = os.listdir(data_path)\n# ngs_files = [x for x in files if 'NGS' in x]\n# ngs_raw = None\n# for idx, file in enumerate(ngs_files):\n#     print(file)\n#     df = pd.read_csv(os.path.join(data_path, file),\n#                     low_memory=False)\n#     df = df.set_index(['GameKey', 'PlayID'])\n#     df = df.loc[df.index.isin(concussion_data.index)]\n#     ngs_raw = df if ngs_raw is None else ngs_raw.append(df)\n#     del df\n#     gc.collect()\n# ngs_raw['GSISID'] = ngs_raw['GSISID'].astype(float)\n# ngs_raw = ngs_raw.merge(player_role_data[['Role', 'GSISID']], left_on=['GameKey', 'PlayID', 'GSISID'], right_on=['GameKey', 'PlayID', 'GSISID'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3ecbec0e89a50f25614dea8fecefe8355a769760"},"cell_type":"markdown","source":"First we wanted to look at if any specific player activity, impact type, or position accounted for more concussions during punt plays than others:"},{"metadata":{"trusted":true,"_uuid":"ff71f3ec2fc63f14ee38e3d51d3d039332093ff6","_kg_hide-input":true},"cell_type":"code","source":"# Get breakdown of concussion data by player activity\nfig = plt.figure(figsize=(15, 10))\nax = fig.add_subplot(221)\ncon_by_activity = concussion_data.Player_Activity_Derived.value_counts()\nplt.bar(con_by_activity.index, con_by_activity.values)\nplt.title('Concussion by Player Activity')\n\n# Get breakdown of concussion data by impact type\nax = fig.add_subplot(222)\ncon_by_impact = concussion_data.Primary_Impact_Type.value_counts()\nplt.bar(con_by_impact.index, con_by_impact.values)\nplt.title('Concussion by Impact Type')\n\n# Get breakdown of player positions\n# getting concussions and giving concussions\nax = fig.add_subplot(223)\ncon_by_pos = concussion_data.P1.value_counts()\nplt.bar(con_by_pos.index, con_by_pos.values)\nplt.xticks(rotation=45)\nplt.title('Players Receiving Concussions by Position')\n\nax = fig.add_subplot(224)\ngive_con_by_pos = concussion_data.P2.value_counts()\nplt.bar(give_con_by_pos.index, give_con_by_pos.values)\nplt.xticks(rotation=45)\nplt.title('Players Giving Concussions by Position')\nplt.show()\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"44f1fe26ef18e02242b8feecdcd2d442e80bdbe5"},"cell_type":"code","source":"con_by_pos = concussion_data.P1.value_counts()\ngive_con_by_pos = concussion_data.P2.value_counts()\ninvolved = con_by_pos.add(give_con_by_pos, fill_value=0).sort_values(ascending=False)\nplt.bar(involved.index, involved.values)\nplt.xticks(rotation=45)\nplt.title('Players Involved in Concussions by Position')\nplt.show()\n\ntrench = ['PRG', 'PDR1', 'PRT', 'PLT', 'PLW', 'PLG', 'PLS', 'PRW', 'PPR', 'PDL1', 'PDR2', 'PDL2', 'PDR3', 'PLL', 'PLL1']\nskilled = ['PR', 'GR', 'GL', 'P', 'VR', 'PFB', 'VLo']\ninvolved_trench = involved.loc[involved.index.isin(trench)].sum()\ninvolved_skilled = involved.loc[involved.index.isin(skilled)].sum()\nskill_vs_unskill = pd.Series(index=['Skilled', 'Unskilled'], data=[[involved_skilled], [involved_trench]])\nplt.bar(skill_vs_unskill.index, skill_vs_unskill.values)\nplt.xticks(rotation=45)\nplt.title('Players Involved in Concussions by Position Type')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"529b2c4464f125f28d6f1bdefc5e0251fc3323a0"},"cell_type":"markdown","source":"From this, we were able to see that concussions seemed to be clustered mostly around the PR and most offensive lineman.\n\nThis helped us see that we needed to focus on two separate rules involving position-specific changes."},{"metadata":{"_uuid":"2fcd4c070eea50fbb35ea5e86516c3eb2f4fac96"},"cell_type":"markdown","source":"**Illegal Formations**\n\nWe also used the below cells to analyze how differing defensive punt recovery packages could potentially increase the risk of concussions."},{"metadata":{"trusted":true,"_uuid":"2732dab46b0ed142a9597084a7348036616669b4"},"cell_type":"code","source":"# Create pivot table of formations\nformations = pd.pivot_table(player_role_data,index=['GameKey', 'PlayID'],columns=['Role'], aggfunc=lambda x: len(x.unique()))['GSISID'].fillna(0)\nmerged_data = pd.merge(formations, play_information_data, left_index=True, right_index=True)\nmerged_data.loc[merged_data.index.isin(concussion_data.index), 'concussed'] = 1\nmerged_data.concussed.fillna(0, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":false,"trusted":true,"_uuid":"4ede0a6922ed9f6d197821805742cf8755c622b4"},"cell_type":"code","source":"# Remove punt plays that are blocked or killed by penalties\nyards_list = []\nfor i,yards in enumerate(merged_data.PlayDescription.str.split(' yard').str[0].str[-2:]):\n    try:\n        yards_list.append(float(yards))\n    except ValueError:\n        yards_list.append(-500)\nmerged_data['punt_yards'] = yards_list\nmerged_data['no_play'] = merged_data.PlayDescription.str.contains('No Play', regex=True)\nmerged_data['blocked'] = merged_data.PlayDescription.str.contains('BLOCKED', regex=True)\nmerged_data = merged_data[(merged_data.punt_yards != -500) & (merged_data.no_play == False) & (merged_data.blocked == False)]\nX = (merged_data[defense].values).astype(None)\ny = merged_data['concussed'].values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1ffa8550316df0bab1cea20217c7abcbcb341b29"},"cell_type":"code","source":"# Cluster analysis\nfrom sklearn.cluster import KMeans, AffinityPropagation, AgglomerativeClustering\nfrom sklearn.decomposition import PCA\nn_clusters = 5\npca = PCA(n_components=n_clusters).fit(X) # use pca with kmeans because data are dichotomous \nmodel = KMeans(init=pca.components_, n_clusters=n_clusters, n_init=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b254b4517c05193fab5ad9085e290d46a493abe6"},"cell_type":"code","source":"# Histograms of clusters\nlabels = model.fit_predict(X)\nfig = plt.figure(figsize=(15, 5))\nax = fig.add_subplot(131)\nn1 = plt.hist(labels,bins=range(0,len(np.unique(labels))+1))\nplt.title('All Plays')\nplt.xlabel('Cluster #')\nplt.ylabel('# plays in cluster')\nax = fig.add_subplot(132)\nn2 = plt.hist(labels[y==1],bins=range(0,len(np.unique(labels))+1))\nplt.title('Concussion Plays')\nplt.xlabel('Cluster #')\nplt.ylabel('# plays in cluster')\nax = fig.add_subplot(133)\nplt.bar(np.unique(labels),100*(n2[0]/n1[0]))\nplt.xlabel('Cluster #')\nplt.ylabel('% concussion plays')\nplt.title('% Concussion Plays')\nidxMax = np.argmax(100*(n2[0]/n1[0])) # index of cluster w/ highest % concussions\n\n# Image of raw concussion play formations\nticks = np.arange(0,X.shape[1],1)\nfig = plt.figure(figsize=(20, 9))\nax = fig.add_subplot(121)\nax.matshow(X[y==1],aspect='auto')\nax.set_xticks(ticks)\nax.set_xticklabels(defense,rotation=45)\nplt.ylabel('Concussion Play')\nplt.title('All Concussion Plays')\n\nax = fig.add_subplot(122)\ncax = ax.matshow(X[(y==1) & (labels==idxMax)],aspect='auto')\nfig.colorbar(cax)\nax.set_xticks(ticks)\nax.set_xticklabels(defense,rotation=45)\nplt.ylabel('Concussion Play')\nplt.title('Concussion Plays in Cluster %d' %idxMax)\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"50206926d586842dfedf1085e909afef2e4d00e8"},"cell_type":"code","source":"# Bar plot of differences between mean of cluster idxMax and the rest\n# For statistical significance, run t-tests? ANOVAs? Need to think about this more...\nfig = plt.figure()\nax = fig.add_subplot(111)\nbar_width = 0.2\ncnt = 0\nfor i in range(0,5):\n    if i != idxMax:\n        plt.bar(np.arange(0,X.shape[1])+bar_width*(cnt-1),np.mean(X[labels==idxMax],axis=0)-np.mean(X[labels==i],axis=0),bar_width)\n        cnt += 1\nax.set_xticks(ticks)\nax.set_xticklabels(defense,rotation=45)\nplt.legend(['Cluster 1', 'Cluster 2','Cluster 3','Cluster 4'])\nax.set_title(r'Comparision: $\\mu_{C0}$ - $\\mu_{C1, C2, C3, or C4}$')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"134444702cdbea81d6e9cb67b842bf50e9935dcd"},"cell_type":"code","source":"# gets all high-speed collisions to use for further analysis\n# all_collisions = None\n# coll_data = ngs_raw.reset_index()\n# coll_data.Time = coll_data.Time.astype(np.datetime64)\n# coll_data = coll_data.pivot_table(values=['dis', 'x', 'y'], index=['GameKey', 'PlayID', 'Time'], columns='Role').sort_index()\n# # Converts distance to velocity in mph, courtesy of mtodisco10\n# coll_data.dis = coll_data.dis.mul(20.455)\n# for player in coll_data.x.columns:\n# #     print(player)\n#     player_dist = coll_data.x.sub(coll_data.x[player], axis=0).pow(2).add(coll_data.y.sub(coll_data.y[player], axis=0).pow(2)).pow(.5)\n#     player_dist = player_dist[offense] if player in defense else player_dist[defense]\n#     player_dist = player_dist[(player_dist < .5)].dropna(how='all', axis=0).stack().to_frame()\n#     player_dist.columns = ['DIST']\n#     player_vel = coll_data.dis\n#     this_player_vel = player_vel.loc[:, player]\n#     this_player_vel.columns = ['V2']\n#     this_player_vel_change = this_player_vel.groupby(level=['GameKey', 'PlayID']).pct_change()\n#     player_vel = player_vel[(player_vel > 7)].dropna(how='all', axis=0)\n#     player_vel = player_vel[(this_player_vel_change < -.20)].stack().to_frame()\n#     player_vel.columns = ['V1']\n#     collisions = player_dist.merge(player_vel, left_index=True, right_index=True).reset_index(level='Role')[['Role', 'V1']]\n#     collisions.columns = ['P1', 'V1']\n#     collisions['V2'] = this_player_vel\n#     collisions['P2'] = player\n#     collisions.P1 = collisions.P1.astype(str)\n#     collisions = collisions.loc[collisions.P1 != player]\n#     collisions = collisions.reset_index().groupby(by=['GameKey', 'PlayID', 'P1']).first()\n#     collisions = collisions.reset_index(['P1'])\n#     all_collisions = collisions if all_collisions is None else all_collisions.append(collisions)\n# all_collisions.to_csv('collisions.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"52f7a4a9ce25345ea64443c3e0d38870f1d41b8a"},"cell_type":"code","source":"# Collision data, I changed the 7mph threshold to 10mph. If a play contains a\n# high speed impact, it is assigned a label of \"1\"\ncollision_data = pd.read_csv('../input/collisions/collisions.csv', index_col=['GameKey', 'PlayID'])\ncollision_data_thresh = collision_data[collision_data['V1'] > 10]\n# collision_idx = np.unique(collision_data_thresh[['GameKey','PlayID']],axis=0)\n# merged_data = merged_data.reset_index()\n# merged_data['collided'] = 0\n# for i in collision_idx:\n#     if sum((merged_data['GameKey']==i[0]) & (merged_data['PlayID']==i[1]))==1:\n#         idx = merged_data[(merged_data['GameKey']==i[0]) & (merged_data['PlayID']==i[1])].index[0]\n#         merged_data['collided'][idx] = 1\nmerged_data.loc[merged_data.index.isin(collision_data.index), 'collided'] = 1\nmerged_data.loc[~merged_data.index.isin(collision_data.index), 'collided'] = 0\n\n# How many of the high speed impacts are in cluster idxMax\ncollisions = merged_data['collided']\nfig = plt.figure()\nax = fig.add_subplot(121)\nn3 = plt.hist(labels[collisions==1],bins=range(0,len(np.unique(labels))+1))\nplt.title('High Speed Plays')\nplt.xlabel('Cluster #')\nplt.ylabel('# plays in cluster')\n           \nax = fig.add_subplot(122)\nplt.bar(np.unique(labels),100*(n3[0]/n1[0]))\nplt.xlabel('Cluster #')\nplt.ylabel('% high speed plays')\nplt.title('% high speed plays')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"74b29ac34d76983e0532b9eeba44be6642f9da74"},"cell_type":"markdown","source":"**Plotting Functions**"},{"metadata":{"trusted":true,"_uuid":"2147be656c7cfe042c5b3c073aa0e1788fd2568b"},"cell_type":"code","source":"# Creates an image of the starting punt formation\ndef create_formation(gameKey, playId):\n    \n    # get data\n    formation = ngs_raw.loc[(gameKey, playId)]\n    print(formation.Event.unique())\n    formation = formation.loc[formation.Event == 'line_set']    \n    \n    fig, ax = plt.subplots(1,1,figsize=(15,10))\n    ax.set_facecolor('green')\n#     img = plt.imread(os.path.join(images_path, 'footballfield.png'))\n#     ax.imshow(img, extent=[0, 120, 0, 53.3])\n    ax.set_xlim([0, 120])\n    ax.set_ylim([0, 53.3])\n    formation['Pos'] = list(zip(formation.x,formation.y))\n    formation_o = formation.loc[formation.Role.isin(offense), :]\n    formation_d = formation.loc[formation.Role.isin(defense), :]\n    formation_o = formation_o.pivot(values=['x', 'y'], index='Time', columns='Role').sort_index().fillna(method='ffill')\n    formation_d = formation_d.pivot(values=['x', 'y'], index='Time', columns='Role').sort_index().fillna(method='ffill')\n\n    formation_o = formation_o.loc[formation_o.index.isin(formation_d.index)]\n    formation_d = formation_d.loc[formation_d.index.isin(formation_o.index)]\n    colors_o = ['#0103f4' for x in formation_o.loc[:, 'x'].columns]\n    colors_d = ['black' for x in formation_d.loc[:, 'x'].columns]\n    sc_o = ax.scatter([], [], s=75, marker='o')\n    sc_d = ax.scatter([], [], s=75, marker='x')\n#     print(formation_o.x)\n    sc_o.set_offsets(np.c_[formation_o.iloc[0].x.values, formation_o.iloc[0].y.values])\n    sc_o.set_facecolor('none')\n    sc_o.set_edgecolor(colors_o)\n    \n    sc_d.set_offsets(np.c_[formation_d.iloc[0].x.values, formation_d.iloc[0].y.values])\n    sc_d.set_color(colors_d)\n\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7e5757a33e2145edda5cde9ecfaea1947a9e7ce1"},"cell_type":"code","source":"# Creates and saves a gif of a given punt play\ndef create_punt_gif(gameKey, playId):\n    %matplotlib inline\n    ngs = ngs_raw.loc[(gameKey, playId)]\n    fig, ax = plt.subplots(1,1,figsize=(15,10))\n    img = plt.imread(os.path.join(images_path, 'footballfield.png'))\n    ax.imshow(img, extent=[0, 120, 0, 53.3])\n    ax.set_xlim([0, 120])\n    ax.set_ylim([0, 53.3])\n    ngs['Pos'] = list(zip(ngs.x, ngs.y))\n    ngs_o = ngs.loc[ngs.Role.isin(offense), :]\n    ngs_d = ngs.loc[ngs.Role.isin(defense), :]\n    ngs_o = ngs_o.pivot(values=['x', 'y'], index='Time', columns='Role').sort_index().fillna(method='ffill')\n    ngs_d = ngs_d.pivot(values=['x', 'y'], index='Time', columns='Role').sort_index().fillna(method='ffill')\n#     play_df = play_df.fillna(method='ffill')\n#     play_df = play_df.iloc[::5, :]\n    ngs_o = ngs_o.loc[ngs_o.index.isin(ngs_d.index)]\n    ngs_d = ngs_d.loc[ngs_d.index.isin(ngs_d.index)]\n    colors_o = ['blue' for x in ngs_o.loc[:, 'x'].columns]\n    colors_d = ['black' for x in ngs_d.loc[:, 'x'].columns]\n    sc_o = ax.scatter([], [], marker='o')\n    sc_d = ax.scatter([], [], marker='x')\n    ngs.Event = ngs.Event.astype(str)\n    ngs = ngs.groupby('Time').first()\n    title = ax.text(60, 57, \"\")\n    time = ax.text(110, 57, \"\")\n\n    def animate(i):\n#         print(ngs_o.loc[i, 'x'])\n        sc_o.set_offsets(np.c_[ngs_o.loc[i, 'x'].values, ngs_o.loc[i, 'y'].values])\n        sc_o.set_facecolor('none')\n        sc_o.set_edgecolor(colors_o)\n        sc_d.set_offsets(np.c_[ngs_d.loc[i, 'x'].values, ngs_d.loc[i, 'y'].values])\n        sc_d.set_color(colors_d)\n        time.set_text(i)\n        if ngs.loc[i, 'Event'] != 'nan':\n            title.set_text(ngs.loc[i, 'Event'].upper())\n\n    ani = FuncAnimation(fig, animate, frames=ngs_o.index, interval=100, repeat=True)\n    ani.save('footballplay{:s}-{:s}.gif'.format(str(gameKey), str(playId)), writer='imagemagick')\n    plt.clf()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}