{"cells":[{"metadata":{"_uuid":"ad02a9c2c2366797d358af1a0d18ea3d4905b1f9"},"cell_type":"markdown","source":"Below you will find my results and recommendations for the Punt Analytics Competition.\n\n\nI was able to articulate more details, provide more visuals, and discuss pros and cons in my powerpoint presentation attached.\n\n\nThe aim of this study is to analyze league punting data from 2016 and 2017, and recommend rules that will reduce concussions on punting plays while maintaining the integrity of the game.\n"},{"metadata":{"trusted":true,"_uuid":"73e312435d0cc2e149ea3bbacf68dcbaf61fe975"},"cell_type":"code","source":"import pandas as pd\nimport scipy\nimport matplotlib.pyplot as plt\nimport numpy as np\n\ngame_data = pd.read_csv('../input/game_data.csv')\nplay_info = pd.read_csv('../input/play_information.csv')\nplay_player_role = pd.read_csv('../input/play_player_role_data.csv')\nplayer_punt = pd.read_csv('../input/player_punt_data.csv')\ninjuries = pd.read_csv('../input/video_review.csv')\npre_2016 = pd.read_csv('../input/NGS-2016-pre.csv')\nreg_2016_1 = pd.read_csv('../input/NGS-2016-reg-wk1-6.csv')\nreg_2016_2 = pd.read_csv('../input/NGS-2016-reg-wk7-12.csv')\nreg_2016_3 = pd.read_csv('../input/NGS-2016-reg-wk13-17.csv')\npost_2016 = pd.read_csv('../input/NGS-2016-post.csv')\npre_2017 = pd.read_csv('../input/NGS-2017-pre.csv')\nreg_2017_1 = pd.read_csv('../input/NGS-2017-reg-wk1-6.csv')\nreg_2017_2 = pd.read_csv('../input/NGS-2017-reg-wk7-12.csv')\nreg_2017_3 = pd.read_csv('../input/NGS-2017-reg-wk13-17.csv')\npost_2017 = pd.read_csv('../input/NGS-2017-post.csv')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"975446276bc88e067f5ac5bca10beb61c6205a2b"},"cell_type":"markdown","source":"Step 1 - Assess the problem\n\nHere I am checking to see what the magnitude of the problem is, and creating a direction for the analysis"},{"metadata":{"trusted":false,"_uuid":"55f69f2442c6bad1111c6d0f7f6127cb7be44ddd"},"cell_type":"code","source":"punt_count = len(play_info)\ncon_count = len(injuries)\n\nprint(\"Number of punt plays: \")\nprint(punt_count) \nprint(\"Number of concussion plays: \")\nprint(con_count)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"25c53aad5df6b9056000a18e98895576b1debf81"},"cell_type":"markdown","source":"It is interesting to note that while concussions are definitely a problem, the actual number is very small (37). Making the problem even more difficult to solve. "},{"metadata":{"_uuid":"36d17cf264dea62b8c0bb0f1e67afe116fab26c1"},"cell_type":"markdown","source":"Within punting plays, let's investigate which are the most dangerous. To do so, we will crawl through the play description and categorize each play as a fair catch, out of bounds, touchback, return, muff, or downed."},{"metadata":{"trusted":false,"_uuid":"3efe16fc02d1a803ea337e2ee3c3a6cc41cef5cb"},"cell_type":"code","source":"#Outcomes of punts\n#1- Fair Catch\n#2- Out of bounds\n#3- Touchback\n#4- Return\n#5- Muff\n#6- Downed\n\nunique_con_id = []\n\nfor row in range(len(injuries)):\n    game_id = str(injuries.iloc[row,1])\n    play_id = str(injuries.iloc[row,2])\n    unique_con_id.append(game_id+play_id)\n\nfc = 0\nob = 0\ntb = 0\nrt = 0\nmf = 0\ndn = 0\nfc_c = 0\nob_c = 0\ntb_c = 0\nrt_c = 0\nmf_c = 0\ndn_c = 0\n\nids = []\nrt_ids = []\nrt_con_ids = []\n\n\nfor i in range(len(play_info)):\n    game_id_1 = str(play_info.iloc[i,2])\n    play_id_1 = str(play_info.iloc[i,5])\n    uni_id = game_id_1+play_id_1\n    \n    ids.append(uni_id)\n    \n    if \"fair catch\" in play_info.iloc[i,13]:\n        fc += 1\n    if \"out of bounds\" in play_info.iloc[i,13]:\n        ob += 1\n    if \"Touchback\" in play_info.iloc[i,13]:\n        tb += 1\n    if \"for\" in play_info.iloc[i,13]:\n        rt += 1\n        rt_ids.append(uni_id)\n    if \"MUFFS\" in play_info.iloc[i,13]:\n        mf += 1\n    if \"downed\" in play_info.iloc[i,13]:\n        dn += 1\n    if uni_id in unique_con_id:\n        if \"fair catch\" in play_info.iloc[i,13]:\n            fc_c += 1\n        if \"out of bounds\" in play_info.iloc[i,13]:\n            ob_c += 1\n        if \"Touchback\" in play_info.iloc[i,13]:\n            tb_c += 1\n        if \"for\" in play_info.iloc[i,13]:\n            rt_c += 1\n            rt_con_ids.append(uni_id)\n        if \"MUFFS\" in play_info.iloc[i,13]:\n            mf_c += 1\n        if \"downed\" in play_info.iloc[i,13]:\n            dn_c += 1\n\noutcomes = ['Fair Catch', 'OB', 'Touchback', 'Return', 'Muff', 'Downed']\ntotal_plays = [fc,ob,tb,rt,mf,dn]\nconc_plays = [fc_c,ob_c,tb_c,rt_c,mf_c,dn_c]\n\ndc = {'Outcomes': outcomes, 'Concussion Plays': conc_plays, 'Total Plays': total_plays, \n      'Percent of Inj': np.array(conc_plays)*100/np.array(total_plays)}\ndf = (pd.DataFrame(dc))\n\nprint(df)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7139628dbae4a5de26adf526df851bdc826e0b69"},"cell_type":"markdown","source":"Punt plays that result in returns appear to be more dangerous than the rest"},{"metadata":{"_uuid":"869b051f2c081f747db0db21807a3a0a017001bf"},"cell_type":"markdown","source":"Next I want to investiage who is most likely to suffer a concussion on a punting play"},{"metadata":{"trusted":false,"_uuid":"c3739a82b333cd20e4480f6dbf85f3a377080254"},"cell_type":"code","source":"freq = injuries.groupby('Player_Activity_Derived').count()\nplt.bar(freq.index.values,freq.iloc[:,0])\nplt.title(\"Activity of Injured Player During Punt Returns\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"50ca3c7e1377b35e2ae14e2de62ae913dcf94351"},"cell_type":"markdown","source":"There is a fairly even distribution among the players who get injured. I was surprised to see that the returner isn't at much at risk as the blockers and tacklers.\n\nThere does appear to be a high proportion of players injured who are on the punting team (blocked and tackling). It's evident that something needs to change to prevent those players from sustaining injuries."},{"metadata":{"_uuid":"222df43a43aa892583403fbe0120b24e42b628cd"},"cell_type":"markdown","source":"Next I hope to investigate if there are any other factors in the dataset that are correlated with concussions. To do so, I am creating a giant correlation matrix of each play using the data in the various spreadsheets. The factors I am including are:\n\n1) Number of box defenders\n\n2) Number of outside blockers\n\n3) Number of returners \n\n4) Return yards\n\n5) Punt distance\n\n6) Punt hangtime\n\n7) Result of play (fair catch, return, out of bounds, etc.)\n\n8) Distance from center of field ball is snapped from\n\n9) Line of scrimmage (defined as distance from end zone)\n\n10) Whether there was any injury on the play or not\n\n11) Whether there was any penalty on the play or not\n\nThe categorical response variable is was there a concussion on the play - yes(1) or no(0)\n\n---\n\nMany factors that are not involved in the actual play were not included, such as weather, score, indoor/outdoor, home team, away team, year, regular season/post season/pre season, etc. I chose not to include these because there is nothing actionable that can come out of including these variables in the analysis. None of them can be changed with a rule change. While I'm sure something like weather affects if there is a return or not, nothing can be done about this.\n"},{"metadata":{"trusted":false,"_uuid":"c5b6ed7a953f39f9d7eaf2c0711c5ec2a7d20911"},"cell_type":"code","source":"#Create unique IDs of all punt plays\nids = []\n\nfor i in range(len(play_info)):\n    game_id_1 = str(play_info.iloc[i,2])\n    play_id_1 = str(play_info.iloc[i,5])\n    uni_id = game_id_1+play_id_1    \n    ids.append(uni_id)\n\n#Part 1 - Get Formations\n\nformation = {}\nfor i in ids:\n    formation[i] = []\n\nfor j in range(len(play_player_role)):\n    game_id = str(play_player_role.iloc[j,1])\n    play_id = str(play_player_role.iloc[j,2])\n    uni_id = game_id+play_id\n    \n    if uni_id in ids:\n        formation[uni_id].append(play_player_role.iloc[j,4])\n\ndef formation_matrix(dictionary_of_formations):\n    \n    fomration_mat = {}\n    \n    for i in dictionary_of_formations:\n        \n        fomration_mat[i] = []\n        \n        box = 0\n        OBs = 0\n        RTs = 0\n        \n        for j in dictionary_of_formations[i]:\n            #Linemen\n            if j == 'PDL2':\n                box += 1\n            elif j == 'PDR2':\n                box += 1\n            elif j == 'PDR1':\n                box += 1\n            elif j == 'PDL1':\n                box += 1\n            elif j == 'PDR4':\n                box += 1\n            elif j == 'PDL4':\n                box += 1        \n            elif j == 'PDR5':\n                box += 1\n            elif j == 'PDL5':\n                box += 1\n            elif j == 'PDL3':\n                box += 1\n            elif j == 'PDR3':\n                box += 1\n            \n            \n            #Linebackers\n            elif j == 'PLM':\n                box += 1\n            elif j == 'PLR':\n                box += 1\n            elif j == 'PLL':\n                box += 1\n            elif j == 'PLL1':\n                box += 1\n            elif j == 'PLR1':\n                box += 1\n            elif j == 'PLL2':\n                box += 1\n            elif j == 'PLR2':\n                box += 1\n            elif j == 'PLL3':\n                box += 1\n            elif j == 'PLR3':\n                box += 1\n\n            #Outsiders\n            elif j == 'VL':\n                OBs += 1\n            elif j == 'VRi':\n                OBs += 1\n            elif j == 'VRo':\n                OBs += 1\n            elif j == 'VR':\n                OBs += 1\n            elif j == 'VLi':\n                OBs += 1\n            elif j == 'VLo':\n                OBs += 1\n\n                \n            #Returnmen\n            elif j == 'PR':\n                RTs += 1\n            elif j == 'PFB':\n                RTs += 1\n            elif j == 'PFB1':\n                RTs += 1\n            elif j == 'PFB2':\n                RTs += 1\n        \n        fomration_mat[i].append(box)\n        fomration_mat[i].append(OBs)\n        fomration_mat[i].append(RTs)\n    \n    return fomration_mat\n\nTHEmatrix = formation_matrix(formation)\n\n#Part 2 - Get Return Yards\n\nfor i in range(len(play_info)):\n    game_id = str(play_info.iloc[i,2])\n    play_id = str(play_info.iloc[i,5])\n    pid = game_id+play_id\n    play = (str(play_info.iloc[i,-1]))\n    if \" for \" in play:\n        result = play.split(\" for \",1)[1]\n        yards = result[0:2]\n        if yards == 'no':\n            yards_ = 0\n        if yards == 'on':\n            yards = yards_  \n        if yards == 'L.':\n            yards = yards_  \n        if yards != 'no':\n            yards_ = int(yards)\n    else:\n        yards_ = 0\n    if pid in THEmatrix:\n        THEmatrix[pid].append(yards_)\n\n#Part 3 - Get Punt Dist\n\nfor i in range(len(play_info)):\n    game_id = str(play_info.iloc[i,2])\n    play_id = str(play_info.iloc[i,5])\n    pid = game_id+play_id\n    play = (str(play_info.iloc[i,-1]))\n    if \" punts \" in play:\n        result = play.split(\" punts \",1)[1]\n        yards = result[0:2]\n        if yards != 'no':\n            yards_ = int(yards) \n        if pid in THEmatrix:\n            THEmatrix[pid].append(yards_)\n            \n#Part 4 - Get Hangtime\n\ndef hangtime(df, start_event='punt', *stop_events):\n    \n    punt_event = df.loc[df.Event==start_event] \\\n        .groupby(['Season_Year', 'GameKey','PlayID'], as_index = False)['Time'].min()\n    punt_event.rename(columns = {'Time':'punt_time'}, inplace=True)\n    punt_event['punt_time'] = pd.to_datetime(punt_event['punt_time'],\\\n                                             format='%Y-%m-%d %H:%M:%S.%f')\n    \n    receiving_event = df.loc[df.Event.isin(stop_events)] \\\n        .groupby(['Season_Year', 'GameKey','PlayID'], as_index = False)['Time'].min()\n    receiving_event.rename(columns = {'Time':'receiving_time'}, inplace=True)\n    receiving_event['receiving_time'] = pd.to_datetime(receiving_event['receiving_time'],\\\n                                             format='%Y-%m-%d %H:%M:%S.%f')\n    \n    punt_df = punt_event.merge(receiving_event, how='inner', on = ['Season_Year','GameKey','PlayID']) \\\n                .reset_index(drop=True)\n    \n    punt_df['hang_time'] = (punt_df['receiving_time'] - punt_df['punt_time']).dt.total_seconds()\n    \n    return punt_df\n\ndef add_ht_to_matrix(data):\n    for i in range(len(data)):\n        game_id = str(data.iloc[i,1])\n        play_id = str(data.iloc[i,2])\n        pid = game_id+play_id\n        \n        if pid in THEmatrix:\n            THEmatrix[pid].append(data.iloc[i,-1])\n            \nlist2016 = [pre_2016, reg_2016_1, reg_2016_2, reg_2016_3, post_2016]\nlist2017 = [pre_2017, reg_2017_1, reg_2017_2, reg_2017_3, post_2017]\n\nfor i in list2016:\n    add_ht_to_matrix(hangtime(i, 'punt', 'punt_received', 'fair_catch', 'punt_downed', 'touchback','out_of_bounds'))\n    \nfor j in list2017:\n    add_ht_to_matrix(hangtime(j, 'punt', 'punt_received', 'fair_catch', 'punt_downed', 'touchback','out_of_bounds'))\n\n#Part 5 - Get Result and add to matrix\nfor i in range(len(play_info)):\n    game_id_1 = str(play_info.iloc[i,2])\n    play_id_1 = str(play_info.iloc[i,5])\n    uni_id = game_id_1+play_id_1\n        \n    if \"fair catch\" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n    \n    if \"out of bounds\" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n    \n    if \"Touchback\" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n\n\n    if \" for \" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n\n    if \"MUFFS\" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n\n    if \"downed\" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n    if \"PENALTY\" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n    if \" injured \" in play_info.iloc[i,13]:\n        THEmatrix[uni_id].append(1)\n    else:\n        THEmatrix[uni_id].append(0)\n\n#Part 6 - Add y coordinate to matrix\n\nLS = player_punt['Position']=='LS'\nLS_list = list(player_punt[LS]['GSISID'])\n\ndef ls_coordinates(NGSdata, matrix):\n    LS = NGSdata['GSISID'].isin(LS_list)\n    LS_matrix = NGSdata[LS]\n        \n    pid_old = ''\n    for i in range(len(LS_matrix)):\n        game_id = str(LS_matrix.iloc[i,1])\n        play_id = str(LS_matrix.iloc[i,2])\n        pid = game_id+play_id\n        y_coord = LS_matrix.iloc[i,6]\n        midfield = 53.3 / 2\n        dist_from_mid = abs(y_coord - midfield)\n        \n        if pid != pid_old:\n            if pid in matrix:\n\n                matrix[pid].append(dist_from_mid)\n\n        pid_old = pid\n\nfor i in list2016:\n    ls_coordinates(i, THEmatrix)\n    \nfor j in list2017:\n    ls_coordinates(j, THEmatrix)\n    \n#Part 7 - Add line of scrimmage(distance from end zone)\n\nfor row in range(len(play_info)):\n    game_id = str(play_info.iloc[row,2])\n    play_id = str(play_info.iloc[row,5])\n    pid = game_id+play_id\n\n    poss = play_info.iloc[row,10]\n    ydline = int((play_info.iloc[row,7])[-2:])\n    \n    if poss in play_info.iloc[row,7]:\n        THEmatrix[pid].append(100 - ydline)\n    else:\n        THEmatrix[pid].append(ydline)\n\n#Add Concussion Y-variable to data\n\ninjury_list = []\n\nfor i in range(len(injuries)):\n    game_id = str(injuries.iloc[i,1])\n    play_id = str(injuries.iloc[i,2])\n    pid = game_id+play_id\n    injury_list.append(pid)\n\nfor j in THEmatrix:\n    if j in injury_list:\n        THEmatrix[j].append(1)\n    else:\n        THEmatrix[j].append(0)\n        \n#Create matrix of dictionary THEmatrix\n\nplayid_mat = []\nbox_mat = []\nout_mat = []\nRT_mat = []\nret_yds_mat = []\npunt_yds_mat = []\nht_mat = []\nfc_mat = []\nob_mat = []\ntb_mat = []\nrt_mat = []\nmf_mat = []\ndn_mat = []\npen_mat = []\ninj_mat = []\ncen_mat = []\nlos_mat = []\ncon_mat = []\n\nfor i in THEmatrix:\n    if len(THEmatrix[i]) == 17:\n        playid_mat.append(i)\n        box_mat.append(THEmatrix[i][0])\n        out_mat.append(THEmatrix[i][1])\n        RT_mat.append(THEmatrix[i][2])\n        ret_yds_mat.append(THEmatrix[i][3])\n        punt_yds_mat.append(THEmatrix[i][4])\n        ht_mat.append(THEmatrix[i][5])\n        fc_mat.append(THEmatrix[i][6])\n        ob_mat.append(THEmatrix[i][7])\n        tb_mat.append(THEmatrix[i][8])\n        rt_mat.append(THEmatrix[i][9])\n        mf_mat.append(THEmatrix[i][10])\n        dn_mat.append(THEmatrix[i][11])\n        pen_mat.append(THEmatrix[i][12])\n        inj_mat.append(THEmatrix[i][13])\n        cen_mat.append(THEmatrix[i][14])\n        los_mat.append(THEmatrix[i][15])\n        con_mat.append(THEmatrix[i][16])\n\nd = {'PlayID': playid_mat,\n    'Box D': box_mat,\n    'Outside D': out_mat,\n    'Return D': RT_mat,\n    'Return Yds': ret_yds_mat,\n    'Punt Dist': punt_yds_mat,\n    'Hangtime': ht_mat,\n    'Dist From Center': cen_mat,\n    'Distance From End Zone': los_mat,\n    'Fair Catch': fc_mat,\n    'Out Of Bounds': ob_mat,\n    'Touchback': tb_mat,\n    'Return': rt_mat,\n    'Penalty': pen_mat,\n    'Injury': inj_mat,\n    'Muff': mf_mat,\n    'Downed': dn_mat,\n    'Concussion': con_mat}\n\ndf = pd.DataFrame(data=d)\n\nimport seaborn as sns\n\ncorr = df.corr()\nf, ax = plt.subplots(figsize = (16,16))\nax.matshow(corr)\nsns.heatmap(corr, mask=np.zeros_like(corr, dtype=np.bool), cmap=sns.diverging_palette(220, 10, as_cmap=True),\n            square=True, ax=ax)\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"95d12532b1bc36e416af238f8dcfbdf8dea07fd8"},"cell_type":"markdown","source":"There are a few observations that are important to note from the correlation matrix:\n\n1) None of the features I've identified are positive or negatively correlated with concussions. This isn't overly surprising, as there are few observations with concussions. What this tells me is that concussions are more or less random occurances and can occur on any play.\n\n2) Second, I looked at 'Return' because we saw earlier that concussions are more likely to happen on returns than any other play. There are a couple of key insights here:\n\na. A punt is more likely to be returned if there are fewer defenders in the box and more defenders on the outside.\n\nb. A punt is more likely to be returned if it is a longer punt. Return yards are also positively correlated with punt distance.\n\nc. A punt is less likely to be returned if there is more hangtime.\n\n3) Box defenders are and outside defenders are highly correlated (negative). This means that the amount of returners is generally constant."},{"metadata":{"_uuid":"08c45921ce0bf18b0a53dcaf3368e7353aa2f316"},"cell_type":"markdown","source":"The first rule I chose to investigate is the amount of outside defenders the receiving team is allowed to have. What happened if we made it constant at 2"},{"metadata":{"trusted":false,"_uuid":"fd556e7a22bb8960b9487802e1015cc92bb9c294"},"cell_type":"code","source":"pivot = pd.pivot_table(df, values=['Fair Catch', 'Return', 'Injury', 'Punt Dist'], index=['Outside D'],\n                   aggfunc={'Fair Catch': np.mean,\n                           'Return': np.mean,\n                            'Injury': np.mean,\n                            'Punt Dist': np.mean\n                           })\n\nprint(pivot)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"63d157eaab29cec3b4ce09a975b763f928e140f5"},"cell_type":"markdown","source":"When there are 2 outside defenders there are more fair catches and fewer returns than when there are 3 and 4 outside returners. Punts are also shorter Next, I did some modeling to see what the potential impact of changing all returns to the 8-2-1 formation. "},{"metadata":{"_uuid":"374e9604106491fcb26f6d01b680292e90a457d2"},"cell_type":"markdown","source":"In the first model, I created a linear regression model to preidct punt distance. Then I set all formations to 8-2-1 and predicted the punt distance of these formations based on the regression formula.\n\nThe factors used for this model were - defensive formation, yard line, distance from center of the field."},{"metadata":{"trusted":false,"_uuid":"214027d16f82f68ca8397e79ad67e62e632b047d"},"cell_type":"code","source":"from sklearn.linear_model import LinearRegression\n\n#defender analysis\n\n#Change Box Defenders\nnew_box = [8] * len(df)\nnew_out = [2] * len(df)\nnew_back = [1] * len(df)\n\ndf['new_box'] = new_box\ndf['new_out'] = new_out\ndf['new_back'] = new_back\n\n#Predict Impact on punt distance\nX = df.iloc[:,[1,2,3,7,8]]\nY = df.iloc[:,[5]]\n\nX_new = df.iloc[:,[18,19,20,7,8]]\n\nreg = LinearRegression().fit(X, Y)\n\nnew_punt = reg.predict(X_new)\n\nprint(\"Old Punt Dist: \" +str(np.mean(np.array(Y))))\n\nprint(\"New Punt Dist: \" +str(np.mean(new_punt)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"efa9b378cab720d4db6f0d4f39d9548205268dad"},"cell_type":"markdown","source":"Next, given the projected punt distances of the new formation I created a Random Forest model to predict whether a punt would be returned or not given the formation, yard line, distance from cetner, and punt distance. The model was trained on the old data and applied to the new dataset (8-2-1 formation, new punt distances)."},{"metadata":{"trusted":false,"_uuid":"52610b32e39c8448ccd0c34c36db011ec70834e3"},"cell_type":"code","source":"#predict returns catches\n\nfrom sklearn.ensemble import RandomForestClassifier\n\ndf['Punt dist_new'] = new_punt\n\nX = df.iloc[:,[1,2,3,5,7,8]]\nY = df.iloc[:,[12]]\n\nX_new = df.iloc[:,[18,19,20,21,7,8]]\n\nclf = RandomForestClassifier(n_estimators=50)\nclf.fit(X, Y)\npred = (clf.predict(X_new))\n\nprint(\"Old RT Avg: \" +str(np.mean(np.array(Y))))\n\nprint(\"New RT Avg: \" +str(np.mean(pred)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4ee4169993e1fc387ac5c26ae0a88ea2fdb3f88b"},"cell_type":"markdown","source":"Eureka! The implementation of this new rule would result in fewer returns.\n\nLastly, we'll see if there is a major impact on concussions. The factors used in this dataset are formation, yard line, distance from cetner, punt distance, and whether there was a return."},{"metadata":{"trusted":false,"_uuid":"b8e613119bdd719abd7dede76513f1eff7a65cca"},"cell_type":"code","source":"#predict concussions\n\ndf['Return_new'] = pred\n\nX = df.iloc[:,[1,2,3,5,7,8,12]]\nY = df.iloc[:,[17]]\n\nX_new = df.iloc[:,[18,19,20,21,7,8,22]]\n\nclf = RandomForestClassifier(n_estimators=50)\nclf.fit(X, Y)\npred = (clf.predict(X_new))\n\nprint(\"Old Concussion Rate: \" +str(np.mean(np.array(Y))))\n\nprint(\"New Concussion Rate: \" +str(np.mean(pred)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bfe62ac74781ee2484acb57dfc36205b5a0c0739"},"cell_type":"markdown","source":"We see a very significant reduction in concussions here! However, because of the nature of the dataset, it would be best to rebalance the data using SMOTE. This will make it easier to predict instances in the minority class. "},{"metadata":{"trusted":false,"_uuid":"d341087ece359c8397429b4a068cca2fe64a161d"},"cell_type":"code","source":"from collections import Counter\nfrom imblearn.over_sampling import SMOTE\nfrom sklearn.model_selection import train_test_split\nfrom sklearn import metrics\n\n\nprint('Original dataset shape %s' % Counter(Y))\n\nx2 = df.iloc[:,[1,2,3,5,7,8,12,18,19,20,21,22]]\ny2 = df.iloc[:,[17]]\n\nsm = SMOTE(random_state=42)\n\nX_res, y_res = sm.fit_resample(x2, y2)\n\nprint('Resampled dataset shape %s' % Counter(y_res))\n\ndf_x = pd.DataFrame(X_res)\n\nx_train = df_x.iloc[:,[0,1,2,3,4,5,6]]\nx_test = df_x.iloc[:,[7,8,9,10,4,5,6]]\n\n\ntraining_data, test_data, training_data_label, test_data_label = train_test_split(x_train, y_res, test_size = .3, random_state = 42)\n\nclf = RandomForestClassifier()\nclf = clf.fit(training_data, training_data_label)\n\npredictions = clf.predict(test_data)\n\nlearning_accuracy = clf.score(test_data, test_data_label)\n\nprint(\"Learning Accuracy\")\nprint(learning_accuracy)\nprint()\nprint(\"Scores\")\nprint(metrics.classification_report(predictions, test_data_label))\n\n\nclf = RandomForestClassifier(n_estimators=50)\nclf.fit(x_train, y_res)\npred = (clf.predict(x_test))\n\nprint(\"Old Concussion Rate: \" +str(np.mean(np.array(y_res))))\n\nprint(\"New Concussion Rate: \" +str(np.mean(pred)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5f2c3df936fda4cbcfce430c4cd56b353f56517e"},"cell_type":"markdown","source":"After rebalancing the dataset, I ran a proper train-test study to check the accuracy of the new model. We see a very high learning accuracy!\n\nWhen applying the Random Forest model to the new dataset, we see a 78% reduction in concussions."},{"metadata":{"_uuid":"08a5785e47cf3f918874abf8c6e6bd1a12bb89d1"},"cell_type":"markdown","source":"Next, using linear regression I want to check the impact on average punt return distance. While I suspect there will be fewer return yards, I'm curious if there will be more big plays. "},{"metadata":{"trusted":false,"_uuid":"642d2cf6d16f12ac07edc597c619ed43d8b45db7"},"cell_type":"code","source":"#predict return averages\n\nfrom sklearn.linear_model import LinearRegression\n\n#Predict Impact on return distance\n\nX = df.iloc[:,[1,2,3,5,7,8,10]]\nY = df.iloc[:,[4]]\n\nX_new = df.iloc[:,[18,19,20,21,7,8,22]]\n\nreg = LinearRegression().fit(X, Y)\n\nnew_ret = reg.predict(X_new)\n\nprint(\"Old Ret Dist: \" +str(np.mean(np.array(Y))))\n\nprint(\"New Ret Dist: \" +str(np.mean(new_ret)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1d7fed29a187106b4d678668e34744c994d5b1cd"},"cell_type":"markdown","source":"We see slightly less yards per return using the new rules. \n\nLinear regression has a hard time predicting outliers (big returns), so instead I will take a look at some of the plays and make observations on potential return distance based on field spacing. This should be interpreted as more of an observation than a science as I've only looked at a handful of plays. In the examples below, I've isolated 2 similar plays. Both punts started from around the 25 and were 45-50 yard punts that were returned."},{"metadata":{"_uuid":"9ba04f03f3379f048f6111706745dabbccae81dc"},"cell_type":"markdown","source":"Play 1 is a 7-3-1 formation. The first visual shows the players on the field when the punt happens. The second visual shows the players on the field when the punt is received.\n\n\nIt is interesting to note the lack of spacing on the field in visual 2. This play ended up as a 10 yard gain, which makes sense given where all the players are positioned on the field."},{"metadata":{"trusted":false,"_uuid":"b10ae0c4695185071e2c03123545b9bbeca10888"},"cell_type":"code","source":"def visualize_players(game_id, play_id, action, matrix):\n    punt_game = matrix.loc[matrix.GameKey==game_id]\n    punt_play = punt_game.loc[punt_game.PlayID==play_id]\n    action = punt_play.loc[punt_play.Event==action]\n    \n    plt.scatter(action.x,action.y)\n\nvisualize_players(192,2801,'punt',reg_2016_2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"cb12034996c2e08a3fd60ac7871a637d0b9bf9c4"},"cell_type":"code","source":"visualize_players(192,2801,'punt_received',reg_2016_2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9f6a6e7b9addd9e6ac74d36934eaecf4606aed0c"},"cell_type":"markdown","source":"Play 2 is a 8-2-1 formation. The first visual shows the players on the field when the punt happens. The second visual shows the players on the field when the punt is received.\n\n\nIt is interesting to note the increased spacing on the field in visual 2. This play ended up as a 27 yard gain. This is because he was able to make the initial defenders miss and find open space afterward for the big gain.\n\nThis is a common trend amongst returns in the 8-2-1 formation."},{"metadata":{"trusted":false,"_uuid":"80b3bba02a1405be3e995f45223c5d8b3ca5551e"},"cell_type":"code","source":"visualize_players(582,3079,'punt',reg_2017_3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"b728ed81e9ae565809241ebee8eba4f52bf645ba"},"cell_type":"code","source":"visualize_players(582,3079,'punt_received',reg_2017_3)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"28b7ce0cab70fe81fee032d633c65273210049ee"},"cell_type":"markdown","source":"The following analyses were potential rules that I looked into but did not have sufficient enough evidence to include in my final recommendations"},{"metadata":{"_uuid":"ba9b3ce3d1103d7cda06a1662e89499e59629d34"},"cell_type":"markdown","source":"1 - Investigating whether 'gunners' are at more risk than other positions. If so, there could be potential to remove that position or modify it in some way."},{"metadata":{"trusted":false,"_uuid":"394082dabc3bc461d593bc9f5c014d1b910a4a28"},"cell_type":"code","source":"injured_players = pd.merge(injuries, play_player_role, on=['GSISID','PlayID','GameKey'])\n\ndef get_positions(lst):\n    PRW = 0\n    PDL2 = 0\n    PRG = 0\n    PLS = 0 \n    PLW = 0\n    P = 0\n    PLM = 0\n    PLG = 0\n    GL = 0\n    VL = 0\n    PFB = 0\n    PRT = 0\n    PLT = 0\n    GR = 0\n    PPL = 0\n    VRi = 0\n    PDR2 = 0\n    PDR1 = 0\n    VRo = 0\n    PR = 0\n    PDL1 = 0\n    PDR3 = 0\n    PLR = 0\n    PDR4 = 0\n    PPR = 0\n    VR = 0\n    VLi = 0\n    PDR5 = 0\n    PLL1 = 0\n    PDL3 = 0\n    PLL2 = 0\n    PLR2 = 0\n    PLL = 0\n    VLo = 0\n    PLR1 = 0\n    PDL4 = 0\n\n    for j in lst:\n        if j == 'PRW':\n            PRW += 1\n        elif j == 'PDL2':\n            PDL2 += 1\n        elif j == 'PRG':\n            PRG += 1\n        elif j == 'PLS':\n            PLS += 1\n        elif j == 'PLW':\n            PLW += 1\n        elif j == 'P':\n            P += 1\n        elif j == 'PLM':\n            PLM += 1\n        elif j == 'PLG':\n            PLG += 1\n        elif j == 'GL':\n            GL += 1\n        elif j == 'VL':\n            VL += 1\n        elif j == 'PFB':\n            PFB += 1\n        elif j == 'PRT':\n            PRT += 1\n        elif j == 'PLT':\n            PLT += 1\n        elif j == 'GR':\n            GR += 1\n        elif j == 'PPL':\n            PPL += 1\n        elif j == 'VRi':\n            VRi += 1\n        elif j == 'PDR2':\n            PDR2 += 1\n        elif j == 'PDR1':\n            PDR1 += 1\n        elif j == 'VRo':\n            VRo += 1\n        elif j == 'PR':\n            PR += 1\n        elif j == 'PDL1':\n            PDL1 += 1\n        elif j == 'PLR':\n            PLR += 1\n        elif j == 'PDR4':\n            PDR4 += 1\n        elif j == 'PPR':\n            PPR += 1\n        elif j == 'VR':\n            VR += 1\n        elif j == 'VLi':\n            VLi += 1\n        elif j == 'PDR5':\n            PDR5 += 1\n        elif j == 'PLL1':\n            PLL1 += 1\n        elif j == 'PDL3':\n            PDL3 += 1\n        elif j == 'PLL3':\n            PLL2 += 1\n        elif j == 'PLR2':\n            PLR2 += 1\n        elif j == 'PLL':\n            PLL += 1\n        elif j == 'VLo':\n            VLo += 1\n        elif j == 'PLR1':\n            PLR1 += 1\n        elif j == 'PDL4':\n            PDL4 += 1        \n\n    p_gunners = (GL + GR)\n    p_linemen = (PLT+PLG+PLS+PRG+PRT)\n    p_wings = (PLW+PRW)\n    p_protect = (PPR+PPL)\n    p_punter = (P)\n    r_outside = (VRo+VRi+VR+VLo+VLi+VL)\n    r_linemen = (PDR3+PDR2+PDR1+PDR4+PDR5+PDL3+PDL2+PDL1+PDL4)\n    r_lbs = (PLR+PLR1+PLR2+PLL+PLL1+PLL2+PLM)\n    r_ret = PR\n    \n    tot = [p_gunners,p_linemen,p_wings,p_protect,p_punter,r_outside,r_linemen,r_lbs,r_ret]\n    positions = ['Punt_Gunners', 'Punt_Linemen','Punt_Wings','Punt_Protect','Punter','Return_Outside','Return_Linemen','Return_LBs','Returner']\n    \n    plt.bar(positions,tot)\n    plt.xticks(rotation=90)\nget_positions(injured_players.iloc[:,-1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"40c14916578537ad90d85b2de625871894173a6a"},"cell_type":"markdown","source":"Only 5 gunners have been concussed, which does not appear to be big enough to make an impact. The reducting in returns coming from the 8-2-1 rule will help reduce concussions of gunners, without a significant change to that position."},{"metadata":{"_uuid":"1fa88548901e2b71f24683538421a4cbce394afe"},"cell_type":"markdown","source":"2 - Checking the impact of the ball originating on the hash marks versus the middle of the field. The hypothesis here was that if the punt is on the hash mark, a punter is more likely to punt out of bounds (a safer play) versus punt to the middle of the field. In the correlation matrix above where the ball is on the field along the y-axis doesn't have any relationship with if the punt goes out of bounds or not. "},{"metadata":{"_uuid":"5c5fb7bb84501877245ca4448e8f62e43df5dad4"},"cell_type":"markdown","source":"3 - Implementing the same touchback rule that was implemented on the kickoff (25 yard line)"},{"metadata":{"trusted":false,"_uuid":"55fd991e6b923ab843ee2f5ab76af523a32a8c02"},"cell_type":"code","source":"import numpy as np\n\ndistfromendzone = []\npuntdist = []\nretdist = []\ndescr = []\n\nfor row in range(len(play_info)):\n    poss = play_info.iloc[row,10]\n    ydline = int((play_info.iloc[row,7])[-2:])\n    play = (str(play_info.iloc[row,-1]))\n    \n    if poss in play_info.iloc[row,7]:\n        distfromendzone.append(100 - ydline)\n    else:\n        distfromendzone.append(ydline)\n\n    if \" punts \" in play:\n        result = play.split(\" punts \",1)[1]\n        yards = result[0:2]\n        if yards != 'no':\n            puntdist.append(int(yards))\n    else:\n        puntdist.append('NA')\n    \n    if \" for \" in play:\n        result = play.split(\" for \",1)[1]\n        yards = result[0:2]\n        if yards == 'no':\n            yards_ = 0\n        if yards == 'on':\n            yards = yards_  \n        if yards == 'L.':\n            yards = yards_  \n        if yards != 'no':\n            yards_ = int(yards)\n    else:\n        yards_ = 0\n    \n    retdist.append(yards_)\n    descr.append(play)\n\nD = {'Yard Line': distfromendzone, 'Punt Distance': puntdist, 'Return Distance': retdist, 'Description': descr}\ndf = pd.DataFrame(data=D)\nnotna = df['Punt Distance']!='NA'\n\ndf2 = df[notna]\n\ndf2['Landed'] = np.array(df2['Yard Line']) - np.array(df2['Punt Distance'])\n\nclose = df2['Yard Line']>0\ndf3 = df2[close]\n\nclose_2 = df2['Yard Line']<50\ndf4 = df3[close_2]\n\nfc = 0\nob = 0\ntb = 0\nrt = 0\nmf = 0\ndn = 0\n\nfor i in range(len(df4)):\n    \n    if \"fair catch\" in df4.iloc[i,3]:\n        fc += 1\n    if \"out of bounds\" in df4.iloc[i,3]:\n        ob += 1\n    if \"Touchback\" in df4.iloc[i,3]:\n        tb += 1\n    if \" for \" in df4.iloc[i,3]:\n        rt += 1\n    if \"MUFFS\" in df4.iloc[i,3]:\n        mf += 1\n    if \"downed\" in df4.iloc[i,3]:\n        dn += 1\n\noutcomes = ['Fair Catch', 'OB', 'Touchback', 'Return', 'Muff', 'Downed']\ntotal_plays = [fc,ob,tb,rt,mf,dn]\n\ndc = {'Outcomes': outcomes, 'Total Plays': total_plays}\ndf50 = (pd.DataFrame(dc))\n\nprint(str(round(rt/len(df4),4)*100)+\"% of punts in coffin corner situations (punts on the plus side of the field) were returned\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a433785a290c46abe486b4304cc36119fa11ac60"},"cell_type":"markdown","source":"The general rule of thumb is that returners are told not to field a punt inside the 10 because there is a good chance it will bounce into the end zone for a touch back. Only 5% of the time do they actually do this. Changing the touchback and further incentivizing not fielding the punt would have minimal affect on the big picture. "}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}