{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# A Data-Driven Evolution to Punt Return Strategy\n## Rahul Amara, Sohum Shah, and Rohit Tanikella","metadata":{}},{"cell_type":"markdown","source":"## Introduction: \"What Causes Muffed Punts?\"","metadata":{}},{"cell_type":"markdown","source":"The stage is set: it's the 2011 NFC Conference Championship game and the 49ers lead over the Giants in the 4th quarter. Their defense pushes hard to preserve their lead and forces the Giants to punt. It seems like a dream come true: the opportunity to secure a ticket to Superbowl 46. Unfortunately, this dream became too good to be true when Kyle Williams muffed the punt return and gave the Giants the ball back, leading to a scoring drive and an overtime victory for them.\n\nNFL fans are not strangers to critical games being determined by punt return mistakes. Though only occurring on 4.4% of fieldable punts, misfielded punt returns (muffed or bobbled) have caused drastic swings in the momentum and outcome of countless games.","metadata":{}},{"cell_type":"markdown","source":"Conventional football strategy primarily attributes misfielded punts to:\n1. the motion of the returner prior to making the catch \n2. the maximum height of the ball during the punt\n3. the proximity of the nearest gunner. \n\nWe aim to let the data speak to the causes of misfielding punts and then utilize these data to guide both punt team and punt return team strategy.","metadata":{}},{"cell_type":"markdown","source":"## EDA: What attributes are common among misfielded punts?","metadata":{}},{"cell_type":"code","source":"#Loading Data and creating dataframes that we will go on to use\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn import metrics\nimport seaborn as sns\nfrom sklearn.metrics import precision_score\nfrom sklearn.metrics import recall_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.model_selection import cross_val_score\nfrom sklearn.feature_selection import SelectKBest\nfrom sklearn import tree\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import GridSearchCV\nfrom sklearn.svm import SVC\nfrom sklearn.feature_selection import RFE\nimport warnings\nfrom sklearn.model_selection import RepeatedStratifiedKFold\nfrom imblearn.over_sampling import RandomOverSampler\nfrom imblearn.pipeline import Pipeline\nwarnings.filterwarnings('ignore')\n\ntracking2018 = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2018.csv')\ntracking2019 = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2019.csv')\ntracking2020 = pd.read_csv('../input/nfl-big-data-bowl-2022/tracking2020.csv')\ntrackingYears = {2018: tracking2018, 2019: tracking2019, 2020:tracking2020}\ngames = pd.read_csv('../input/nfl-big-data-bowl-2022/games.csv')\nplays = pd.read_csv('../input/nfl-big-data-bowl-2022/plays.csv')\npff = pd.read_csv('../input/nfl-big-data-bowl-2022/PFFScoutingData.csv')\n\n\n\npff = pff[pff['hangTime'].notna()]\npff = pff[pff['operationTime'].notna()]\npunts = plays[plays['specialTeamsPlayType'] == 'Punt']\npunts = punts.merge(games, on = 'gameId', how = 'inner')\npunts = punts.merge(pff, on = ['gameId', 'playId'], how = 'inner')\n\n#Include only punts where a return was possible\npunts = punts[punts['specialTeamsResult'].isin(['Return', 'Fair Catch', 'Downed', 'Touchback', 'Muffed'])]\n\n#Look at punts that were touched\nfieldedPunts = punts[punts['specialTeamsResult'].isin(['Return', 'Fair Catch', 'Muffed'])]\n\n#Remove R style punts due to a low sample size\nfieldedPunts = fieldedPunts[fieldedPunts['kickType'] != 'R']\n\n#Include only punts that the designated returner had a chance to catch in the air\nfieldedPunts = fieldedPunts[fieldedPunts['kickContactType'].isin(['CC', 'BC', 'MBDR'])]\n\nfieldedPunts = fieldedPunts[fieldedPunts['returnerId'].notna()]\nfieldedPunts = fieldedPunts[~fieldedPunts['returnerId'].str.contains(';')]\nfieldedPunts = fieldedPunts[fieldedPunts['gunners'].notna()]\nfieldedPunts = fieldedPunts[fieldedPunts['gunners'].str.count(';') == 1]\nfieldedPunts = fieldedPunts[fieldedPunts['vises'].notna()]\nfieldedPunts.reset_index(inplace = True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-06T23:09:04.473174Z","iopub.execute_input":"2022-01-06T23:09:04.473580Z","iopub.status.idle":"2022-01-06T23:10:28.296006Z","shell.execute_reply.started":"2022-01-06T23:09:04.473531Z","shell.execute_reply":"2022-01-06T23:10:28.295365Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def distance(x1, y1, x2, y2):\n    return (np.sqrt((x2-x1)**2 + (y2-y1)**2))\n\ndef viseTeam(gunnerTeam):\n    if gunnerTeam == 'home':\n        return 'away'\n    return 'home'\n\n#Generates a dataframe of the desired features for each punt play in our sample set\ndef featureGeneration(row, totalFeatures):\n    output = [str(row.gameId), row.playId]\n    \n    #kick type: 1 if Normal and 0 if Aussie\n    if row.kickType == 'N':\n        output.append(1)\n    else:\n        output.append(0)\n        \n    #hang time and operation time of punt (operation time is time from snap to punt)\n    output.append(row.hangTime)\n    output.append(row.operationTime)\n    \n    #Magnitude of Direction Match:\n    #   Vectorized for the purpose of attaining magnitude of offset\n    #   If how the team lined up to return is how they actually returned, equalDirection is True\n    #   If the team's actual return is 1 to the left or right of how they lined up, off1Direction is True\n    #   If team lined up Left and returned Right (or reversed), off2Direction is True\n    if row.returnDirectionIntended == row.returnDirectionActual:\n        output.extend([1, 0, 0])\n    elif row.returnDirectionIntended == 'C' and (row.returnDirectionActual == 'L' or row.returnDirectionActual == 'R'):\n        output.extend([0, 1, 0])\n    elif (row.returnDirectionIntended == 'L' or row.returnDirectionIntended == 'R') and row.returnDirectionActual == 'C':\n        output.extend([0, 1, 0])\n    else:\n        output.extend([0, 0, 1])\n    \n    tracking = trackingYears[row.season]\n    puntRec = tracking[(tracking['gameId'] == row.gameId) & (tracking['playId'] == row.playId)]\n    returnerTrack = puntRec[puntRec['nflId'] == float(row.returnerId)]\n    received = returnerTrack[returnerTrack['event'].isin(['fair_catch', 'punt_muffed', 'punt_received'])]\n    \n    if len(received) == 0: #Missing information about when punt was fielded (applies to just 11 punts)\n        return [None] * totalFeatures\n    \n    #Some received are longer than 1, but this only applies to 55 punts so we just take first one\n    received = received.sort_values(by = 'frameId')\n    \n    #Acceleration and Speed of returner at time of catch\n    output += [received.iloc[0].a, received.iloc[0].s]\n    \n    #Stationary: set a threshold of 1.5 yards per second as the mark for whether a returner is stationary\n    if (received.iloc[0].s <= 1.5):\n        output.append(1)\n    else:\n        output.append(0)\n    \n    \n    #Fair catch called by returner\n    if 'fair_catch' in returnerTrack['event'].unique():\n        output.append(1)\n    else:\n        output.append(0)\n    \n    #Direction of retuner's motion: Set to be zero'd when facing desired endzone\n    #Orientation of returner: Set to be zero'd when facing desired endzone\n    #AbsOrientation: absolute value of orientation\n    if received.iloc[0].playDirection == 'right':\n        direction, orientation = (received.iloc[0].dir+90)%360, (received.iloc[0].o+90)%360\n        if direction > 180:\n            direction = direction - 360\n        if orientation > 180:\n            orientation = orientation - 360\n        output.append(direction)\n        output.append(orientation)\n        output.append(abs(orientation))\n    else:\n        direction, orientation = (received.iloc[0].dir-90)%360, (received.iloc[0].o-90)%360\n        if direction > 180:\n            direction = direction - 360\n        if orientation > 180:\n            orientation = orientation - 360\n        output.append(direction)\n        output.append(orientation)\n        output.append(abs(orientation))\n    \n    output.append(min(53.3- received.iloc[0].y, received.iloc[0].y))\n    output.append(min(110 - received.iloc[0].x, received.iloc[0].x-10))\n    \n    \n    #Gunner 1 information: Gunner 1 distance, Gunner 1 speed, Gunner 1 Accel, \n    #                      G1 Direction, G1 Orientation, G1 Abs Orientation\n    #Note that orientation for gunner is zerod at their desired enzone (other side of returner's desired endzone)\n    returner_x = received.iloc[0].x\n    returner_y = received.iloc[0].y\n    \n    gunners = row.gunners.split('; ')\n    gunner_team = gunners[0][0:gunners[0].index(\" \")]\n    gunner1_number = int(gunners[0][gunners[0].index(\" \") + 1: ])\n    gunner2_number = int(gunners[1][gunners[1].index(\" \") + 1: ])\n    \n    homeTeam = games[games['gameId'] == row.gameId].iloc[0].homeTeamAbbr\n    team_home_away = 'home'\n    if homeTeam != gunner_team:\n        team_home_away = 'away'\n    \n    gunner1_track = puntRec[(puntRec['team'] == team_home_away) & (puntRec['jerseyNumber'] == gunner1_number)]\n    g1AtContact = gunner1_track[gunner1_track['frameId'] == received.iloc[0].frameId]\n    g1AtContact = g1AtContact.sort_values(by = 'frameId')\n    if (len(g1AtContact) == 0):\n        return [None] * totalFeatures\n    \n    gunner1_x = g1AtContact.iloc[0].x\n    gunner1_y = g1AtContact.iloc[0].y\n    gunner1Distance = distance(returner_x, returner_y, gunner1_x, gunner1_y)\n    gunner1Speed = g1AtContact.iloc[0].s\n    gunner1Accel = g1AtContact.iloc[0].a\n    gunner1Dir = 0\n    gunner1Orientation = 0\n    \n    output.append(gunner1Distance)\n    output.append(gunner1Speed)\n    output.append(gunner1Accel)\n    \n    if received.iloc[0].playDirection == 'right':\n        gunner1Dir, gunner1Orientation = (g1AtContact.iloc[0].dir-90)%360, (g1AtContact.iloc[0].o-90)%360\n        if gunner1Dir > 180:\n            gunner1Dir = gunner1Dir - 360\n        if gunner1Orientation > 180:\n            gunner1Orientation = gunner1Orientation - 360\n        output.append(gunner1Dir)\n        output.append(gunner1Orientation)\n        output.append(abs(gunner1Orientation))\n    else:\n        gunner1Dir, gunner1Orientation = (g1AtContact.iloc[0].dir+90)%360, (g1AtContact.iloc[0].o+90)%360\n        if gunner1Dir > 180:\n            gunner1Dir = gunner1Dir - 360\n        if orientation > 180:\n            gunner1Orientation = gunner1Orientation - 360\n        output.append(gunner1Dir)\n        output.append(gunner1Orientation)\n        output.append(abs(gunner1Orientation))\n        \n\n    #Gunner 2 information: Gunner 2 distance, Gunner 2 speed, Gunner 2 Accel\n    #                      G2 Direction, G2 Orientation, G2 Abs Orientation\n    gunner2_track = puntRec[(puntRec['team'] == team_home_away) & (puntRec['jerseyNumber'] == gunner2_number)]\n    g2AtContact = gunner2_track[gunner2_track['frameId'] == received.iloc[0].frameId]\n    g2AtContact = g2AtContact.sort_values(by = 'frameId')\n    if (len(g2AtContact) == 0):\n        return [None] * totalFeatures\n    gunner2_x = g2AtContact.iloc[0].x\n    gunner2_y = g2AtContact.iloc[0].y\n    gunner2Distance = distance(returner_x, returner_y, gunner2_x, gunner2_y)\n    gunner2Speed = g2AtContact.iloc[0].s\n    gunner2Accel = g2AtContact.iloc[0].a\n    gunner2Dir = 0\n    gunner2Orientation = 0\n    \n    output.append(gunner2Distance)\n    output.append(gunner2Speed)\n    output.append(gunner2Accel)\n    \n    if received.iloc[0].playDirection == 'right':\n        gunner2Dir, gunner2Orientation = (g2AtContact.iloc[0].dir-90)%360, (g2AtContact.iloc[0].o-90)%360\n        if gunner2Dir > 180:\n            gunner2Dir = gunner2Dir - 360\n        if gunner2Orientation > 180:\n            gunner2Orientation = gunner2Orientation - 360\n        output.append(gunner2Dir)\n        output.append(gunner2Orientation)\n        output.append(abs(gunner2Orientation))\n    else:\n        gunner2Dir, gunner2Orientation = (g2AtContact.iloc[0].dir+90)%360, (g2AtContact.iloc[0].o+90)%360\n        if gunner2Dir > 180:\n            gunner2Dir = gunner2Dir - 360\n        if gunner2Orientation > 180:\n            gunner2Orientation = gunner2Orientation - 360\n        output.append(gunner2Dir)\n        output.append(gunner2Orientation)\n        output.append(abs(gunner2Orientation))\n    \n    \n    #Nearest Gunner Information: Distance, speed, acceleration, time to contact, direction,\n    #                            orientation, abs orientation\n    nearestGunnerDistance = 0\n    if gunner1Distance < gunner2Distance:\n        nearestGunnerDistance = gunner1Distance\n        output.append(gunner1Distance)\n        output.append(gunner1Speed)\n        output.append(gunner1Accel)\n        output.append((gunner1Speed + np.sqrt(gunner1Speed**2 + 2*gunner1Accel*gunner1Distance))/(gunner1Accel))\n        output.append(gunner1Dir)\n        output.append(gunner1Orientation)\n        output.append(abs(gunner2Orientation))\n        \n    else:\n        nearestGunnerDistance = gunner2Distance\n        output.append(gunner2Distance)\n        output.append(g2AtContact.iloc[0].s)\n        output.append(g2AtContact.iloc[0].a)\n        output.append((g2AtContact.iloc[0].s + np.sqrt(g2AtContact.iloc[0].s**2 + 2*g2AtContact.iloc[0].a*gunner2Distance))/(g2AtContact.iloc[0].a))\n        output.append(gunner2Dir)\n        output.append(gunner2Orientation)\n        output.append(abs(gunner2Orientation))\n    \n    #Average Gunner Distance:\n    output.append((gunner1Distance + gunner2Distance)/2)\n    \n    #Average Vise Information: Average distance of all vises\n    #Closest Vise Information: Distance, speed, acceleration, direction, and orientation\n    vises = row.vises.split('; ')\n    nearestViseDistance = np.sqrt(100000)\n    nearestViseSpeed = 0\n    nearstViseAccel = 0\n    nearestViseDirection = 0\n    nearestViseOrientation = 0\n    foundVice = False\n    \n    for vise in vises:\n        viseNumber = int(vise[vise.index(\" \") + 1:])\n        viseTrack = puntRec[(puntRec['team'] == viseTeam(team_home_away)) & (puntRec['jerseyNumber'] == viseNumber)]\n        viseAtContact = viseTrack[viseTrack['frameId'] == received.iloc[0].frameId]\n        viseAtContact = viseAtContact.sort_values(by = 'frameId')\n        if (len(viseAtContact) == 0):\n            continue\n        foundVice = True\n        vise_x = viseAtContact.iloc[0].x\n        vise_y = viseAtContact.iloc[0].y\n        viseDistance = distance(returner_x, returner_y, vise_x, vise_y)\n        if viseDistance < nearestViseDistance:\n            nearestViseDistance = viseDistance\n            nearestViseSpeed = viseAtContact.iloc[0].s\n            nearstViseAccel = viseAtContact.iloc[0].a\n            if viseAtContact.iloc[0].playDirection == 'right':\n                nearestViseDirection, nearestViseOrientation = (viseAtContact.iloc[0].dir+90)%360, (viseAtContact.iloc[0].o+90)%360\n                if nearestViseDirection > 180:\n                    nearestViseDirection = nearestViseDirection - 360\n                if nearestViseOrientation > 180:\n                    nearestViseOrientation = nearestViseOrientation - 360\n            else:\n                nearestViseDirection, nearestViseOrientation= (viseAtContact.iloc[0].dir-90)%360, (viseAtContact.iloc[0].o-90)%360\n                if nearestViseDirection > 180:\n                    nearestViseDirection = nearestViseDirection - 360\n                if nearestViseOrientation > 180:\n                    nearestViseOrientation = nearestViseOrientation - 360\n    if not foundVice:\n        return [None] * totalFeatures    \n    output.append(nearestViseDistance)\n    output.append(nearestViseSpeed)\n    output.append(nearstViseAccel)\n    output.append(nearestViseDirection)\n    output.append(nearestViseOrientation)\n    \n    #Result\n    if row.kickContactType == 'CC':\n        output.append(0)\n    else:\n        output.append(1)\n        \n    return output\n\ncolumnNames = ['gameId', 'playId', 'kickType', 'hangTime', 'opTime', 'equalDirection', 'off1Direction', 'off2Direction',\n               'accel', 'speed', 'stationary', 'fair_catch', 'returnerDir', 'returnerOr', 'returnerOrAbs', 'distanceFromSideline', 'distanceFromEndzone', 'gunner1Distance', \n                'gunner1Speed', 'gunner1Accel', 'gunner1Dir', 'gunner1Or', 'gunner1OrAbs', 'gunner2Distance', \n                'gunner2Speed', 'gunner2Accel', 'gunner2Dir', 'gunner2Or', 'gunner2OrAbs', 'closestGunnerDistance',\n                'closestGunnerSpeed', 'closestGunnerAccel','closestGunnerTimeToContact', 'closestGunnerDir', \n                'closestGunnerOr', 'closestGunnerOrAbs', 'avgGunnerDistance', \n                'nearestViseDistance', 'nearestViseSpeed', 'nearestViseAccel', 'nearestViseDirection', \n                'nearestViseOr', 'result']\ndf = pd.DataFrame(columns = columnNames)\ndf[columnNames] = fieldedPunts.apply(lambda x: featureGeneration(x, len(columnNames)), axis = 1, result_type=\"expand\")\ndf.dropna(inplace = True)\ndf.reset_index(inplace = True)","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:10:28.301695Z","iopub.execute_input":"2022-01-06T23:10:28.302146Z","iopub.status.idle":"2022-01-06T23:13:09.073560Z","shell.execute_reply.started":"2022-01-06T23:10:28.302108Z","shell.execute_reply":"2022-01-06T23:13:09.072784Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As part of our EDA Process, we first sought to identify trends in various features between when a punt was clean-caught and when a punt was misfielded. Among others, some of the critical features that we examined were\n1. The rate of fair catches depending on whether a punt was misfielded\n2. The orientation of the returner at time of catch depending on whether a punt was misfielded\n3. The speed of the returner at the time of catch depending on whether a punt was misfielded\n4. The distance between the returner and the gunner at time of catch depending on whether a punt was misfielded\n\nWhile we understood that this isn't presenting any evidence for causality, our findings supported that there were common characteristics among plays that were misfielded and that these characteristics could likely be used in creating a misfielded return risk model.","metadata":{}},{"cell_type":"code","source":"pd.pivot_table(df, values=\"accel\", index=\"fair_catch\", columns=\"result\", aggfunc=\"count\")","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.076587Z","iopub.execute_input":"2022-01-06T23:13:09.078362Z","iopub.status.idle":"2022-01-06T23:13:09.109756Z","shell.execute_reply.started":"2022-01-06T23:13:09.078300Z","shell.execute_reply":"2022-01-06T23:13:09.108839Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the pivot table above, we define a result of 1 to represent a misfielded return and a fair_catch value of 1 to represent a fair catching being called. As we see above, the vast majority of punts that were misfielded occured when a fair catch was not called. Specifically, the probability of sampling a misfielded punt on a fair catch is","metadata":{}},{"cell_type":"code","source":"print(str(round(df[df['fair_catch']==1]['result'].mean()*100, 2)) + \"%\")","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.110911Z","iopub.execute_input":"2022-01-06T23:13:09.111184Z","iopub.status.idle":"2022-01-06T23:13:09.119338Z","shell.execute_reply.started":"2022-01-06T23:13:09.111114Z","shell.execute_reply":"2022-01-06T23:13:09.118617Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"and the probability of sampling a misfielded punt when no fair catch is called is ","metadata":{}},{"cell_type":"code","source":"print(str(round(df[df['fair_catch']==0]['result'].mean()*100, 1)) + \"%\")","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.120370Z","iopub.execute_input":"2022-01-06T23:13:09.121095Z","iopub.status.idle":"2022-01-06T23:13:09.130824Z","shell.execute_reply.started":"2022-01-06T23:13:09.121044Z","shell.execute_reply":"2022-01-06T23:13:09.129814Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In summary, nearly 10 times more misfielded punts occur on plays where a fair catch was not called.\n\n","metadata":{}},{"cell_type":"code","source":"plt.title(\"Distance to Gunner on Clean vs. Misfielded Punts\", fontsize=14)\nax=sns.violinplot(x='result', y='closestGunnerDistance', data=df[df['fair_catch']==0], inner='quartile')\nplt.xlabel('Fair Catch Called')\nplt.xticks([0, 1], ['Clean Caught', 'Misfielded'])\nplt.ylabel(\"Distance from Returner to Nearest Gunner\")\nplt.ylim(-5, 35);","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.131902Z","iopub.execute_input":"2022-01-06T23:13:09.132589Z","iopub.status.idle":"2022-01-06T23:13:09.395966Z","shell.execute_reply.started":"2022-01-06T23:13:09.132556Z","shell.execute_reply":"2022-01-06T23:13:09.395402Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the above graphic, fair catch (in the legend) refers to whether the returner elected to call a fair caught with 0 referring to no fair catch called and 1 referring to a fair catch.\n\nBecause fair catches confound so heavily with the distance from the returner to the nearest gunner, it was important to separate them out in understanding the significance of this attribute. As such, what we see above is the distance to the nearest gunner exclusively on plays where a fair catch was not called.\n\nAs general football intuition would guide us, the gunner was significantly closer to the returner on punts that ended up being misfielded.\n\nSpecifically, we see that on plays where a fair catch was not called, the median distance from the returner to the gunner on clean caught punts was ","metadata":{}},{"cell_type":"code","source":"print(str(round(df[(df['fair_catch']==0) & (df['result']==0)]['closestGunnerDistance'].median(), 1)) + \" yards\")","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.396986Z","iopub.execute_input":"2022-01-06T23:13:09.397529Z","iopub.status.idle":"2022-01-06T23:13:09.404631Z","shell.execute_reply.started":"2022-01-06T23:13:09.397430Z","shell.execute_reply":"2022-01-06T23:13:09.403777Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"and the median distance from the returner to the gunner on muffed punts was ","metadata":{}},{"cell_type":"code","source":"print(str(round(df[(df['fair_catch']==0) & (df['result']==1)]['closestGunnerDistance'].median(), 1)) + \" yards.\")","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.406100Z","iopub.execute_input":"2022-01-06T23:13:09.406371Z","iopub.status.idle":"2022-01-06T23:13:09.418106Z","shell.execute_reply.started":"2022-01-06T23:13:09.406341Z","shell.execute_reply":"2022-01-06T23:13:09.417217Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, (ax1, ax2) = plt.subplots(1, 2, sharey=False)\nplt.subplots_adjust(left=0.1,\n                    bottom=0.1, \n                    right=0.9, \n                    top=0.9, \n                    wspace=0.4, \n                    hspace=0.4)\n\nsns.violinplot(x=df['result'], y=(df['speed'] - df['speed'].mean())/df['speed'].std(), inner='quartile', ax=ax1)\nax1.set_title('Misfielded vs. Speed')\nax1.set_ylabel('Speed (Normalized)')\nax2.set_title('Misfielded vs. Normalized Orientation')\nax2.set_ylabel('Orientation (Normalized)')\nf.set_size_inches(9, 5)\nsns.violinplot(x=df['result'], y=(df['returnerOrAbs'] - df['returnerOrAbs'].mean())/df['returnerOrAbs'].std(), inner='quartile', ax=ax2);\n","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.420588Z","iopub.execute_input":"2022-01-06T23:13:09.420831Z","iopub.status.idle":"2022-01-06T23:13:09.778563Z","shell.execute_reply.started":"2022-01-06T23:13:09.420805Z","shell.execute_reply":"2022-01-06T23:13:09.776311Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we see above, on misfielded punts, the median speed of a returner as well as the median angular displacement from center of a returner's orientation is roughly 0.75 standard deviations higher than when a punt is clean-caught.","metadata":{}},{"cell_type":"markdown","source":"We have included a correlation matrix below of all the features we identified as potentially causal. Note that the limitation of a matrix such as this one is that it does not demonstrate the correlation between features and the outcome (labeled \"result\" here) in conjunction with other features.","metadata":{}},{"cell_type":"code","source":"featuresToShow = ['kickType', 'equalDirection', 'off1Direction', 'off2Direction',\n                'speed', 'fair_catch', 'returnerDir', 'returnerOr', 'returnerOrAbs', 'distanceFromSideline', \n                'closestGunnerDistance', 'result']\n\ncorrGraph = df[featuresToShow].copy()\n\ncorr = corrGraph.corr()\n\nplt.figure(figsize=[20,20])\nax = plt.axes()\nplt.rcParams.update({'font.size': 12})\nax.set_title('Punt Data Correlation Matrix', fontsize=40)\nsns.heatmap(corr, xticklabels=corr.columns, yticklabels=corr.columns, cmap='RdBu', annot=True, fmt='.2f', vmin=-1.0)\nplt.show();","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:13:09.779551Z","iopub.execute_input":"2022-01-06T23:13:09.779742Z","iopub.status.idle":"2022-01-06T23:13:10.810072Z","shell.execute_reply.started":"2022-01-06T23:13:09.779717Z","shell.execute_reply":"2022-01-06T23:13:10.808560Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Classification Model: Clean-Caught vs. Misfielded","metadata":{}},{"cell_type":"markdown","source":"Using the insights gained from the EDA above, we sought to produce a model that would predict if a punt was misfielded or not.\nThe first important characteristic of our data that we noticed was the imbalance in the dataset. This meant that our model could have high accuracy while not producing any useful insight by simply predicting every return as clean-caught. This led to us to focus on recall as the primary metric for measuring our model’s performance.","metadata":{}},{"cell_type":"markdown","source":"Recall = $\\frac{TP}{TP+FN}$","metadata":{}},{"cell_type":"markdown","source":"We determined that we care about recall over precision specifically here because our goal was to predict returns that were likely to be misfielded even if that meant misclassifying a few borderline clean catches as misfielded.","metadata":{}},{"cell_type":"markdown","source":"We first used the data provided to produce a naive logistic regression model. Using that model as a baseline we then improved its performance by both reformulating/adding features as well as setting class weights to correct the inherent imbalance in the dataset. Our final model and its performance are displayed below.\n\nThe reason we chose to use a logistic regression model as our classifier is because we wanted to generate a model whose formula is extractable and utilizable to generate coaching insights on a feature-by-feature basis as opposed to using a black-box model. ","metadata":{}},{"cell_type":"markdown","source":"$Pr(Y_i=1|X_i) = {\\frac{exp(\\beta_0 + \\beta_1X_i + \\beta_2X_2 + \\beta_3X_3 + \\beta_4X_4 + \\beta_5X_5...)}{1 + exp (\\beta_0 + \\beta_1X_i + \\beta_2X_2 + \\beta_3X_3 + \\beta_4X_4 + \\beta_5X_5...)}}$","metadata":{"execution":{"iopub.status.busy":"2022-01-06T22:34:53.816747Z","iopub.execute_input":"2022-01-06T22:34:53.817622Z","iopub.status.idle":"2022-01-06T22:34:53.821888Z","shell.execute_reply.started":"2022-01-06T22:34:53.817575Z","shell.execute_reply":"2022-01-06T22:34:53.820964Z"}}},{"cell_type":"code","source":"def runClassifier(clf, params):\n    cv = GridSearchCV(clf, params, error_score=-1, scoring = 'f1', verbose = 0)\n    cv.fit(X_train, Y_train)\n    best = cv.best_estimator_\n\n    acc = best.score(X_test, Y_test)\n    print(\"Model Accuracy: \" + str(acc))\n\n    preds = best.predict(X_test)\n    prec = precision_score(Y_test, preds)\n    recall = recall_score(Y_test, preds)\n    f1 = f1_score(Y_test, preds)\n\n    print(\"Model Precision: \" + str(prec))\n    print(\"Model Recall: \" + str(recall))\n    #print(\"Model F1 Score: \" + str(f1))\n    \n    print(cv.best_params_)\n    return cv.best_estimator_\n\n\nfeatureNames = ['kickType', 'hangTime', 'equalDirection', 'off1Direction', 'off2Direction', 'accel', \n                'speed','fair_catch', 'returnerDir', 'returnerOr', 'returnerOrAbs', 'gunner1Distance',  \n                'gunner1Speed', 'gunner1Dir', 'gunner1Or', 'gunner2Distance', \n                'gunner2Speed', 'gunner2Accel', 'gunner2Dir', 'gunner2Or', 'gunner2OrAbs', 'closestGunnerDistance',\n                'closestGunnerSpeed', 'closestGunnerAccel','closestGunnerTimeToContact', 'closestGunnerDir', \n                'closestGunnerOr', \n                'nearestViseDistance', 'nearestViseSpeed', 'nearestViseAccel', 'nearestViseDirection', \n                'nearestViseOr']\nX = df[featureNames].copy()\nY = df['result']\n\ntoNormalize = ['hangTime', 'accel', 'speed' ,'returnerDir', 'returnerOr', 'returnerOrAbs', 'gunner1Distance',\n              'gunner1Speed', 'gunner1Dir', 'gunner1Or', 'gunner2Distance', 'gunner2Speed',\n              'gunner2Accel', 'gunner2Dir', 'gunner2Or', 'closestGunnerDistance', 'closestGunnerSpeed', \n              'closestGunnerAccel', 'closestGunnerTimeToContact', 'closestGunnerDir', 'closestGunnerOr', \n               'nearestViseDistance', 'nearestViseSpeed', 'nearestViseAccel', \n               'nearestViseDirection', 'nearestViseOr']\nfor f in toNormalize:\n    X[f] = (X[f] - X[f].mean()) / X[f].std()\n    \n\nselectedFeatures = featureNames #Try removing highly correlated features and look at correlation matrix\n#Feature selection code\nX = X[selectedFeatures]\nX_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=0)","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:13:10.812086Z","iopub.execute_input":"2022-01-06T23:13:10.812551Z","iopub.status.idle":"2022-01-06T23:13:10.864921Z","shell.execute_reply.started":"2022-01-06T23:13:10.812502Z","shell.execute_reply":"2022-01-06T23:13:10.864141Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params = {'solver': ['liblinear', 'sag', 'saga', 'lbfgs'], 'C': [0.001,0.01,0.1,1,10,100,1000], \n          'class_weight': [{0: 1, 1: 6},{0: 1, 1: 6.5}, {0: 1, 1: 7}],\n          'penalty': ['l1', 'l2', 'elasticnet', 'none', 'newton-cg']}\nlogReg = runClassifier(LogisticRegression(), params)","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:13:10.866902Z","iopub.execute_input":"2022-01-06T23:13:10.867810Z","iopub.status.idle":"2022-01-06T23:15:06.413625Z","shell.execute_reply.started":"2022-01-06T23:13:10.867772Z","shell.execute_reply":"2022-01-06T23:15:06.412456Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With our final model, we can take a look at which features of our data were deemed most important to predicting whether a kick would be misfielded. We see that calling a fair catch as well as how a player was lined up (among several other factors) were very important in determining the likelihood of a misfield. The most important features from our model are displayed below with their relative importance","metadata":{}},{"cell_type":"code","source":"d = {\"Feature\": featureNames, \"Coefficent\": logReg.coef_[0], \"Absolute Value of Coefficent\": np.absolute(logReg.coef_[0])}\nweights = pd.DataFrame(d)\n\nweights = weights[weights[\"Absolute Value of Coefficent\"] > 0.3]\nsns.set(rc = {'figure.figsize':(15,8)})\nax = sns.barplot(x=\"Feature\", y=\"Coefficent\", data=weights, order=weights.sort_values('Absolute Value of Coefficent',ascending = False).Feature)\nax.set_title('Logistic Regression Feature Weights', fontsize=16);","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:15:06.415906Z","iopub.execute_input":"2022-01-06T23:15:06.416634Z","iopub.status.idle":"2022-01-06T23:15:06.701484Z","shell.execute_reply.started":"2022-01-06T23:15:06.416582Z","shell.execute_reply":"2022-01-06T23:15:06.700505Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As can be seen from the precision from above, our model sometimes incorrectly classifies a clean-caught return as misfielded. This occurs when a play is still \"high risk\", but the returner manages to catch it without misfielding it. To demonstrate this, we have included two visualizations: \n1. The model correctly called a misfield and one \n2. Model incorrectly called a misfield","metadata":{}},{"cell_type":"code","source":"from IPython.display import Image\n\nImage(open('../input/gifsfornfl/muff.gif', 'rb').read())","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:15:06.702653Z","iopub.execute_input":"2022-01-06T23:15:06.702895Z","iopub.status.idle":"2022-01-06T23:15:06.921875Z","shell.execute_reply.started":"2022-01-06T23:15:06.702865Z","shell.execute_reply":"2022-01-06T23:15:06.918797Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(open('../input/gifsfornfl/riskyButClean.gif', 'rb').read())","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:15:06.923806Z","iopub.execute_input":"2022-01-06T23:15:06.924115Z","iopub.status.idle":"2022-01-06T23:15:07.125354Z","shell.execute_reply.started":"2022-01-06T23:15:06.924084Z","shell.execute_reply":"2022-01-06T23:15:07.123125Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we see, the first visualization shows a clearly risky play that resulted in a misfield as the retuner was both running backwards in conjunction with the fact that the was very close to him.\n\nThe second visualization shows a clearly risky play that happened to not lead to a misfield despite the fact that the gunner was right next to the returner and ready to make contact.","metadata":{}},{"cell_type":"markdown","source":"## Transitioning to a risk assessment model","metadata":{}},{"cell_type":"markdown","source":"To make our model more granular, we converted the prediction of a clean catch vs. a misfield to a risk factor between 0 and 1 where a higher risk refers to a higher chance of misfielding. This was done by using the numerical output of our logistic regression model.\n\nWhat is seen below is a distribution of 'risk factors' for all of the plays in our sample set. We have made plays that were misfielded red.","metadata":{}},{"cell_type":"code","source":"logistic_probabilities = logReg.predict_proba(X)\nrisks = []\nfor touple in logistic_probabilities:\n    risks.append(touple[1])\n\nwithRisk = X.copy()\nwithRisk['gameId'] = df['gameId'].astype(int)\nwithRisk['playId'] = df['playId'].astype(int)\nwithRisk['risk'] = risks\nwithRisk['result'] = Y\n\nplt.figure(figsize=[14,6])\n#sns.histplot(x='risk', data=toGraph, hue='result', palette=sns.color_palette(['#00CCFF', '#FF3300']));\nsns.histplot(x='risk', data=withRisk, binrange=(0,1), binwidth=0.2/14)\nsns.histplot(x='risk', data=withRisk[(withRisk['result']==1) & (withRisk['risk']<0.4)], binrange=(0,1), binwidth=0.2/14, color='red')\nsns.histplot(x='risk', data=withRisk[(withRisk['result']==1) & (withRisk['risk']>0.4)], binrange=(0,1), binwidth=0.4/14, color='red')\nplt.title('Disribution of Modeled Risk Values for Punt Plays')\nplt.xlabel('Modeled Risk Value')\nplt.ylabel('Count (Red Bars refer to misfielded punts)');","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:15:07.127308Z","iopub.execute_input":"2022-01-06T23:15:07.127798Z","iopub.status.idle":"2022-01-06T23:15:07.877201Z","shell.execute_reply.started":"2022-01-06T23:15:07.127755Z","shell.execute_reply":"2022-01-06T23:15:07.876302Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you can see above, the clear majority of misfielded punts occurred on plays that we modeled to be high risk.","metadata":{}},{"cell_type":"markdown","source":"## Defining Situational Risk-Reward","metadata":{}},{"cell_type":"markdown","source":"In the context of misfielded punts, there is no way to calculate a meaningful expected value. This is because if a punt is successfully fielded, the reward is yards gained. On the other hand, if a punt is misfielded, the risk is loss of possession. As such, we have chosen to present two metrics for any given situation:\n1. The percentage of punts with similar risk values that were misfielded\n2. The expected yards to be gained if the punt is not misfielded","metadata":{}},{"cell_type":"code","source":"riskAndOutcome = withRisk.merge(plays, on = ['gameId', 'playId'], how = 'inner').drop(columns=['passResult', \n                                'preSnapVisitorScore', 'preSnapHomeScore', 'penaltyJerseyNumbers', 'penaltyCodes',\n                                'gameClock', 'yardlineSide', 'kickBlockerId', 'specialTeamsPlayType', 'kickerId',\n                                'yardsToGo', 'possessionTeam', 'down', 'quarter', 'penaltyYards'])\nx_vals = []\ny1_vals = []\ny2_vals = []\nfor i in np.arange(10):\n    x = i/10 + 0.05\n    y1 = riskAndOutcome[(riskAndOutcome['risk'] < x + 0.05) & (riskAndOutcome['risk'] >= x - 0.05)]['kickReturnYardage'].mean()\n    y2 = riskAndOutcome[(riskAndOutcome['risk'] < x + 0.05) & (riskAndOutcome['risk'] >= x - 0.05)]['result'].mean()\n    x_vals.append(x)\n    y1_vals.append(y1)\n    y2_vals.append(y2)\n    \nfig, ax = plt.subplots()\nfig.set_size_inches(8, 6)\nsns.regplot(x=x_vals, y=y1_vals, ax=ax)\nax2 = ax.twinx()\nsns.regplot(x=x_vals, y=y2_vals, ax=ax2, color='r')\nax.set_ylabel('Mean Return for Similar Risk')\nax2.set_ylabel(\"Misfield Rate for Similar Risk\")\nax.set_xlabel(\"Risk\")\nax.set_title(\"Risk vs. Mean Return Yardage and Misfielded Return Rate\");","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:15:07.878336Z","iopub.execute_input":"2022-01-06T23:15:07.878569Z","iopub.status.idle":"2022-01-06T23:15:08.543563Z","shell.execute_reply.started":"2022-01-06T23:15:07.878541Z","shell.execute_reply":"2022-01-06T23:15:08.542538Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see above that as a play becomes riskier, not only does the likelyhood of a misfield increase, but the yards gained on the return decrease. This emphasizes further that returners should call a fair catch or allow the ball to bounce on risky plays.","metadata":{}},{"cell_type":"markdown","source":"## Punt Return Strategy Cheat-Sheet","metadata":{}},{"cell_type":"markdown","source":"Using the model decomposition and analysis in this section, we then set out to define some implementable strategies that coaches and special teams players can use to mitigate the risk of misfielding.","metadata":{}},{"cell_type":"markdown","source":"### 1) If the returner is running to catch the punt, he should think first to call fair catch","metadata":{}},{"cell_type":"markdown","source":"From the model decomposition above, it is clear that two of the largest risk factors for a misfield are not calling a fair catch as well as running while fielding the punt. To put this into numbers using our risk metric","metadata":{}},{"cell_type":"code","source":"riskForNoFairCatch = riskAndOutcome[(riskAndOutcome['speed'] > 2) & (riskAndOutcome['fair_catch'] == 0)]['risk'].mean()\nriskForFairCatch = riskAndOutcome[(riskAndOutcome['speed'] > 2) & (riskAndOutcome['fair_catch'] == 1)]['risk'].mean()\n\nnoFairCatchMisfield = riskAndOutcome[(riskAndOutcome['risk'] > riskForNoFairCatch - 0.05) & (riskAndOutcome['risk'] > riskForNoFairCatch + 0.05)]['result'].mean()*100\nnoFairCatchReturn = riskAndOutcome[(riskAndOutcome['risk'] > riskForNoFairCatch - 0.05) & (riskAndOutcome['risk'] > riskForNoFairCatch + 0.05)]['kickReturnYardage'].mean()\nfairCatchMisfield = riskAndOutcome[(riskAndOutcome['risk'] > riskForFairCatch - 0.05) & (riskAndOutcome['risk'] > riskForFairCatch + 0.05)]['result'].mean()*100\n\nprint(\"Returns as risky as a moving catch with no fair catch called, \" + str(round(noFairCatchMisfield, 1)) + \"% were misfielded and the average return yardage was \" + str(round(noFairCatchReturn, 2)) + \" yards\")\nprint(\"If the returner simply called for a fair catch, only \" + str(round(fairCatchMisfield, 1)) + \"% of similarly risky plays were misfielded, but the average return yardage was 0 yards\")\n","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:15:08.545044Z","iopub.execute_input":"2022-01-06T23:15:08.545500Z","iopub.status.idle":"2022-01-06T23:15:08.564338Z","shell.execute_reply.started":"2022-01-06T23:15:08.545451Z","shell.execute_reply":"2022-01-06T23:15:08.563214Z"},"jupyter":{"source_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In essence, the returner is nearly doubling his risk of misfielding for a very small expected number of yards gained on the return.","metadata":{}},{"cell_type":"markdown","source":"### 2) If the nearest gunner is within 5 yards of you, should always call for a fair catch","metadata":{}},{"cell_type":"markdown","source":"From our model decomposition and supported by our EDA, another large risk factor is electing to return catches when the gunner is nearby. Putting this into numbers using our risk metric we see that","metadata":{}},{"cell_type":"code","source":"nearGunnerNoFairCatch = riskAndOutcome[(riskAndOutcome['fair_catch'] == 0) & (riskAndOutcome['closestGunnerDistance'] < 5)]['risk'].mean()\nnearGunnerFairCatch = riskAndOutcome[(riskAndOutcome['fair_catch'] == 1) & (riskAndOutcome['closestGunnerDistance'] < 5)]['risk'].mean()\n\nnearGunnerNoFairCatchMisfield = riskAndOutcome[(riskAndOutcome['risk'] > nearGunnerNoFairCatch - 0.05) & (riskAndOutcome['risk'] > nearGunnerNoFairCatch + 0.05)]['result'].mean()*100\nnearGunnerNoFairCatchReturn = riskAndOutcome[(riskAndOutcome['risk'] > nearGunnerNoFairCatch - 0.05) & (riskAndOutcome['risk'] > nearGunnerNoFairCatch + 0.05)]['kickReturnYardage'].mean()\n\nnearGunnerFairCatchMisfield = riskAndOutcome[(riskAndOutcome['risk'] > nearGunnerFairCatch - 0.05) & (riskAndOutcome['risk'] > nearGunnerFairCatch + 0.05)]['result'].mean()*100\nnearGunnerFairCatchReturn = riskAndOutcome[(riskAndOutcome['risk'] > nearGunnerFairCatch - 0.05) & (riskAndOutcome['risk'] > nearGunnerFairCatch + 0.05)]['kickReturnYardage'].mean()\n\n\n\nprint(\"For returns as risky as fielded return with a nearby gunner \" + str(round(nearGunnerNoFairCatchMisfield, 1)) + \"% were misfielded and the average return yardage was \" + str(round(nearGunnerNoFairCatchReturn, 2)) + \" yards\")\nprint(\"If the returner simply called for a fair catch, only \" + str(round(nearGunnerFairCatchMisfield, 1)) + \"% of similarly risky plays were misfielded, but the average return yardage was 0 yards\")\n","metadata":{"jupyter":{"source_hidden":true},"execution":{"iopub.status.busy":"2022-01-06T23:15:08.566314Z","iopub.execute_input":"2022-01-06T23:15:08.567169Z","iopub.status.idle":"2022-01-06T23:15:08.591046Z","shell.execute_reply.started":"2022-01-06T23:15:08.567122Z","shell.execute_reply":"2022-01-06T23:15:08.590417Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Once again, the returner is more than doubling his risk of misfielding for a very small expected number of yards gained on the return.","metadata":{}},{"cell_type":"markdown","source":"### 3) It is ill advised to field a ball that you have run across the field to catch","metadata":{}},{"cell_type":"markdown","source":"From our model decomposition, another large risk factor is fielding punts that are 'off2Direction'. Putting this into numbers using our risk metric we see that","metadata":{}},{"cell_type":"code","source":"riskForCrossCatch = riskAndOutcome[(riskAndOutcome['off2Direction'] == 1) & (riskAndOutcome['speed'] > 1.5)]['risk'].mean()\nriskForNotCrossCatch = riskAndOutcome[(riskAndOutcome['off2Direction'] == 0)]['risk'].mean()\n\ncrossCatchMisfield = riskAndOutcome[(riskAndOutcome['risk'] > riskForCrossCatch - 0.05) & (riskAndOutcome['risk'] > riskForCrossCatch + 0.05)]['result'].mean()*100\ncrossCatchReturn = riskAndOutcome[(riskAndOutcome['risk'] > riskForCrossCatch - 0.05) & (riskAndOutcome['risk'] > riskForCrossCatch + 0.05)]['kickReturnYardage'].mean()\n\nnotCrossCatchMisfield = riskAndOutcome[(riskAndOutcome['risk'] > riskForNotCrossCatch - 0.05) & (riskAndOutcome['risk'] > riskForNotCrossCatch + 0.05)]['result'].mean()*100\nnotCrossCatchReturn = riskAndOutcome[(riskAndOutcome['risk'] > riskForNotCrossCatch - 0.05) & (riskAndOutcome['risk'] > riskForNotCrossCatch + 0.05)]['kickReturnYardage'].mean()\n\n\n\n\nprint(\"For returns as risky as those that were fielded in motion on the opposite side of the field \" + str(round(crossCatchMisfield, 1)) + \"% were misfielded and the average return yardage was \" + str(round(crossCatchReturn, 2)) + \" yards\")\nprint(\"On the other hand, for returns as risky as returns that weren't fielded across the field \" + str(round(notCrossCatchMisfield, 1)) + \"% of similarly risky plays were misfielded, but the average return yardage was \"+ str(round(notCrossCatchReturn, 2)) + \" yards\")","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:15:08.594106Z","iopub.execute_input":"2022-01-06T23:15:08.594373Z","iopub.status.idle":"2022-01-06T23:15:08.613254Z","shell.execute_reply.started":"2022-01-06T23:15:08.594343Z","shell.execute_reply":"2022-01-06T23:15:08.612491Z"},"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The returner is significantly increasing his risk for an even smaller return. As such, it is best that the returer avoids fielding the ball altogether.","metadata":{}},{"cell_type":"markdown","source":"While we have presented a few strategies here, please feel free to play around with the calculator at the bottom of the notebook that we have included below to try to create and assess your own strategies.","metadata":{}},{"cell_type":"markdown","source":"## Next Steps\nWhile our results above do lead to some valid and interesting strategies for returners, there are a couple areas where our team would like to explore further. One idea was to incorporate the idea of max ball height into our model and risk calculator. Conventional wisdom says that a higer punt increases the odds of a misfield, so the addition of height to our model may increase accuracy and recall. Another idea is to create a more robust risk calculator (like the one below) so that coaches and players can create more detailed simulations to test out situations they may find themselves in. \n","metadata":{}},{"cell_type":"markdown","source":"## Works Cited\nhttps://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc\n\nhttps://www.sciencedirect.com/topics/medicine-and-dentistry/logistic-regression-analysis\n\nhttps://machinelearningmastery.com/what-is-imbalanced-classification/\n\nhttps://www.analyticsvidhya.com/blog/2020/07/10-techniques-to-deal-with-class-imbalance-in-machine-learning/\n\nhttps://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052\n\nhttps://web.engr.oregonstate.edu/~tgd/publications/mcs-ensembles.pdf\n\nhttps://builtin.com/data-science/precision-and-recall","metadata":{}},{"cell_type":"markdown","source":"## Risk-Reward Calculator","metadata":{}},{"cell_type":"code","source":"from ipywidgets import interact, interactive, fixed, interact_manual, FloatSlider\nimport ipywidgets as widgets","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:15:09.245559Z","iopub.execute_input":"2022-01-06T23:15:09.245952Z","iopub.status.idle":"2022-01-06T23:15:09.249704Z","shell.execute_reply.started":"2022-01-06T23:15:09.245919Z","shell.execute_reply":"2022-01-06T23:15:09.249182Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def status(not_fair_catch, equal_direction, off_one_direction, off_two_direction, gunner_distance_less_than_5_yards, returner_orientation_greater_than_15_degrees):\n    notFairCatch = riskAndOutcome['gameId'] > 0\n    equal = riskAndOutcome['gameId'] > 0\n    offOne = riskAndOutcome['gameId'] > 0\n    offTwo = riskAndOutcome['gameId'] > 0\n    gunnerNear = riskAndOutcome['gameId'] > 0\n    gunnerTurned = riskAndOutcome['gameId'] > 0\n    \n    if not_fair_catch:\n        notFairCatch = riskAndOutcome['fair_catch'] == 0\n    if equal_direction:\n        equal = riskAndOutcome['equalDirection'] == equal_direction\n    if off_one_direction:\n        offOne = riskAndOutcome['off1Direction'] == off_one_direction\n    if off_two_direction:\n        offTwo = riskAndOutcome['off2Direction'] == off_two_direction\n    if gunner_distance_less_than_5_yards:\n        gunnerNear = riskAndOutcome['closestGunnerDistance'] < 5\n    if returner_orientation_greater_than_15_degrees:\n        gunnerTurned = riskAndOutcome['returnerOrAbs'] > 10\n    \n    situationalRisk = riskAndOutcome[notFairCatch & equal & offOne & offTwo & gunnerNear & gunnerTurned]['risk'].mean()\n    situationalMisfield = riskAndOutcome[(riskAndOutcome['risk'] > situationalRisk - 0.05) & (riskAndOutcome['risk'] < situationalRisk + 0.05)]['result'].mean()\n    situationalReturn = riskAndOutcome[(riskAndOutcome['risk'] > situationalRisk - 0.05) & (riskAndOutcome['risk'] < situationalRisk + 0.05)]['kickReturnYardage'].mean()\n    print(\"Based on the inputs, for similar risk plays, \" + str(round(situationalMisfield, 2)) + \"% were misfielded and the average return was \" + str(round(situationalReturn,2)) + \" yards\")\n    \n\ninteract(status, not_fair_catch=False, equal_direction=False, off_one_direction=False, off_two_direction=False, \n         gunner_distance_less_than_5_yards=False, returner_orientation_greater_than_15_degrees=False)\n\n","metadata":{"execution":{"iopub.status.busy":"2022-01-06T23:52:25.009863Z","iopub.execute_input":"2022-01-06T23:52:25.010597Z","iopub.status.idle":"2022-01-06T23:52:25.092604Z","shell.execute_reply.started":"2022-01-06T23:52:25.010558Z","shell.execute_reply":"2022-01-06T23:52:25.091730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}