{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# A New Metric for Kick Returns - Yardage Pressure Ratio (YPR)\n\nWe were giddy with excitment at the possibility of contributing to the NFL experience since the start of this challenge. As fans of the League who watch their home teams regularly (we are Dolphins and Seahawks fans), we were thrilled with the abundance of spatial data from *Next Gen Stats* that would allow us to focus on ways to measure special team plays. We decided to narrow our focus specifically on kick returns for simplicity, but believe that our approach can be adapted to other special team plays. \n\nThere were countless directions we considered taking, but ultimately we asked ourselves how we could distinguish two kick return plays where the returners both ran the same distance. We wanted to find a way to compare the difficulty of kick returns that took into account not only the kick returner's ability, but also the performance of the defense.  \n\nIn this notebook, we focus on kick return plays and introduce a novel metric to assess the difficulty of a kick return.\n\nWe call the metric the **Yardage Pressure Ratio** or the **YPR**. The **YPR** quantifies how far the kick returner was able to run in relation to how close the other players were to him--in other words, how much *pressure* the defense placed on average during the play. The metric reflects the kick returner's ability to cover more yards while overcoming tighter defense.\n\nThe **YPR** is calculated by dividing the total distance the kick returner ran during a kick return in yards by the sum of the distances of all opponents to the kick returner for the play in yards, divided by time of play in seconds, all multiplied by 100 for easier interpretation. \n\nThe remainder of the notebook will shortly walk through methodology and compare some of the best kick returners in the NFL over the 2018-2020 seasons.","metadata":{}},{"cell_type":"markdown","source":"## Code Set up\n\nThe following section will load in the datasets, perform data cleaning and wrangling and do the necessary calculations to get a dataframe with the **YPR**s.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom datetime import datetime\nimport re\nimport os\nimport math\nimport matplotlib.pyplot as plt \nimport seaborn as sns\n\n# import data\n# players\nplayers_path = \"../input/nfl-big-data-bowl-2022/players.csv\"\nplayers_raw = pd.read_csv(players_path)\n\n# tracking 2018\nnfl_2018_path = \"../input/nfl-big-data-bowl-2022/tracking2018.csv\"\nnfl_2018 = pd.read_csv(nfl_2018_path)\n\n# tracking 2019\nnfl_2019_path = \"../input/nfl-big-data-bowl-2022/tracking2019.csv\"\nnfl_2019 = pd.read_csv(nfl_2019_path)\n\n# tracking 2020\nnfl_2020_path = \"../input/nfl-big-data-bowl-2022/tracking2020.csv\"\nnfl_2020 = pd.read_csv(nfl_2020_path)\n\n# get play data -- only look at kick returns\nplays_path = \"../input/nfl-big-data-bowl-2022/plays.csv\"\nplays_raw = pd.read_csv(plays_path)\n\n\n# Get a subset of the *plays* data where the special team play is **Kickoff** and the result \n# is a **Return**. We also don't want plays where the kick was blocked, so we \n# need **KickBlockerId** to be `Nan`.\n\nkickoffReturns = plays_raw.loc[\n    (plays_raw[\"specialTeamsPlayType\"] == \"Kickoff\") &  #### Can change to include punts ####\n    (plays_raw[\"specialTeamsResult\"] == \"Return\") & \n    (np.isnan(plays_raw[\"kickBlockerId\"]))\n]\n\n# grab only the columns we need from each.\nkickoffReturnCols = [\n    \"gameId\", \"playId\", \"returnerId\"\n]\n\nkickoffReturns = kickoffReturns[kickoffReturnCols]\n\nnflTrackingCols = [\n    \"gameId\", \"playId\", \"x\", \"y\", \n    \"nflId\", \"time\", \"team\", \"frameId\", \n    \"dis\"\n]\n\nnfl_2018 = nfl_2018[nflTrackingCols]\nnfl_2019 = nfl_2019[nflTrackingCols]\nnfl_2020 = nfl_2020[nflTrackingCols]\n\n\n# Must join all years by both gameId and playId \ndef mergeWithTracking(nfl_df, kickoffReturns = kickoffReturns):\n    \"\"\"Merges nfl tracking dataframe with plays dataframe \n    subset for kickoff returns and no blocked kicks. \n    \n    Returns merged dataframes\n    \"\"\"\n    merged_df = pd.merge(\n        kickoffReturns, \n        nfl_df, \n        on = [\"gameId\", \"playId\"], \n        how = \"inner\"\n    )\n    return merged_df\n\nnfl_2018_tracking = mergeWithTracking(nfl_2018)\nnfl_2019_tracking = mergeWithTracking(nfl_2019)\nnfl_2020_tracking = mergeWithTracking(nfl_2020)\n\nkickReturnTracking = pd.concat(\n    [nfl_2018_tracking, nfl_2019_tracking, nfl_2020_tracking]\n).reset_index(\n    drop = True\n).drop_duplicates()\n\n\n# data cleanup\n# kickReturnTracking.dtypes\n\n# parse times\nkickReturnTracking[\"time\"] = kickReturnTracking[\"time\"].apply(\n    lambda t: datetime.strptime(t, \"%Y-%m-%dT%H:%M:%S.%f\")\n)\n\n# drop rows where there is a semi-colon for `returnerId`. Multiple returners. < 1% of data and where \n# is `NaN`\nrows_to_drop = kickReturnTracking.loc[\n    kickReturnTracking[\"returnerId\"].str.contains(\";\", na = False)\n]\n\nkickReturnTracking = kickReturnTracking[~kickReturnTracking.isin(rows_to_drop)]\n\nassert kickReturnTracking[\"returnerId\"].str.contains(\";\", na = False).sum() == 0\n\n# make `returnerId` numeric\nkickReturnTracking[\"returnerId\"] = pd.to_numeric(kickReturnTracking[\"returnerId\"])\n\nkickReturnTracking = kickReturnTracking[\n    ~np.isnan(kickReturnTracking[\"returnerId\"])\n]\n\nassert np.isnan(kickReturnTracking[\"returnerId\"]).sum() == 0\n\n# Creating boolean for kick returner\nkickReturnTracking[\"isKickReturner\"] = kickReturnTracking[\"returnerId\"].eq(kickReturnTracking[\"nflId\"])\n\n# Start creating dictionary to hold key value pairs of game and plays for each game.\nunique_game_ids = kickReturnTracking[\"gameId\"].unique()\n# drop nan cases\nunique_game_ids = unique_game_ids[~np.isnan(unique_game_ids)]\n\n# Make empty dictionary that will hold the play IDs for each Game ID.\ngame_plays_dict = {}\n\n# Loop through the game IDs\nfor game in unique_game_ids:\n    # start with an empty array to hold the play IDs\n    play_arr = np.empty(0)\n    \n    # add the play IDs from each Game ID\n    play_arr = np.append(\n        play_arr,\n        kickReturnTracking.loc[\n            kickReturnTracking[\"gameId\"] == game\n        ][\"playId\"]\n    )\n    play_arr = np.unique(play_arr)\n    \n    # Assign the key value pair to the dictionary\n    game_plays_dict[game] = play_arr\n    \n    \n### Helper Functions ###\n# function to get the desired game and play from dataframe\ndef getPlay(df, game, play):\n    \"\"\"\n    Returns single game and single play subset of dataframe. Meant to be\n    used with the kickReturnTracking dataframe.\n    \n    Args: \n        df: should be kickReturnTracking dataframe \n        game: gameId desired\n        play: playId desired. Will be subset of game.\n    \"\"\"\n    game_df = df.loc[df[\"gameId\"] == game]\n    play_df = game_df.loc[game_df[\"playId\"] == play]\n    return play_df\n\n# Write distance formula\ndef distanceFormula(x1, x2, y1, y2):\n    \"\"\"Distance formula \n    Args:\n        x1: x coordinate of player 1 \n        x2: x coordinate of player 2\n        y1: y coordinate of player 1 \n        y2: y coordinate of player 2\n    \"\"\"\n    d = math.sqrt(\n        ((x2 - x1) ** 2) + ((y2 - y1) ** 2)\n    )\n    return d\n\n# Function to get distance covered by kickReturner for each play\ndef getKickReturnerDistanceForPlay(play_df):\n    \"\"\"\n    Returns dataframe of kickReturner distance covered for each play\n    Args:\n        play_df: should be returned from getPlay() function using\n        kickReturner dataframe.\n    \"\"\"\n    \n    # get the row of the kick returner\n    kick_returner_row = play_df.loc[\n        play_df[\"isKickReturner\"] == True\n    ]\n    \n    kickReturnerDistance = kick_returner_row[\"dis\"].sum()\n    \n    # makes dataframe \n    kr_df = pd.DataFrame({\n        \"gameId\" : play_df[\"gameId\"], \n        \"playId\" : play_df[\"playId\"],\n        \"nflId\" : play_df[\"nflId\"],\n        \"time\" : play_df[\"time\"], \n        \"kickReturnerDistance\" : kickReturnerDistance\n    })\n    \n    return kr_df\n        \n    \n# Function to get the opponent's distances for each play\ndef getOpponentsDistanceForPlay(play_df):\n    \"\"\"\n    Calculates the distance of each kick returner to opposing team\n    players.\n    \n    Args:\n        play_df: should be from getPlay function\n    \"\"\"\n    # Get all the frames per play\n    frames = play_df[\"frameId\"]\n    \n    # For loop of each frame of the play_df subset. \n    # make an array to hold the distances of each frame \n    distance = np.empty(0)\n\n    for frame in frames:\n        play_i_df = play_df.loc[\n            play_df[\"frameId\"] == frame\n        ]\n    \n        # get the row of the kick returner\n        kick_returner_row = play_i_df.loc[\n            play_i_df[\"isKickReturner\"] == True\n        ]\n        \n        # Some issue where there are plays without a kick returner. We will drop those for now \n        # and revisit later.\n        try:\n            assert len(kick_returner_row) == 1\n        except:\n            problem_game = play_i_df[\"gameId\"]\n            problem_play = play_i_df[\"playId\"]\n            \n            distance = np.append(distance, 0)\n            return np.sum(distance) \n            break  # Does this break statement need to be here?\n       \n        # capture kick returner team in a variable\n        kick_returner_team = kick_returner_row[\"team\"]\n\n        kick_returner_x = kick_returner_row[\"x\"]\n        kick_returner_y = kick_returner_row[\"y\"]\n\n        # filter for rows of opposing team players\n        opposing_team_players = play_i_df.loc[\n            (play_i_df[\"team\"] != \"football\") & \n            (play_i_df[\"team\"] != str(kick_returner_team))\n        ]\n        \n        # Itterate through the opposing players to get each of their \n        # x and y positions and calculate distance from kick returner\n        for opposing_player_i in range(len(opposing_team_players)):\n            opposing_player_x = opposing_team_players.iloc[opposing_player_i][\"x\"]\n            opposing_player_y = opposing_team_players.iloc[opposing_player_i][\"y\"]\n            \n            # Will use distance formula; euclidean distance \n            distance_diff = distanceFormula(\n                kick_returner_x, \n                opposing_player_x, \n                kick_returner_y, \n                opposing_player_y\n            )\n            distance = np.append(distance, distance_diff)\n            \n        all_opposing_player_dist = np.sum(distance)\n        \n        # Make a dataframe to return \n        distance_df = pd.DataFrame({\n            \"gameId\" : play_df[\"gameId\"],\n            \"playId\" : play_df[\"playId\"],\n            \"nflId\" : play_df[\"nflId\"], \n            \"frameId\" : play_df[\"frameId\"],\n            \"time\" : play_df[\"time\"],\n            \"allOpposingPlayerDist\" : all_opposing_player_dist \n        })\n        \n        return distance_df    \n    \n\n# Function to get time for play\ndef getTimeForPlay(play_df):\n    \"\"\"\n    Function to return duration of play column for dataframes returned from \n    getKickReturnerDistanceForPlay() function and getOpponentsDistanceForPlay()\n    function.\n    This should produce the final output dataframe needed for the remainder of \n    of the analysis.\n    \"\"\"\n    kr_distance_df = getKickReturnerDistanceForPlay(play_df)\n    opp_distance_df = getOpponentsDistanceForPlay(play_df)\n\n    # append kr distance Column to opponent df\n    opp_distance_df[\"kickReturnerDistance\"] = kr_distance_df[\"kickReturnerDistance\"]\n    \n    # remove cases where kr distance is 0\n    opp_distance_df = opp_distance_df[\n        opp_distance_df[\"kickReturnerDistance\"] > 0\n    ]\n\n    # Get play duration\n    play_duration = opp_distance_df[\"time\"].iloc[-1] - opp_distance_df[\"time\"].iloc[0] \n\n    play_duration_seconds = play_duration.value / 1e9\n    opp_distance_df[\"playDurationSec\"] = play_duration_seconds\n    \n    return opp_distance_df\n\n# itterate through all the plays and get the final nfl_kr_opp_df data frame needed.\nnfl_kr_opp_df = pd.DataFrame()\nfor g, p in zip(game_plays_dict.keys(), game_plays_dict.values()):\n    for i in range(len(p)):\n        # Get the ith play from game\n        play_df = getPlay(kickReturnTracking, g, p[i])\n        # get the time, opponent distance, and kr distance df\n        kr_dist_df = getTimeForPlay(play_df)\n        # append the first row \n        nfl_kr_opp_df = nfl_kr_opp_df.append(kr_dist_df.iloc[0])\n        \n        \nnfl_kr_opp_df.reset_index(drop = True, inplace = True)\n\n# get YPR metric\nnfl_kr_opp_df[\"ypr\"] = (\n    100 * \\\n    ((nfl_kr_opp_df[\"kickReturnerDistance\"] / nfl_kr_opp_df[\"allOpposingPlayerDist\"]) \\\n     / nfl_kr_opp_df[\"playDurationSec\"])\n)\n\n# merge player names to nfl_kr_opp_df \nnfl_players_ypr_df = pd.merge(\n    nfl_kr_opp_df, \n    players_raw, \n    on = \"nflId\", \n    how = \"left\"\n)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-03T02:05:18.463329Z","iopub.execute_input":"2022-01-03T02:05:18.463806Z","iopub.status.idle":"2022-01-03T02:10:08.650298Z","shell.execute_reply.started":"2022-01-03T02:05:18.463759Z","shell.execute_reply":"2022-01-03T02:10:08.649196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Methodology \n\nTo calculate the **YPR**, we looked only at kick return special team plays and calculated the sum of the distances of the opponents to the kick returner for each play. The *Next Gen Stats* data provided the `X` and `Y` coordinates of each player on the field which we used to calculate the distances using the distance formula. The ratio is multiplied by 100 for easier interpretation. We compiled all the data in a dataframe that holds the **YPR**s for each available play.","metadata":{}},{"cell_type":"markdown","source":"## Analysis\n\nBelow is a table of the top 10 **YPR** values. There are some impressive runs above 1.0! ","metadata":{}},{"cell_type":"code","source":"nfl_players_ypr_df.sort_values(\"ypr\", ascending = False).head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-03T02:10:08.654149Z","iopub.execute_input":"2022-01-03T02:10:08.654516Z","iopub.status.idle":"2022-01-03T02:10:08.683276Z","shell.execute_reply.started":"2022-01-03T02:10:08.654472Z","shell.execute_reply":"2022-01-03T02:10:08.681509Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We also wanted to consider players who performed well *consistently*. So we filtered for players who recorded at least five kick returns. Below are the top **YPR**s from that subset.","metadata":{}},{"cell_type":"code","source":"nflIdValueCounts = nfl_players_ypr_df[\"nflId\"].value_counts()\n\nnfl_players_ypr_df_gt5 = nfl_players_ypr_df[\n    nfl_players_ypr_df[\"nflId\"].isin(nflIdValueCounts.index[nflIdValueCounts.gt(4)])\n]\n\nnfl_players_ypr_df_gt5.sort_values(\"ypr\", ascending = False).head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-03T02:20:56.855506Z","iopub.execute_input":"2022-01-03T02:20:56.856003Z","iopub.status.idle":"2022-01-03T02:20:56.890347Z","shell.execute_reply.started":"2022-01-03T02:20:56.855966Z","shell.execute_reply":"2022-01-03T02:20:56.889701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Comparison\n\nTo further drive the point home, we wanted to show how players with the highest average yards per kick return (also with at least five runs) are different from those with the highest average **YPR**. Below are those respective lists.","metadata":{}},{"cell_type":"code","source":"avg_run_dist_per_player = nfl_players_ypr_df_gt5.groupby(\n    \"displayName\"\n)[\"kickReturnerDistance\"].mean().sort_values(ascending = False)\n\navg_ypr_per_player = nfl_players_ypr_df_gt5.groupby(\n    \"displayName\"\n)[\"ypr\"].mean().sort_values(ascending = False)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-03T02:24:49.595945Z","iopub.execute_input":"2022-01-03T02:24:49.596654Z","iopub.status.idle":"2022-01-03T02:24:49.605164Z","shell.execute_reply.started":"2022-01-03T02:24:49.596599Z","shell.execute_reply":"2022-01-03T02:24:49.60442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Top kick return distances on average: ","metadata":{}},{"cell_type":"code","source":"avg_run_dist_per_player","metadata":{"execution":{"iopub.status.busy":"2022-01-03T02:24:51.172241Z","iopub.execute_input":"2022-01-03T02:24:51.173091Z","iopub.status.idle":"2022-01-03T02:24:51.179766Z","shell.execute_reply.started":"2022-01-03T02:24:51.173048Z","shell.execute_reply":"2022-01-03T02:24:51.178841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Average **YPR** per player:","metadata":{}},{"cell_type":"code","source":"avg_ypr_per_player","metadata":{"execution":{"iopub.status.busy":"2022-01-03T02:25:05.902673Z","iopub.execute_input":"2022-01-03T02:25:05.903894Z","iopub.status.idle":"2022-01-03T02:25:05.911083Z","shell.execute_reply.started":"2022-01-03T02:25:05.903843Z","shell.execute_reply":"2022-01-03T02:25:05.910453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Even though Nate Stupar had the highest average kick return distance among players in our subset, the **YPR** uncovers that Derek Carrier's runs had more pressure on average. The **YPR** surfaces Derek Carrier's otherwise hidden accomplishment. In fact, Derek Carrier is **eigth from the bottom in average kick return distance**. Simply focusing on average kick return distance would rank players like Derek Carrier significantly below their true ability. The **YPR** uncovers a player's ability to overcome tighter defense--something coaches might want to consider if they are playing a notoriously defense-heavy team, or a play near the goal line.","metadata":{}},{"cell_type":"code","source":"avg_run_dist_per_player.loc[\"Derek Carrier\":]","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2022-01-03T02:36:10.181249Z","iopub.execute_input":"2022-01-03T02:36:10.181722Z","iopub.status.idle":"2022-01-03T02:36:10.191248Z","shell.execute_reply.started":"2022-01-03T02:36:10.181684Z","shell.execute_reply":"2022-01-03T02:36:10.190259Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Descriptive Statistics\n\nFor a more complete picture of the **YPR**, see below for the distribution of **YPR**s and the median and mean values for plays in the subset.\n\nThe distribution of **YPR**s is slightly right skewed, but otherwise unimodal with a peak around 0.3.","metadata":{}},{"cell_type":"code","source":"sns.set(rc = {\"figure.figsize\": (10, 6)})\nsns.set_theme(style = \"whitegrid\")\nhist = sns.histplot(\n    nfl_players_ypr_df_gt5[\"ypr\"], \n    bins = 30\n)\nhist.set_title(\"Distribution of YPRs\")\nhist.set_xlabel(\"YPR\");","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-03T02:43:29.047952Z","iopub.execute_input":"2022-01-03T02:43:29.048235Z","iopub.status.idle":"2022-01-03T02:43:29.432048Z","shell.execute_reply.started":"2022-01-03T02:43:29.0482Z","shell.execute_reply":"2022-01-03T02:43:29.431095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'The mean YPR value is {np.mean(nfl_players_ypr_df_gt5[\"ypr\"])}\\nThe median YPR value is {np.median(nfl_players_ypr_df_gt5[\"ypr\"])}')","metadata":{"execution":{"iopub.status.busy":"2022-01-03T02:43:41.461227Z","iopub.execute_input":"2022-01-03T02:43:41.461738Z","iopub.status.idle":"2022-01-03T02:43:41.468836Z","shell.execute_reply.started":"2022-01-03T02:43:41.461694Z","shell.execute_reply":"2022-01-03T02:43:41.467794Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion\n\nWe hope this brief introduction into the **Yardage Pressure Ratio** highlights a potential new way to assess special team plays. We envision the **YPR** can be a useful metric for coaches and the NFL to asses performances of not just the kick returner, but also the defense. The **YPR** uncovers otherwise ignored performances by kick returners who consistently are able to run farther with tighter defense. Conversely, this could also be helpful for teams tracking their defense's performance across plays. \n\nThis is just the tip of the iceburg in terms of ways to use the **YPR**. One could also imagine the **YPR** used to assess punts, field goals, and even general defensive plays. ","metadata":{}}]}