{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Returner Quality \nKnowing the quality of a returner is more complex than simply how many yards are gained on returns. For example, a return of a few positive yards may have been at the expense of greater yardage if a player did not choose an optimal path away from tacklers. Rather than using total yardage for a measure of returner quality, we can estimate the return yardage using two metrics: a tackle probability metric and an expected return yards metric. Combining the outputs of the two metrics and creating a composite metric with tackle probability and expected return yards, we can evaluate the returner on a metric more statistically advanced than just total yardage. I call this metric Returner Quality. \n\n## Measuring Returner Quality  \nA primary responsibility of a returner is to gain yardage beyond the initial reception point. A player can take all the available space in front of them and a player can increase available space by making potential tacklers miss tackle attempts. Players who can excel in positive return yardage and also avoid tackles can be considered \"good\" returners. Players who are below-average in these measures could be considered \"bad\" returners. Players who excel in positive return yardage but do not avoid tackles can be labeled \"trains\", as they theoretically are straight-line (or vertical) runners while players who excel in avoiding tackles but are below-average in return yardage can be labeled \"dancers\", as they theoretically make many cut moves (horizontal runners). We can illustrate this by taking the average return yardage and average missed tackles per return (min 20 returns).  \n![](https://i.imgur.com/ogcXuLI.png)  \n\nHowever, per-play averages alone suffer when contextualized over the course of a season. Players with more returns will inevitably have more return yardage and avoided tackles. For example, 2018 Tyler Lockett is considered a bad returner based on his average return yards (15.3) and average tackles avoided (0.3) but over the course of a season he is considered a good returner because he had 38 returns over the season (583 yards gained, 10 tackles avoided).  \n![](https://i.imgur.com/LKVKaTY.png)  \n\nSo is Tyler Lockett good or bad? Rather than using season totals, I created a model that estimated return yardage and a model that estimated tackle probability. Each of these models estimated per frame of tracking data, allowing for precise event-based measurements over time. The estimated return yardage at the time a returner receives the ball is what I consider \"expected return yardage\". Kicking team members within 4 yards of the returner who do _not_ tackle the returner are considered avoided tackles and their tackle probability is considered the amount of a tackle \"avoided\". \n\nBelow is an example of a play that tracks these metrics. \n![](https://i.imgur.com/01KioxM.gif) \n\nThe dotted black line is the line of scrimmage, solid red line is the actual return yards gained, the dotted red line is a frame-by-frame estimated return yards, and the dotted blue line is the expected return yards at the point of reception. In the example above, Lockett gains 9 yards but was estimated to gain 12 yards. He also avoids 0 tackles in this example.  \n\nUsing this expected return yards value based on nearly all return plays from 2018-2020, I can subtract actual from expected to get \"return yards over expected\" where returns with positive return yards over expected would be good for the returner. Likewise, using the estimated tackles avoided metric, we can assess the elusiveness of a player. \n\nWe can plot these two new measures of returner quality just like we plotted return yardage and tackles avoided while still categorizing each player based off of their original returner types:  \n![](https://i.imgur.com/ovouW66.png)  \n\nBased on Lockett's per-return averages, he would classify as a bad returner based on return quality. We can also see if this is the case in the season total:  \n![](https://i.imgur.com/q6FFjzK.png) \n\nOver the course of the 2018 season, he would technically qualify as a returner who dances but does not generate above-average return yardage.  \n\nScaling and combining average return yards over expected and average estimated tackles avoided results in a new metric, returner quality. \n### Top 10 player-seasons, by returner quality \nRank |\tPlayer |\t# Returns |\tReturner Quality |\tReturner Type  \n--- |--- |--- |--- |---  \n1 |\t2018  Darius Jennings |\t23 |\t1.11 |\ttrain\n2 |\t2018  Desmond King |\t42 |\t0.89 |\tgood\n3 |\t2018  Andre Roberts |\t62 |\t0.81 |\tgood\n4 |\t2018  Cordarrelle Patterson |\t22 |\t0.80 |\ttrain\n5 |\t2018  Tremon Smith |\t32 |\t0.73 |\ttrain\n6 |\t2019  Cordarrelle Patterson |\t27 |\t0.68 |\tgood\n7 |\t2019  Desmond King |\t30 |\t0.63 |\tdances\n8 |\t2018  Alex Erickson |\t59 |\t0.61 |\ttrain\n9 |\t2020  Isaiah Rodgers |\t25 |\t0.60 |\tgood\n10 |\t2020  Deonte Harris |\t32 |\t0.58 |\tgood   \n  \n### Bottom 10 player-seasons, by returner quality \nRank |\tPlayer |\t# Returns |\tReturner Quality |\tReturner Type  \n--- |--- |--- |--- |---  \n81 |\t2018 Isaiah McKenzie |\t26 |\t-0.48 |\tbad\n82 |\t2020 Braxton Berrios |\t20 |\t-0.49 |\tbad\n83 |\t2020 Brandon Powell |\t30 |\t-0.53 |\tbad\n84 |\t2019 Richie James |\t44 |\t-0.54 |\tbad\n85 |\t2020 Greg Ward |\t20 |\t-0.56 |\tbad\n86 |\t2019 T.J. Logan |\t23 |\t-0.59 |\tdances\n87 |\t2018 Adam Humphries |\t21 |\t-0.61 |\tbad\n88 |\t2019 Tyler Lockett |\t27 |\t-0.66 |\tbad\n89 |\t2020 Christian Kirk |\t22 |\t-0.82 |\tbad\n90 |\t2018 Pharoh Cooper |\t22 |\t-0.93 |\tbad  \n\n2018 Tyler Lockett ranks 67th in returner quality (-0.20).  \n\nReturner quality has an average of 0.076 (sd: 0.404). The distribution for qualified returners ranges between -0.94 and 1.12.  \n![](https://i.imgur.com/AGSwUwy.png)   \n\nReturner quality also correlates positively with actual return yardage and tackles avoided as determined by PFF.  \n![](https://i.imgur.com/KikFY6u.png)  \n\nOverall, if a player had to choose between a vertical runner or a horizontal runner, vertical running has a stronger positive correlation with returner quality than horizontal running. In more general terms, juking your opponent out is no replacement for gaining positive yardage. \n\nReturner quality does have limitations. One area of limitation is blocker usage by the returner. \n![](https://i.imgur.com/fDkRnmw.gif)  \n\nThe tackle probability model considers blocker distance as well as kicking teammate distance relative to each other and the ball carrier but has trouble assigning tackle probability when a returner's best path requires blocker usage. In the example above, the returner passes behind their blocker even though the nearest tackler is identified with having a high likelihood to tackle. This perhaps indicates overfitting tackler distance from returner and may be best solved with a model that takes into account future areas controlled by the returning team.  \n\n## Returner Quality, applied  \nTwo application areas are return optimization and identifying players who may be overlooked in their return abilities. Since this is a per-play metric, each play can be evaluated for returner quality. Each play can also assess whether returner quality depended on return yards over expected or estimated missed tackles. Simple returner optimization could rely on whether a returner followed an optimal path and when the return failed to be optimal over time.  \n\nIdentifying players who have positive returner quality but perhaps are underappreciated is also a potential application. For example, 2018 Chester Rogers classifies as a bad returner based on below-average return yards and below-average tackles avoided. Using returner quality, he scores a 0.181 which is above average in returner quality. This is similar to 2020 Cordarrelle Patterson (0.187). The discrepancy between the two players as identified by returner quality is Patterson gains 2.17 yards over expected whereas Rogers gains 1.72 yards over expected. Although small, this is the difference between a 1st team all-pro year and not, respectively. \n\nThere are many areas of improvement for returner quality but this model provides good context for a returner using basic kick/punt return concepts. Future steps would include model improvement and predictability of return quality over seasons. \n\n## Model \n\nFor the data inputs to the tackle probability model, I used the majority of player tracking information (time, space, speed) as well as player-specific statistics (height/weight, average speed metrics) and play-context information (kicking team player distance from ball carrier, closest teammate, nearest teammate to ball carrier, and nearest opponent player). The target of the model was the player who tackled the ball carrier (the target label is static throughout each play, i.e. non-tacklers get a 0 for every row whereas the tackler gets a 1 for every row for the target feature column). The model is a `LightGBM` gradient boosted decision tree classifer, hyperparameter tuned using `optuna`. \n\nAfter the model tuning, predicted probabilities are generated on all available plays using a 5-fold cross validation, saving the out-of-sample predictions per iteration. \n\nThese outputs are used in conjunction with a feature set similar to the one The Zoo used in their 2020 Big Data Bowl [expected rush yards notebook](https://www.kaggle.com/jccampos/nfl-2020-winner-solution-the-zoo). Rather than using a convolutional neural net, I simplified the approach by using data only from the highest probability tackler, as opposed to all 11 kicking team players in a `LightGBM` gradient boosted decision tree regressor model. This was in addition to using the information regarding the returner, which also includes the distance to nearest tackler as well as the probability of tackle from the previous model. The target is yards gained on the return. This produces a prediction of yards, where outputs were also saved through out-of-sample 5-fold cross validation. \n\nThis results in two predictions: return yards and tackle probability. I scale both of these values using a z-transform and add the results together to create returner quality. \n![](https://i.imgur.com/7sNC8xc.jpeg)  \n\n\n\n","metadata":{"id":"tPA42e0NTwYx"}},{"cell_type":"markdown","source":"# installs","metadata":{"id":"-V0SGvlk3WGg","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"!apt install imagemagick\n!pip install optuna\n!pip uninstall lightgbm -y\n!git clone --recursive --branch v3.2.1 https://github.com/Microsoft/LightGBM\n!apt-get install -y -qq libboost-all-dev","metadata":{"id":"gc6-DYii42NT","execution":{"iopub.status.busy":"2022-01-06T14:18:34.416431Z","iopub.execute_input":"2022-01-06T14:18:34.41685Z","iopub.status.idle":"2022-01-06T14:18:35.389299Z","shell.execute_reply.started":"2022-01-06T14:18:34.416783Z","shell.execute_reply":"2022-01-06T14:18:35.385844Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%bash\ncd LightGBM\nrm -rf build\nmkdir build\ncd build\ncmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..\nmake -j$(nproc)","metadata":{"id":"jLVRHdGv48ZE","execution":{"iopub.status.busy":"2022-01-06T14:18:35.391458Z","iopub.status.idle":"2022-01-06T14:18:35.391922Z","shell.execute_reply.started":"2022-01-06T14:18:35.391637Z","shell.execute_reply":"2022-01-06T14:18:35.391688Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cd LightGBM/python-package/;python3 setup.py install --precompile\n!mkdir -p /etc/OpenCL/vendors && echo \"libnvidia-opencl.so.1\" > /etc/OpenCL/vendors/nvidia.icd\n!rm -r LightGBM\n!apt install imagemagick\n!pip install optuna","metadata":{"id":"HQ-UqT8P494m","execution":{"iopub.status.busy":"2022-01-06T14:18:35.39425Z","iopub.status.idle":"2022-01-06T14:18:35.394763Z","shell.execute_reply.started":"2022-01-06T14:18:35.394496Z","shell.execute_reply":"2022-01-06T14:18:35.394523Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# imports ","metadata":{"id":"uzBe6tM_5BTL","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"import pandas as pd \nimport numpy as np \nimport matplotlib.pyplot as plt \nimport seaborn as sns \nimport os \nfrom tqdm.notebook import tqdm\n\nimport optuna\nimport optuna.integration.lightgbm as lgb\nimport lightgbm as lgb_vanilla\nfrom sklearn.metrics import mean_squared_error, log_loss, accuracy_score, roc_auc_score, roc_curve, auc\nfrom sklearn.model_selection import GroupKFold\nfrom sklearn.preprocessing import minmax_scale, scale\nfrom scipy.stats import pearsonr, spearmanr, percentileofscore\nfrom scipy.spatial import distance\n\n# for mpl animation\nimport matplotlib.animation as animation\nfrom matplotlib import rc\nrc('animation', html='html5')\n","metadata":{"id":"snb3Wanw5Dqc","execution":{"iopub.status.busy":"2022-01-06T14:18:35.397407Z","iopub.status.idle":"2022-01-06T14:18:35.397938Z","shell.execute_reply.started":"2022-01-06T14:18:35.397645Z","shell.execute_reply":"2022-01-06T14:18:35.397673Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# functions ","metadata":{"id":"KQ5hoUZd5oqU","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"def get_play_by_frame(fid, ax, los, one_play):\n  ax.cla()\n  gid = one_play['gameId'].unique()[0]\n  pid = one_play['playId'].unique()[0]\n  one_frame = one_play.loc[one_play['frameId']==fid]\n  try:\n    fig1 = sns.scatterplot(x='x',y='y',data=one_frame, hue='team', ax=ax, size='target_scaled', sizes=(60,200))\n  except:\n    fig1 = sns.scatterplot(x='x',y='y',data=one_frame, hue='team', ax=ax, s=100)\n  fig1.axvline(los, c='k', ls=':')\n  fig1.axvline(0, c='k', ls='-')\n  fig1.axvline(100, c='k', ls='-')\n  fig1.set_title(f\"game {gid} play {pid}\")\n  fig1.legend([]).set_visible(False)\n  sns.despine(left=True)\n  fig1.set_ylabel('')\n\n  fig1.set_yticks([])\n  fig1.set_xlim(-10,110)    \n  fig1.set_ylim(0,54) \n\ndef animate_predicted_play(one_play, suffix='_pred', use_pred_values=False, \n                           pred_value=None):    \n  gid = one_play['gameId'].unique()[0]\n  pid = one_play['playId'].unique()[0]\n\n  scaled = pd.Series()\n  if use_pred_values:\n    if pred_value is None:\n      raise ValueError('no pred value provided bruv')\n      return None\n    for idx, row in one_play.loc[:, ['team','frameId']].drop_duplicates().iterrows():\n      fid = row['frameId']\n      tid = row['team']\n      _one_play = one_play.loc[(one_play['team']==tid)&(one_play['frameId']==fid), pred_value]\n      scaled = scaled.append(pd.Series(minmax_scale(_one_play, feature_range=(0,1)), index=_one_play.index))\n      \n    one_play['target_scaled'] = scaled.fillna(scaled.quantile(.3))\n    one_play.loc[one_play['team']=='football', 'target_scaled'] = scaled.quantile(.15) \n\n  los = one_play.loc[(one_play['frameId']==1) & (one_play['team']=='football'), 'x'].values[0]\n\n  fig = plt.figure(figsize=(14.4, 6.4))\n  ax = fig.gca()\n  ani = animation.FuncAnimation(fig, get_play_by_frame, \n                                frames=one_play['frameId'].sort_values().unique().shape[0],\n                                interval=100, repeat=True, fargs=(ax,los,one_play,))\n\n  plt.close()\n  ani.save(f'{gid}_{pid}.gif', writer='imagemagick', fps=10)\n  return ani    \n\ndef tuple2range(df):\n  x = range(int(df[0]),int(df[1]))\n  return x\n\ndef is_event_str(event):\n  return type(event) == str\n\ndef is_event_int(event):\n  return type(event) == int\n\ndef is_event_math(event):\n  event = event.split(' ')  \n  return event[0] in ['add', 'sub', 'mul', 'div']\n\ndef do_event_math(event, df):\n  new_series = None\n  event = event.split(' ')  \n  if event[0] == 'add':\n    new_series = df['frameId'].add(int(event[1]))\n  elif event[0] == 'sub':\n    new_series = df['frameId'].sub(int(event[1]))\n  elif event[0] == 'mul':\n    new_series = df['frameId'].mul(int(event[1]))\n  elif event[0] == 'div':\n    new_series = df['frameId'].div(int(event[1]))\n  return new_series\n\ndef explode_dataframe(df, from_event, to_event):\n  if is_event_str(from_event):\n    _df1 = df.loc[(df['event']==from_event), ['gameId', 'playId', 'frameId']].drop_duplicates()\n\n  elif is_event_int(from_event):\n    _df1 = df.loc[(df['frameId']==from_event), ['gameId', 'playId', 'frameId']].drop_duplicates()\n\n  if is_event_str(to_event): \n    if is_event_math(to_event): \n      _df2 = _df1.loc[:, ['gameId', 'playId', 'frameId']].drop_duplicates() \n      _df2['frameId'] = do_event_math(to_event, _df1)\n    else:\n      _df2 = df.loc[(df['event']==to_event), ['gameId', 'playId', 'frameId']].drop_duplicates()\n\n  elif is_event_int(to_event): \n    _df2 = df.loc[(df['frameId']==to_event), ['gameId', 'playId', 'frameId']].drop_duplicates() \n  \n  _df1 = _df1.merge(_df2, on=['gameId', 'playId'], suffixes=('_from', '_to'))\n  _df1['explode'] = pd.Series(_df1.loc[:, ['frameId_from', 'frameId_to']].values.tolist(), index=_df1.index)\n  _df1['explode'] = _df1['explode'].apply(tuple).apply(tuple2range)\n  _df1 = _df1.explode('explode')\n  _df = _df1.drop(['frameId_from', 'frameId_to'], axis=1).rename(columns={'explode':'frameId'})\n  return _df \n\ndef clean_raw_data(df):\n  df['is_football'] = 0\n  df.loc[df['team']=='football', 'is_football'] = 1\n  _df = df.loc[df['position'].isin(['P','K']), ['gameId', 'playId', 'team']].drop_duplicates()\n  _df['is_kicking_team'] = 1\n  df = df.merge(_df, how='left')\n  df['is_kicking_team'] = df['is_kicking_team'].fillna(0).astype(int)\n  df['is_going_left'] = df['playDirection'].replace(['left', 'right'], [1,0])\n  return df\n\ndef clean_players_data(players):\n  _height = players.loc[players['height'].str.contains('-'),'height'].str.split('-', expand=True)\n  _height['inches'] = _height[0].astype(int).mul(12).add(_height[1].fillna(0).astype(int))\n  players['inches'] = _height['inches']\n  _height = players.loc[~players['height'].str.contains('-'),'height'].astype(int)\n  players['inches'] = players['inches'].fillna(_height)\n  return players      \n\ndef generate_tackles_df(df, plays, pff):\n  _df = df.loc[(df['is_kicking_team']==1) & (df['event']=='tackle'), ['gameId', 'playId', 'frameId', 'nflId', 'jerseyNumber']].drop_duplicates()\n  _df = plays.loc[:, ['gameId', 'playId', 'possessionTeam']].merge(_df)\n  _df['jerseyNumber'] = _df['jerseyNumber'].astype(int)\n  _df['tackler'] = _df['possessionTeam'].add(' ').add(_df['jerseyNumber'].astype(str).str.rjust(2,'0'))\n  _pff = pff.loc[pff['tackler'].notnull(), ['gameId', 'playId', 'tackler']]\n  _pff = _pff.append(pff.loc[pff['tackler'].notnull(), ['gameId', 'playId', 'assistTackler']].rename(columns={'assistTackler':'tackler'})).dropna(subset=['tackler'])\n  _pff['makes_tackle'] = 1\n  _df = _df.merge(_pff)\n  df_tackles = _df.loc[:, ['gameId', 'playId']].drop_duplicates().merge(df)\n  df_tackles = df_tackles.merge(_df, how='left')\n  df_tackles['makes_tackle'] = df_tackles['makes_tackle'].fillna(0).astype(int)\n  return df_tackles\n\ndef generate_agg_bullshit(df, players, plays):\n  _df = explode_dataframe(df, 'ball_snap', 'add 3')\n  df_agg = df.merge(_df)\n  df_agg = df_agg.loc[:, ['nflId', 's', 'a']].groupby(['nflId'], as_index=False).mean()\n  \n  df_agg = players.loc[:, ['nflId', 'Position', 'inches', 'weight']].merge(df_agg)\n  df_agg = df_agg.rename(columns={'s':'snapBurstS', 'a':'snapBurstA'})\n\n  _df = explode_dataframe(df, 1, 'ball_snap')\n  _df_agg = df.merge(_df)\n  _df_agg = _df_agg.loc[:, ['gameId', 'playId','nflId', 'dis']].groupby(['gameId', 'playId','nflId'], as_index=False).sum()\n  _df_agg = _df_agg.loc[:, ['nflId', 'dis']].groupby('nflId', as_index=False).mean()\n  _df_agg = _df_agg.rename(columns={'dis':'preSnapDis'})  \n  df_agg = df_agg.merge(_df_agg)\n  df_agg = players.loc[:, ['nflId','displayName']].merge(df_agg)\n\n  pids = df.loc[~df['nflId'].isin(df_agg['nflId'].unique()), 'nflId'].unique()\n  _df = explode_dataframe(df, 'autoevent_kickoff', 'add 3')\n  _df1 = explode_dataframe(df, 'ball_snap', 'add 3')\n  _df2 = explode_dataframe(df, 'kickoff', 'add 3')\n  _df3 = explode_dataframe(df, 'onside_kick', 'add 3')\n  _df4 = explode_dataframe(df, 'drop_kick', 'add 3')\n  _df5 = explode_dataframe(df, 'free_kick', 'add 3')\n  _df = _df.append(_df1).append(_df2).append(_df3).append(_df4).append(_df5).drop_duplicates()\n  burst_fill = df.loc[df['nflId'].isin(pids)].merge(_df)\n  burst_fill = burst_fill.loc[:, ['nflId', 's', 'a']].groupby(['nflId'], as_index=False).mean()\n\n  _df = explode_dataframe(df, 1, 'autoevent_kickoff')\n  _df1 = explode_dataframe(df, 1, 'ball_snap')\n  _df2 = explode_dataframe(df, 1, 'kickoff')\n  _df3 = explode_dataframe(df, 1, 'onside_kick')\n  _df4 = explode_dataframe(df, 1, 'drop_kick')\n  _df5 = explode_dataframe(df, 1, 'free_kick')\n  _df = _df.append(_df1).append(_df2).append(_df3).append(_df4).append(_df5).drop_duplicates()\n  motion_fill = df.loc[df['nflId'].isin(pids)].merge(_df)\n  motion_fill = motion_fill.loc[:, ['gameId', 'playId','nflId', 'dis']].groupby(['gameId', 'playId', 'nflId'], as_index=False).sum()\n  motion_fill = motion_fill.loc[:, ['nflId', 'dis']].groupby(['nflId'], as_index=False).mean()\n\n  agg_fill = players.loc[:, ['nflId', 'displayName', 'Position', 'inches', 'weight']].merge(burst_fill).merge(motion_fill).rename(columns={'s':'snapBurstS', 'a':'snapBurstA', 'dis':'preSnapDis'})\n\n  df_agg = df_agg.append(agg_fill, ignore_index=True).drop_duplicates(subset=['nflId'])\n\n  _plays = plays.loc[plays['specialTeamsResult']=='Return', ['returnerId']].rename(columns={'returnerId':'nflId'}).dropna().drop_duplicates()\n  _plays['nflId'] = _plays['nflId'].str.split(';')\n  _plays = _plays.explode('nflId')\n  _plays['nflId'] = _plays['nflId'].astype(int)\n\n  _df = explode_dataframe(df, 'kick_received', 'add 3')\n  _df_agg = df.merge(_df)\n  _df_agg = _df_agg.loc[:, ['nflId', 's', 'a']].groupby('nflId', as_index=False).mean()\n  _df_agg1 = _df_agg.rename(columns={'s':'kickReturnBurstS', 'a':'kickReturnBurstA'})  \n\n  _df = explode_dataframe(df, 'kick_received', 'add 3')\n  _df_agg = df.merge(_df)\n  _df_agg = _df_agg.loc[:, ['gameId', 'playId','nflId', 'dis']].groupby(['gameId', 'playId','nflId'], as_index=False).sum()\n  _df_agg = _df_agg.loc[:, ['nflId', 'dis']].groupby('nflId', as_index=False).mean()\n  _df_agg2 = _df_agg.rename(columns={'dis':'kickReturnBurstDis'})  \n\n  _df = explode_dataframe(df, 'punt_received', 'add 3')\n  _df_agg = df.merge(_df)\n  _df_agg = _df_agg.loc[:, ['nflId', 's', 'a']].groupby('nflId', as_index=False).mean()\n  _df_agg3 = _df_agg.rename(columns={'s':'puntReturnBurstS', 'a':'puntReturnBurstA'})  \n\n  _df = explode_dataframe(df, 'punt_received', 'add 3')\n  _df_agg = df.merge(_df)\n  _df_agg = _df_agg.loc[:, ['gameId', 'playId','nflId', 'dis']].groupby(['gameId', 'playId','nflId'], as_index=False).sum()\n  _df_agg = _df_agg.loc[:, ['nflId', 'dis']].groupby('nflId', as_index=False).mean()\n  _df_agg4 = _df_agg.rename(columns={'dis':'puntReturnBurstDis'})  \n\n  _df_agg = _df_agg1.merge(_df_agg2).merge(_df_agg3).merge(_df_agg4)\n  _df_agg = _plays.merge(_df_agg)\n\n  df_agg = df_agg.merge(_df_agg, how='left')\n  return df_agg\n\ndef generate_distance2football(df):\n  df_dist2fb = pd.DataFrame()\n  total = df.loc[:, ['gameId', 'playId']].drop_duplicates().shape[0]\n  for idx, row in tqdm(df.loc[:, ['gameId', 'playId']].drop_duplicates().iterrows(), total=total, desc='dist2fb', leave=False):\n    gid, pid = row['gameId'], row['playId']\n    one_play = df.loc[(df['gameId']==gid) & (df['playId']==pid)]\n    defteam_ids = one_play.loc[(one_play['is_kicking_team']==1), 'nflId'].astype(int).unique()\n    id2generic = {x:idx for idx,x in enumerate(defteam_ids)}\n    generic2id = {v:k for k,v in id2generic.items()}\n    n_defenders = defteam_ids.shape[0]\n    def_loc = one_play.loc[one_play['nflId'].isin(defteam_ids), ['x', 'y']]\n    football_loc = one_play.loc[one_play['is_football']==1, ['x', 'y']]\n    _dist = pd.DataFrame(distance.cdist(football_loc, def_loc, 'euclidean')) \n\n    _stage = pd.DataFrame()\n    for generic_id in range(len(defteam_ids)):\n      nflId = generic2id[generic_id]\n      player_cols = _dist.index.values + (generic_id * one_play['frameId'].unique().shape[0])\n      _dist1 = _dist.loc[:, player_cols]\n      _dist1.columns = range(_dist1.shape[1])\n      _df = pd.DataFrame(np.diag(_dist1), index=[_dist1.index, _dist1.columns]).reset_index(drop=True)\n      _df['frameId'] = range(1,_df.shape[0]+1)\n      _df['nflId'] = nflId\n      _df['gameId'] = gid\n      _df['playId'] = pid\n      _df = _df.rename(columns={0:'distanceToFootball'})\n      _stage = _stage.append(_df, ignore_index=True)\n    df_dist2fb = df_dist2fb.append(_stage, ignore_index=True)\n  return df_dist2fb       \n\ndef generate_nearest_blocker(df):\n  nearest_blocker = pd.DataFrame()\n  total = df.loc[:, ['gameId', 'playId']].drop_duplicates().shape[0]\n  for idx, row in tqdm(df.loc[:, ['gameId', 'playId']].drop_duplicates().iterrows(), total=total, desc='nearest blocker', leave=False):\n    gid, pid = row['gameId'], row['playId']\n    one_play = df.loc[(df['gameId']==gid) & (df['playId']==pid)]\n\n    n_frames = one_play['frameId'].max()\n    posteam_ids = one_play.loc[(one_play['is_kicking_team']==0) & (one_play['is_football']==0), 'nflId'].astype(int).unique()\n    defteam_ids = one_play.loc[(one_play['is_kicking_team']==1), 'nflId'].astype(int).unique()\n\n    defid2generic = {x:idx for idx,x in enumerate(defteam_ids)}\n    defgeneric2id = {v:k for k,v in defid2generic.items()}\n\n    posid2generic = {x:idx for idx,x in enumerate(posteam_ids)}\n    posgeneric2id = {v:k for k,v in posid2generic.items()}\n\n    n_footguys = posteam_ids.shape[0]\n    n_defenders = defteam_ids.shape[0]\n\n    pos_loc = one_play.loc[one_play['nflId'].isin(posteam_ids), ['x', 'y']]\n    def_loc = one_play.loc[one_play['nflId'].isin(defteam_ids), ['x', 'y']]\n    _dist = pd.DataFrame(distance.cdist(pos_loc, def_loc, 'euclidean')) \n    diagonal = _dist.copy()\n    for col in diagonal.columns:\n      diagonal[col].values[:] = 0\n    _stage = pd.DataFrame()\n    for def_generic_id in range(len(defteam_ids)):\n      def_nflId = defgeneric2id[def_generic_id]\n      def_player_cols = np.array(range(n_frames)) + (def_generic_id * one_play['frameId'].unique().shape[0])\n      _dist1 = _dist.loc[:, def_player_cols]\n      _dist1.columns = range(_dist1.shape[1])\n      \n      _diagonal = diagonal.copy()\n      for pos_generic_id in range(len(posteam_ids)):\n        pos_nflId = posgeneric2id[pos_generic_id]\n        pos_player_cols = np.array(range(n_frames)) + (pos_generic_id * one_play['frameId'].unique().shape[0])\n        _dist2 = _dist1.loc[pos_player_cols]\n        _dist2.columns = pos_player_cols\n        _diagonal.loc[pos_player_cols, pos_player_cols] = _dist2\n      _df = pd.DataFrame(np.diag(_diagonal))\n      _df['frameId'] = list(range(1,n_frames+1)) * len(posteam_ids)\n      _df['posId'] = np.repeat(posteam_ids, n_frames)\n      _df = pd.pivot_table(_df, index=['frameId'], columns=['posId'])\n      _df.columns = [x[1] for x in _df.columns.to_flat_index()]\n      _df['nearestBlockerDistance'] = _df.min(axis=1)\n      _df['nearestBlockerId'] = _df.idxmin(axis=1)\n      _nearest_blocker = _df.loc[:, ['nearestBlockerDistance', 'nearestBlockerId']].reset_index()\n      _nearest_blocker['nflId'] = def_nflId\n      _nearest_blocker['gameId'] = gid\n      _nearest_blocker['playId'] = pid  \n      _stage = _stage.append(_nearest_blocker)\n    nearest_blocker = nearest_blocker.append(_stage)    \n  nearest_blocker = nearest_blocker.reset_index(drop=True)  \n  return nearest_blocker  \n\ndef standardization(df):\n  df['dir_rad'] = np.mod(90 - df['dir'], 360) * math.pi/180.0\n  df['x_std'] = df['x']\n  df.loc[df['is_going_left']==1, 'x_std'] = 120 - df.loc[df['is_going_left']==1, 'x_std']\n  df['y_std'] = df['y']\n  df.loc[df['is_going_left']==1, 'y_std'] = 160/3 - df.loc[df['is_going_left']==1, 'y']\n  df['dir_std'] = df['dir_rad']\n  df.loc[df['is_going_left']==1, 'dir_std'] = np.mod(np.pi + df.loc[df['is_going_left']==1, 'dir_rad'], 2*np.pi)\n  \n  # kinda crude. could make this sharper (aka eliminate those who come back into the end zone)\n  df['touchback_possible'] = 0\n  df.loc[df['x_std']>100, 'touchback_possible'] = 1\n\n  #Replace Null in Dir_rad\n  df.loc[(df['is_kicking_team']==1) & (df['dir_std'].isna()), 'dir_std'] = 0\n  df.loc[(df['is_kicking_team']==0) & (df['is_football']==0) & (df['dir_std'].isna()), 'dir_std'] = np.pi\n\n  # speed relative of direction \n  df['sx'] = df['s']*df['dir_std'].apply(math.cos)\n  df['sy'] = df['s']*df['dir_std'].apply(math.sin)\n  return df  \n\ndef get_play_desc(gid, pid):\n  return plays.loc[(plays['gameId']==gid) & (plays['playId']==pid), 'playDescription'].values[0]\n\ndef get_xret(one_play, fid):  \n  received = 1\n  is_going_left = one_play['is_going_left'].unique()[0]\n  received_at = one_play.loc[(one_play['event'].str.contains('receive')) & (one_play['is_football']==1), 'x'].values[0]\n  received_frame = one_play.loc[(one_play['event'].str.contains('receive')) & (one_play['is_football']==1), 'frameId'].values[0]\n  received_est_yards = one_play.loc[(one_play['frameId']==received_frame), 'kickReturnYardage_pred'].values[0]\n  if received_frame >= fid:\n    received = 0\n  if is_going_left:\n    est_yards = received_at + one_play.loc[one_play['frameId']==fid, 'kickReturnYardage_pred'].values[0]\n    actual_yards = received_at + one_play.loc[one_play['frameId']==fid, 'kickReturnYardage'].values[0]    \n    received_est_yards = received_at + received_est_yards\n  else:\n    est_yards = received_at - one_play.loc[one_play['frameId']==fid, 'kickReturnYardage_pred'].values[0]\n    actual_yards = received_at - one_play.loc[one_play['frameId']==fid, 'kickReturnYardage'].values[0]\n    received_est_yards = received_at - received_est_yards\n\n  return est_yards, actual_yards, received, received_est_yards\n\ndef get_play_by_frame(fid, ax, los, one_play):\n  ax.cla()\n  gid = one_play['gameId'].unique()[0]\n  pid = one_play['playId'].unique()[0]\n  play_desc = get_play_desc(gid, pid)\n\n  one_frame = one_play.loc[one_play['frameId']==fid]    \n  # print([fid, est_yards, actual_yards])\n\n  try:\n    fig1 = sns.scatterplot(x='x',y='y',data=one_frame, hue='team', ax=ax, size='target_scaled', sizes=(60,200))\n    est_yards, actual_yards, received, received_est_yards = get_xret(one_play, fid)\n    fig1.axvline(received_est_yards, c='b', ls=':', alpha=received)\n    fig1.axvline(est_yards, c='r', ls=':')\n    fig1.axvline(actual_yards, c='r', ls='-')\n  except:\n    fig1 = sns.scatterplot(x='x',y='y',data=one_frame, hue='team', ax=ax, s=100)\n  \n  \n  fig1.axvline(los, c='k', ls=':')\n  fig1.axvline(los, c='k', ls=':')\n  fig1.axvline(0, c='k', ls='-')\n  fig1.axvline(100, c='k', ls='-')\n  fig1.set_title(f\"{play_desc}\\ngame {gid} play {pid}\")\n  fig1.legend([]).set_visible(False)\n  sns.despine(left=True)\n  fig1.set_ylabel('')\n\n  fig1.set_yticks([])\n  fig1.set_xlim(-10,110)    \n  fig1.set_ylim(0,54) \n\ndef animate_predicted_play(one_play, suffix='_pred', use_pred_values=False, \n                           pred_value=None):    \n  gid = one_play['gameId'].unique()[0]\n  pid = one_play['playId'].unique()[0]\n\n  scaled = pd.Series()\n  if use_pred_values:\n    if pred_value is None:\n      raise ValueError('no pred value provided bruv')\n      return None\n    for idx, row in one_play.loc[:, ['team','frameId']].drop_duplicates().iterrows():\n      fid = row['frameId']\n      tid = row['team']\n      _one_play = one_play.loc[(one_play['team']==tid)&(one_play['frameId']==fid), pred_value]\n      scaled = scaled.append(pd.Series(minmax_scale(_one_play, feature_range=(0,1)), index=_one_play.index))\n      \n    one_play['target_scaled'] = scaled.fillna(scaled.quantile(.3))\n    one_play.loc[one_play['team']=='football', 'target_scaled'] = scaled.quantile(.15) \n\n  los = one_play.loc[(one_play['frameId']==1) & (one_play['team']=='football'), 'x'].values[0]\n\n  fig = plt.figure(figsize=(14.4, 6.4))\n  ax = fig.gca()\n  ani = animation.FuncAnimation(fig, get_play_by_frame, \n                                frames=one_play['frameId'].sort_values().unique().shape[0],\n                                interval=100, repeat=True, fargs=(ax,los,one_play,))\n\n  plt.close()\n  ani.save(f'{gid}_{pid}.gif', writer='imagemagick', fps=10)\n  return ani      ","metadata":{"id":"RepL_MK-5p4C","execution":{"iopub.status.busy":"2022-01-06T14:18:35.400147Z","iopub.status.idle":"2022-01-06T14:18:35.400845Z","shell.execute_reply.started":"2022-01-06T14:18:35.40055Z","shell.execute_reply":"2022-01-06T14:18:35.400578Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# data load and preprocessing ","metadata":{"id":"f5_aHw4Q8V_M","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"markdown","source":"load data in, including `nflfastR` 2018-2020 data ","metadata":{"id":"GuOLJah58qJy","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"project_dir = 'drive/My Drive/2021bdb'\nfastr_dir = 'drive/My Drive/nflfastR-data/data'\n\nfns = [x for x in os.listdir(fastr_dir) if ('csv.gz' in x)]\nfns = [f\"{fastr_dir}/{x}\" for x in fns if x[-11:-7] in ['2018', '2019', '2020']]\nfastr_data = pd.DataFrame()\nfor fn in fns:\n  _fastr_data = pd.read_csv(fn)\n  fastr_data = fastr_data.append(_fastr_data)\nfastr_data = fastr_data.reset_index(drop=True)\n\nplays = pd.read_csv(f\"{project_dir}/data/plays.csv.zip\", compression='zip')\nplays['specialTeamsPlayType_code'] = plays['specialTeamsPlayType'].astype('category').cat.codes\n\n_df = fastr_data.loc[:, ['old_game_id', 'play_id', 'ep', 'epa']].rename(columns={'old_game_id':'gameId', 'play_id':'playId'})\nplays = plays.merge(_df)\n\nplayers = pd.read_csv(f\"{project_dir}/data/players.csv\")\nplayers = clean_players_data(players)\npff = pd.read_csv(f\"{project_dir}/data/PFFScoutingData.csv.zip\", compression='zip')","metadata":{"id":"NTwI8g8E8kOv","execution":{"iopub.status.busy":"2022-01-06T14:18:35.402609Z","iopub.status.idle":"2022-01-06T14:18:35.403128Z","shell.execute_reply.started":"2022-01-06T14:18:35.402857Z","shell.execute_reply":"2022-01-06T14:18:35.402886Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"preprocess raw data (~3 hrs) ","metadata":{"id":"18G0XLDT8_x5","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"tracking_fns = [x for x in os.listdir(f\"{project_dir}/data\") if 'tracking' in x]\nfor season in tqdm(range(2018,2021), desc='preprocessing'):\n  df = pd.read_csv(f\"{project_dir}/data/tracking{season}.csv.zip\", compression='zip')\n  df = clean_raw_data(df)  \n  \n  df_tackles = generate_tackles_df(df, plays, pff)\n  df_tackles.to_csv(f'{project_dir}/processed/{season}_df_tackles.csv', index=False)      \n\n  df_agg = generate_agg_bullshit(df, players, plays)\n  df_agg.to_csv(f'{project_dir}/processed/{season}_df_agg.csv', index=False)      \n\n  df_dist2fb = generate_distance2football(df)\n  df_dist2fb.to_csv(f'{project_dir}/processed/{season}_kicking_team_dist2fb.csv', index=False)\n  \n  nearest_blocker = generate_nearest_blocker(df)\n  nearest_blocker.to_csv(f'{project_dir}/processed/{season}_nearest_blocker.csv', index=False)\n  ","metadata":{"id":"5A1CBweZ8pru","execution":{"iopub.status.busy":"2022-01-06T14:18:35.404581Z","iopub.status.idle":"2022-01-06T14:18:35.405297Z","shell.execute_reply.started":"2022-01-06T14:18:35.404965Z","shell.execute_reply":"2022-01-06T14:18:35.405022Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"merge preprocessed data into one dataset (~30 mins) ","metadata":{"id":"ZRMDRbbO9eE6","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"for season in tqdm(range(2018,2021), desc='merging and saving'):\n  df = pd.read_csv(f\"{project_dir}/data/tracking{season}.csv.zip\", compression='zip')\n  df = clean_raw_data(df)  \n  df_tackles = pd.read_csv(f'{project_dir}/processed/{season}_df_tackles.csv')      \n  df_agg = pd.read_csv(f'{project_dir}/processed/{season}_df_agg.csv')      \n  df_dist2fb = pd.read_csv(f'{project_dir}/processed/{season}_kicking_team_dist2fb.csv')\n  nearest_blocker = pd.read_csv(f'{project_dir}/processed/{season}_nearest_blocker.csv')\n  df_with_agg = (df\n  .merge(df_dist2fb, how='left')\n  .merge(df_agg\n          .loc[:, ['nflId', 'Position', \n                  'inches', 'weight', 'snapBurstS', \n                  'snapBurstA', 'preSnapDis']]\n          .drop_duplicates(), how='left')\n  .merge(nearest_blocker.drop_duplicates(), how='left')\n  .merge(df_tackles, how='left')\n  .merge(plays.loc[:, ['gameId', 'playId', 'specialTeamsPlayType_code']]))\n  _df_min = df_with_agg.loc[:, ['gameId', 'playId', 'frameId', 'distanceToFootball']].groupby(['gameId', 'playId', 'frameId'], as_index=False).min()\n  df_with_agg = df_with_agg.merge(_df_min.rename(columns={'distanceToFootball':'closestDefenderDistance'}))\n  df_with_agg['distanceToLikelyTackler'] = df_with_agg['distanceToFootball'].sub(df_with_agg['closestDefenderDistance'])\n  df_with_agg['is_going_left'] = df_with_agg['playDirection'].replace(['left', 'right'], [1,0])\n  df_with_agg['makes_tackle'] = df_with_agg['makes_tackle'].fillna(0)\n  df_with_agg.to_csv(f'{project_dir}/processed/{season}_processed_data.csv', index=False) ","metadata":{"id":"SFZZcn1A9Jeu","execution":{"iopub.status.busy":"2022-01-06T14:18:35.407119Z","iopub.status.idle":"2022-01-06T14:18:35.407653Z","shell.execute_reply.started":"2022-01-06T14:18:35.40739Z","shell.execute_reply":"2022-01-06T14:18:35.407417Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Tackle probability model","metadata":{"id":"-zD3eE0UJs6V","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"markdown","source":"parse data into tackle probability model data ","metadata":{"id":"b9cwQWcl9r48","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"id_cols = ['gameId', 'playId','nflId', 'displayName','position']\nmodel_feats = ['frameId', 'specialTeamsPlayType_code', 'is_going_left', 'x', 'y', 's', 'a', 'dis', 'o', \n               'dir', 'distanceToFootball', 'inches', 'weight', 'snapBurstS', \n               'snapBurstA', 'preSnapDis', 'closestDefenderDistance',\n               'distanceToLikelyTackler', 'nearestBlockerDistance']\ntarget = 'makes_tackle'\ntackle_model_data = pd.DataFrame()\nfor season in tqdm(range(2018,2021), desc='make tackle model data'):\n  df_with_agg = pd.read_csv(f'{project_dir}/processed/{season}_processed_data.csv') \n  df_with_agg['frame_of_tackle'] = df_with_agg['makes_tackle']\n\n  tackle_model_data = tackle_model_data.append(df_with_agg.loc[(df_with_agg['is_kicking_team']==1), id_cols+model_feats+[target]].drop_duplicates().dropna(subset=model_feats+[target]))\n\ntackle_model_data.to_csv(f'{project_dir}/processed/tackle_model_data.csv', index=False)","metadata":{"id":"Ge4ZhliE9j2w","execution":{"iopub.status.busy":"2022-01-06T14:18:35.409616Z","iopub.status.idle":"2022-01-06T14:18:35.410245Z","shell.execute_reply.started":"2022-01-06T14:18:35.409847Z","shell.execute_reply":"2022-01-06T14:18:35.409874Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"load tackle prob model data","metadata":{"id":"ak5IEdEAJ6Cj","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"model_data = pd.read_csv(f'{project_dir}/processed/tackle_model_data.csv') \n_df1 = model_data.loc[:, ['gameId', 'playId', 'nflId', 'frameId']].reset_index()\n_df2 = model_data.loc[model_data['makes_tackle']==1, ['gameId', 'playId', 'nflId']]\n_df = _df1.merge(_df2)\nidx = _df['index'].unique()\n\n_s = pd.Series(index=idx).fillna(1)\nmodel_data['frame_of_tackle'] = model_data['makes_tackle']\nmodel_data['makes_tackle'] = _s\n\n_tacklers_only = model_data.loc[(model_data['makes_tackle']==1), ['gameId', 'playId', 'nflId']].drop_duplicates()\nmodel_data['gameIdPlayId'] = model_data['gameId'].astype(str).add(model_data['playId'].astype(str))\nmodel_data['gameIdPlayId'] = model_data['gameIdPlayId'].astype(int)\ntacklers_only = _tacklers_only.merge(model_data)\nid_cols = ['gameId', 'playId','nflId', 'displayName','position']\nmodel_feats = ['frameId', 'specialTeamsPlayType_code', 'is_going_left', 'x', 'y', 's', 'a', 'dis', \n               'o', 'dir', 'distanceToFootball', 'inches', 'weight', \n               'snapBurstS', 'snapBurstA', 'preSnapDis', \n               'closestDefenderDistance', 'distanceToLikelyTackler', \n               'nearestBlockerDistance']\ntarget = 'makes_tackle'","metadata":{"id":"k3VK1j2pJmC8","execution":{"iopub.status.busy":"2022-01-06T14:18:35.411846Z","iopub.status.idle":"2022-01-06T14:18:35.412365Z","shell.execute_reply.started":"2022-01-06T14:18:35.412057Z","shell.execute_reply":"2022-01-06T14:18:35.412133Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"produce tackle probabilities for all kicking team players","metadata":{"id":"DP0TdilIPccs","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"df_pred = pd.DataFrame()\nfolds = 5\nkf = GroupKFold(folds)\n\n# learned from hyperparam tuning with optuna \np = {'bagging_fraction': 0.9911851261849397,\n 'bagging_freq': 3,\n 'device': 'gpu',\n 'feature_fraction': 0.948,\n 'feature_pre_filter': False,\n 'gpu_device_id': 0,\n 'gpu_platform_id': 0,\n 'lambda_l1': 1.0919192786057997e-05,\n 'lambda_l2': 0.23713111023668204,\n 'learning_rate': 0.1,\n 'max_bin': 63,\n 'min_child_samples': 20,\n 'num_leaves': 3,\n 'num_thread': 28,\n 'objective': 'binary',\n 'verbosity': -1}\n\n for train_idx, test_idx in tqdm(kf.split(model_data, groups=model_data['gameIdPlayId']), total=folds):\n  train_data = model_data.iloc[train_idx]\n  test_data = model_data.iloc[test_idx]\n\n  lgb_train = lgb_vanilla.Dataset(train_data.loc[:, model_feats], train_data[target])\n  lgb_test = lgb_vanilla.Dataset(test_data.loc[:, model_feats], test_data[target])\n  _model = lgb_vanilla.train(p,lgb_train,num_boost_round=4000, valid_sets=lgb_test, early_stopping_rounds=200, verbose_eval=0)\n  test_data[f'{target}_pred'] = pd.Series(_model.predict(test_data.loc[:, model_feats]), index=test_data.index)\n  df_pred = df_pred.append(test_data)\n\ndf_pred.to_csv(f'{project_dir}/processed/tackle_pred.csv', index=False) \ndf_pred = pd.read_csv(f'{project_dir}/processed/tackle_pred.csv') \ndf_pred['season'] = df_pred['gameId'].astype(str).str[:4].astype(int)\nfor season in range(2018,2021):\n  df_pred.loc[df_pred['season']==season, ['gameId', 'playId', 'frameId', 'nflId', 'makes_tackle_pred']].to_csv(f'{project_dir}/processed/{season}_tackle_pred.csv', index=False) ","metadata":{"id":"PwlJQ3vXMM6o","execution":{"iopub.status.busy":"2022-01-06T14:18:35.414158Z","iopub.status.idle":"2022-01-06T14:18:35.414747Z","shell.execute_reply.started":"2022-01-06T14:18:35.41449Z","shell.execute_reply":"2022-01-06T14:18:35.414517Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# xreturn model ","metadata":{"id":"Afv5xP-ncy59","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"markdown","source":"preprocess the xreturn model data ","metadata":{"id":"Ao09q1jueTLI","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"id_cols = ['gameId', 'playId','nflId', 'displayName','position']\nreturner_feats = ['frameId', 'x_std', 'y_std', 'sx', 'sy', 'touchback_possible', 'specialTeamsPlayType_code']\ntackler_feats = ['frameId', 'x_std', 'y_std', 'sx', 'sy', \n                 'nearestBlockerDistance', 'makes_tackle_pred']\ntarget = 'kickReturnYardage'\n\nfor season in tqdm(range(2018,2021)):\n  df_with_agg = pd.read_csv(f'{project_dir}/processed/{season}_processed_data.csv') \n  tackle_pred = pd.read_csv(f'{project_dir}/processed/{season}_tackle_pred.csv') \n  df_with_agg = standardization(df_with_agg)\n  _returner = plays.loc[plays['specialTeamsResult']=='Return', ['gameId', 'playId', 'returnerId', 'kickReturnYardage']].rename(columns={'returnerId':'nflId'})\n  x = _returner['nflId']\n  y = x[x.str.contains(';', na=False)].str.split(';')\n  x = x[~x.str.contains(';', na=False)].str.split(';')\n  x = x.append(y)\n  _returner['nflId'] = x\n  _returner = _returner.explode('nflId')\n  _returner['nflId'] = _returner['nflId'].astype(float)\n  _returner['is_returner'] = 1\n  df_with_agg = df_with_agg.merge(_returner, how='left')\n  df_with_agg['is_returner'] = df_with_agg['is_returner'].fillna(0)\n  df_with_agg = df_with_agg.merge(tackle_pred, how='left')\n  df_with_agg['uid'] = df_with_agg['gameId'].astype(str).add(df_with_agg['playId'].astype(str))\n\n\n\n  _tackler = df_with_agg.loc[df_with_agg['distanceToLikelyTackler']==0, id_cols+tackler_feats]\n  _tackler['makes_tackle_pred'] = _tackler['makes_tackle_pred'].fillna(_tackler['makes_tackle_pred'].median())\n  _returner = df_with_agg.loc[df_with_agg['is_returner']==1, id_cols+returner_feats+[target]]\n  drop_plays = _returner.loc[_returner['sx'].isnull(), ['gameId', 'playId']].drop_duplicates()\n  drop_plays['uid'] = drop_plays['gameId'].astype(str).add(drop_plays['playId'].astype(str))\n  df_with_agg = df_with_agg.loc[~df_with_agg['uid'].isin(drop_plays['uid'].unique())]\n  _returner = _returner.dropna()\n  rename_cols = {'x_std':'kick_x_std', 'y_std':'kick_y_std', 'sx':'kick_sx', 'sy':'kick_sy'}\n  _tackler = _tackler.loc[:, ['gameId', 'playId', 'frameId', 'x_std', 'y_std', 'sx', 'sy', 'nearestBlockerDistance', 'makes_tackle_pred']].rename(columns=rename_cols)\n  xret_model_data = _returner.merge(_tackler)\n  xret_model_data['x_diff'] = xret_model_data['x_std'].sub(xret_model_data['kick_x_std'])\n  xret_model_data['y_diff'] = xret_model_data['y_std'].sub(xret_model_data['kick_y_std'])\n  xret_model_data['sx_diff'] = xret_model_data['sx'].sub(xret_model_data['kick_sx'])\n  xret_model_data['sy_diff'] = xret_model_data['sy'].sub(xret_model_data['kick_sy'])\n  xret_model_data.to_csv(f'{project_dir}/processed/{season}_xret_model_data.csv', index=False)","metadata":{"id":"pvrAi71RJgo-","execution":{"iopub.status.busy":"2022-01-06T14:18:35.41855Z","iopub.status.idle":"2022-01-06T14:18:35.419182Z","shell.execute_reply.started":"2022-01-06T14:18:35.418897Z","shell.execute_reply":"2022-01-06T14:18:35.418928Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"make data model-ready ","metadata":{"id":"rQgDAAnieWWf","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"model_data = pd.DataFrame()\nfor season in range(2018,2021):\n  _model_data = pd.read_csv(f'{project_dir}/processed/{season}_xret_model_data.csv')\n  model_data = model_data.append(_model_data)\nmodel_data = model_data.reset_index(drop=True)  \nmodel_data['gameIdPlayId'] = model_data['gameId'].astype(str).add(model_data['playId'].astype(str))\nmodel_data['gameIdPlayId'] = model_data['gameIdPlayId'].astype(int)\nid_cols = ['gameId', 'playId','nflId', 'displayName','position']\nmodel_feats = ['frameId', 'specialTeamsPlayType_code', 'x_std', 'y_std', 'sx', 'sy', 'touchback_possible',\n               'kick_x_std', 'kick_y_std', 'kick_sx', 'kick_sy','x_diff',\n               'y_diff', 'sx_diff', 'sy_diff', 'nearestBlockerDistance', \n               'makes_tackle_pred']\ntarget = 'kickReturnYardage'","metadata":{"id":"cXgsQpB8eV2b","execution":{"iopub.status.busy":"2022-01-06T14:18:35.421019Z","iopub.status.idle":"2022-01-06T14:18:35.421554Z","shell.execute_reply.started":"2022-01-06T14:18:35.421236Z","shell.execute_reply":"2022-01-06T14:18:35.421271Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_pred = pd.DataFrame()\nfolds = 5\nkf = GroupKFold(folds)\n\n# params from optuna\np ={'bagging_fraction': 1.0,\n 'bagging_freq': 0,\n 'device': 'gpu',\n 'feature_fraction': 1.0,\n 'feature_pre_filter': False,\n 'gpu_device_id': 0,\n 'gpu_platform_id': 0,\n 'lambda_l1': 0.0,\n 'lambda_l2': 0.0,\n 'learning_rate': 0.1,\n 'max_bin': 63,\n 'metric': 'rmse',\n 'min_child_samples': 20,\n 'num_leaves': 251,\n 'num_thread': 28,\n 'objective': 'regression'}\n\nfor train_idx, test_idx in tqdm(kf.split(model_data, groups=model_data['gameIdPlayId']), total=folds):\n  train_data = model_data.iloc[train_idx]\n  test_data = model_data.iloc[test_idx]\n\n  lgb_train = lgb_vanilla.Dataset(train_data.loc[:, model_feats], train_data[target])\n  lgb_test = lgb_vanilla.Dataset(test_data.loc[:, model_feats], test_data[target])\n  _model = lgb_vanilla.train(p,lgb_train,num_boost_round=4000, valid_sets=lgb_test, early_stopping_rounds=200, verbose_eval=0)\n  test_data[f'{target}_pred'] = pd.Series(_model.predict(test_data.loc[:, model_feats]), index=test_data.index)\n  df_pred = df_pred.append(test_data)\n\ndf_pred.to_csv(f'{project_dir}/processed/xret.csv', index=False) \ndf_pred['season'] = df_pred['gameId'].astype(str).str[:4].astype(int)\nfor season in range(2018,2021):\n  df_pred.loc[df_pred['season']==season, ['gameId', 'playId', 'frameId', 'nflId', 'makes_tackle_pred']].to_csv(f'{project_dir}/processed/{season}_tackle_pred.csv', index=False) ","metadata":{"id":"Q44worsneaqx","execution":{"iopub.status.busy":"2022-01-06T14:18:35.423392Z","iopub.status.idle":"2022-01-06T14:18:35.423932Z","shell.execute_reply.started":"2022-01-06T14:18:35.423656Z","shell.execute_reply":"2022-01-06T14:18:35.423683Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# munge model outputs together ","metadata":{"id":"5ZDCV9oke1nl","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"markdown","source":"take outputs and munge into one dataframe ","metadata":{"id":"aFezgvQXelui","_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"_merge = pd.DataFrame()\nfor season in [2018,2019,2020]:\n  df_with_agg = pd.read_csv(f'{project_dir}/processed/{season}_processed_data.csv') \n  _df = df_with_agg.loc[:, ['gameId', 'playId', 'frameId', 'distanceToFootball', 'makes_tackle']].sort_values('distanceToFootball').drop_duplicates(subset=['gameId', 'playId', 'frameId'])\n  _df['season'] = season\n  _merge = _merge.append(_df)\n\n\nreceived_frames = pd.DataFrame()\nfor season in [2018,2019,2020]:\n  df = pd.read_csv(f\"{project_dir}/data/tracking{season}.csv.zip\", compression='zip')\n  df = clean_raw_data(df)  \n  df = standardization(df)\n  _received_frames = df.loc[df['event'].str.contains('receive'), ['gameId', 'playId', 'frameId']]\n  _received_frames = _received_frames.drop_duplicates()\n  _received_frames['receives_ball'] = 1\n  received_frames = received_frames.append(_received_frames)\n\nxret = pd.read_csv(f'{project_dir}/processed/xret.csv')\nx = pff['missedTackler']\ny = x[x.str.contains(';', na=False)].str.split(';')\nx = x[~x.str.contains(';', na=False)].str.split(';')\nz = x.append(y)\npff['tacklesAvoided'] = z.fillna(\"\").apply(list).apply(len)\nxret = xret.merge(pff.loc[:, ['gameId', 'playId', 'tacklesAvoided']].drop_duplicates())\nxret = xret.merge(_merge, on=['gameId', 'playId', 'frameId'], how='left')\nxret = xret.merge(received_frames, how='left')\nxret['receives_ball'] = xret['receives_ball'].fillna(0)\nxret = xret.merge(plays.loc[:, ['gameId', 'playId','specialTeamsPlayType']])\nxret['kickReturnYardage_oe'] = xret['kickReturnYardage'].sub(xret['kickReturnYardage_pred'])\nxret_oe = xret.loc[xret['receives_ball']==1, ['season','nflId', 'kickReturnYardage', 'tacklesAvoided', 'kickReturnYardage_oe']].groupby(['season','nflId'], as_index=False).sum()\n_xret_oe = xret.loc[xret['receives_ball']==1, ['season','nflId', 'kickReturnYardage_oe']].groupby(['season','nflId'], as_index=False).count().rename(columns={'kickReturnYardage_oe':'n_returns'})\nxret_oe = players.loc[:, ['nflId', 'displayName']].drop_duplicates(subset=['nflId'], keep='last').merge(xret_oe).merge(_xret_oe)\nxret_oe = xret_oe.sort_values('kickReturnYardage_oe', ascending=False).reset_index(drop=True)\n\n_ep = plays.loc[:, ['returnerId', 'ep', 'epa']].rename(columns={'returnerId':'nflId'}).dropna()\nx = _ep['nflId']\ny = x[x.str.contains(';', na=False)].str.split(';')\nx = x[~x.str.contains(';', na=False)].str.split(';')\nx = x.append(y)\n_ep['nflId'] = x\n_ep = _ep.explode('nflId')\n_ep['nflId'] = _ep['nflId'].astype(int)\n_ep = _ep.groupby('nflId', as_index=False).sum()\n\nxret_oe = xret_oe.merge(_ep)\n\ntarget = 'makes_tackle_pred'\n_df1 = xret.loc[xret['receives_ball']==1, ['gameId', 'playId', 'season', 'nflId', 'displayName', 'kickReturnYardage', 'tacklesAvoided', 'kickReturnYardage_oe']]\n_df2 = xret.loc[(xret['makes_tackle']!=1)&(xret['distanceToFootball']<=4), ['gameId', 'playId','season','nflId', target]].groupby(['gameId', 'playId','season','nflId'], as_index=False).sum()\nwtf = _df1.merge(_df2)\nfor col in ['kickReturnYardage_oe', 'makes_tackle_pred']:\n  wtf[f\"{col}_scaled\"] = scale(wtf[col])\n\nWINNING_METRIC_NAME = \"returner quality\" \nwtf[WINNING_METRIC_NAME] = wtf.loc[:, ['kickReturnYardage_oe_scaled', 'makes_tackle_pred_scaled']].sum(axis=1)  \n\ncols = ['player', 'n_returns', 'kickReturnYardage',\n       'tacklesAvoided', 'kickReturnYardage_oe', 'makes_tackle_pred',\n       WINNING_METRIC_NAME]\n\n_df = wtf.loc[:, ['season','nflId', 'displayName', WINNING_METRIC_NAME]].groupby(['season','nflId', 'displayName'], as_index=False).count().rename(columns={WINNING_METRIC_NAME:'n_returns'})\ndf_agg = wtf.loc[:, ['season','nflId', 'displayName', 'kickReturnYardage', 'tacklesAvoided', 'kickReturnYardage_oe', 'makes_tackle_pred', WINNING_METRIC_NAME]].groupby(['season','nflId', 'displayName'], as_index=False).mean()\ndf_agg = df_agg.merge(_df)\ndf_agg['player'] = df_agg['season'].astype(str).add(' ').add(df_agg['displayName'])\ndf_agg = df_agg.loc[:, cols]","metadata":{"id":"pKmSqlk6e_I3","execution":{"iopub.status.busy":"2022-01-06T14:18:35.42629Z","iopub.status.idle":"2022-01-06T14:18:35.426763Z","shell.execute_reply.started":"2022-01-06T14:18:35.426508Z","shell.execute_reply":"2022-01-06T14:18:35.426532Z"},"_kg_hide-input":false,"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]}]}