{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Playing Surface Analysis\n![](https://i.imgur.com/ZsERLAT.png)\n## Lateral movement and increased injury risk.\n\nThis report contains my analysis for the 2019 NFL 1st and Future competition. The competition tasked data scientists to investigate the relationship between the playing surface and the injury and performance of National Football League (NFL) athletes and to examine factors that may contribute to lower extremity injuries.\n\nI propose a metric for measuring the angle between athlete's body orientation and movement direction. I categorize player movements into three groups: forward, lateral, and backpedaling. I find that increased lateral player movement is strongly correlated with plays involving non-contact (NC) lower limb injuries. I then look to see if we can find a measurable link between increased lateral movement and playing surface.\nI then propose a hypothesis that if injury rate is linked to playing surface, then we may see a link between playing surface and lateral movement. My findings show that there is no link between lateral movement and playing surface.\n\nLast, I offer three suggestions that I believe may help to identify players at high risk of injuries: 1) Monitor the percentage of lateral movement of players during game play and provide summary statistics on players, and play types. This could be integrated into already existing monitoring protocols.  2) Monitor  the orientation of players’ hips, shoulders and head. I also suggest collecting data that might elucidate the link between playing surface and injuries: player cleat data and dampness of playing surface could be a place to start. 3) Allow strength and conditioning coaches to play a role in injury prevention by educating them about the link between lateral movement and injuries for Linebackers, Wide Receivers and Defensive backs.  They may be able to use  data that I’ve suggested  they develop individualized workout plans."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pylab as plt\nimport os\nimport psutil\n\nimport lightgbm as lgb\nfrom sklearn.model_selection import train_test_split\nimport random\nimport sklearn\nfrom itertools import cycle, islice\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\nfrom tqdm.notebook import tqdm\npd.set_option('max_columns', 500)\nplt.style.use('fivethirtyeight')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Read in data\ntracks = pd.read_csv('../input/nfl-playing-surface-analytics/PlayerTrackData.csv',\n                        dtype={'time':'float64',\n                                'x':'float16',\n                                'y':'float16',\n                                'dir': 'float16',\n                                'dis': 'float16',\n                                'o':'float16',\n                                's':'float16'})\n\nplays = pd.read_csv('../input/nfl-playing-surface-analytics/PlayList.csv')\ninjury = pd.read_csv('../input/nfl-playing-surface-analytics/InjuryRecord.csv')\n\n# Create injury detailed by merging on play information\ninjury_detailed = injury.merge(plays, how='left')\ninjury_detailed = injury_detailed.merge(plays[['PlayerKey','RosterPosition']].drop_duplicates() \\\n                                            .rename(columns={'RosterPosition':'RosterPosition_notplay'}))\ninjury_detailed['RosterPosition_notplay'] = injury_detailed['RosterPosition_notplay'] \\\n    .replace({'Safety':'Defensive Back',\n              'Cornerback' : 'Defensive Back'})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Background\nThere has long been professional and academic interest in the differences regarding athletic performance measures and injury occurrence on various surface types (Meyers and Barnhill, 2004; Powell and Schootman, 1992). Recent studies have specifically examined synthetic turf (Mack et al., 2018; Loughran et al., 2019). These observational studies only show a correlation in injury rate-- they don’t make assertions about the specific mechanisms associated with lower body injury. My analysis attempts to find a specific link between a movement pattern and NC injury. Once I’ve identified a movement pattern link, can I find a measurable difference in that  movement pattern between playing surfaces (synthetic turf vs. natural turf)?\n\nMy analysis is based on data that the NFL has provided with the full player tracking of on-field position for 250 players over two regular season schedules. Of the 250 players provided, one hundred of them sustained one or more injuries during the study period. The other 150 serve as a control group (representing a sample of the non-injured NFL population).\nAt first look, it’s provocative to note that, within the data set provided, there are numerically more NC injuries on synthetic turf. I hope to go beyond this purely observational approach, though, by also finding mechanisms associated with these injuries."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.style.use('fivethirtyeight')\n# Find Injury Rate by Surface\ninjury_playkeys = injury['PlayKey'].unique()\nplays['counter'] = 1 # Column used when grouping to count\nplays['isInjuryPlay'] = False\nplays.loc[plays['PlayKey'].isin(injury_playkeys), 'isInjuryPlay'] = True\n\n# Plot Results\nfig, ax = plt.subplots(1, 1, figsize=(8, 6))\n(plays.groupby('FieldType')[['isInjuryPlay']].mean() * 100000).plot(kind='bar', ax=ax)\nax.get_legend().remove()\nax.set_xlabel('')\nax.set_title('Lower Body Injury Rate by Surface')\nax.set_ylabel('Injury per \\n 100,000 plays', rotation=0, fontsize=13, color='darkgrey')\nax.yaxis.set_label_coords(-0.12,0.85)\nfor p in ax.patches:\n    ax.annotate(f'{int(p.get_height())}',\n                (p.get_x() + 0.20,\n                 p.get_height() - 3),\n                 fontsize=20,\n                color='white')\nax.axhline(0, color='k')\nplt.xticks(rotation=0)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Focusing on the players and plays at high risk\n\nIt’s intuitive, and quickly reveals itself in the data that certain players are at higher risk than others when it comes to NC lower body injuries. Among  the 100+ injuries from  the 2 seasons of game play, 70% (74/105) were sustained by just three positions (Defensive Backs, Linebackers and Wide Receivers). When thinking about why these positions might be at higher risk, I envisioned the movements commonly made by these players. I was struck by the fact that these men are spending a lot of time moving in open space and making quick changes in direction."},{"metadata":{"_kg_hide-output":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#### USED FOR SLIDES\n\n# ax = injury_detailed.groupby('RosterPosition_notplay')['PlayerKey'].count().sort_values() \\\n#     .plot(kind='barh', title='Non-Contact Injuries', figsize=(8, 5))\n# count = 0\n# for x in ax.patches:\n#     if count > 3:\n#         x.set_color('orange')\n#     count += 1\n# # plt.annotate('Three positions account for 2/3 of injuries', xy=(15, 1.5), fontsize=15, color='brown')\n# # plt.arrow(15, 2, -0.3, 1, color='brown', head_width=0.2, head_length=0.2, lw=3)\n# ax.set_title('All Non-Contact Injuries', fontsize=15)\n# ax.grid(b=None, axis='y')\n# ax.set_ylabel('')\n# ax.set_xlabel('Injury Count', fontsize=15)\n# plt.legend(['Excluded from study'])\n# ax.axvline(0.05, color='black')\n\n# rects = ax.patches\n# # For each bar: Place a label\n# for rect in rects:\n#     # Get X and Y placement of label from rect.\n#     x_value = rect.get_width()\n#     y_value = rect.get_y() + rect.get_height() / 2\n\n#     # Number of points between bar and label. Change to your liking.\n#     space = -20\n#     # Vertical alignment for positive values\n#     ha = 'left'\n\n#     # If value of bar is negative: Place label left of bar\n#     if x_value < 0:\n#         # Invert space to place label to the left\n#         space *= -1\n#         # Horizontally align label at right\n#         ha = 'right'\n\n#     # Use X value as label and format number with one decimal place\n#     label = \"{:.0f}\".format(x_value)\n\n#     # Create annotation\n#     ax.annotate(\n#         label,                      # Use `label` as label\n#         (x_value, y_value),         # Place label at end of the bar\n#         xytext=(space, 0),          # Horizontally shift label by `space`\n#         textcoords=\"offset points\", # Interpret `xytext` as offset in points\n#         va='center',                # Vertically center label\n#         ha=ha,\n#         fontsize=14,\n#         color='white')                      # Horizontally align label differently for\n#                                     # positive and negative values.\n\n# plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, axes = plt.subplots(1, 2, figsize=(15, 5))\nax = axes[0]\ninjury_detailed.groupby('RosterPosition_notplay')['PlayerKey'].count().sort_values() \\\n    .plot(kind='barh', title='Non-Contact Injuries', figsize=(10, 5), ax=ax)\ncount = 0\nfor x in ax.patches:\n    if count > 3:\n        x.set_color('orange')\n    count += 1\n# plt.annotate('Three positions account for 2/3 of injuries', xy=(15, 1.5), fontsize=15, color='brown')\n# plt.arrow(15, 2, -0.3, 1, color='brown', head_width=0.2, head_length=0.2, lw=3)\nax.set_title('All Non-Contact Injuries', fontsize=15)\nax.grid(b=None, axis='y')\nax.set_ylabel('')\nax.set_xlabel('Injury Count', fontsize=15)\nax.axvline(0.1, color='black')\nax2 = axes[1]\ninjury_detailed.query('DM_M7 == 1').groupby('RosterPosition_notplay')['PlayerKey'] \\\n    .count() \\\n    .sort_values() \\\n    .plot(kind='barh', figsize=(15, 5), ax=ax2)\ncount = 0\nfor x in ax2.patches:\n    if count > 3:\n        x.set_color('orange')\n    count += 1\nax2.set_title('Non-Contact Injury > 1 week missed', fontsize=15)\nax2.grid(b=None, axis='y')\nax2.set_xlabel('Injury Count', fontsize=15)\nax2.set_ylabel('')\nplt.subplots_adjust(wspace = 0.4)\nfig.suptitle('3 Positions account for 70% of all injuries', fontsize=20)\nplt.subplots_adjust(top=0.83)\n\nrects = ax.patches\n# For each bar: Place a label\nfor rect in rects:\n    # Get X and Y placement of label from rect.\n    x_value = rect.get_width()\n    y_value = rect.get_y() + rect.get_height() / 2\n\n    # Number of points between bar and label. Change to your liking.\n    space = -20\n    # Vertical alignment for positive values\n    ha = 'left'\n\n    # If value of bar is negative: Place label left of bar\n    if x_value < 0:\n        # Invert space to place label to the left\n        space *= -1\n        # Horizontally align label at right\n        ha = 'right'\n\n    # Use X value as label and format number with one decimal place\n    label = \"{:.0f}\".format(x_value)\n\n    # Create annotation\n    ax.annotate(\n        label,                      # Use `label` as label\n        (x_value, y_value),         # Place label at end of the bar\n        xytext=(space, 0),          # Horizontally shift label by `space`\n        textcoords=\"offset points\", # Interpret `xytext` as offset in points\n        va='center',                # Vertically center label\n        ha=ha,\n        fontsize=14,\n        color='white')                      # Horizontally align label differently for\n                                    # positive and negative values.\n        \n        \nrects = ax2.patches\n# For each bar: Place a label\nfor rect in rects:\n    # Get X and Y placement of label from rect.\n    x_value = rect.get_width()\n    y_value = rect.get_y() + rect.get_height() / 2\n\n    # Number of points between bar and label. Change to your liking.\n    space = -20\n    # Vertical alignment for positive values\n    ha = 'left'\n\n    # If value of bar is negative: Place label left of bar\n    if x_value < 0:\n        # Invert space to place label to the left\n        space *= -1\n        # Horizontally align label at right\n        ha = 'center'\n\n    # Use X value as label and format number with one decimal place\n    label = \"{:.0f}\".format(x_value)\n\n    # Create annotation\n    ax2.annotate(\n        label,                      # Use `label` as label\n        (x_value, y_value),         # Place label at end of the bar\n        xytext=(space, 0),          # Horizontally shift label by `space`\n        textcoords=\"offset points\", # Interpret `xytext` as offset in points\n        va='center',                # Vertically center label\n        ha=ha,\n        fontsize=14,\n        color='white')                      # Horizontally align label differently for\n                                    # positive and negative values.\nplt.legend(['Excluded from study'])\nax2.axvline(0.05, color='black')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Special teams plays are inherently different from a normal play. As one would expect, most of the injuries (roughly 2/3) in the data were for non-special team plays."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"injury_detailed['PlayType_simple'] = injury_detailed['PlayType'] \\\n    .replace({'Kickoff Not Returned' : 'Kickoff',\n              'Kickoff Returned' : 'Kickoff',\n              'Punt Not Returned' : 'Punt',\n              'Punt Returned' : 'Punt'})\n\n\nax = injury_detailed.groupby('PlayType_simple') \\\n    .count()['PlayKey'] \\\n    .sort_values().plot(kind='barh',\n                        figsize=(8, 4),\n                       title='Non-Contact Injury count by Play Type')\ncount = 0\n#ax.text(16, 1.5, '*Over 70% occured during non-special teams plays', fontsize=12, color='brown')\nfor x in ax.patches:\n    if count > 1:\n        x.set_color('orange')\n    count += 1\nax.grid(b=None, axis='y')\nax.set_ylabel('')\nplt.xlabel('Injury Count', fontsize=15)\n\nrects = ax.patches\n# For each bar: Place a label\nfor rect in rects:\n    # Get X and Y placement of label from rect.\n    x_value = rect.get_width()\n    y_value = rect.get_y() + rect.get_height() / 2\n\n    # Number of points between bar and label. Change to your liking.\n    space = -20\n    # Vertical alignment for positive values\n    ha = 'center'\n\n    # If value of bar is negative: Place label left of bar\n    if x_value < 0:\n        # Invert space to place label to the left\n        space *= -1\n        # Horizontally align label at right\n        ha = 'right'\n\n    # Use X value as label and format number with one decimal place\n    label = \"{:.0f}\".format(x_value)\n\n    # Create annotation\n    ax.annotate(\n        label,                      # Use `label` as label\n        (x_value, y_value),         # Place label at end of the bar\n        xytext=(space, 0),          # Horizontally shift label by `space`\n        textcoords=\"offset points\", # Interpret `xytext` as offset in points\n        va='center',                # Vertically center label\n        ha=ha,\n        fontsize=15,\n        color='white')                      # Horizontally align label differently for\n                                    # positive and negative values.\nplt.legend(['Excluded from study'])\nax.axvline(0.05, color='black')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With these findings in mind, I decided to focus my analysis on **Defensive Backs, Linebackers, and Wide Receivers during rushing and passing plays**. Narrowing the scope of my analysis allows me to clearly identify player movements that involve high risk of injury for these positions and play types while removing the \"noise\" of plays and positions that have very different movement patterns. "},{"metadata":{},"cell_type":"markdown","source":"# Data Cleaning and Standardizing the Orientation Feature\n\nIt was noted in the data description that the orientation feature may not be completely reliable when considering \"geography\". I have some experience working with the NGS data in the 2019 Big Data Bowl. During that competition, it was found that the orientation was shifted 90 degrees for one of the seasons' data. To correct for this orientation difference, I determined the plays which appeared to be shifted and standardized them as to be consistent with the direction feature. This approach was first shared by John Miller in his [kernel](https://www.kaggle.com/jpmiller/how-to-adjust-orientation).\n\nWhile not perfect, I believe these corrections and standardization techniques are appropriate for the purposes of my analysis. As the NFL continues to collect tracking data, I'm sure the quality of the orientation feature will increase, strengthing the quality of similar analyses.  \n\nNext, I verified that the orientation was reasonable by plotting orientation at the moment of ball snap for all players. As you would expect, most players were facing towards the line of scrimmage at this time.\n\nI also did some cleaning to the NGS data in order to hone in on moments of gameplay. I removed the portion of NGS data not associated with play time (0.1 second prior to the ball snap up until the moment of the final \"event\" in the play). I also capped play length to 25 seconds in order to exclude outliers.\n\nLastly, I computed some features to the NGS data later in my analysis, including the time since the snap and acceleration (difference in speed over 0.1 second). I also added binary indicators to the tracking data for plays involving injured players and plays involving an injury."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Remove any data for a play 0.1 second before snap\n# print(tracks.shape)\ntracks_snap = tracks[['PlayKey','x','y','time','event']].query('event == \"ball_snap\"')\ntracks_snap = tracks_snap[['PlayKey','x','y','time']] \\\n    .rename(columns={'x':'x_snap',\n                     'y':'y_snap',\n                     'time':'time_snap'}).copy()\ntracks = tracks.merge(tracks_snap, on='PlayKey', how='left')\ntracks = tracks.query('time >= (time_snap - 0.1)')\n# print(tracks.shape)\n\n# Remove any data for a play 0.1 second after last event\n# print(tracks.shape)\ntracks_max_event = tracks.loc[~tracks['event'].isna()] \\\n    .groupby('PlayKey')['time'] \\\n    .max().reset_index()\ntracks_max_event = tracks_max_event.rename(columns={'time': 'time_last_event'}).copy()\ntracks = tracks.merge(tracks_max_event)\ntracks['max_event'] = tracks.loc[tracks['time_last_event'] == tracks['time']]['event'].values[0]\ntracks = tracks.query('time <= (time_last_event + 0.1)')\n# print(tracks.shape)\n\n# Fix orientation\n# Reference: https://www.kaggle.com/jpmiller/how-to-adjust-orientation\n# print(tracks.shape)\ntough_guys = plays.loc[plays.PlayerDay >= 350, 'PlayerKey'].unique()\nplaylist_tough = plays[plays.PlayerKey.isin(tough_guys)].copy()\ndays = playlist_tough.groupby('PlayerDay')['PlayerGamePlay'].mean()\n\nplaylist_tough['Season'] = np.where(playlist_tough.PlayerDay<350, 1, 2)\ngames = playlist_tough.drop_duplicates('GameID')[['GameID', 'Season']]\n\ntracks = tracks.merge(playlist_tough[['GameID', 'PlayKey']], on='PlayKey', how='left')\ntracks = tracks.merge(games, on='GameID', how='left')\ntracks['Season'] = tracks['Season'].fillna(-999) # Unknown season as -999\n\n# Assume other seasons based on direction at snap - if orientation at snap is outside normal range, shift.\ns1 = tracks.query('event == \"ball_snap\" and o < 50 and Season < 0')['PlayKey'].unique().tolist()\ns2 = tracks.query('event == \"ball_snap\" and o > 325 and Season < 0')['PlayKey'].unique().tolist()\ns3 = tracks.query('event == \"ball_snap\" and o < 225 and o > 125 and Season < 0')['PlayKey'].unique().tolist()\n\ntracks.loc[(tracks['Season'] < 0) &\n           (tracks['PlayKey'].isin(s1+s2+s3)), 'Season'] = 1\ntracks.loc[(tracks['Season'] < 0) &\n           (~tracks['PlayKey'].isin(s1+s2+s3)), 'Season'] = 2\n\n# Change orientation for season 1\ntracks['o'] = np.where(tracks.Season == 1,\n                            np.mod(tracks.o+90, 360),\n                            tracks.o\n                            )\n# print(tracks.shape)\n\n# Previous speed, acceleration, absolute acceleration\ntracks['s_prev1'] = tracks.groupby('PlayKey')['s'].shift(1)\ntracks['a'] = tracks['s'] - tracks['s_prev1']\ntracks['a_abs'] = np.abs(tracks['a'])\n\n# Add playerkey\ntracks = tracks.merge(plays[['PlayKey','PlayerKey']])\n\n# Binary Features for track data\n# If tracks is for injured player, play where injury occured, \ntracks = tracks.merge(plays[['PlayKey','RosterPosition','PositionGroup','FieldType','PlayType']], how='left')\ntracks.loc[tracks['PositionGroup'].isin(['DB','WR','LB']), 'isInjuryPronePos'] = True\ntracks['isInjuryPlay'] = False\ntracks.loc[tracks['PlayKey'].isin(injury['PlayKey'].unique()), 'isInjuryPlay'] = True\ntracks['isRushPass'] = False\ntracks.loc[tracks['PlayType'].isin(['Rush','Pass']), 'isRushPass'] =  True\ntracks['isInjuredPlayer'] = False\ntracks.loc[tracks['PlayerKey'].isin(injury['PlayerKey'].unique()), 'isInjuredPlayer'] =  True\n\n# Generalized Position groups focus on high injury roles\ntracks['Position_inj'] = tracks['PositionGroup']\ntracks.loc[~tracks['Position_inj'].isin(['LB','WR','DB']), 'Position_inj'] = 'Other'\ntracks['Position_inj'] = tracks['Position_inj'].replace({'LB':'Linebacker',\n                                'WR':'Wide Receiver',\n                                'DB':'Defensive Back',\n                                'Other':'Other Positions'})\n\n# Time since the snap\ntracks['time_since_snap'] = tracks['time']- tracks['time_snap']\ntracks['time_since_snap'] = tracks['time_since_snap'].round(2)\n# print(tracks.shape)\ntracks = tracks.loc[tracks['time_since_snap'] < 25]\n# print(tracks.shape)\ntracks['counter'] = True # Used for aggregating counts","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"tracks.query('event == \"ball_snap\"')['o'] \\\n    .plot(kind='hist',\n          bins=50,\n          figsize=(15, 5),\n          title='Distribution of Orientation during Snap after Data Cleaning')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Computing the Orientation-Movement Angle\n\nDefensive backs and Linebackers have responsibilities to track offensive players- they’re mirroring quick directional movements. As compared to their offensive opponents, they aren’t putting a lot of forethought into their path. Could this be placing statistically more strain on their knees and ankles? Wide receivers make quick movements down field and then make cuts in their routes to create space between themselves and defenders. Could lateral movements during their “cut” moments be an underlying factor in their NC injuries?\n\nIn this video clip, you can see the movements of Derrick Johnson moments before he tears his achilles tendon. He’s almost exclusively moving sideways before he goes down. This gave me an idea of what to focus on in the data.\n\n![](https://media.giphy.com/media/XDErlBCeSQ8TFsRx6n/giphy.gif)"},{"metadata":{},"cell_type":"markdown","source":"To quantify these types of lateral movements, I can calculate the angle between the direction a player is moving and the orientation he is facing.\n\n![](https://i.imgur.com/OmtVL37.png)\n\nI created three specific movement groups based on the calculated angle: **Forward Movement** is when a player’s orientation is generally in line with the direction he’s moving; **Lateral Movement** is when a player is moving from side to side; and **Backpedaling** is when a player is moving in the opposite direction of his orientation.\n\n<table style='font-family:\"Courier\", Courier, monospace; font-size:120%; boarder=10px'>\n  <tr>\n    <th>Orientation Movement Angle</th>\n    <th>Movement Type </th>\n  </tr>\n  <tr>\n    <td>0°-75°</td>\n    <td>Forward Movement</td>\n  </tr>\n  <tr>\n    <td>75°-105°</td>\n    <td>Lateral Movement</td>\n  </tr>\n  <tr>\n    <td>105°-180°</td>\n    <td>Backpedaling</td>\n  </tr>\n</table>\n"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"# O vs Dir feature\ntracks['o_dir_diff1'] = np.abs(tracks['o'] - tracks['dir'])\ntracks['o_dir_diff2'] = np.abs(tracks['o'] - (tracks['dir'] - 360))\ntracks['o_dir_diff3'] = np.abs(tracks['o'] - (tracks['dir'] + 360))\ntracks['o_dir_diff'] = tracks[['o_dir_diff1','o_dir_diff2','o_dir_diff3']].min(axis=1)\ntracks = tracks.drop(['o_dir_diff1','o_dir_diff2','o_dir_diff3'], axis=1)\n\n# Create movement groups\ntracks['OffsetAngleGroup'] = 'Forward'\ntracks.loc[tracks['o_dir_diff'] >= 75, 'OffsetAngleGroup'] = 'Lateral'\ntracks.loc[tracks['o_dir_diff'] >= 105, 'OffsetAngleGroup'] = 'Backpedal'\ntracks['isLateralMovement'] = False\ntracks.loc[tracks['OffsetAngleGroup'] == 'Lateral', 'isLateralMovement'] = True","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Analysis of movement category over duration of play\n\nI found it useful to visualize the percentage of time a given position spends using each of these three movement categories during a play. Wide Receivers in our dataset tend to move forward right after the snap use lateral movement and backpedaling 2 to 3 seconds after the play. Defensive Backs spend much of the beginning of a play backpedaling and then quickly change to lateral or forward movement. Linebackers appear to have a mix of all three types of movements with a slight decrease in forward movement around 2-3 seconds after the snap."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"fig, axes = plt.subplots(1, 3, figsize=(15, 4))\n# Linebacker\nax=axes[0]\nt_group = tracks.query('isRushPass and Position_inj == \"Linebacker\" and time_since_snap < 5 and s > 0') \\\n    .groupby(['time_since_snap','OffsetAngleGroup'])['OffsetAngleGroup'] \\\n    .count() \\\n    .unstack('OffsetAngleGroup')\nt_group.apply(lambda x: 100 * x / float(x.sum()), axis=1) \\\n    .plot(kind='area', stacked=True, alpha=0.5, ax=ax, title='Linebacker')\nfor tick in ax.get_xticklabels():\n    tick.set_rotation(0)\n# Shrink current axis by 20%\nbox = ax.get_position()\nax.set_position([box.x0, box.y0, box.width * 0.8, box.height])\nax.set_xlabel('seconds after snap', fontsize=14)\nax.set_ylabel('% time in \\n movement \\n category', rotation=0, fontsize=10, color='darkgrey')\nax.yaxis.set_label_coords(-0.25,0.82)\n\nax.get_legend().remove()\n\n# Defensive Back\nax=axes[1]\nt_group = tracks.query('isRushPass and Position_inj == \"Defensive Back\" and time_since_snap < 5 and s > 0') \\\n    .groupby(['time_since_snap','OffsetAngleGroup'])['OffsetAngleGroup'] \\\n    .count() \\\n    .unstack('OffsetAngleGroup')\nt_group.apply(lambda x: 100 * x / float(x.sum()), axis=1).plot(kind='area', stacked=True, alpha=0.5, ax=ax, title='Defensive Back')\nfor tick in ax.get_xticklabels():\n    tick.set_rotation(0)\n# Shrink current axis by 20%\nbox = ax.get_position()\nax.set_position([box.x0, box.y0, box.width * 0.8, box.height])\nax.set_xlabel('seconds after snap', fontsize=14)\nax.set_ylabel('% time in \\n movement \\n category', rotation=0, fontsize=10, color='darkgrey')\nax.yaxis.set_label_coords(-0.25,0.82)\nax.get_legend().remove()\n\n# Wide Receiver\nax=axes[2]\nt_group = tracks.query('isRushPass and Position_inj == \"Wide Receiver\" and time_since_snap < 5 and s > 0') \\\n    .groupby(['time_since_snap','OffsetAngleGroup'])['OffsetAngleGroup'] \\\n    .count() \\\n    .unstack('OffsetAngleGroup')\nt_group.apply(lambda x: 100 * x / float(x.sum()), axis=1).plot(kind='area', stacked=True, alpha=0.5, ax=ax, title='Wide Receiver')\nfor tick in ax.get_xticklabels():\n    tick.set_rotation(0)\n# Shrink current axis by 20%\nbox = ax.get_position()\nax.set_position([box.x0, box.y0, box.width * 0.8, box.height])\nax.set_xlabel('seconds after snap', fontsize=14)\nax.set_ylabel('% time in \\n movement \\n category', rotation=0, fontsize=10, color='darkgrey')\nax.yaxis.set_label_coords(-0.25,0.82)\n# # Put a legend to the right of the current axis\nax.legend(loc='center left', bbox_to_anchor=(1, 0.5))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Lateral Movement During Plays is Linked to Injury\nAfter looking at the data, it was clear to me that one factor closely linked to player injury was the amount of time during a play that he spent in lateral movement."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":false},"cell_type":"code","source":"t_group = tracks[['s','isRushPass','isInjuryPronePos',\n                  'PlayKey','OffsetAngleGroup','isInjuryPlay']] \\\n    .loc[tracks['isRushPass'] & tracks['isInjuryPronePos']] \\\n    .groupby(['OffsetAngleGroup','isInjuryPlay'])['PlayKey'] \\\n    .count() \\\n    .unstack(['OffsetAngleGroup'])\n\nfig, ax = plt.subplots(1,1, figsize=(8, 8))\nt_group.apply(lambda x: 100 * x / float(x.sum()), axis=1)['Lateral'] \\\n    .plot(kind='bar',\n          title='Time spent in Lateral Movement',\n          figsize=(10, 4),\n         ax=ax)\nax.xaxis.set_label('')\nax.set_ylabel('% of \\n play time', rotation=0, color='darkgrey', fontsize=12)\nax.yaxis.set_label_coords(-0.08, 0.85)\nax.set_xlabel('')\nfor p in ax.patches:\n    ax.annotate(f'{round(p.get_height(),1)}%', (p.get_x() + 0.2, p.get_height() - 1.5), color='white')\nax.set_xticklabels(['Non Injury Play', 'Injury Play'], rotation=0)\nax.axhline(0, color='black')\nax.axhline(10.1, linestyle='--', linewidth=2, color='orange')\n# fig.annotate('*Linebackers, Defensive Backs, and Cornerbacks, not including special teams', (0, -0.001), fontsize=8)\n\nplt.figtext(0.99, 0.01,\n            '*LB, DB, and WRs not including special teams',\n            fontsize=6,\n            horizontalalignment='right')\nplt.annotate(r\"$\\{$\",fontsize=60,\n            xy=(0.58, 0.7), xycoords='figure fraction'\n            )\nplt.annotate('40% increase', xy=(0.3, 12))\nplt.grid(b=None, axis='x')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we look closer at the lateral movement by position, I found this relationship between lateral movement and injury is found across all three of my focus positions (Wide Receivers, Linebackers and Defensive Backs)."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Numbers used in powerpoint presentation.\n# (tracks.query('isRushPass') \\\n#     .groupby(['Position_inj','isInjuryPlay'])['isLateralMovement'].mean() * 100) \\\n#     .unstack('isInjuryPlay')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injury_prone_pos = ['Wide Receiver', 'Linebacker', 'Defensive Back']\nax = (tracks.query('Position_inj in @injury_prone_pos and isRushPass') \\\n    .groupby(['Position_inj','isInjuryPlay'])['isLateralMovement'].mean() * 100) \\\n    .unstack('isInjuryPlay').plot(kind='barh', figsize=(10, 5),\n                                  title='Time Spent in Lateral Movement')\n\n# set individual bar lables using above list\nfor i in ax.patches:\n    # get_width pulls left or right; get_y pushes up or down\n    ax.text(i.get_width()-0.7, i.get_y()+.08, \\\n            str(int((i.get_width())))+'%', fontsize=10,\n            color='white')\n\nplt.legend(['Non-Injury Play', 'Injury Play'])\nplt.xlabel('% of Play')\nplt.ylabel('')\nplt.grid(b=None, axis='y')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# No evidence to support a link between playing surface and lateral movement\n\nPrevious observational studies have shown a link between injury and playing surface. My analysis for this report identifies an association between NC injuries and lateral movement during game play. Now I wonder: is there a measurable difference between time spent in lateral movement based  on playing surface? My hypothesis would be that, if players gain more traction when interacting with a type of turf, then I will observe a difference in lateral movement (and therefore a difference in NC injuries) on that turf.\n\nOne of the most straightforward ways to test my hypothesis is to examine the distribution of Orientation-Movement angles over plays on Natural and Synthetic playing surfaces. When I compared these distributions, I couldn't visually see any difference between turf type and the time spent in specific Orientation-Movement angles. To test if these distributions are different I calculated the Kolmogorov-Smirnov statistic on 2 samples. Since this statistic is very small (0.0096) then I cannot reject the hypothesis that the distributions of the two samples are the same."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax= plt.subplots(1,1, figsize=(15, 5))\nsns.distplot(tracks.query('FieldType == \"Natural\"')['o_dir_diff'].dropna(),\n             hist=False, label='Natural', color='darkgreen')\nsns.distplot(tracks.query('FieldType == \"Synthetic\"')['o_dir_diff'].dropna(),\n             hist=False, label='Synthetic', color='mediumseagreen')\nax.set_ylabel('% of play time')\nax.set_xlabel('Orientation-Movement Angle')\nax.set_title('Player movement angle by Turf Type')\nax.legend(['Natural Turf', 'Synthetic Turf'])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from scipy.stats import ks_2samp\n\nks_stat = ks_2samp(tracks.query('FieldType == \"Natural\"')['o_dir_diff'].dropna(),\n        tracks.query('FieldType == \"Synthetic\"')['o_dir_diff'].dropna())[0]\nprint(f'The Kolmogorov-Smirnov statistic on 2 samples is {ks_stat:0.4f}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I also compared the differences of time spent in lateral movement by field type. I broke down the percentages by positions in a similar manner to how I compared injury plays to non-injury plays. I found more commonalities than differences. The difference for each position  were all less than 1%."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"injury_prone_pos = ['Wide Receiver', 'Linebacker', 'Defensive Back']\nmy_colors = list(islice(cycle(['darkgreen','mediumseagreen']), None, 3))\nax = (tracks.query('Position_inj in @injury_prone_pos and isRushPass') \\\n    .groupby(['Position_inj','FieldType'])['isLateralMovement'].mean() * 100) \\\n    .unstack('FieldType').plot(kind='barh', figsize=(10, 5),\n                               title='Time Spent in Lateral Movement',\n                               color=my_colors)\n\n# set individual bar lables using above list\nfor i in ax.patches:\n    # get_width pulls left or right; get_y pushes up or down\n    ax.text(i.get_width()-0.8, i.get_y()+.08, \\\n            str(round(i.get_width(), 2))+'%', fontsize=10,\n            color='white')\n\nplt.xlabel('% of Play')\nplt.ylabel('')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I  created a regression model to capture any difference in the percent of lateral movement by player, based on features about the play. The regression included:\n- Position (one-hot-encoded)\n- PlayerDay (integer sequence reflecting timeline of a players participation in games)\n- PlayerGame (Uniquely identifies player’s games)\n- PlayerGamePlay (Ordered interger denoting the running count of plays the player has participated in during the game)\n- Synthetic (binary indicator if field type is Synthetic)\n- isInjuryPlay (binary indicator)\n\nThe regression coefficients from this model give us a view into which features are most closely tied to lateral movement. These coefficents tell us how much the dependent variable (percentage of lateral movement in a play) is impacted by a change in this feature."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Data Prep for linear model\n\nplay_movement_dir = tracks.loc[tracks['Position_inj'].isin(injury_prone_pos) &\n                               tracks['isRushPass']] \\\n    .groupby(['PlayKey','PlayerKey','OffsetAngleGroup','Position_inj','FieldType',\n              'isInjuredPlayer','isInjuryPlay']) \\\n    .count()['counter'] \\\n    .unstack('OffsetAngleGroup') \\\n    .fillna(0)\nplay_mov_pct = play_movement_dir.apply(lambda x: 100 * x / float(x.sum()), axis=1)\nplay_mov_pct = play_mov_pct.reset_index()\n\n# Add play features and make dummy variables for categoricals\nplay_mov_pct_w_feats = plays.merge(play_mov_pct)\nplay_mov_pct_w_feats = pd.concat([play_mov_pct_w_feats, pd.get_dummies(play_mov_pct_w_feats['PlayType'])], axis=1)\nplay_mov_pct_w_feats = pd.concat([play_mov_pct_w_feats, pd.get_dummies(play_mov_pct_w_feats['Position_inj'])], axis=1)\nplay_mov_pct_w_feats = pd.concat([play_mov_pct_w_feats, pd.get_dummies(play_mov_pct_w_feats['FieldType'])], axis=1)\n\n# Replace -999 Temperature values with average temperature\nplay_mov_pct_w_feats.loc[play_mov_pct_w_feats['Temperature'] < -100, 'Temperature'] = np.nan\nplay_mov_pct_w_feats['Temperature'] = play_mov_pct_w_feats['Temperature'].fillna(play_mov_pct_w_feats['Temperature'].mean())","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from sklearn.linear_model import LinearRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_absolute_error\n\nimport statsmodels.api as sm\nfrom scipy import stats\n\n\nFEATURES = ['PlayerDay','PlayerGame','PlayerGamePlay','Rush','Temperature','Defensive Back',\n            'Linebacker', # , DB and WR as 0 is equalt\n            'Wide Receiver',\n            'Synthetic','isInjuryPlay']\nX = play_mov_pct_w_feats[FEATURES]\nX['isInjuryPlay'] = X['isInjuryPlay'].astype(int)\ny = play_mov_pct_w_feats['Lateral']\n\n\n# Create and fit OLS Model\nX2 = sm.add_constant(X)\nest = sm.OLS(y, X2)\nest2 = est.fit()\n\n# Display results of model\ncoeff_df = pd.DataFrame(est2.params, columns=['Regression Coefficient'])\n# coeff_df['p-value'] = est2.get_robustcov_results().pvalues\ndef color_negative_red(val):\n    color = 'red' if val < 0 else 'black'\n    return 'color: %s' % color\ncoeff_df.sort_values('Regression Coefficient', ascending=False) \\\n    .drop('const') \\\n    .round(4) \\\n    .style.applymap(color_negative_red) \\\n    .set_table_attributes('style=\"font-size: 17px\"') \\\n    .set_precision(4) \\\n    .apply(lambda x: ['background: lightgreen' if x.name == 'Synthetic' else '' for i in x], axis=1) \\\n    .apply(lambda x: ['background: lightgreen' if x.name == 'isInjuryPlay' else '' for i in x], axis=1)\n\nplt.style.use('fivethirtyeight')\nax = coeff_df.sort_values('Regression Coefficient') \\\n     .drop('const') \\\n    .plot(kind='barh', figsize=(10, 5), title='Regression coefficents modeling Lateral Movement as % of play')\nax.get_legend().remove()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This model wasn’t created for its predictive power, but rather as  an attempt to quantify the impact of each feature and its statistical significance. The R-squared value is quite low, however we can still gain insights from the model. The main insights are as follows:\n1. The position type (Defensive Back, Linebacker, or Wide Receiver) has the strongest relationship with lateral movement.\n2. Injury Play is also a strong indicator, this supports our analysis above.\n3. Synthetic playing surface  demonstrated a (weakly) negative relationship with amount of lateral movement. Simply put, with all things taken into account synthetic turf decreases the percent of time spent in lateral movement by roughly 0.4%\n\nTo quickly summarize: My analysis shows we are not able to prove that there is a substantial link between playing surface and the amount of lateral movement of an athlete during a play."},{"metadata":{},"cell_type":"markdown","source":"## Conclusion and Recommendations\n\nSo there you have it. I've shown that high lateral movements have a strong relationship with injury plays. I've also shown that there isn't a relationship between playing surface and lateral movement. Because of that we can't conclude that turf type plays a role in increasing this specific type of high risk movement.\n\nGiven my findings I have the following suggestions for the NFL:\n1. Monitor the percentage of lateral movement of players during game play and provide summary statistics on players, and play types. This could be integrated into already existing monitoring protocols.\n2. Increase the quality of the orientation data. If possible include it to monitor the orientation of players’ hips, shoulders and head. I also suggest  collecting data that might elucidate the link between playing surface and injuries: player cleat data and dampness of playing surface could be a place to start.\n3. Allow strength and conditioning coaches to play a role in injury prevention by educating them about the link between lateral movement and injuries for Linebackers, Wide Receivers and Defensive backs. They may be able to use data that I’ve suggested  they develop individualized workout plans.\n\nThanks for taking the time to read my analysis."},{"metadata":{},"cell_type":"markdown","source":"# References\n- Murphy, D F. “Risk Factors for Lower Extremity Injury: a Review of the Literature.” British Journal of Sports Medicine, vol. 37, no. 1, Jan. 2003, pp. 13–29., doi:10.1136/bjsm.37.1.13.\n- Stockman, J.a. “Incidence, Causes, and Severity of High School Football Injuries On FieldTurf Versus Natural Grass: A 5-Year Prospective Study.” Yearbook of Pediatrics, vol. 2006, 2006, pp. 333–335., doi:10.1016/s0084-3954(07)70202-6.\n- Powell, John W., and Mario Schootman. “A Multivariate Risk Analysis of Selected Playing Surfaces in the National Football League: 1980 to 1989.” The American Journal of Sports Medicine, vol. 20, no. 6, 1992, pp. 686–694., doi:10.1177/036354659202000609.\n- Mack, Christina D., et al. “Higher Rates of Lower Extremity Injury on Synthetic Turf Compared With Natural Turf Among National Football League Athletes: Epidemiologic Confirmation of a Biomechanical Hypothesis.” The American Journal of Sports Medicine, vol. 47, no. 1, 2018, pp. 189–196., doi:10.1177/0363546518808499.\n- Mcmurtry, Shea, and Goeran Fiedler. “Comparison of Lower Limb Segment Forces during Running on Artificial Turf and Natural Grass.” Journal of Rehabilitation and Assistive Technologies Engineering, vol. 6, 2019, p. 205566831983570., doi:10.1177/2055668319835701."},{"metadata":{},"cell_type":"markdown","source":"# Appendix\n\n## Details of Regression analysis\n\nExpand the cell below to see the full details of the regression model."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"print(est2.summary())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Machine Learning Model\nBelow is an approach I worked on that did not provide conclusive results. I created three machine learning model based on players movement patterns. These models tried to predict using only player movement:\n- What surface type was the player on\n- If it was raining\n- If it was snowing\n\nI chose these three models because I assumed that players movements would be impacted by rain and snow. If I was able to show similar accuracy when trying to predict playing surface then I could conclude that the NGS data had enough information to show surfaces. Unfortunately I wasn't able to determine any conclusive results.\n\nUnfortunately, while machine learning models like the one I used in this analysis have very strong predictive power, they are in a sense black boxes where we can't easily understand how the model makes its predictions. Due to the lack of transparency I determined this analysis wouldn't be helpful for my report.\n\n    Model Features:\n        - From -0.1 second prior to snap until 5 seconds after the snap at 0.1 second intervals:\n            1. Speed of the player\n            2. Acceleration of the player\n            3. Orientation-Direction Angle of the player\n        - If the play is Pass or Rushing\n        - The PlayerID"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":false},"cell_type":"code","source":"# Preparing model training data\ntracks_model = tracks.loc[(tracks['time_since_snap'] < 5) &\n                           tracks['isRushPass'] &\n                           tracks['isInjuryPronePos']]\n\ntracks_model['a'] = tracks_model['a'].astype('float32')\ntracks_model['s'] = tracks_model['s'].astype('float32')\n\n# Every Play is a row, - create features for every 10th of a second up until 5 seconds after snap\npp_piv = tracks_model[['PlayKey',\n                       'time_since_snap',\n                       'o_dir_diff',\n                       's',\n                       'a',\n                       'RosterPosition',\n                       'Position_inj',\n                       'FieldType']].groupby(['PlayKey','time_since_snap']).sum().unstack('time_since_snap')\npp_piv.columns = [col[0] + str(col[1]) for col in pp_piv.columns.to_flat_index()]\npp_piv = pp_piv.reset_index()\npp_piv = pp_piv.merge(plays[['PlayKey','FieldType','Temperature','PlayType','PlayerDay','PlayerGame','Weather','PlayerKey']])\npp_piv['isSynthetic'] = False\npp_piv.loc[pp_piv['FieldType'] == 'Synthetic', 'isSynthetic'] = True\n\no_dir_diff_cols = [f for f in pp_piv.columns if 'o_dir_diff' in f]\na_cols = [x for x in pp_piv.columns if 'a' in x and '.' in x]\ns_cols = [x for x in pp_piv.columns if 's' in x and '.' in x]\n\n# pp_piv['max_o_dir_diff'] = pp_piv[o_dir_diff_cols].max(axis=1)\n# pp_piv['min_o_dir_diff'] = pp_piv[o_dir_diff_cols].min(axis=1)\n# pp_piv['avg_o_dir_diff'] = pp_piv[o_dir_diff_cols].mean(axis=1)\n# pp_piv['std_o_dir_diff'] = pp_piv[o_dir_diff_cols].std(axis=1)\npp_piv['isPass'] = pd.get_dummies(pp_piv['PlayType'])['Pass']\npp_piv = pp_piv.fillna(0)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.style.use('default')\nax = pp_piv[o_dir_diff_cols].head(50).T \\\n    .plot(title='Orientation-Movement Angle over first 5 seconds of play',\n          figsize=(15, 4), color='grey')\nax.get_legend().remove()\nplt.show()\nax = pp_piv[a_cols].head(50).T \\\n    .plot(title='Acceleration over first 5 seconds of play',\n         figsize=(15, 4), color='brown')\nax.get_legend().remove()\nplt.show()\nax = pp_piv[s_cols].sample(50).T \\\n    .plot(title='Speed over first 5 seconds of play',\n          figsize=(15, 4), color='lightblue')\nax.get_legend().remove()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Add features for percipitation and snow based on weather feature\nweather_percip_mapping = {\n    'Controlled Climate' : False,\n    'Sunny' : False,\n    0 : False,\n    'Cloudy' : False,\n    'Clear' : False,\n    'N/A Indoor' : False,\n    'Partly sunny' : False,\n    'N/A (Indoors)' : False,\n    'Sunny and clear' : False,\n    'Partly Cloudy' : False,\n    'Snow' : True,\n    'Indoor' : False,\n    'Indoors' : False,\n    'Showers' : True,\n    'Rain' : True,\n    'Clear and warm' : False,\n    'Mostly Cloudy' : False,\n    'Mostly Sunny' : False,\n    'Clear skies' : False,\n    'Party Cloudy' : False,\n    'Hazy' : False,\n    'Partly Clouidy': False,\n    'Sunny Skies' : False,\n    'Overcast' : False,\n    'Rain likely, temps in low 40s.' : True,\n    'Cloudy, 50% change of rain' : True,\n    'Sunny and warm' : False,\n    'Partly cloudy' : False,\n    'Clear and Cool' : False,\n    'Clear and cold' : False,\n    'Sunny and cold' : False,\n    'Cloudy, fog started developing in 2nd quarter' : False,\n    'Scattered Showers' : True,\n    'Heat Index 95' : False,\n    'Mostly cloudy' : False,\n    'Sunny, highs to upper 80s' : False,\n    'Fair' : False,\n    'Partly Sunny' : False,\n    'Sunny, Windy' : False,\n    'Mostly Sunny Skies' : False,\n    'Cloudy and Cool' : False,\n    'Mostly Coudy' : False,\n    'Mostly sunny' : False,\n    'Rainy' : True,\n    'Cloudy and cold' : False,\n    'Rain Chance 40%' : True,\n    '30% Chance of Rain' : True,\n    '10% Chance of Rain' : True,\n    'Cloudy, chance of rain' : True,\n    'Light Rain' : True, \n    'cloudy' : False,\n    'Clear and Sunny': False,\n    'Partly clear' : False,\n    'Coudy' : False,\n    'Cloudy, Rain' : True,\n    'Sun & clouds' : False,\n    'Clear to Partly Cloudy' : False,\n    'Heavy lake effect snow' : True,\n    'Clear and sunny' : False,\n    'Rain shower' : True,\n    'Cloudy, light snow accumulating 1-3\"' : True,\n    'Cloudy with periods of rain, thunder possible. Winds shifting to WNW, 10-20 mph.' : True,\n    'Clear Skies' : False,\n    'Cold': False\n}\n\npp_piv['Precipitation'] = pp_piv['Weather'].map(weather_percip_mapping)\n\nweather_snow_mapping = {\n    'Controlled Climate' : False,\n    'Sunny' : False,\n    0 : False,\n    'Cloudy' : False,\n    'Clear' : False,\n    'N/A Indoor' : False,\n    'Partly sunny' : False,\n    'N/A (Indoors)' : False,\n    'Sunny and clear' : False,\n    'Partly Cloudy' : False,\n    'Snow' : True,\n    'Indoor' : False,\n    'Indoors' : False,\n    'Showers' : False,\n    'Rain' : False,\n    'Clear and warm' : False,\n    'Mostly Cloudy' : False,\n    'Mostly Sunny' : False,\n    'Clear skies' : False,\n    'Party Cloudy' : False,\n    'Hazy' : False,\n    'Partly Clouidy': False,\n    'Sunny Skies' : False,\n    'Overcast' : False,\n    'Rain likely, temps in low 40s.' : False,\n    'Cloudy, 50% change of rain' : False,\n    'Sunny and warm' : False,\n    'Partly cloudy' : False,\n    'Clear and Cool' : False,\n    'Clear and cold' : False,\n    'Sunny and cold' : False,\n    'Cloudy, fog started developing in 2nd quarter' : False,\n    'Scattered Showers' : False,\n    'Heat Index 95' : False,\n    'Mostly cloudy' : False,\n    'Sunny, highs to upper 80s' : False,\n    'Fair' : False,\n    'Partly Sunny' : False,\n    'Sunny, Windy' : False,\n    'Mostly Sunny Skies' : False,\n    'Cloudy and Cool' : False,\n    'Mostly Coudy' : False,\n    'Mostly sunny' : False,\n    'Rainy' : False,\n    'Cloudy and cold' : False,\n    'Rain Chance 40%' : False,\n    '30% Chance of Rain' : False,\n    '10% Chance of Rain' : False,\n    'Cloudy, chance of rain' : False,\n    'Light Rain' : False, \n    'cloudy' : False,\n    'Clear and Sunny': False,\n    'Partly clear' : False,\n    'Coudy' : False,\n    'Cloudy, Rain' : False,\n    'Sun & clouds' : False,\n    'Clear to Partly Cloudy' : False,\n    'Heavy lake effect snow' : True,\n    'Clear and sunny' : False,\n    'Rain shower' : False,\n    'Cloudy, light snow accumulating 1-3\"' : True,\n    'Cloudy with periods of rain, thunder possible. Winds shifting to WNW, 10-20 mph.' : False,\n    'Clear Skies' : False,\n    'Cold': False\n}\n\npp_piv['Snow'] = pp_piv['Weather'].map(weather_snow_mapping)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Show the first few rows of features used in the model\n- s -> speed features\n- a -> acceleration features\n- o_dir_diff -> angle difference between orientation and direction"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Add position\npp_piv = pp_piv.merge(tracks_model[['Position_inj','PlayKey']].drop_duplicates(), on='PlayKey', how='left')\npp_piv = pd.concat([pp_piv, pd.get_dummies(pp_piv['Position_inj'])], axis=1)\npp_piv['Position_inj'] = pp_piv['Position_inj'].astype('category')\n# Perform a train / test split\nX = pp_piv.drop(['FieldType','PlayKey','isSynthetic','PlayType','Temperature',\n                 'PlayerGame','PlayerDay','Weather','PlayerKey',\n                 'Precipitation', 'Snow'], axis=1)\ny = pp_piv['isSynthetic']\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=529)\nX.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":false},"cell_type":"code","source":"params = {}\nparams['max_bin'] = 50\nparams['learning_rate'] = 0.01\nparams['boosting_type'] = 'gbdt'\nparams['objective'] = 'binary'\nparams['metric'] = 'auc'\n\n##################################################\n# TRAIN MODEL TO PREDICT FIELD TYPE (isSynthetic)\n##################################################\n\nX = pp_piv.drop(['FieldType','PlayKey','isSynthetic','PlayType','Temperature',\n                 'PlayerGame','PlayerDay','Weather','PlayerKey',\n                 'Precipitation', 'Snow'], axis=1)\ny = pp_piv['isSynthetic']\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=529)\n\nd_train = lgb.Dataset(X_train, label=y_train) #, categorical_feature=['PlayerKey'])\nd_test = lgb.Dataset(X_test, label=y_test)\nlgbm_model_FieldType = lgb.train(params,\n                       d_train,\n                       valid_sets=(d_train, d_test),\n                       num_boost_round=5000,\n                       verbose_eval=False,\n                       early_stopping_rounds=200)\n\ny_pred_train = lgbm_model_FieldType.predict(X_train)\ny_pred_test = lgbm_model_FieldType.predict(X_test)\n\ntrain_acc = sklearn.metrics.accuracy_score(y_train, y_pred_train.round())\ntest_acc = sklearn.metrics.accuracy_score(y_test, y_pred_test.round())\ntrain_auc = lgbm_model_FieldType.best_score['training']['auc']\ntest_auc = lgbm_model_FieldType.best_score['valid_1']['auc']\nprint('Turf Type Prediction Model:')\nprint(f'Training accuracy: {train_acc:0.4}')\nprint(f'Test accuracy:     {test_acc:0.4}')\nprint(f'Training AUC:      {train_auc:0.4}')\nprint(f'Test AUC:          {test_auc:0.4}')\n\nfi_df_lgbm_model_FieldType = pd.DataFrame(index=lgbm_model_FieldType.feature_name(),\n             data=lgbm_model_FieldType.feature_importance(),\n             columns=['importance']\n            )","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":false},"cell_type":"code","source":"##################################################\n# TRAIN MODEL TO PREDICT PRECIPITATION\n##################################################\n\nX = pp_piv.drop(['FieldType','PlayKey','isSynthetic','PlayType','Temperature',\n                 'PlayerGame','PlayerDay','Weather','PlayerKey',\n                 'Precipitation', 'Snow'], axis=1)\ny = pp_piv['Precipitation']\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=529)\n\nd_train = lgb.Dataset(X_train, label=y_train) #, categorical_feature=['PlayerKey'])\nd_test = lgb.Dataset(X_test, label=y_test)\nlgbm_model_PRECIP = lgb.train(params,\n                       d_train,\n                       valid_sets=(d_train, d_test),\n                       num_boost_round=5000,\n                       verbose_eval=False,\n                       early_stopping_rounds=200)\n\ny_pred_train = lgbm_model_PRECIP.predict(X_train)\ny_pred_test = lgbm_model_PRECIP.predict(X_test)\n\ntrain_acc = sklearn.metrics.accuracy_score(y_train, y_pred_train.round())\ntest_acc = sklearn.metrics.accuracy_score(y_test, y_pred_test.round())\ntrain_auc = lgbm_model_PRECIP.best_score['training']['auc']\ntest_auc = lgbm_model_PRECIP.best_score['valid_1']['auc']\nprint('Raining Prediction Model:')\nprint(f'Training accuracy: {train_acc:0.4}')\nprint(f'Test accuracy:     {test_acc:0.4}')\nprint(f'Training AUC:      {train_auc:0.4}')\nprint(f'Test AUC:          {test_auc:0.4}')\n\nfi_df_lgbm_model_PRECIP = pd.DataFrame(index=lgbm_model_PRECIP.feature_name(),\n             data=lgbm_model_PRECIP.feature_importance(),\n             columns=['importance']\n            )","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":false},"cell_type":"code","source":"#####################################\n# TRAIN MODEL TO PREDICT IF SNOWING\n#####################################\n\nX = pp_piv.drop(['FieldType','PlayKey','isSynthetic','PlayType','Temperature',\n                 'PlayerGame','PlayerDay','Weather','PlayerKey',\n                 'Precipitation', 'Snow'], axis=1)\ny = pp_piv['Snow']\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=529)\n\nd_train = lgb.Dataset(X_train, label=y_train)#, categorical_feature=['PlayerKey'])\nd_test = lgb.Dataset(X_test, label=y_test)\nlgbm_model_SNOW = lgb.train(params,\n                       d_train,\n                       valid_sets=(d_train, d_test),\n                       num_boost_round=5000,\n                       verbose_eval=False,\n                       early_stopping_rounds=200)\n\ny_pred_train = lgbm_model_SNOW.predict(X_train)\ny_pred_test = lgbm_model_SNOW.predict(X_test)\n\ntrain_acc = sklearn.metrics.accuracy_score(y_train, y_pred_train.round())\ntest_acc = sklearn.metrics.accuracy_score(y_test, y_pred_test.round())\ntrain_auc = lgbm_model_SNOW.best_score['training']['auc']\ntest_auc = lgbm_model_SNOW.best_score['valid_1']['auc']\nprint('Snowing Prediction Model:')\nprint(f'Training accuracy: {train_acc:0.4}')\nprint(f'Test accuracy:     {test_acc:0.4}')\nprint(f'Training AUC:      {train_auc:0.4}')\nprint(f'Test AUC:          {test_auc:0.4}')\n\nfi_df_lgbm_model_SNOW = pd.DataFrame(index=lgbm_model_SNOW.feature_name(),\n             data=lgbm_model_SNOW.feature_importance(),\n             columns=['importance']\n            )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Concat results\nfi_df = pd.concat([fi_df_lgbm_model_SNOW, fi_df_lgbm_model_FieldType, fi_df_lgbm_model_PRECIP], axis=1)\nfi_df.columns = ['snow_importance','fieldtype_importance','precipitation_importance']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, axes = plt.subplots(1,3,figsize=(15, 5))\nfi_df.sort_values('fieldtype_importance', ascending=True).tail(10)[['fieldtype_importance']] \\\n    .plot(kind='barh', ax=axes[0], title='Top 10 Features to predict FieldType')\nfi_df.sort_values('precipitation_importance', ascending=True).tail(10)[['precipitation_importance']] \\\n    .plot(kind='barh', ax=axes[1], title='Top 10 Features to predict precipitation')\nfi_df.sort_values('snow_importance', ascending=True).tail(10)[['snow_importance']] \\\n    .plot(kind='barh', ax=axes[2], title='Top 10 Features to predict snow')\naxes[0].get_legend().remove()\naxes[1].get_legend().remove()\naxes[2].get_legend().remove()\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}