{"cells":[{"metadata":{},"cell_type":"markdown","source":"By: Nick Tarsi and Brandon Snyder"},{"metadata":{},"cell_type":"markdown","source":"# Problem Definition\n\n> Your challenge is to characterize any differences in player movement between the playing surfaces and identify specific scenarios (e.g., field surface, weather, position, play type, etc.) that interact with player movement to present an elevated risk of injury. The NFL is challenging Kagglers to help them examine the effects that playing on synthetic turf versus natural turf can have on player movements and the factors that may contribute to lower extremity injuries.\n\n### Hypotheses:\n* There is a significant increase in injury occurence for plays on synthetic turf vs. plays on natural turf.\n* However, due to the extremely low proportion of plays with injuries overall, turf type alone will barely explain any of the variation in injury occurence. \n* While no other individual play-level factor will explain much more of the variation in injury occurence, those that do will have significant interaction effects with turf type, suggesting that synthetic turf's elevates injury risk through amplifying other risk factors.\n\n### Plan:\n1. Import playtrack data (with some creative workarounds to converse memory), calculate additional columns representing change between observations (acceleration, rotation, etc.)\n1. Group playtrack data into play-level aggregate features, as well as features representing the end of movement during the play (assuming players do not continue tracked movement after experiencing an injury)\n1. Join play-level aggregate features to player-level data (roster role) and game-level data (turf type, stadium type, weather, play role, play role group)\n1. Statistical significance tests for differences in injury occurence:\n    * Split play-level records by turf type and conduct two-sample t-test\n    * Further split play-level records by other categorical/binary features and conduct multiple logistic regressions to identify significant predictors among categorical features, continuous features, and interaction effects."},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","scrolled":false,"trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plot\nimport datetime\n\nfrom scipy.stats import ttest_ind\nimport statsmodels.api as sm\nfrom statsmodels.formula.api import ols\nimport statsmodels.api as sm\nfrom statsmodels.graphics.factorplots import interaction_plot\nimport statistics\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import roc_auc_score\nfrom sklearn.metrics import roc_curve\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.\n\npd.options.display.float_format = '{:.3f}'.format #prevent scientific notation in dataframes, display #.### instead\n\n%whos #outputs table of variables and their info\n\n## used to expand Jupyter Notebook to full browser width for easier reading of long lines\nfrom IPython.core.display import display, HTML\ndisplay(HTML(\"<style>.container { width:100% !important; }</style>\"))\n\n## template for monitoring/recording run-times for blocks of code\noverall_startDT = datetime.datetime.now()\nprint('Started at', overall_startDT)\n#code\nprint(datetime.datetime.now() - overall_startDT)\n# usually around 0:##:##.# on my laptop, ##m##s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Import playtrack data with some initial pre-processing"},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"## import playtrack data\nstartDT = datetime.datetime.now()\nprint('Starting import of playtrackDF at', startDT)\n#original key without padding\ncol_dtypes = {'time':float, \n              'x':float, 'y':float, \n              'dir':float, 'dis':float, 'o':float, 's':float}\nuse_cols = ['PlayKey', 'time', 'dir', 'dis', 'o', 's']\n#playtrackDF = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayerTrackData.csv', dtype=col_dtypes, usecols=use_cols)\nplaytrackDF = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayerTrackData.csv', dtype=col_dtypes)\n\n\n## there are 2 rows with NaN values for 'dir' and 'o', they appear to be meaningless glitches and deletable\n#print(playtrackDF.columns)\n#print(playtrackDF.isna().sum())\n#dropableRows = list(playtrackDF[playtrackDF['dir'].isna()].index)\n#for row in dropableRows:\n#    display(playtrackDF[row-2:row+3])\nplaytrackDF.dropna(axis='index', subset=['dir', 'o'], inplace=True)\n\nprint(playtrackDF.shape)\nprint(datetime.datetime.now() - startDT, 'to import playtrack data and drop NA rows')\nprint(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs for initial import')\n#usually around 1m45s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Because this dataframe utilizes a large amount of memory and Kaggle's cloud session restarts if it runs out of memory, some workarounds are implemented to use smaller datatypes to store the values. Specifically, floats (i.e. decimals) that are only precise to the hundredth's place (#.##) are multipied by 100 and then converted to integers. That mutliplication will be reversed once playtrack data has been grouped to play-level data, which requires a much smaller record count and therefore can contain floats without utilizing too much memory. (https://docs.scipy.org/doc/numpy/user/basics.types.html)\n\nAdditionally, the \"PlayKey\" valyes are reformatted so they can be sorted properly (01,02...11,12,etc. instead of 1,11,12,2,etc.) and also stored as numbers rather than strings."},{"metadata":{"trusted":true},"cell_type":"code","source":"## reducing memory usage of this massive dataframe by converting columns to *100 ints\nstartDT = datetime.datetime.now()\nfor col in playtrackDF.columns:\n    if playtrackDF[col].dtype != object:\n        print(col, playtrackDF[col].min(), playtrackDF[col].max())\n        playtrackDF[col] = playtrackDF[col]*100\n        if playtrackDF[col].min() < 0:\n            playtrackDF[col] = playtrackDF[col].astype(np.int16) #stores -32,768 to 32,767\n        else:\n            playtrackDF[col] = playtrackDF[col].astype(np.uint16) #stores 0-65,535\nprint(datetime.datetime.now() - startDT, 'to reduce memory usage') ## usually around 15s on Kaggle\nprint(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs for *100 ints')\n\n## reformatting the PlayKey from 12345-1-1 to 1234512123 to enable storage as number with proper sorting\nstartDT = datetime.datetime.now()\nprint('Starting rekeying at', startDT)\nplaytrackDF.loc[:, 'PlayKey'] = playtrackDF['PlayKey'].apply(lambda v: '{0:0>5}{1:0>2}{2:0>3}'.format(*v.split('-') ) )\nplaytrackDF.loc[:, 'PlayKey'] = playtrackDF['PlayKey'].astype(np.int64) #uint32's max value is just below the largest key\nprint(datetime.datetime.now() - startDT, 'to rekey DF') ## usually around 2m on Kaggle\nprint(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs for reduced/rekeyed')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# import injury and plays data\n\nThe same rekeying from the playtrack data is applied to the \"PlayKey\", \"GameID\", and \"PlayerKey\" columns in these dataframes."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","scrolled":false,"trusted":true},"cell_type":"code","source":"#import injuries and plays data\ninjuriesDF = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nprint(\"injuriesDF shape\", injuriesDF.shape)\nplaysDF = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')\nprint(\"playsDF shape\", playsDF.shape)\n\nstartDT = datetime.datetime.now()\ninjuriesDF['PlayerKey'] = injuriesDF['PlayerKey'].apply(lambda v: '{0:0>5}'.format(v) )\ninjuriesDF['GameID'] = injuriesDF['GameID'].apply(lambda v: '{0:0>5}{1:0>2}'.format(*v.split('-') ) )\ninjuriesDF['PlayKey'].fillna(value='0-0-0', inplace=True)\ninjuriesDF['PlayKey'] = injuriesDF['PlayKey'].astype(str).apply(lambda v: '{0:0>5}{1:0>2}{2:0>3}'.format(*v.split('-') ) )\nplaysDF['PlayerKey'] = playsDF['PlayerKey'].apply(lambda v: '{0:0>5}'.format(v) )\nplaysDF['GameID'] = playsDF['GameID'].apply(lambda v: '{0:0>5}{1:0>2}'.format(*v.split('-') ) )\nplaysDF['PlayKey'] = playsDF['PlayKey'].apply(lambda v: '{0:0>5}{1:0>2}{2:0>3}'.format(*v.split('-') ) )\ninjuriesDF.loc[:, 'PlayKey'] = injuriesDF['PlayKey'].astype(np.int64)\nplaysDF.loc[:, 'PlayKey'] = playsDF['PlayKey'].astype(np.int64)\nprint(datetime.datetime.now() - startDT, 'to rekey smaller dataframes') # usually around 1s on Kaggle\n\nprint('Dataframes created from injuries and plays CSVs (playtrack CSV to be handled separately due to size)')","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"#collapse player-level info into smaller DF\nplayersDF = playsDF.groupby(by=['PlayerKey', 'RosterPosition']).size().reset_index().rename(columns={0:'Plays'})\nprint(\"playersDF shape\", playersDF.shape)\n\n#collapse game-level info into smaller DF\ngroupCols = [playsDF['PlayerKey'], playsDF['GameID'], playsDF['StadiumType'].fillna('unknown'), playsDF['FieldType'], playsDF['Temperature'], playsDF['Weather'].fillna('unknown'), playsDF['PlayerDay']]\ngamesDF = playsDF.groupby(by=groupCols).size().reset_index().rename(columns={0:'Plays'})\nprint(\"gamesDF shape\", gamesDF.shape)\ngamesDF = gamesDF.sort_values(by=['PlayerKey', 'PlayerDay'])\ngamesDF.reset_index(drop=True, inplace=True)\n\nprint(len(injuriesDF['PlayerKey'].unique()), 'unique players in injuriesDF')\n\ndisplay(gamesDF.head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Some data cleaning is needed for the \"StadiumType\", \"Weather\", and \"Temperature\" columns. \n\nGiven the focus on the turf, new columns are created that contain simpler categorical variables representing whether the field is exposed and/or wet, rather than overwrite the original values for \"StadiumType\" and \"Weather\". \n\nSome \"Temperature\" values needed to be imputed as well, specifically for indoor games that originally were reported to have a temperature of -999. These invalid values will be replaced by the average temperature of other indoor games."},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"injuryGames = pd.DataFrame(injuriesDF.groupby(by=['GameID']).size())\ninjuryGames.reset_index(inplace=True)\ngamesDF['InjOcc'] = 0\nfor idVal in injuryGames['GameID']:\n    gamesDF.loc[gamesDF['GameID'] == idVal, 'InjOcc'] = 1\n## there's 104 unique GameIDs for injured players and 5712 unique GameIDs in the play data (gamesDF)\n\n## cleaning StadiumType and determining FieldExposed\n#print(gamesDF['StadiumType'].unique())\noutdoorList = ['Open', 'Outdoor', 'Oudoor', 'Outdoors', 'Ourdoor', 'Outddors', 'Heinz Field', 'Outdor', 'Outside']\nindoorList = ['Indoors', 'Closed Dome', 'Domed, closed', 'Dome', 'Indoor', 'Domed', 'Retr. Roof-Closed', 'Outdoor Retr Roof-Open', 'Indoor, Roof Closed', 'Retr. Roof - Closed', 'Retr. Roof-Open', 'Dome, closed', 'Indoor, Open Roof', 'Domed, Open', 'Domed, open', 'Retr. Roof - Open', 'Retr. Roof Closed', 'Retractable Roof']\nunknownList = ['unknown', 'Bowl', 'Cloudy']\ngamesDF.loc[gamesDF['StadiumType'].isin(outdoorList), 'FieldExposed'] = 1\ngamesDF.loc[gamesDF['StadiumType'].isin(indoorList), 'FieldExposed'] = 0\ngamesDF.loc[gamesDF['StadiumType'].isin(unknownList), 'FieldExposed'] = 1 ## assuming exposed b/c not specified\nprint('FieldExposed vals:', gamesDF['FieldExposed'].unique())\n\n## cleaning Weather and determining FieldWet\nwetDescs = []\ndryDescs = []\nunkDescs = []\nfor desc in list(gamesDF['Weather'].unique()):\n    if 'rain' in desc or 'Rain' in desc or 'showers' in desc or 'Showers' in desc or 'snow' in desc or 'Snow' in desc:\n        wetDescs.append(desc)\n    else: \n        if 'unknown' in desc or 'Unknown' in desc:\n            unkDescs.append(desc)\n        else:\n            dryDescs.append(desc)\ngamesDF.loc[(gamesDF['Weather'].isin(wetDescs)) & (gamesDF['FieldExposed'] == 1), 'FieldWet'] = 1\ngamesDF.loc[gamesDF['Weather'].isin(dryDescs), 'FieldWet'] = 0\ngamesDF.loc[gamesDF['FieldExposed'] == 0, 'FieldWet'] = 0\ngamesDF.loc[gamesDF['Weather'].isin(unkDescs), 'FieldWet'] = 0 ## assuming field is not wet b/c not specified\nprint('FieldWet vals:', gamesDF['FieldWet'].unique())\n\n## cleaning Temperature\navgIndoorTemp = gamesDF.loc[(gamesDF['FieldExposed'] != 1) & (gamesDF['Temperature'] != -999), 'Temperature'].mean()\n#print('Avg (valid) temp for indoor games is', round(avgIndoorTemp, 0))\ngamesDF.loc[(gamesDF['FieldExposed'] != 1) & (gamesDF['Temperature'] == -999), 'Temperature'] = round(avgIndoorTemp, 0)\n\n## making a one-hot variable for turf type\nprint('FieldType vals:', gamesDF['FieldType'].unique())\ngamesDF['Synthetic'] = 1 * (gamesDF['FieldType'] == 'Synthetic')\n\ndisplay(gamesDF.head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# further pre-processing of ALL playtrackDF rows"},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"print(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs before additional columns')\nstartDT = datetime.datetime.now()\nplaytrackDF.loc[:, 'twist'] = abs(playtrackDF['dir'].astype(np.int32) - playtrackDF['o'].astype(np.int32))\nplaytrackDF.loc[playtrackDF['twist'] > 18000, 'twist'] = 36000 - playtrackDF.loc[playtrackDF['twist'] > 18000, 'twist']\nprint(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs with twist col added')\nplaytrackDF.loc[:, 'twist'] = playtrackDF['twist'].astype(np.int16)\nprint(datetime.datetime.now() - startDT, 'to calculate twist column') ## usually around 5s on Kaggle\nprint(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs with twist col as int16')\n\ncols_for_diffs = ['dir', 'o', 's', 'twist']\n\nfor col in cols_for_diffs:\n    startDT = datetime.datetime.now()\n    playtrackDF.loc[:, 'd_'+col ] = abs(playtrackDF[col].astype(np.int32).diff())\n    playtrackDF.loc[0, 'd_'+col ] = 0 ## these rows represent a new play and shouldn't be compared to the row above\n    playtrackDF.loc[:, 'd_'+col] = playtrackDF['d_'+col].astype(np.int32)\n    print(datetime.datetime.now() - startDT, 'to calculate', 'd_'+col, 'column') ## each around 2-5s on Kaggle\n    \nprint(round(playtrackDF.memory_usage().sum() / 1024**3, 3), 'GBs currently')\n\ndisplay(playtrackDF.head())","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"for col in playtrackDF.columns:\n    if playtrackDF[col].dtype != object:\n        print(col, playtrackDF[col].dtype, round(playtrackDF[col].memory_usage() / 1024**3, 3), 'GBs', playtrackDF[col].min(), playtrackDF[col].max() )\n    else:\n        print(col, playtrackDF[col].dtype, round(playtrackDF[col].memory_usage() / 1024**3, 3), 'GBs' )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Memory utilization is starting to creep up again with the additional columns added, but the playtrack data is going to be grouped into play-level records now, so this huge dataframe won't need to be processed much longer and doesn't threaten to overwhelm Kaggle's memory limit.\n\n## Calculating play-level features and joining to player-level and game-level features"},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\n\nplayLens = pd.DataFrame(playtrackDF[['PlayKey']].groupby(by=['PlayKey']).size())\nplayLens.columns = ['obs']\n\n## any stats here are divided by 100 to account for earlier *100 for sake of reduced memory via int datatypes \n\n## identifying the field \"coordinates\" for the player's final location per play\nplayFinalLocs = playtrackDF[['PlayKey', 'x', 'y']].groupby(by=['PlayKey']).tail(1).set_index('PlayKey')/100\n## assuming each quadrant of the field is interchangeable, the location values should be \n## relative to midfield rather than a corner of the field\nplayFinalLocs.loc[:, 'x'] = abs(playFinalLocs['x'] - (120/2)) # converting to yds away from midfield\nplayFinalLocs.loc[:, 'y'] = abs(playFinalLocs['y'] - (53.3/2)) # converting to yds away from midfield\nplayFinalLocs.columns = 'rel_' + playFinalLocs.columns + '_final'\n\n## aggregate between-observation changes per play, first as sums...\nplaySums = playtrackDF[['PlayKey','d_dir', 'd_o', 'd_s', 'd_twist']].groupby(by=['PlayKey']).sum()/100\nplaySums.columns = playSums.columns + '_sum'\n\n## ... then as means\nplayAvgs = playtrackDF[['PlayKey','d_dir', 'd_o', 'd_s', 'd_twist']].groupby(by=['PlayKey']).mean()/100\nplayAvgs.columns = playAvgs.columns + '_avg'\n\n## join all features together\nplayStats = playsDF[['PlayKey', 'FieldType']].drop_duplicates() ## assumed to be consistent for all plays with same gameID prefix in PlayKey\nplayStats = playStats.merge(playFinalLocs, on='PlayKey')\nplayStats = playStats.merge(playLens, on='PlayKey')\nplayStats = playStats.merge(playSums, on='PlayKey')\nplayStats = playStats.merge(playAvgs, on='PlayKey')\n\nprint(datetime.datetime.now() - startDT, 'to generate playStats') ## about 20s on Kaggle\n\ndisplay(playStats.head(2))","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"## just verifying no weird values resulted from the calculations above\nfor col in playStats.columns:\n    if playStats[col].dtype != 'object':\n        print(col, '\\t', playStats[col].dtype, '\\t', round(playStats[col].min(),3), '\\t', round(playStats[col].max(), 3) )","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"# identifying injury plays\nprint(playStats.shape)\nstartDT = datetime.datetime.now()\ninjPlays = list(injuriesDF['PlayKey'].unique())\nplayStats['Inj'] = 0\nplayStats.loc[playStats['PlayKey'].isin(injPlays), 'Inj'] = 1\nprint(datetime.datetime.now() - startDT, 'to identify Inj plays') ## about 0.05s on Kaggle\nprint(playStats.shape)\n\ndisplay(playStats[playStats['Inj'] == 1].head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Initial simple analysis"},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"display(playStats.groupby(['Inj', 'FieldType']).size().round(3))","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"display(playStats.groupby(['Inj', 'FieldType']).mean().round(3))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"WITHOUT FILTERING PRE- AND POST-PLAY OBSERVATIONS\n* Injury plays tended to end further outside the hashmarks (3.0833 yds rel_y_final = on hashmark), especially on synturf\n* plays slightly longer on natturf for non inj plays, but shorter for inj plays; exaggerates differences in _sum stats\n    * simplest explanation would be that synthetic turf is dangerous when plays are drawn out\n* Movement _avg stats all increase in injury plays\n    * increase for synturf vs natturf is exaggerated for d_twist_avg (torque?) and d_dir_avg (re-routing)\n    * natturf actually had higher increases in d_o_avg (pivoting) and d_s_avg (accel)\n"},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"display(playStats.groupby(['Inj', 'FieldType']).median().round(3))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"WITHOUT FILTERING PRE- AND POST-PLAY OBSERVATIONS\n* Medians almost universally lower than averages, suggesting more outlier plays with very high stats skewing avg higher\n* Same trends in median play end location (relative to hashmarks) and lengths as was found in means\n* _avg stats increase for injury plays again\n    * d_twist_avg (torque?) larger increase for synturf\n    * d_dir_avg (re-routing), d_o_avg (pivoting)  larger increase for natturf\n    * d_s_avg (accel) equivalent increase for both fieldtypes"},{"metadata":{},"cell_type":"markdown","source":"To help better highlight the interaction of \"FieldType\" and \"Inj\" upon these metrics, interaction plots can be generated of the levels among the 4 segments created by these two binary categorical features. \n\nOne categorical feature forms the x-axis, while the other determines which points are connected by lines. Comparing the slopes of these lines shows the \"difference of differences\" that defines the interaction of the variables. Plots where the lines have drastically different slopes illustrate the most noteworthy interaction effects - such as for \"rel_y_final\" (second plot)."},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nplot_fts = playStats.columns[2:-1]\nfor i_1, feature_1 in enumerate( plot_fts ):\n    simple_stats = playStats[['Inj', 'FieldType', feature_1]].copy(deep=True)\n    simple_stats['plays'] = 1\n    pivot_for_interax = simple_stats.groupby(['Inj', 'FieldType']).mean()\n    #features = list(pivot_for_interax.columns)\n    axes = list(pivot_for_interax.index.names)\n    pivot_for_interax.reset_index(inplace=True)\n    for col in axes:\n        pivot_for_interax.loc[:, col] = pivot_for_interax[col].astype('str')\n    #print(pivot_for_interax)\n    ## https://www.statsmodels.org/dev/generated/statsmodels.graphics.factorplots.interaction_plot.html\n    interax_plot = interaction_plot(x = pivot_for_interax['FieldType'], trace = pivot_for_interax['Inj'], response = pivot_for_interax[feature_1], plottype = 'both')\n    interax_plot.suppressComposite #output plot only once, otherwise two copies are displayed\nprint(datetime.datetime.now() - startDT, 'to plot interactions between FieldType and Inj') ## about 1.0s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Further processing"},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"# adding additional columns for player-level, game-level, and play-level info\nmlDF = playStats.copy(deep=True)\nprint(mlDF.shape)\nmlDF['PlayerKey'] = mlDF['PlayKey'].astype(str).str[0:5]\nmlDF['GameID'] = mlDF['PlayKey'].astype(str).str[0:7]\nmlDF['endOutsideHash'] = 1*(mlDF['rel_y_final'] > (18.5/3/2) ) ## 3.0833 yds from midfield to hashmarks\n\nmlDF = mlDF.merge(gamesDF[['GameID', 'Temperature', 'FieldExposed', 'FieldWet']], left_on='GameID', right_on='GameID')\nmlDF = mlDF.merge(playersDF[['PlayerKey', 'RosterPosition']], left_on='PlayerKey', right_on='PlayerKey')\nmlDF = mlDF.merge(playsDF[['PlayKey', 'Position', 'PositionGroup']], left_on='PlayKey', right_on='PlayKey')\n\nprint(mlDF.shape)\nmlDF.head()","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"## this isn't strictly necessary but it helps with keeping the data concise\n\nmlDF.loc[mlDF['RosterPosition'] == 'Quarterback', 'RosterPosition'] = 'QB'\nmlDF.loc[mlDF['RosterPosition'] == 'Wide Receiver', 'RosterPosition'] = 'WR'\nmlDF.loc[mlDF['RosterPosition'] == 'Linebacker', 'RosterPosition'] = 'LB'\nmlDF.loc[mlDF['RosterPosition'] == 'Running Back', 'RosterPosition'] = 'RB'\nmlDF.loc[mlDF['RosterPosition'] == 'Defensive Lineman', 'RosterPosition'] = 'DL'\nmlDF.loc[mlDF['RosterPosition'] == 'Tight End', 'RosterPosition'] = 'TE'\nmlDF.loc[mlDF['RosterPosition'] == 'Safety', 'RosterPosition'] = 'S'\nmlDF.loc[mlDF['RosterPosition'] == 'Cornerback', 'RosterPosition'] = 'CB'\nmlDF.loc[mlDF['RosterPosition'] == 'Offensive Lineman', 'RosterPosition'] = 'OL'\nmlDF.loc[mlDF['RosterPosition'] == 'Kicker', 'RosterPosition'] = 'K'\n\nmlDF['RosterPosition'].unique()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To enable greater analysis flexibility, the categorical variables \"FieldType\", \"RosterPosition\", \"Position\" (i.e. role for play), and \"PositionGroup\" (i.e role group for play) will need to be converted one-hot variables (aka dummy variables)."},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"## Because this process needs to be repeated for three very similar variables, the resulting column names need to be prefixed.\n## pd.get_dummies() could have been used, but it was using too much memory so this semi-manual process was used instead.\n\n## convert position categorical features into one-hot feature sets\nprint('Converting RosterPosition...')\nfor rosterPos in mlDF['RosterPosition'].unique():\n    #print(rosterPos)\n    mlDF['ros_' + rosterPos] = 0\n    mlDF.loc[mlDF['RosterPosition'] == rosterPos, 'ros_' + rosterPos] = 1\nmlDF.drop('RosterPosition', axis='columns', inplace=True)\n\nprint('Converting (play) Position...')\nfor playPos in mlDF['Position'].unique():\n    #print(playPos)\n    mlDF['play_' + playPos] = 0\n    mlDF.loc[mlDF['Position'] == playPos, 'play_' + playPos] = 1\nmlDF.drop('Position', axis='columns', inplace=True)\n\nprint('Converting (play) PositionGroup...')\nfor playPosGrp in mlDF['PositionGroup'].unique():\n    #print(playPosGrp)\n    mlDF['playGrp_' + playPosGrp] = 0\n    mlDF.loc[mlDF['PositionGroup'] == playPosGrp, 'playGrp_' + playPosGrp] = 1\nmlDF.drop('PositionGroup', axis='columns', inplace=True)\n\nmlDF.drop(['PlayerKey', 'GameID'], axis='columns', inplace=True)\n\nmlDF['SynTurf'] = 0\nmlDF.loc[mlDF['FieldType'] == 'Synthetic', 'SynTurf'] = 1\nmlDF.drop('FieldType', axis='columns', inplace=True)\n\nprint('Done!')\n\n## verifying results with a side-scrollable display of the dataframe\nHTML(mlDF.head().to_html())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Statistical analysis using significance tests\n\nFirst, it should be pointed out that this problem is pretty much the definition of \"needle in a haystack\" due to the incredibly small variance that will be nearly impossible to model effectively."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Variance in whether plays result in injury:', mlDF['Inj'].var() )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Testing the first hypothesis is the simplest - the plays are segmented into samples based on whether they were on synthetic turf, then the proportion of injurious plays in each sample is tested to determine if the difference is statistically significant based on the size of the samples."},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.options.display.float_format = '{:.10f}'.format ## need  greater level of detail due to tiny injury occurence proportions\n\ndisplay(mlDF[['Inj', 'SynTurf']].groupby('SynTurf').mean())\ndisplay(ttest_ind(mlDF.loc[mlDF['SynTurf']==1,'Inj'], mlDF.loc[mlDF['SynTurf']==0,'Inj']))\n\n## p-value of 0.0436 suggests statistically significant difference in likelihood of injury based on turf type\n\ninj_turf_corr = mlDF['Inj'].corr(mlDF['SynTurf'])\nprint('The correlation of turf type and injury occurence is only', round(inj_turf_corr, 6), 'which means only', round((inj_turf_corr**2), 10), ' of the variance in injury occurence would be explainable by turf type (possibly through other factors)' )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In order to determine if other features might be better predictors, either alone or in interaction with turf type, further testing is necessary. First, do any features correlate better with injury occurence than turf type?"},{"metadata":{"trusted":true},"cell_type":"code","source":"corrDF = pd.DataFrame(columns = ['Feature', 'Corr_w_Inj'])\n\nfeat_for_corr = list(mlDF.columns)[1:-1] #exclude PlayKey and SynTurf\nfeat_for_corr.remove('Inj')\n\nfor i_col, col in enumerate(feat_for_corr):\n    if col == 'Inj':\n        print(col)\n    corrDF.loc[i_col] = [col, mlDF[col].corr(mlDF['Inj'])]\n\ncorrDF['R2'] = corrDF['Corr_w_Inj']**2\n\ndisplay(corrDF.sort_values('R2').loc[corrDF['R2'] > (inj_turf_corr**2)])\n\nfeatures_to_test = list(corrDF.sort_values('R2').loc[corrDF['R2'] > (inj_turf_corr**2)]['Feature'])\n\nfeatures_cat = list()\nfeatures_con = list()\n\nfor ft in features_to_test:\n    if(mlDF[ft].max() == 1 and mlDF[ft].min() == 0):\n        features_cat.append(ft)\n    else:\n        features_con.append(ft)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Several features had stronger correlation with injury occurence than turf type:\n* All 8 of the play-level aggregate features (4 sums, 4 avgs), \n* The continuous \"rel_y_final\" (lateral position relative to the midpoint between the hashmarks, inverse of proximity to sideline)\n* Temperature\n* Three inherently similar roles (rostered Offensive Lineman, play Offensive Guard (specific spot on line), and play Offensive Line)\n\n## Categorical features, individual significance testing\n\nThe categorical variables can be tested in the same way that turf type was."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Categorical:', features_cat)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\ncatFtsDF = pd.DataFrame(columns=['Feature', 'FalseInj', 'TrueInj', 'p-val'])\n\nfor i_cat, cat_ft in enumerate(features_cat):\n    inj_ps = mlDF[['Inj', cat_ft]].groupby(cat_ft).mean()['Inj']\n    pval = ttest_ind(mlDF.loc[mlDF[cat_ft]==1,'Inj'], mlDF.loc[mlDF[cat_ft]==0,'Inj']).pvalue\n    catFtsDF.loc[i_cat] = [cat_ft, inj_ps[0], inj_ps[1], pval]\n    \ndisplay(catFtsDF)\nprint(datetime.datetime.now() - startDT, 'to test all categorical features') ## about 0.1s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Individually, each categorical feature segments plays into samples with statistically significant injury likelihoods, all four more significant than turf type (p-values below 0.0436).\n\n## Categorical features, interaction effects\n\nGiven that some of these variables are obviously (or at least intuitively) connected to each other or turf type, isolating interaction effects between pairs of categorical features might help determine whether the significance of any of them depends on another."},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nmodel_InjCouple = sm.Logit.from_formula('Inj ~ SynTurf', data=mlDF).fit()\nprint(model_InjCouple.pvalues)\nprint()\n\nfor cat_ft in features_cat:\n    model_InjCouple = sm.Logit.from_formula('Inj ~ SynTurf*' + cat_ft, data=mlDF).fit()\n    print(model_InjCouple.pvalues)\n    print()\n    \nfor cat_ft1 in features_cat:\n    for cat_ft2 in features_cat:\n        if cat_ft1 != cat_ft2:\n            print(cat_ft1, '*', cat_ft2)\n            both1 = mlDF[(mlDF[cat_ft1] == 1) & (mlDF[cat_ft2] == 1)].shape[0]\n            both0 = mlDF[(mlDF[cat_ft1] == 0) & (mlDF[cat_ft2] == 0)].shape[0]\n            first1 = mlDF[(mlDF[cat_ft1] == 1) & (mlDF[cat_ft2] == 0)].shape[0]\n            second1 = mlDF[(mlDF[cat_ft1] == 0) & (mlDF[cat_ft2] == 1)].shape[0]\n            if(both1 == 0 or both0 == 0 or first1 == 0 or second1 == 0):\n                print(\"One feature is a subset of the other, no interaction effect necessary\")\n            else:\n                model_InjCouple = sm.Logit.from_formula('Inj ~' + cat_ft1 + '*' + cat_ft2, data=mlDF).fit()\n                print(model_InjCouple.pvalues)\n            print()\nprint(datetime.datetime.now() - startDT, 'to test interactions between categorical features') ## about 15s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lots of output to read through, but the key takeaways were:\n* Each of the 3 offensive line/guard variables depend heavily on turf type for their significance, based on their p-values rising while the p-value for \"SynTurf\" remains significant when the interaction effect is isolated\n* As expected, those 3 variables are subsets of one another, which means that can't be modeled together; for example, there are no record where the player was an offensive guard for the play but not on the offensive line.\n\nThese interactions can also easily be visualized with the same plot format as was used earlier:"},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nplot_fts = ['SynTurf'] + features_cat\nfor i_1, feature_1 in enumerate( plot_fts ):\n    for i_2, feature_2 in enumerate( plot_fts ):\n        if i_1 < i_2:\n            simple_mlDF = mlDF[['Inj', feature_1, feature_2]].copy(deep=True)\n            simple_mlDF['plays'] = 1\n\n            pivot_for_interax = simple_mlDF.groupby([feature_1, feature_2]).sum()\n\n            for col in pivot_for_interax.columns:\n                pivot_for_interax.loc[:, col] = pivot_for_interax[col] / pivot_for_interax['plays']\n\n            pivot_for_interax.drop(labels='plays', axis='columns', inplace=True)\n            #features = list(pivot_for_interax.columns)\n            axes = list(pivot_for_interax.index.names)\n            pivot_for_interax.reset_index(inplace=True)\n            for col in axes:\n                pivot_for_interax.loc[:, col] = pivot_for_interax[col].astype('str')\n\n            #print(pivot_for_interax)\n\n            ## https://www.statsmodels.org/dev/generated/statsmodels.graphics.factorplots.interaction_plot.html\n\n            interax_plot = interaction_plot(x = pivot_for_interax[feature_1], trace = pivot_for_interax[feature_2], response = pivot_for_interax['Inj'], plottype = 'both')\n            interax_plot.suppressComposite #output plot only once, otherwise two copies are displayed\nprint(datetime.datetime.now() - startDT, 'to plot interactions between categorical features') ## about 1.0s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Continuous variables, individual significance testing\n\nThe continuous variables that were strongly correlated with injury occurences on an individual basis can be tested for significance in a similar manner:"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Continuous:', features_con)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"First, the significance of each continuous feature's relationship, in terms of whether an increase in that feature is associated with an increase in injury occurence, can be tested individually."},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nfor con_ft in features_con:\n    model_InjCouple = sm.Logit.from_formula('Inj ~ ' + con_ft, data=mlDF).fit()\n    print(model_InjCouple.pvalues)\n    print()\nprint(datetime.datetime.now() - startDT, 'to test each continuous feature') ## about 10s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Again, lots of output due the number of features, but each continuous variable tested appears to have a significant relationship with injury occurence. There are noteworthy tiers within these variables though: \n* 'd_twist_avg', 'd_dir_avg', 'Temperature', 'd_twist_sum', and 'd_dir_sum' each had p-values between 0.02 to 0.04\n* 'd_s_avg', 'd_s_sum', and 'rel_y_final' had p-values an order lower (0.00#... ; more significant)\n    * the two 'd_s' features are measures of acceleration experienced, which matches a basic physics intuition that higher acceleration equals higher force equals more injuries\n* 'd_o_sum' and 'd_o_avg' had p-values even lower p-values\n    * these features are measures of spinning experienced, which matches a biological common sense of lower-body injuries generally occuring when a player's feet/legs aren't kept parallel to one another\n\n## Continuous features, interaction effects\n\nSame as the categorical variables, the interactions between these continuous variables and turf type, as well as with each other, should be examined to determine if any feature depends on another for significance. Interaction plots are helpful for continuous variables as well, but with 250k+ records, they would be resource-intensive to produce."},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nmodel_InjCouple = sm.Logit.from_formula('Inj ~ SynTurf', data=mlDF).fit()\nprint(model_InjCouple.pvalues)\nprint()\n\nfor con_ft in features_con:\n    model_InjCouple = sm.Logit.from_formula('Inj ~ SynTurf*' + con_ft, data=mlDF).fit()\n    print(model_InjCouple.pvalues)\n    print()\nprint(datetime.datetime.now() - startDT, 'to test interaction effects between turf type and continuous features') ## about 10s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Again, key takeaways from that wall of output:\n* The significance of almost every continuous variable is muddled when interacting with turf type (\"SynTurf\"), although some were still close to being significant after the interaction effect was isolated, with others were almost entirely insignificant (p-values near 1.0)\n* \"d_o_sum\" and \"d_o_avg\" were both still significant after their interactions with turf type were isolated; while neither have a significant interaction with turf type, they differ in that the significane of turf type is actually lessened by \"d_o_sum\", while \"d_o_avg\" appears to be independent from turf type\n    * This suggests that turf type has a relatively stronger correlation with \"d_o_sum\" than \"d_o_avg\", which would likely be due to another relatively strong correlation with the length of plays (\"obs\" in mlDF). Both mini-hypotheses will be checked briefly below via correlation calculation and box-plot distribution comparison."},{"metadata":{"trusted":true},"cell_type":"code","source":"print(mlDF['SynTurf'].corr(mlDF['d_o_avg']) )\nbox_data = [mlDF.loc[mlDF['SynTurf'] == 0, 'd_o_avg'], mlDF.loc[mlDF['SynTurf'] == 1, 'd_o_avg']]\ndisplay(plot.boxplot(box_data, vert=False, showfliers = False))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(mlDF['SynTurf'].corr(mlDF['d_o_sum']) )\nbox_data = [mlDF.loc[mlDF['SynTurf'] == 0, 'd_o_sum'], mlDF.loc[mlDF['SynTurf'] == 1, 'd_o_sum']]\ndisplay(plot.boxplot(box_data, vert=False, showfliers = False))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(mlDF['SynTurf'].corr(mlDF['obs']) )\nbox_data = [mlDF.loc[mlDF['SynTurf'] == 0, 'obs'], mlDF.loc[mlDF['SynTurf'] == 1, 'obs'] ]\ndisplay(plot.boxplot(box_data, vert=False, showfliers = False))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Both mini-hypotheses appear to be confirmed! All 3 correlations are all pretty weak in absolute terms, but binary variables and continuous variables are difficult to assess with correlation. The box-plots illustrate the difference between natural turf and synthetic turf plays is much more noticable for \"d_o_sum\" and \"obs\" than for \"d_o_avg\".\n\nOne more extremely long output is needed to examine the interaction effects between each of the continuous variables - see below, with takeaways to follow."},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nfor i_ft1, con_ft1 in enumerate(features_con):\n    for i_ft2, con_ft2 in enumerate(features_con):\n        if i_ft1 < i_ft2:\n            print(con_ft1, '*', con_ft2)\n            model_InjCouple = sm.Logit.from_formula('Inj ~' + con_ft1 + '*' + con_ft2, data=mlDF).fit()\n            print(model_InjCouple.pvalues)\n            print()\nprint(datetime.datetime.now() - startDT, 'to test interactions within continuous features') ## about 45s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Similar to the takeaways for the categorical variables' interaction effects with one another:\n* Most variable pairs muddled each other's significance, although to varying degrees\n* The following variables' relationship with injury occurence remained significant after isolating interaction effect with at least one other variable (note that these are not necessarily reciprocal):\n    * \"d_twist_avg\": \"d_s_avg\", \"rel_y_final\", \n    * \"d_s_avg\": \"d_twist_avg\", \"d_dir_avg\"\n    * \"d_o_avg\": \"d_twist_avg\", \"d_dir_avg\", \"d_s_avg\", \"rel_y_final\", \"d_o_sum\"\n    * \"d_dir_avg\": \"rel_y_final\", \"d_o_avg\"\n    * \"rel_y_final\": \"d_dir_avg\", \"d_dir_sum\", \"d_twist_sum\", \"d_s_sum\", \"d_o_sum\"\n    * \"Temperature\": \"d_s_sum\"\n    * \"d_s_sum\": \"Temperature\"\n    * \"d_twist_sum\": \"rel_y_final\"\n    * \"d_o_sum\": \"d_twist_sum\", \"d_dir_sum\", \"d_s_avg\", \"d_s_sum\", \"rel_y_final\", \"d_o_avg\"\n    * \"d_dir_sum\": \"rel_y_final\"\n    * \"d_s_avg\": \"rel_y_final\", \"d_o_sum\", \"d_o_avg\"\n    * \"d_s_sum\": \"rel_y_final\", \"d_o_sum\"\n* The only interaction that was significant was d_s_sum:rel_y_final (p-value 0.008801), but both variables' relationship with injury occurence was still easily significant after isolating their interaction.\n    * This suggests a very intuitive relationship between a players' experienced acceleration and their proximity to the sidelines; i.e. the players who are changing speeds most often are generally playing closer to the sidelines, while lineman on both offense and defense stay closer to the middle and don't change speeds as much. This correlation will be calculated below to confirm this mini-hypothesis.\n    * The only other two interactions that were even close to being significant were d_twist_avg:d_s_avg (p-value 0.089105) and d_twist_sum:rel_y_final (p-value 0.093208); again, both are intuitive relationships, but do not need to be explored further due to lack of true significance."},{"metadata":{"trusted":true},"cell_type":"code","source":"mlDF['d_s_sum'].corr(mlDF['rel_y_final'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Another mini-hypothesis confirmed!\n\n## Final classified model effectiveness\n\nBefore attempting to a final model, it's worth remembering the miniscule amount of variation in injury occurence that is attempting to be explained. The signal-to-noise ratio is extremely low, and none of the individual factors reviewed so far have been truly strong predictors individually.\n\nBased on the factors identified as the (relatively) strongest predictors, a logistic regression model can be fitted to a training subset of the plays and then tested on the remaining test set. As a logistic regression, its predictions are probabilities of injuries occuring on the test plays, and the actual classification as Inj = 0 or Inj = 1 depends on the threshold. \n\nTo evaluate the model's performance, many thresholds are processed to calculate numbers of True Positives (TPs) and False Positives (FP), which are then plotted against each other to form a Receiver Operating Characteristic (ROC) curve. Ideally a classifier has a greater proportion of TP than FP at all thresholds and will be above a diagonal line on this plot. Even in that case though, the statistical metric for classifier quality is the Area Under the Curve (AUC), which can be interpreted on a scale from 1 to 0:\n* 1.0 is best, perfect alignment of predicted classification to actual classification\n* 0.0 is worst, where every record is classified incorrectly\n* 0.5 is NOT considered middling performance, rather it means the model is unable to differentiate the classes at all"},{"metadata":{"trusted":true},"cell_type":"code","source":"startDT = datetime.datetime.now()\nmodel_fts = ['SynTurf', 'd_o_avg', 'd_o_sum', 'rel_y_final', 'd_s_sum', 'd_s_avg']\nX_train, X_test, y_train, y_test = train_test_split(mlDF[model_fts], mlDF['Inj'], test_size=0.33, random_state=42)\nprint(sum(y_train == 1), 'injuried in training set,', sum(y_test == 1), 'injuried in test set',)\nlogreg = LogisticRegression(penalty='none', solver='lbfgs')\nlogreg.fit(X_train, y_train)\ny_pred = logreg.predict(X_test)\nlogit_roc_auc = roc_auc_score(y_test, logreg.predict(X_test))\nprint('AUC:', logit_roc_auc)\nfpr, tpr, thresholds = roc_curve(y_test, logreg.predict_proba(X_test)[:,1])\nplot.figure()\nplot.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % logit_roc_auc)\nplot.plot([0, 1], [0, 1],'r--')\nplot.xlim([0.0, 1.0])\nplot.ylim([0.0, 1.05])\nplot.xlabel('False Positive Rate')\nplot.ylabel('True Positive Rate')\nplot.title('Receiver operating characteristic')\nplot.legend(loc=\"lower right\")\nplot.savefig('Log_ROC')\nplot.show()\nprint(datetime.datetime.now() - startDT, 'to fit, test, and evaluate logistic regression model') ## about 10s on Kaggle","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As expected, the AUC of this model is 0.5.\n\n## Conclusions\n\nSpeaking of hypotheses, it time to revisiting those stated at the beginning of the workbook:\n* There is a significant increase in injury occurence for plays on synthetic turf vs. plays on natural turf.\n    * CONFIRMED based on test result p-value of 0.04355063587737655\n* However, due to the extremely low proportion of plays with injuries overall, turf type alone will barely explain any of the variation in injury occurence. \n    * CONFIRMED based on an R^2 value of 0.0000152604; although it is worth noting, as was stated immediately prior to the calculation of that R^2, there's not much variance in injury occurence among the plays to be explained (\"needle-in-a-haystack\" problem)\n* While no other individual play-level factor will explain much more of the variation in injury occurence, those that do may have significant interaction effects with turf type, suggesting that synthetic turf's elevates injury risk through amplifying other risk factors.\n    * UNCONFIRMED\n    * No game-level factors such as whether the field was exposed (based on StadiumType) or wet (based on weather) were found to be noteworthy predictors\n    * No interactions with turf type were found to be significant, but most variables' significance was reduced when their interaction with turf type was isolated and factored out, suggesting that there is some sort of more complex interaction involving additional confounding variables affecting the variables' relationship with injury occurence. Modeling interactions between 3+ variables is a much more involved process and was left for future exploration.\n    * When including interactions between variables other than turf type, the only interaction found to be significant was between 'd_s_sum' and 'rel_y_final'.\n    * The best features by correlation strength and significance testing p-value were:\n        * 'd_o_avg', 'd_o_sum' - how much a player \"spun\" during a play\n            * These were more important injury risks than turf type and didn't appear to be amplified by synthetic turf\n        * 'rel_y_final' - distance from the middle of the field (i.e. a line perpendicular to the yardlines), inverse of proximity to sideline\n            * This was also an important injury risk, but possibly related to turf type\n        * 'd_s_sum', 'd_s_avg' - how much a player accelerated during a play\n            * These were important injury risks, but it was unclear whether they were related-to / independent-from turf type\n        * (no categorical features were as predictive as these continuous variables, not even turf type)\n        \n## Final Thoughts\n\nSynthetic turf does appear to be related to a higher risk for non-contact lower-extremity injuries, but a few other simple features calculated from player movement data have relationships of equal or greater statistical significance. Without access to cost information on the removal/replacement of synthetic turf vs. these injuries, it's difficult to say whether a decision can or should be made regarding turf types. \n\nIt is very clear though, that removing synthetic turf from the NFL is not a silver-bullet solution to this injury issue. More research is needed to monitor the injury risk of players' spin, proximity to the sideline, and acceleration. While there is probably no way to reduce these tendencies without harming the game of football; there may be policies, equipment, or other steps that can be taken to make them safer. If nothing else, based on this movement tracking data, players who experience these risk factors at extreme levels should be examined soon after for subtle injuries that might be exacerbated by further risky movement."},{"metadata":{},"cell_type":"markdown","source":"# OVERALL RUN TIME"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Entire workbook runs in ', datetime.datetime.now() - overall_startDT)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.4"}},"nbformat":4,"nbformat_minor":1}