{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Using Deep Learning on Movement Data for Injury Prediction\nNow that we have an understanding of what variables correlate with acute injuries amongst NFL players, we investigate how repeated movement patterns can lead to physical injury.\n\nGiven just historical movement and play data of a player over the course of several seasons, **our method is able to predict with almost 70% accuracy whether or not that player was injured over the course of those seasons.**\n\nOur pipeline can be broken down into three stages:\n1. Pre-processing: loads, cleans and stores the data that will be used in our model\n2. Timeseries feature extraction: extracts relevant data from our cleaned timeseries data\n3. Model training: uses features extracted in the previous step to train a deep learning model"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"## Step 1: Preprocessing\n\n### Loading data\nWe begin by reading in our data files and transforming them into a format that we can process"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\nfrom tqdm import tqdm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# load in our data\ntracking_df = pd.read_csv(\"../input/nfl-playing-surface-analytics/PlayerTrackData.csv\")\ninjury_df = pd.read_csv(\"../input/nfl-playing-surface-analytics/InjuryRecord.csv\")\nplay_df = pd.read_csv(\"../input/nfl-playing-surface-analytics/PlayList.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"players = play_df.PlayerKey.unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tracking_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"play_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Data Cleaning\nFirst we take just a subset of our columns to avoid training our model on redundant features."},{"metadata":{"trusted":true},"cell_type":"code","source":"cat_names = ['StadiumType', 'FieldType', 'Weather', 'PlayType', 'PositionGroup', 'RosterPosition']\n\nmodel_data_df = play_df[cat_names + ['PlayerKey', 'PlayKey']]\nmodel_data_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we transform columns representing categorical data into numpy's categorical data type. This allows us to treet the column labels as numbers when building deep learning models. This is particularly useful for embeddings."},{"metadata":{"trusted":true},"cell_type":"code","source":"# convert all categories to integers and normalize such that the minimum value is 0\nfrom fastai.tabular.transform import Categorify\n\ntfm = Categorify(cat_names, [])\ntfm(model_data_df)\nfor cat_name in cat_names:\n    codes = model_data_df[cat_name].cat.codes\n    model_data_df[cat_name] = codes - min(codes)\nmodel_data_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Exposing data by player\nHere we define helper functions that make it easy to access all of the information relevant to a particular player"},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_plays_for_player(player):\n    \"\"\"\n    Returns data for all of the plays associated with the player\n    \"\"\"\n    return model_data_df[model_data_df.PlayerKey == player].drop(['PlayerKey'], axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`tracking_df` is massive and incredibly slow to access. To speed up our calculations we cache movement data associated with every player to disk"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nPLAYER_DATA_CACHE_PATH = \"../input/tracking-cache/player_data_tracking\"\n\n# makes sure the local directory for the cache exists\nif not os.path.exists(PLAYER_DATA_CACHE_PATH):\n    os.makedirs(PLAYER_DATA_CACHE_PATH)\n\ndef get_tracking_for_player(player):\n    \"\"\"\n    Returns tracking data stored in a local json file. If the file\n    doesn't exist the function creates it\n    \"\"\"\n    file_cache_path = f\"{PLAYER_DATA_CACHE_PATH}/{player}.csv\"\n    if not os.path.exists(file_cache_path):\n        small_tracking = tracking_df[tracking_df.PlayKey.str.startswith(str(player))]\n        small_tracking.to_csv(file_cache_path)\n    return pd.read_csv(file_cache_path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"get_tracking_for_player(players[0]).head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_data_for_player(player):\n    \"\"\"\n    Combines all of the data associated with the plays for an individual player\n    \"\"\"\n    plays = get_plays_for_player(player)\n    tracking = get_tracking_for_player(player)\n    \n    group = tracking.groupby(\"PlayKey\")\n    avg_speed = group.mean()['s'] * 100\n    total_distance = group.sum()['dis']\n    \n    data = plays.merge(avg_speed, on=\"PlayKey\").merge(total_distance, on=\"PlayKey\").drop('PlayKey', axis=1)\n    data['id'] = player\n    data['time'] = data.index\n    \n    return data\n\nget_data_for_player(players[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Iterate over all of the players in our dataset and populate the local cache"},{"metadata":{},"cell_type":"markdown","source":"## Step 2: Timeseries Feature Extraction\nWe use [TSFresh](https://tsfresh.readthedocs.io/en/latest/) to automatically extract relevant features from our dataset. This converts our 3-dimensional data to 2-dimensional data.\n\nWe begin by converting our data into the format needed by TSFResh."},{"metadata":{"trusted":true},"cell_type":"code","source":"fresh_data = pd.concat([get_data_for_player(player) for player in tqdm(players)])\nfresh_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from tsfresh import extract_relevant_features\nfrom tsfresh.feature_extraction import MinimalFCParameters","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We approach the problem as one of binary classification, labeling players that were injured with \"1\" and players that were not injured with \"0\"."},{"metadata":{"trusted":true},"cell_type":"code","source":"injured_players = set(injury_df.PlayerKey)\ny = pd.DataFrame(index=players)\ny['target'] = [int(player in injured_players) for player in players]\ny.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We use the `extract_relevant_features` method provided by the `TSFresh` library to automatically extract hundreds of features from our timeseries data. `extract_relevant_features` automatically runs correlation tests to determine which features are relevant and retains only the important features."},{"metadata":{"trusted":true},"cell_type":"code","source":"extracted_features = extract_relevant_features(\n    fresh_data,\n    y.target,\n    column_id=\"id\",\n    column_sort=\"time\",\n    default_fc_parameters=MinimalFCParameters()\n)\nextracted_features['target'] = y.target\nextracted_features.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Part 3: Model Training\nWe use tabular deep learning to build a binary classifier. Our model is built on top of [FastAI](https://www.fast.ai/)'s tabular deep learning architecture."},{"metadata":{"trusted":true},"cell_type":"code","source":"from fastai.tabular import *\nfrom fastai.callbacks import SaveModelCallback\nfrom sklearn.model_selection import train_test_split","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creating our model\nFirst we create a `DataBunch`, a concept in the fastai library that acts as a wrapper around your dataset. We choose a random subset of our players to be the validation set"},{"metadata":{"trusted":true},"cell_type":"code","source":"_, valid = train_test_split(range(len(players)))\ndata = TabularDataBunch.from_df(\"./models\", extracted_features, 'target', valid_idx=valid)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# this choice of layer sizes comes from experimental success in similar tabular deep learning problems\nLAYERS = [200, 100]\n\ndef create_learner():\n    \"\"\"\n    We define this as a function as it allows us to later load models from disk easily\n    later in the notebook\n    \"\"\"\n    return tabular_learner(data, layers=LAYERS, metrics=accuracy)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = create_learner()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We follow [Leslie Lamport's technique](https://arxiv.org/pdf/1506.01186.pdf) for learning rate finding to determine the optimal hyperparamters.\n\nWe follow common practice in the data science community of:\n1. Finding the lowest point in the graph below\n2. Dividing that value by 10\n3. Using that value as our learning rate\nIn the graph below we see that the minimum occurs around 1e-1, so we set our learning rate to 1e-2."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()\nlearn.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lr = 1e-2\nN_CYCLES = 10","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# clear the existing model cache\n!rm -rf ./models/models","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training our model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# train our model and save the best result\nlearn.fit_one_cycle(\n    N_CYCLES,\n    lr,\n    callbacks=[SaveModelCallback(learn, every=\"improvement\", monitor=\"accuracy\")]\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We get our best result on epoch 6 with a model that achieves 70% accuracy."},{"metadata":{"trusted":true},"cell_type":"code","source":"best_model = create_learner()\nbest_model.load(\"bestmodel\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Analyzing the results\nLooking at our confusion matrix we come to the following conclusions:\n* Our model has very few false positives\n* Our model is much better at predicting when a player will not get injured\n* We need a lot more data before we can definitively say that this appraoch effectively predicts injury likelihoods"},{"metadata":{"trusted":true},"cell_type":"code","source":"interpretation = best_model.interpret()\ninterpretation.plot_confusion_matrix()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}