{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np\nimport pandas as pd \n\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sb\nsb.set_style('whitegrid')\nsb.set(font_scale = 1.4)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:22:59.534832Z","iopub.execute_input":"2021-11-03T00:22:59.535647Z","iopub.status.idle":"2021-11-03T00:22:59.545699Z","shell.execute_reply.started":"2021-11-03T00:22:59.535599Z","shell.execute_reply":"2021-11-03T00:22:59.544832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#games = '../input/nfl-big-data-bowl-2022/games.csv'\nplayers = '../input/nfl-big-data-bowl-2022/players.csv'\n#plays = '../input/nfl-big-data-bowl-2022/plays.csv'\ntracking_2018 = '../input/nfl-big-data-bowl-2022/tracking2018.csv'\n#tracking_2019 = '../input/nfl-big-data-bowl-2022/tracking2019.csv'\n#tracking_2020 = '../input/nfl-big-data-bowl-2022/tracking2020.csv'","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:00.439229Z","iopub.execute_input":"2021-11-03T00:23:00.440085Z","iopub.status.idle":"2021-11-03T00:23:00.446344Z","shell.execute_reply.started":"2021-11-03T00:23:00.44003Z","shell.execute_reply":"2021-11-03T00:23:00.444647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Towards ranking players \n\nThis notebook focuses on the players and tracking data and hopefully will help people who will work on ranking players and their performance. Some new features are derived from the tracked data and merged to the players data frame. Finally, a few visualisations and some observations can be found at towards the end. \n\nGood luck to all 🏈\n\n","metadata":{}},{"cell_type":"code","source":"players_df = pd.read_csv(players)\nplayers_df","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:02.039866Z","iopub.execute_input":"2021-11-03T00:23:02.040202Z","iopub.status.idle":"2021-11-03T00:23:02.075942Z","shell.execute_reply.started":"2021-11-03T00:23:02.040167Z","shell.execute_reply":"2021-11-03T00:23:02.075074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"'birthDate' is not usefull as it is, it is coverted to the age (int) for each player. Also, the height feature is of string type with inconsistencies in its format (i.e. not always <feet> - <inches>). I am converting it to inches for now.","metadata":{}},{"cell_type":"code","source":"# convert birthday to years old\nplayers_df['birthDate'] = pd.to_datetime(players_df['birthDate'], infer_datetime_format = True) \nplayers_df['birthDate'] = np.round((pd.Timestamp.now() - players_df['birthDate']).dt.days/365)\nplayers_df = players_df.rename(columns={'birthDate': 'age'})\n\n# fix and conver'height' to number (inches)\ni =0\nfor h in players_df['height']:\n    if '-' in h:\n        h.split('-')\n        tot_inch = int(h[0])*12+int(h[2])\n        players_df.at[i,'height'] = tot_inch\n        \n    else:\n        tot_inch = int(h[0])*12+int(h[1])\n        players_df.at[i,'height'] = tot_inch     \n    i=i+1","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:03.116636Z","iopub.execute_input":"2021-11-03T00:23:03.117192Z","iopub.status.idle":"2021-11-03T00:23:03.163002Z","shell.execute_reply.started":"2021-11-03T00:23:03.117154Z","shell.execute_reply":"2021-11-03T00:23:03.161807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"players_df","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:03.446916Z","iopub.execute_input":"2021-11-03T00:23:03.448064Z","iopub.status.idle":"2021-11-03T00:23:03.471323Z","shell.execute_reply.started":"2021-11-03T00:23:03.448008Z","shell.execute_reply":"2021-11-03T00:23:03.469659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, let's take a look at the tracking data for 2018:","metadata":{}},{"cell_type":"code","source":"tracking_2018_df = pd.read_csv(tracking_2018)\ntracking_2018_df","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:04.051667Z","iopub.execute_input":"2021-11-03T00:23:04.052208Z","iopub.status.idle":"2021-11-03T00:23:37.803441Z","shell.execute_reply.started":"2021-11-03T00:23:04.052168Z","shell.execute_reply":"2021-11-03T00:23:37.801721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We observe that the timestep from the tracker data is every one second, from that we can derive information on how much each player played. Follow the code bellow for a step-by-step exploration of the data:","metadata":{}},{"cell_type":"code","source":"df1 = tracking_2018_df.groupby([\"nflId\",'gameId'])\n#how many seconds did each player played in each game?\nans = df1['time'].count().to_frame()\nans","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:37.805743Z","iopub.execute_input":"2021-11-03T00:23:37.806097Z","iopub.status.idle":"2021-11-03T00:23:39.098329Z","shell.execute_reply.started":"2021-11-03T00:23:37.806063Z","shell.execute_reply":"2021-11-03T00:23:39.096757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In order to create a feature which I can merge in the 'players' data frame, we aggregate the above:","metadata":{}},{"cell_type":"code","source":"# In how many games did each player played in total (in a given year)?\nans.groupby('nflId').count()","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-11-03T00:23:39.100454Z","iopub.execute_input":"2021-11-03T00:23:39.100877Z","iopub.status.idle":"2021-11-03T00:23:39.120261Z","shell.execute_reply.started":"2021-11-03T00:23:39.100824Z","shell.execute_reply":"2021-11-03T00:23:39.119471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#merge with players_pd..\nplayers_df = pd.merge(players_df, ans.groupby('nflId').count(), on=['nflId'])\nplayers_df = players_df.rename(columns={'time': 'gamesPlayed2018'})","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:39.122765Z","iopub.execute_input":"2021-11-03T00:23:39.123486Z","iopub.status.idle":"2021-11-03T00:23:39.137309Z","shell.execute_reply.started":"2021-11-03T00:23:39.123437Z","shell.execute_reply":"2021-11-03T00:23:39.136398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# how many minutes did each played played in total (in a given year)?\nnp.round(ans.groupby('nflId').sum()/60, decimals=1)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:39.139512Z","iopub.execute_input":"2021-11-03T00:23:39.140223Z","iopub.status.idle":"2021-11-03T00:23:39.163111Z","shell.execute_reply.started":"2021-11-03T00:23:39.140174Z","shell.execute_reply":"2021-11-03T00:23:39.162211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#merge with players_pd..\nplayers_df = pd.merge(players_df, np.round(ans.groupby('nflId').sum()/60, decimals=1), on=['nflId'])\nplayers_df = players_df.rename(columns={'time': 'minutesPlayed2018'})","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:39.164928Z","iopub.execute_input":"2021-11-03T00:23:39.165657Z","iopub.status.idle":"2021-11-03T00:23:39.188217Z","shell.execute_reply.started":"2021-11-03T00:23:39.165595Z","shell.execute_reply":"2021-11-03T00:23:39.187456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# which plays did he play in each game?  -not using this for now..\ndf1['playId'].unique().to_frame()","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:39.189769Z","iopub.execute_input":"2021-11-03T00:23:39.190604Z","iopub.status.idle":"2021-11-03T00:23:40.723182Z","shell.execute_reply.started":"2021-11-03T00:23:39.190556Z","shell.execute_reply":"2021-11-03T00:23:40.721902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, we will extract some useful features regarding the performance of the players in the game. We assume that: \n\n * The average speed the player has is a good indicator on how much distance he is covering in his play time i.e. how mobile he is overall.  \n * the 90%the quantile of their top acceleration is a good indicatior of their explosiveness and agility in the field - we don't want to consider only the max() value to avoid being influenced by a single extreme outlier measurement. Neverthless, we are interested only on the tail of the distribution (q90%) as it is sensible that explosive moves (= high acceleration) will occur over a small fraction of the tracked period.","metadata":{}},{"cell_type":"code","source":"# What was the average speed (y/sec) of each player in each game ?\ndf1[['s']].mean()","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:40.725622Z","iopub.execute_input":"2021-11-03T00:23:40.726083Z","iopub.status.idle":"2021-11-03T00:23:40.883131Z","shell.execute_reply.started":"2021-11-03T00:23:40.726027Z","shell.execute_reply":"2021-11-03T00:23:40.881486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that, more or less, the average speed of the players stays the same in each game.","metadata":{}},{"cell_type":"code","source":"# What was the average speed (y/sec) of each player in 2018? \ndf1[['s']].mean().groupby('nflId').mean()","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-11-03T00:23:40.885419Z","iopub.execute_input":"2021-11-03T00:23:40.885838Z","iopub.status.idle":"2021-11-03T00:23:41.041503Z","shell.execute_reply.started":"2021-11-03T00:23:40.88579Z","shell.execute_reply":"2021-11-03T00:23:41.04009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#merge with players_pd..\nplayers_df = pd.merge(players_df, df1[['s']].mean().groupby('nflId').mean(), on=['nflId'])\nplayers_df = players_df.rename(columns={'s': 'averageSpeed2018'})","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:41.044846Z","iopub.execute_input":"2021-11-03T00:23:41.045513Z","iopub.status.idle":"2021-11-03T00:23:41.195611Z","shell.execute_reply.started":"2021-11-03T00:23:41.045466Z","shell.execute_reply":"2021-11-03T00:23:41.194461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# What was  the top speed (y/sec) and top acceleration (y/sec^2) for each player in each game in 2018?\ndf1[['s','a']].quantile(q=0.95)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:23:41.197865Z","iopub.execute_input":"2021-11-03T00:23:41.198601Z","iopub.status.idle":"2021-11-03T00:23:52.592148Z","shell.execute_reply.started":"2021-11-03T00:23:41.198541Z","shell.execute_reply":"2021-11-03T00:23:52.590762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# What was  the top speed (y/sec) and top acceleration (y/sec^2) for each player in 2018?\ndf1[['s','a']].quantile(q=0.95).groupby('nflId').mean()","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-11-03T00:23:52.593625Z","iopub.execute_input":"2021-11-03T00:23:52.59393Z","iopub.status.idle":"2021-11-03T00:24:03.949505Z","shell.execute_reply.started":"2021-11-03T00:23:52.593896Z","shell.execute_reply":"2021-11-03T00:24:03.948257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#merge with players_pd..\nplayers_df = pd.merge(players_df, df1[['s']].quantile(q=0.95).groupby('nflId').mean(), on=['nflId'])\nplayers_df = players_df.rename(columns={'s': 'averageTopSpeed2018'})\n\nplayers_df = pd.merge(players_df, df1[['a']].quantile(q=0.95).groupby('nflId').mean(), on=['nflId'])\nplayers_df = players_df.rename(columns={'a': 'averageTopAcc2018'})","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:24:03.950951Z","iopub.execute_input":"2021-11-03T00:24:03.95126Z","iopub.status.idle":"2021-11-03T00:24:15.332168Z","shell.execute_reply.started":"2021-11-03T00:24:03.951228Z","shell.execute_reply":"2021-11-03T00:24:15.330825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see that some of the top speeds are impressive (1yard = 0.9m). For reference, Usain Bolt reached an astounding 12.3 meters per second when he did his record.","metadata":{}},{"cell_type":"code","source":"players_df","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:24:15.333785Z","iopub.execute_input":"2021-11-03T00:24:15.334124Z","iopub.status.idle":"2021-11-03T00:24:15.36962Z","shell.execute_reply.started":"2021-11-03T00:24:15.334087Z","shell.execute_reply":"2021-11-03T00:24:15.368738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"...and now some visualizations - feel free to mix and match features to discover new insigths! ","metadata":{}},{"cell_type":"code","source":"sb.relplot(x=\"weight\", y=\"height\", size=\"age\",\n            sizes=(50, 1000), alpha=0.4, palette=\"muted\",\n            height=12, data=players_df)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:24:15.371056Z","iopub.execute_input":"2021-11-03T00:24:15.371663Z","iopub.status.idle":"2021-11-03T00:24:16.492596Z","shell.execute_reply.started":"2021-11-03T00:24:15.371603Z","shell.execute_reply":"2021-11-03T00:24:16.490982Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* At least 4 body type-clusters can be identified. Palyer have different roles to play in the course of the game and their physical qualities are optimised for that. There seems to be a a distinct cluster of short/light players with well defined height/weight requirements. Most of the players are found in the central area and we can see a distinct group of taller players, distributed over a large ramge of weights. \n* Age seems to be distributed over all body types.","metadata":{}},{"cell_type":"code","source":"sb.relplot(x=\"gamesPlayed2018\", y=\"minutesPlayed2018\", size=\"weight\",\n            sizes=(50, 2000), alpha=0.4, palette=\"muted\",\n            height=12, data=players_df)\n\nsb.relplot(x=\"minutesPlayed2018\", y=\"weight\", size=\"age\",\n            sizes=(50, 3060), alpha=0.3, palette=\"muted\",\n            height=12, data=players_df)\n\nsb.relplot(x=\"minutesPlayed2018\", y=\"height\", size=\"age\",\n            sizes=(50, 3060), alpha=0.3, palette=\"muted\",\n            height=12, data=players_df)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:24:16.494666Z","iopub.execute_input":"2021-11-03T00:24:16.495023Z","iopub.status.idle":"2021-11-03T00:24:20.898406Z","shell.execute_reply.started":"2021-11-03T00:24:16.494987Z","shell.execute_reply":"2021-11-03T00:24:20.897311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This figure might seem obvious because it makes sense that the total time spent in the game would be proportional to the number of games played but I had to make sure that it is.\n* Linear proportionality between the number of games played and the total time spent in game - expected.\n* Heavier players spend less time in the game but weight does not limit how many games they play.\n* Taller players also tend to spend less time on the court as well.","metadata":{}},{"cell_type":"code","source":"sb.relplot(x=\"averageTopSpeed2018\", y=\"averageTopAcc2018\", size=\"weight\",\n            sizes=(50, 300), alpha=1, palette=\"muted\",\n            height=12, data=players_df)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T00:24:20.899952Z","iopub.execute_input":"2021-11-03T00:24:20.900292Z","iopub.status.idle":"2021-11-03T00:24:22.065849Z","shell.execute_reply.started":"2021-11-03T00:24:20.900255Z","shell.execute_reply":"2021-11-03T00:24:22.064508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* The most agile and explosive players tend to also be the fastest palyers on the court. \n* The majority of players seems to be consenrtater to two regions. Players tend to (relatively) either be fast and explosive edge, or slow. There are not a lot of palyers in between those two clusters.\n* Expectedly, heavy players tend to be on the slow end.","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}