{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O \npd.set_option('display.max_columns', 5000)\n\n# pretty pictures\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-10-16T19:12:27.761719Z","iopub.execute_input":"2021-10-16T19:12:27.761999Z","iopub.status.idle":"2021-10-16T19:12:27.771057Z","shell.execute_reply.started":"2021-10-16T19:12:27.761970Z","shell.execute_reply":"2021-10-16T19:12:27.770256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# [NFL BDB](https://www.kaggle.com/c/nfl-big-data-bowl-2022/overview)\n\n### Predicting Penalties on Punts and Kickoffs\n\nThis notebook is meant as an exploration of the NFL Bid Data Bowl data, as well as some general interest feature engineering and building basic classification models. It's purpose isn't to generate a useful model but more for my own Data Science interest and practice. \n\nThe main focus here is feature engineering using four of the NFL's special teams datasets, and using the engineered data to build a model that can predict whether or not a punt or a kickoff will result in a penalty. Of course, most plays *don't* result in a penalty, so as shown below, the models that get built will all be very good at predicting the plays without penalties, but not so good at predicting the plays that *do* result in penalties. Also, the model can't predict *which* team will get a penalty. \n\nLike I said, this isn't meant to be a useful model (yet?).\n\n[Part 1:](#part1) Data Exploration and Feature Engineering\n\n[Part 2:](#modelling) Modelling","metadata":{}},{"cell_type":"code","source":"# get the data\nscouting_data = pd.read_csv(\"../input/nfl-big-data-bowl-2022/PFFScoutingData.csv\")\ngames = pd.read_csv(\"../input/nfl-big-data-bowl-2022/games.csv\")\nplayers = pd.read_csv(\"../input/nfl-big-data-bowl-2022/players.csv\")\nplays = pd.read_csv(\"../input/nfl-big-data-bowl-2022/plays.csv\")","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:27.772739Z","iopub.execute_input":"2021-10-16T19:12:27.773446Z","iopub.status.idle":"2021-10-16T19:12:27.920070Z","shell.execute_reply.started":"2021-10-16T19:12:27.773403Z","shell.execute_reply":"2021-10-16T19:12:27.919363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Part 1: Data Exploration and Feature Engineering \n* [Games](#games-data)\n* [Plays](#plays-data-nav)\n* [Scouting](#scouting-data)","metadata":{}},{"cell_type":"markdown","source":"<a id=\"games-data\"></a>\n## Games Data\nThe Games dataset is the smallest dataset and very straightforward. \n* gameID is a primary key for joins\n* having gameDate and the week number may be redundant\n* gameTime will be changed to afternoon, evening, night","metadata":{}},{"cell_type":"code","source":"games","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:27.921304Z","iopub.execute_input":"2021-10-16T19:12:27.922119Z","iopub.status.idle":"2021-10-16T19:12:27.935828Z","shell.execute_reply.started":"2021-10-16T19:12:27.922086Z","shell.execute_reply":"2021-10-16T19:12:27.935122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"games['gameTimeEastern'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:27.937050Z","iopub.execute_input":"2021-10-16T19:12:27.937455Z","iopub.status.idle":"2021-10-16T19:12:27.953056Z","shell.execute_reply.started":"2021-10-16T19:12:27.937425Z","shell.execute_reply":"2021-10-16T19:12:27.952168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def categorize_start_time(hour):\n    if int(hour)<16:\n        return 'A'  # afternoon                       \n    elif int(hour)>=16 and int(hour)<19:\n        return 'E'  # evening\n    else:\n        return 'N'  # night","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:27.954975Z","iopub.execute_input":"2021-10-16T19:12:27.955187Z","iopub.status.idle":"2021-10-16T19:12:27.961807Z","shell.execute_reply.started":"2021-10-16T19:12:27.955162Z","shell.execute_reply":"2021-10-16T19:12:27.961169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_games(games_df):\n    # make a working copy\n    df = games_df.copy()\n    \n    # add a startPeriod column\n    df['startHour'] = games['gameTimeEastern'].str[0:2]\n    df['startPeriod'] = df['startHour'].map(lambda x: categorize_start_time(x))\n        \n    # remove the unnecessary ones\n    df = df.drop(['startHour', 'gameTimeEastern'], axis=1)\n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:27.963222Z","iopub.execute_input":"2021-10-16T19:12:27.963657Z","iopub.status.idle":"2021-10-16T19:12:27.975403Z","shell.execute_reply.started":"2021-10-16T19:12:27.963624Z","shell.execute_reply":"2021-10-16T19:12:27.974344Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_games(games)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:27.976572Z","iopub.execute_input":"2021-10-16T19:12:27.976921Z","iopub.status.idle":"2021-10-16T19:12:28.001686Z","shell.execute_reply.started":"2021-10-16T19:12:27.976867Z","shell.execute_reply":"2021-10-16T19:12:28.001068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"plays-data-nav\"></a>\n## Plays Data\n\nMuch more interesting. The field descriptions are from the [competition data page](https://www.kaggle.com/c/nfl-big-data-bowl-2022/data).\n\n* **gameId**: Game identifier, unique (numeric)\n* **playId**: Play identifier, not unique across games (numeric)\n* **playDescription**: Description of play (text)\n* **quarter**: Game quarter (numeric)\n* **down**: Down (numeric)\n* **yardsToGo**: Distance needed for a first down (numeric)\n* **possessionTeam**: Team punting, placekicking or kicking off the ball (text)\n* **specialTeamsPlayType**: Formation of play: Extra Point, Field Goal, Kickoff or Punt (text)\n* **specialTeamsPlayResult**: Special Teams outcome of play dependent on play type: Blocked Kick Attempt, Blocked Punt, Downed, Fair Catch, Kick Attempt Good, Kick Attempt No Good, Kickoff Team Recovery, Muffed, Non-Special Teams Result, Out of Bounds, Return or Touchback (text)\n* **kickerId**: nflId of placekicker, punter or kickoff specialist on play (numeric)\n* **returnerId**: nflId(s) of returner(s) on play if there was a special teams return. Multiple returners on a play are separated by a ; (text)\n* **kickBlockerId**: nflId of blocker of kick on play if there was a blocked field goal or blocked punt (numeric)\n* **yardlineSide**: 3-letter team code corresponding to line-of-scrimmage (text)\n* **yardlineNumber**: Yard line at line-of-scrimmage (numeric)\n* **gameClock**: Time on clock of play (MM:SS)\n* **penaltyCodes**: NFL categorization of the penalties that occurred on the play. Multiple penalties on a play are separated by a ; (text)\n* **penaltyJerseyNumbe**r: Jersey number and team code of the player committing each penalty. Multiple penalties on a play are separated by a ; (text)\n* **penaltyYards**: yards gained by possessionTeam by penalty (numeric)\n* **preSnapHomeScore**: Home score prior to the play (numeric)\n* **preSnapVisitorScore**: Visiting team score prior to the play (numeric)\n* **passResult**: Scrimmage outcome of the play if specialTeamsPlayResult is \"Non-Special Teams Result\" (C: Complete pass, I: Incomplete pass, S: Quarterback sack, IN: Intercepted pass, R: Scramble, ' ': Designed Rush, text)\n* **kickLength**: Kick length in air of kickoff, field goal or punt (numeric)\n* **kickReturnYardage**: Yards gained by return team if there was a return on a kickoff or punt (numeric)\n* **playResult**: Net yards gained by the kicking team, including penalty yardage (numeric)\n* **absoluteYardlineNumber**: Location of ball downfield in tracking data coordinates (numeric)","metadata":{}},{"cell_type":"code","source":"plays","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.002799Z","iopub.execute_input":"2021-10-16T19:12:28.003386Z","iopub.status.idle":"2021-10-16T19:12:28.042853Z","shell.execute_reply.started":"2021-10-16T19:12:28.003346Z","shell.execute_reply":"2021-10-16T19:12:28.041999Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"plays-data-nav\"></a>\n### Plays Data Feature Engineering\n\nMy Goal here is to predict whether or not there will be a penalty, so first we're going to add a penalty flag column (no pun intended). I'm also looking only at penalties on kickoffs and punts, so I'll filter out other plays.\nI'm going to get rid of some of these columns:\n* The playID and gameID are going to stay in order to join the other data sets\n* penaltyYards, penaltyCodes, and penaltyJerseyNumbers have to go because they will obviously match up to perfectly to the penaltyFlag column, which I'm trying to predict. This is to avoid leakage. \n* Kickoffs don't have a down and every single punt happened on fourth down, so I'll get rid of the down column\n* I need to test whether absoluteYardlineNumber measures from one team's end or a stadium-specific end [(see below)](#yardline).\n* I'm going to assume that who the kicker or returner is has no bearing on whether or not there was a penalty and get rid of the kickerId  and returnerId column; this is justifiable because I will be including data about each specific kick. I did a brief examination [below](#kickers-returners) but some kickers and returners have been involved with plays with a large amount of penalties. (Note: I'm going to do an exploration on which kickers and returners have been involved with the most plays with penalties. It's not done yet but I am going to link to it [here]()).\n* Going to use quarter as a proxy for the time and get rid of the gameClock column.\n* Such a small quantity of non-Nan's in the passResult column that I'm just going to remove it.\n\nIn addition. the returnYardage column has a bunch of NaN values, that I'll replace with 0. \n\nThe [special teams result](#specialTeamResult) column seems to a somewhat valuable predictor, so it will be kept and one-hot encoded. Some of the encoding of categorical features will need to be done after the dataset is split between kickoffs and punts, and some before. Click [here](#encoding) to jump to the encoding section.\n\nNext, I'm going to split the data into kickoffs and punts, as there are differences between them that should be handled. Click [here](#split) to jump to this section.","metadata":{}},{"cell_type":"code","source":"def kick_punt_filter(specialTeamsPlayType):\n    \"\"\"Used in a map/lambda to get rid of records other than those referring to a kickoff or a punt\"\"\"\n    \n    if specialTeamsPlayType == \"Kickoff\" or specialTeamsPlayType==\"Punt\":\n        return specialTeamsPlayType\n    else:\n        return False","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.043980Z","iopub.execute_input":"2021-10-16T19:12:28.044669Z","iopub.status.idle":"2021-10-16T19:12:28.055593Z","shell.execute_reply.started":"2021-10-16T19:12:28.044634Z","shell.execute_reply":"2021-10-16T19:12:28.054675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_plays(plays_df):\n    \"\"\"Takes in the original plays DataFrame and returns the processed one.\"\"\"\n    \n    df = plays_df.copy()\n    \n    # add penalty flag column\n    df['penaltyFlag'] = df['penaltyCodes'].notnull().astype(int)\n    \n    # filter for kickoffs and punts\n    df['kickType'] = df['specialTeamsPlayType'].map(lambda x: kick_punt_filter(x))\n    df = df.drop('specialTeamsPlayType', axis=1)\n    \n    df = df[df.kickType != False]\n    \n    # add in the yardsFromEndzone column\n    # the reasoning is described in the Yard Line section below\n    yardsFromEndzone = df['yardlineSide'] == df['possessionTeam']\n    df['yardsFromEndzone'] = np.where(yardsFromEndzone, df['yardlineNumber'], 100-df['yardlineNumber']) \n    \n    # fill NaN's\n    df['kickReturnYardage'].fillna(0, inplace=True)\n    df['penaltyYards'].fillna(0, inplace=True)\n    df['penaltyJerseyNumbers'].fillna('None', inplace=True)\n    \n    # combine playId and gameId into a single column\n    # this will be used later to join with the scouting data\n    df['gameId'] = df['gameId'].astype(str)\n    df['playId'] = df['playId'].astype(str)\n    df['play_key'] = df[['gameId', 'playId']].agg('-'.join, axis=1)\n    \n    # drop unnecessary columns\n    df = df.drop(['gameId',\n                  'playId',\n                  'penaltyYards',\n                  'playDescription',\n                  'absoluteYardlineNumber',\n                  'down',\n                  'gameClock',\n                  'passResult',\n                  'yardlineSide',\n                  'yardlineNumber',\n                  'kickerId',\n                  'returnerId',\n                  'kickBlockerId',\n                  'penaltyCodes',\n                  'penaltyJerseyNumbers'],\n                 axis=1)\n    \n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.056801Z","iopub.execute_input":"2021-10-16T19:12:28.057050Z","iopub.status.idle":"2021-10-16T19:12:28.076267Z","shell.execute_reply.started":"2021-10-16T19:12:28.057023Z","shell.execute_reply":"2021-10-16T19:12:28.075343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_plays(plays)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.079378Z","iopub.execute_input":"2021-10-16T19:12:28.080018Z","iopub.status.idle":"2021-10-16T19:12:28.633529Z","shell.execute_reply.started":"2021-10-16T19:12:28.079970Z","shell.execute_reply":"2021-10-16T19:12:28.632668Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"yardline\"></a>\n#### Yard Line \n\nThe next code block will test whether **absoluteYardlineNumber** is fixed from one team's end (e.g. yards from the home team's end zone) or fixed from one end of the stadium (e.g. yards from the south end zone, regardless of which team's that is). We'll have to make subset DataFrames from the relevant columns in the games and the plays df's and then join them on **gameId**. \n\nFrom inspecting the first few rows, it's obvious that **absoluteYardlineNumber** measures from a fixed end in the stadium.  In the first quarter of the Philadelphia-Atlanta game (seen below, gameId=201809600), **absoluteYardlineNumber** measures from Philadelphia's end. When the two teams switch sides in the second quarter, it measures from Atlanta's end. This means we can get rid of the the **absoluteYardline** column and use only **yardlineSide** and **yardlineNumber**.\n\n[Back to Plays Data Navigator](#plays-data-nav)\n","metadata":{}},{"cell_type":"code","source":"def test_yardline(plays, games):\n    # relevant columns from the two dfs:\n    yardlineTestDf1 = plays[['gameId','absoluteYardlineNumber', 'yardlineSide', 'yardlineNumber', 'quarter', 'possessionTeam']]\n    yardlineTestDf2 = games[['gameId', 'homeTeamAbbr']]\n    \n    # join them\n    yardlineTestDF = yardlineTestDf1.set_index('gameId').join(yardlineTestDf2.set_index('gameId'))\n    \n    return yardlineTestDF","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.634616Z","iopub.execute_input":"2021-10-16T19:12:28.634932Z","iopub.status.idle":"2021-10-16T19:12:28.640336Z","shell.execute_reply.started":"2021-10-16T19:12:28.634903Z","shell.execute_reply":"2021-10-16T19:12:28.639508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_yardline(plays, games).head(10)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.641445Z","iopub.execute_input":"2021-10-16T19:12:28.641795Z","iopub.status.idle":"2021-10-16T19:12:28.671370Z","shell.execute_reply.started":"2021-10-16T19:12:28.641764Z","shell.execute_reply":"2021-10-16T19:12:28.670558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now that we've got that, what we want to do is turn the yardlineSide and yardlineNumber features into a single one called yardsFromEndzone, which denotes where the line of scrimmage is from one's own endzone. This will be yardline number if yardlineSide = possessionTeam, if not, it'll be the yardlineNumber subtracted from 100, since the other team's 5 yard line and 25 yard line are 95 and 75 yards from one's own endzone.\n\nThis functionality is implemented within the process_plays() function above.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"specialTeamResult\"></a>\n#### Special Teams Result\nNext, I'm going to do some general exploring of other features and decide whether or not to keep them, based on whether there's any indication of those correlating with there being a penalty. The **specialTeamsResult** column definitely looks like it should be kept.\n\n[Back to Plays Data Navigator](#plays-data-nav)","metadata":{}},{"cell_type":"code","source":"process_plays(plays)['specialTeamsResult'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:28.672457Z","iopub.execute_input":"2021-10-16T19:12:28.672850Z","iopub.status.idle":"2021-10-16T19:12:29.210516Z","shell.execute_reply.started":"2021-10-16T19:12:28.672819Z","shell.execute_reply":"2021-10-16T19:12:29.209689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,14))\ngraph = sns.histplot(process_plays(plays), x=process_plays(plays)['specialTeamsResult'], hue=process_plays(plays)['penaltyFlag'], stat='count')\nfor p in graph.patches:\n    height = p.get_height()\n    graph.text(p.get_x()+p.get_width()/2., height + 0.3, height, ha=\"center\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:29.211643Z","iopub.execute_input":"2021-10-16T19:12:29.211849Z","iopub.status.idle":"2021-10-16T19:12:30.893845Z","shell.execute_reply.started":"2021-10-16T19:12:29.211825Z","shell.execute_reply":"2021-10-16T19:12:30.892852Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_plays(plays)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:30.895195Z","iopub.execute_input":"2021-10-16T19:12:30.895505Z","iopub.status.idle":"2021-10-16T19:12:31.414837Z","shell.execute_reply.started":"2021-10-16T19:12:30.895466Z","shell.execute_reply":"2021-10-16T19:12:31.414027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# only a few non-nan passResult\nplays['passResult'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:31.416273Z","iopub.execute_input":"2021-10-16T19:12:31.416714Z","iopub.status.idle":"2021-10-16T19:12:31.426199Z","shell.execute_reply.started":"2021-10-16T19:12:31.416669Z","shell.execute_reply":"2021-10-16T19:12:31.425331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=kickers-returners></a>\nWhat about keeping the kicker or returner? Let's sum up the number of penalties for each.\n#### Kickers and Returners\nFor some reason, a few kickers have a huge number of penalties resulting from their kicks. This will be explored elsewhere and I'll link to it here when it's done. A surface level examination is below; click here to skip to below it. For example, we can see below that kicker 40113's kicks had 45 penalties; the average was 9.3 (red line) and the median was 8 (yellow line).\n\nThe situation for returners is similar.\n\nFor the time being, I'm going to assume the identity of the kicker or returner is irrelevant and remove it.\n\n[Back to Plays Data Navigator](#plays-data-nav)","metadata":{}},{"cell_type":"code","source":"def whatsup_with_the_kickers(plays_df):\n    \"\"\"Takes in the original plays DataFrame and returns the processed one.\"\"\"\n    \n    df = plays_df.copy()\n    \n    # add penalty flag column\n    df['penaltyFlag'] = df['penaltyCodes'].notnull().astype(int)\n    \n    kickers = df[['kickerId', 'penaltyFlag']]\n    kickers = kickers.groupby(['kickerId']).sum().sort_values('penaltyFlag', ascending=False)\n    kickers = kickers.reset_index()\n    \n    # plot the penalties for each kicker\n    plt.figure(figsize=(14,10))\n    ax = sns.barplot(x=kickers.index, y=kickers[\"penaltyFlag\"])\n    # horizontal lines for median, mean\n    ax.axhline(kickers.describe().loc['mean','penaltyFlag'], color='red')\n    ax.axhline(kickers.describe().loc['50%','penaltyFlag'], color='yellow')\n    ax.set_xticklabels(kickers.kickerId)\n    for item in ax.get_xticklabels(): item.set_rotation(90)\n    plt.show()\n    \n    \n    return kickers, kickers.describe()['penaltyFlag'], kickers.describe().loc['50%','penaltyFlag']","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:31.427403Z","iopub.execute_input":"2021-10-16T19:12:31.427616Z","iopub.status.idle":"2021-10-16T19:12:31.437190Z","shell.execute_reply.started":"2021-10-16T19:12:31.427591Z","shell.execute_reply":"2021-10-16T19:12:31.436292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"whatsup_with_the_kickers(plays)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:31.438381Z","iopub.execute_input":"2021-10-16T19:12:31.438594Z","iopub.status.idle":"2021-10-16T19:12:33.002472Z","shell.execute_reply.started":"2021-10-16T19:12:31.438571Z","shell.execute_reply":"2021-10-16T19:12:33.001619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=encoding></a>\n### Encoding\nI'm going to define a general one-hot encoding function that will take in a dataframe, the column to encode, and a prefix and return a new Dataframe with the column encoded.\n\n[Back to Plays Data Navigator](#plays-data-nav)","metadata":{}},{"cell_type":"code","source":"def onehot_encode(df, column, prefix):\n    df = df.copy()\n    \n    dummies = pd.get_dummies(df[column], prefix=prefix)\n    df = pd.concat([df,dummies],axis=1)\n    df = df.drop(column, axis=1)\n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:33.003527Z","iopub.execute_input":"2021-10-16T19:12:33.003720Z","iopub.status.idle":"2021-10-16T19:12:33.008796Z","shell.execute_reply.started":"2021-10-16T19:12:33.003698Z","shell.execute_reply":"2021-10-16T19:12:33.008005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_plays(plays)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:33.009707Z","iopub.execute_input":"2021-10-16T19:12:33.010045Z","iopub.status.idle":"2021-10-16T19:12:33.540740Z","shell.execute_reply.started":"2021-10-16T19:12:33.010018Z","shell.execute_reply":"2021-10-16T19:12:33.539898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plays_data = onehot_encode(process_plays(plays), 'possessionTeam', 'team')\nplays_data","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:33.542083Z","iopub.execute_input":"2021-10-16T19:12:33.542289Z","iopub.status.idle":"2021-10-16T19:12:34.073102Z","shell.execute_reply.started":"2021-10-16T19:12:33.542266Z","shell.execute_reply":"2021-10-16T19:12:34.072227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"scouting-data\"></a>\n## Scouting Data\n\nThe third and final dataset we'll bring in is the PFF scouting data. Here are the field definitions from the NFL:\n\n* **gameId**: Game identifier, unique (numeric)\n* **playId**: Play identifier, not unique across games (numeric)\n* **snapDetail**: On Punts, whether the snap was on target and if not, provides detail (H: High, L: Low, <: Left, >: Right, OK: Accurate Snap, text)\n* **operationTime**: Timing from snap to kick on punt plays in seconds: (numeric)\n* **hangTime**: Hangtime of player's punt or kickoff attempt in seconds. Timing is taken from impact with foot to impact with the ground or a player. (numeric)\n* **kickType**: Kickoff or Punt Type (text).\n* * Possible values for kickoff plays:\n* * * D: Deep - your normal deep kick with decent hang time\n* * * F: Flat - different than a Squib in that it will have some hang time and no roll but has a lower trajectory and hang time than a Deep kick off\n* * * K: Free Kick - Kick after a safety\n* * * O: Obvious Onside - score and situation dictates the need to regain possession. Also the hands team is on for the returning team\n* * * P: Pooch kick - high for hangtime but not a lot of distance - usually targeting an upman\n* * * Q: Squib - low-line drive kick that bounces or rolls considerably, with virtually no hang time\n* * * S: Surprise Onside - accounting for score and situation an onsides kick that the returning team doesn’t expect. Hands teams probably aren't on the field\n* * * B: Deep Direct OOB - Kickoff that is aimed deep (regular kickoff) that goes OOB directly (doesn't bounce)\n* * Possible values for punt plays:\n* * * N: Normal - standard punt style\n* * * R: Rugby style punt\n* * * A: Nose down or Aussie-style punts\n* **kickDirectionIntended**: Intended kick direction from the kicking team's perspective - based on how coverage unit sets up and other factors (L: Left, R: Right, C: Center, text).\n* **kickDirectionActual**: Actual kick direction from the kicking team's perspective (L: Left, R: Right, C: Center, text).\n* **returnDirectionIntended**: The return direction the punt return or kick off return unit is set up for from the return team's perspective (L: Left, R: Right, C: Center, text).\n* **returnDirectionActual**: Actual return direction from the return team's perspective (L: Left, R: Right, C: Center, text).\n* **missedTacklers**: Jersey number and team code of player(s) charged with a missed tackle on the play. It will be reasonable to assume that he should have brought down the ball carrier and failed to do so. This situation does not have to entail contact, but it most frequently does. Missed tackles on a QB by a pass rusher are also included here. Multiple missed tacklers on a play are separated by a ; (text).\n* **assistTacklers**: Jersey number and team code of player(s) assisting on the tackle. Multiple assist tacklers on a play are separated by a ; (text).\n* **tacklers**: Jersey number and team code of player making the tackle (text).\n* **kickoffReturnFormation**: 3 digit code indicating the number of players in the Front Wall, Mid Wall and Back Wall (text).\n* **gunners**: Jersey number and team code of player(s) lined up as gunner on punt unit. Multiple gunners on a play are separated by a ; (text).\n* **puntRushers**: Jersey number and team code of player(s) on the punt return unit with \"Punt Rush\" role for actively trying to block the punt. Does not include players crossing the line of scrimmage to engage in punt coverage players in a \"Hold Up\" role. Multiple punt rushers on a play are separated by a ; (text).\n* **specialTeamsSafeties**: Jersey number and team code for player(s) with \"Safety\" roles on kickoff coverage and field goal/extra point block units - and those not actively advancing towards the line of scrimmage on the punt return unit. Multiple special teams safeties on a play are separated by a ; (text).\n* **vises**: Jersey number and team code for player(s) with a \"Vise\" role on the punt return unit. Multiple vises on a play are separated by a ; (text).\n* **kickContactType**: Detail on how a punt was fielded, or what happened when it wasn't fielded (text).\n* * Possible values:\n* * * BB: Bounced Backwards\n* * * BC: Bobbled Catch from Air\n* * * BF: Bounced Forwards\n* * * BOG: Bobbled on Ground\n* * * CC: Clean Catch from Air\n* * * CFFG: Clean Field From Ground\n* * * DEZ: Direct to Endzone\n* * * ICC: Incidental Coverage Team Contact\n* * * KTB: Kick Team Knocked Back\n* * * KTC: Kick Team Catch\n* * * KTF: Kick Team Knocked Forward\n* * * MBC: Muffed by Contact with Non-Designated Returner\n* * * MBDR: Muffed by Designated Returner\n* * * OOB: Directly Out Of Bounds","metadata":{}},{"cell_type":"code","source":"scouting_data","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:34.076546Z","iopub.execute_input":"2021-10-16T19:12:34.076767Z","iopub.status.idle":"2021-10-16T19:12:34.102210Z","shell.execute_reply.started":"2021-10-16T19:12:34.076742Z","shell.execute_reply":"2021-10-16T19:12:34.101311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Scouting Data Feature Engineering\n* As with the plays data, the gameId and the playId will be merged to make a primary key \n* I'm going to get rid of the data about the tacklers\n* Gunners, rushers, safeties, and vises will be turned into numeric values (simply the number of each)\n* The times will stay as they are\n* All of the other field are categorical and will get one hot encoded\n\nThe plays that aren't kickoffs or punts will be dropped when this data is inner-joined with the other data.\n","metadata":{}},{"cell_type":"code","source":"def process_scouting(df):\n    df = df.copy()\n    \n    # rename kickType \n    df['typeOfKick'] = df['kickType']\n    \n    # change the player roles columns to the number of players occupying that role\n    df['gunners'] = df['gunners'].map(lambda x: number_players(x))\n    df['puntRushers'] = df['puntRushers'].map(lambda x: number_players(x))\n    df['specialTeamsSafeties'] = df['specialTeamsSafeties'].map(lambda x: number_players(x))\n    df['vises'] = df['vises'].map(lambda x: number_players(x))\n    \n    # combine playId and gameId into a single column\n    df['gameId'] = df['gameId'].astype(str)\n    df['playId'] = df['playId'].astype(str)\n    df['play_key'] = df[['gameId', 'playId']].agg('-'.join, axis=1)\n    \n    # drop unnecessary columns\n    df = df.drop(['gameId',\n                  'playId',\n                  'missedTackler',\n                  'assistTackler',\n                  'tackler',\n                  'kickType'],\n                 axis=1)\n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:34.103456Z","iopub.execute_input":"2021-10-16T19:12:34.103667Z","iopub.status.idle":"2021-10-16T19:12:34.112159Z","shell.execute_reply.started":"2021-10-16T19:12:34.103644Z","shell.execute_reply":"2021-10-16T19:12:34.111201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def number_players(players):\n    \"\"\"function to change the player roles columns to the number of players occupying that role\"\"\"\n    if type(players) != str: # if it's not NaN\n        return 0\n\n      # if there are players who got penalties, we're just returning the number of semicolons + 1\n    else:\n        return players.count(';') + 1\n","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:34.113655Z","iopub.execute_input":"2021-10-16T19:12:34.113869Z","iopub.status.idle":"2021-10-16T19:12:34.122037Z","shell.execute_reply.started":"2021-10-16T19:12:34.113846Z","shell.execute_reply":"2021-10-16T19:12:34.121144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_scouting(scouting_data)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:34.123024Z","iopub.execute_input":"2021-10-16T19:12:34.123216Z","iopub.status.idle":"2021-10-16T19:12:34.843338Z","shell.execute_reply.started":"2021-10-16T19:12:34.123193Z","shell.execute_reply":"2021-10-16T19:12:34.842493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Merging and finalizing the two data sets \nBefore we can go on any farther, the two datasets need to be merged - we get a 28 field dataset with nearly 14000 records.","metadata":{}},{"cell_type":"code","source":"full_data = process_plays(plays).set_index('play_key').join(process_scouting(scouting_data).set_index('play_key'))\nfull_data","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:34.844936Z","iopub.execute_input":"2021-10-16T19:12:34.845239Z","iopub.status.idle":"2021-10-16T19:12:36.060433Z","shell.execute_reply.started":"2021-10-16T19:12:34.845202Z","shell.execute_reply":"2021-10-16T19:12:36.059579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=split></a>\n### Splitting kickoffs and punts\nSome of the fields are only for kickoffs, and some only for punts. Here I'll split them up before we move on.\n\n* The fields only for punts are yardsToGo snapDetail, operationTime, hangTime, gunners, rushers, vises and kickContactType\n* The field only for kickoffs is kickoffReturnFormation","metadata":{}},{"cell_type":"code","source":"kickoffs = full_data[full_data['kickType'] == 'Kickoff']\npunts = full_data[full_data['kickType'] == 'Punt']","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.061914Z","iopub.execute_input":"2021-10-16T19:12:36.062213Z","iopub.status.idle":"2021-10-16T19:12:36.075682Z","shell.execute_reply.started":"2021-10-16T19:12:36.062169Z","shell.execute_reply":"2021-10-16T19:12:36.074756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check columns only for punts are all nans\n# snapDetail, operationTime, snapTime, gunners, rushers, vises and kickContactType should all be 0\nfor column in kickoffs.columns:\n    print(column + ' - '+ str(kickoffs[column].count()))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.076930Z","iopub.execute_input":"2021-10-16T19:12:36.077196Z","iopub.status.idle":"2021-10-16T19:12:36.097186Z","shell.execute_reply.started":"2021-10-16T19:12:36.077163Z","shell.execute_reply":"2021-10-16T19:12:36.096304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for column in punts.columns:\n    print(column + ' - '+ str(punts[column].count()))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.098467Z","iopub.execute_input":"2021-10-16T19:12:36.098714Z","iopub.status.idle":"2021-10-16T19:12:36.117540Z","shell.execute_reply.started":"2021-10-16T19:12:36.098687Z","shell.execute_reply":"2021-10-16T19:12:36.116729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=kickoffs></a>\n    \n### Finishing up the kickoffs data\n\nLast thing is to onehot encode the categorical columns and drop the unnecessary columns.","metadata":{}},{"cell_type":"code","source":"kickoffs","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.118701Z","iopub.execute_input":"2021-10-16T19:12:36.118942Z","iopub.status.idle":"2021-10-16T19:12:36.152821Z","shell.execute_reply.started":"2021-10-16T19:12:36.118916Z","shell.execute_reply":"2021-10-16T19:12:36.152247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_kickoffs(df):\n    df = df.copy()\n    \n    # drop unnecessary columns\n    df = df.drop(['snapDetail',\n                  'snapTime',\n                  'yardsToGo',\n                  'operationTime',\n                  'gunners',\n                  'puntRushers',\n                  'vises',\n                  'kickContactType',\n                  'kickType']\n                , axis=1)\n    \n    # onehot encoding\n    df = onehot_encode(df, 'quarter', 'quarter')\n    df = onehot_encode(df, 'possessionTeam', 'team')\n    df = onehot_encode(df, 'specialTeamsResult', 'result')\n    df = onehot_encode(df, 'kickDirectionIntended', 'kickDirectionIntended')\n    df = onehot_encode(df, 'kickDirectionActual', 'kickDirectionActual')\n    df = onehot_encode(df, 'returnDirectionIntended', 'returnDirectionIntended')\n    df = onehot_encode(df, 'returnDirectionActual', 'returnDirectionActual')\n    df = onehot_encode(df, 'kickoffReturnFormation', 'kickoffReturnFormation')\n    df = onehot_encode(df, 'typeOfKick', 'typeOfKick')\n    \n    # deal with the NaN's in the hangTime Column\n    df['hangTime'] = df['hangTime'].fillna(value=df['hangTime'].mean())\n    \n    return df\n\n","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.153816Z","iopub.execute_input":"2021-10-16T19:12:36.154541Z","iopub.status.idle":"2021-10-16T19:12:36.161006Z","shell.execute_reply.started":"2021-10-16T19:12:36.154509Z","shell.execute_reply":"2021-10-16T19:12:36.160203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_kickoffs(kickoffs)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.162272Z","iopub.execute_input":"2021-10-16T19:12:36.162474Z","iopub.status.idle":"2021-10-16T19:12:36.276067Z","shell.execute_reply.started":"2021-10-16T19:12:36.162450Z","shell.execute_reply":"2021-10-16T19:12:36.275175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are 620 NaN's in the hangTime field for kickoffs, which are simply missing randomly. They'll be filled with the average kickoff hangtime.","metadata":{}},{"cell_type":"code","source":"# the count() function counts the non-NaN's \n7843 - kickoffs['hangTime'].count()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.277460Z","iopub.execute_input":"2021-10-16T19:12:36.277678Z","iopub.status.idle":"2021-10-16T19:12:36.283858Z","shell.execute_reply.started":"2021-10-16T19:12:36.277652Z","shell.execute_reply":"2021-10-16T19:12:36.282849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# this should now be 7843\nprocess_kickoffs(kickoffs)['hangTime'].count()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.287546Z","iopub.execute_input":"2021-10-16T19:12:36.287780Z","iopub.status.idle":"2021-10-16T19:12:36.358732Z","shell.execute_reply.started":"2021-10-16T19:12:36.287753Z","shell.execute_reply":"2021-10-16T19:12:36.357956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 620 were filled with the average value\nprocess_kickoffs(kickoffs)['hangTime'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.361014Z","iopub.execute_input":"2021-10-16T19:12:36.361303Z","iopub.status.idle":"2021-10-16T19:12:36.431044Z","shell.execute_reply.started":"2021-10-16T19:12:36.361264Z","shell.execute_reply":"2021-10-16T19:12:36.430225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"My final kickoffs data is below:","metadata":{}},{"cell_type":"code","source":"process_kickoffs(kickoffs)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.432212Z","iopub.execute_input":"2021-10-16T19:12:36.432388Z","iopub.status.idle":"2021-10-16T19:12:36.540071Z","shell.execute_reply.started":"2021-10-16T19:12:36.432366Z","shell.execute_reply":"2021-10-16T19:12:36.539344Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# quick null check\nfor col in process_kickoffs(kickoffs).columns:\n    print(col + ' - ' + str(process_kickoffs(kickoffs)[col].isna().sum()))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:36.541257Z","iopub.execute_input":"2021-10-16T19:12:36.541456Z","iopub.status.idle":"2021-10-16T19:12:41.962393Z","shell.execute_reply.started":"2021-10-16T19:12:36.541431Z","shell.execute_reply":"2021-10-16T19:12:41.961571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=punts></a>\n    \n### Finishing up the punts data","metadata":{}},{"cell_type":"code","source":"punts","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:41.963501Z","iopub.execute_input":"2021-10-16T19:12:41.963739Z","iopub.status.idle":"2021-10-16T19:12:42.001312Z","shell.execute_reply.started":"2021-10-16T19:12:41.963712Z","shell.execute_reply":"2021-10-16T19:12:42.000439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_punts(df):\n    df = df.copy()\n    \n    # drop unnecessary columns\n    df = df.drop(['kickoffReturnFormation', 'kickType'], axis=1)\n    \n    # onehot encoding\n    df = onehot_encode(df, 'quarter', 'quarter')\n    df = onehot_encode(df, 'possessionTeam', 'team')\n    df = onehot_encode(df, 'specialTeamsResult', 'result')\n    df = onehot_encode(df, 'snapDetail', 'snap')\n    df = onehot_encode(df, 'kickDirectionIntended', 'kickDirectionIntended')\n    df = onehot_encode(df, 'kickDirectionActual', 'kickDirectionActual')\n    df = onehot_encode(df, 'returnDirectionIntended', 'returnDirectionIntended')\n    df = onehot_encode(df, 'returnDirectionActual', 'returnDirectionActual')\n    df = onehot_encode(df, 'kickContactType', 'kickContactType')\n    df = onehot_encode(df, 'typeOfKick', 'typeOfKick')\n    \n    # drop records with NaN's in the hangTime Column; explained below\n    df = df.dropna()\n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.002437Z","iopub.execute_input":"2021-10-16T19:12:42.003249Z","iopub.status.idle":"2021-10-16T19:12:42.010506Z","shell.execute_reply.started":"2021-10-16T19:12:42.003205Z","shell.execute_reply":"2021-10-16T19:12:42.009580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_punts(punts)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.012221Z","iopub.execute_input":"2021-10-16T19:12:42.012617Z","iopub.status.idle":"2021-10-16T19:12:42.141458Z","shell.execute_reply.started":"2021-10-16T19:12:42.012576Z","shell.execute_reply":"2021-10-16T19:12:42.140664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The NaN values for the punts are a little trickier because some of them are simply missing data, but some are for punt plays for which there is no hangTime, such as blocked punts.","metadata":{}},{"cell_type":"code","source":"temp = punts[['specialTeamsResult','hangTime','penaltyFlag']]\ntemp","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.142631Z","iopub.execute_input":"2021-10-16T19:12:42.143504Z","iopub.status.idle":"2021-10-16T19:12:42.158081Z","shell.execute_reply.started":"2021-10-16T19:12:42.143462Z","shell.execute_reply":"2021-10-16T19:12:42.157065Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp[temp['hangTime'].isna()]['penaltyFlag'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.159846Z","iopub.execute_input":"2021-10-16T19:12:42.160137Z","iopub.status.idle":"2021-10-16T19:12:42.174902Z","shell.execute_reply.started":"2021-10-16T19:12:42.160105Z","shell.execute_reply":"2021-10-16T19:12:42.174106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"temp[temp['hangTime'].isna()]['specialTeamsResult'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.175804Z","iopub.execute_input":"2021-10-16T19:12:42.176083Z","iopub.status.idle":"2021-10-16T19:12:42.192000Z","shell.execute_reply.started":"2021-10-16T19:12:42.176056Z","shell.execute_reply":"2021-10-16T19:12:42.191038Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"However, there's only 118 such records, and only 6 have penalties, so I'm just going to delete those rows for the first go through. A future version of this may deal with those more rigorously.\n\nMy final punts dataset is below:","metadata":{}},{"cell_type":"code","source":"process_punts(punts)","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.193418Z","iopub.execute_input":"2021-10-16T19:12:42.194011Z","iopub.status.idle":"2021-10-16T19:12:42.310810Z","shell.execute_reply.started":"2021-10-16T19:12:42.193950Z","shell.execute_reply":"2021-10-16T19:12:42.309862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# quick null check\nfor col in process_punts(punts).columns:\n    print(col + ' - ' + str(process_punts(punts)[col].isna().sum()))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:42.311867Z","iopub.execute_input":"2021-10-16T19:12:42.312093Z","iopub.status.idle":"2021-10-16T19:12:48.408393Z","shell.execute_reply.started":"2021-10-16T19:12:42.312064Z","shell.execute_reply":"2021-10-16T19:12:48.407481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=modelling></a>\n# Part 2: Modelling\n\nI'm going to put together some classification models that can use these two datasets to predict whether or not there will be a penalty on each play.\n\nFor both the kickoffs and punts, I'll first need to split the data into a training and testing set, fit a standard scaler on the training sets and apply it to both.\n\n### Kickoffs first","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler\nfrom sklearn.model_selection import train_test_split","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.411492Z","iopub.execute_input":"2021-10-16T19:12:48.411721Z","iopub.status.idle":"2021-10-16T19:12:48.415349Z","shell.execute_reply.started":"2021-10-16T19:12:48.411693Z","shell.execute_reply":"2021-10-16T19:12:48.414490Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_for_modelling(df, target):\n    df = df.copy()\n    \n    X = df.drop(target, axis=1)  # feature matrix\n    y = df[target]   # target vector\n    \n    # split it \n    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=13)\n    \n    # scale it \n    scaler = StandardScaler()\n    scaler.fit(X_train) # fit only to the training data\n\n    # apply to both\n    X_train = pd.DataFrame(scaler.transform(X_train), columns=X_train.columns)\n    X_test = pd.DataFrame(scaler.transform(X_test), columns=X_test.columns)\n    \n    return X_train, X_test, y_train, y_test","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.417058Z","iopub.execute_input":"2021-10-16T19:12:48.417293Z","iopub.status.idle":"2021-10-16T19:12:48.432399Z","shell.execute_reply.started":"2021-10-16T19:12:48.417267Z","shell.execute_reply":"2021-10-16T19:12:48.431382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k_X_train, k_X_test, k_y_train, k_y_test = process_for_modelling(process_kickoffs(kickoffs), 'penaltyFlag')","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.433926Z","iopub.execute_input":"2021-10-16T19:12:48.434624Z","iopub.status.idle":"2021-10-16T19:12:48.527795Z","shell.execute_reply.started":"2021-10-16T19:12:48.434562Z","shell.execute_reply":"2021-10-16T19:12:48.526902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k_X_train","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.529233Z","iopub.execute_input":"2021-10-16T19:12:48.529466Z","iopub.status.idle":"2021-10-16T19:12:48.609556Z","shell.execute_reply.started":"2021-10-16T19:12:48.529440Z","shell.execute_reply":"2021-10-16T19:12:48.608765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And we can see the scaler was applied as the standard deviations are all the same:","metadata":{}},{"cell_type":"code","source":"k_X_train.describe()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.610789Z","iopub.execute_input":"2021-10-16T19:12:48.611029Z","iopub.status.idle":"2021-10-16T19:12:48.857288Z","shell.execute_reply.started":"2021-10-16T19:12:48.611004Z","shell.execute_reply":"2021-10-16T19:12:48.856740Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k_y_test.value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.858224Z","iopub.execute_input":"2021-10-16T19:12:48.859031Z","iopub.status.idle":"2021-10-16T19:12:48.865652Z","shell.execute_reply.started":"2021-10-16T19:12:48.858999Z","shell.execute_reply":"2021-10-16T19:12:48.864783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And with the scaled data, we'll do some modelling and see if this mess can predict anything usefully:","metadata":{}},{"cell_type":"code","source":"# import models\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.linear_model import LogisticRegression, Ridge, SGDClassifier\nfrom sklearn.svm import SVC, LinearSVC\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.ensemble import RandomForestClassifier\n\n# import eval stuff\nfrom sklearn.metrics import classification_report\nfrom sklearn.metrics import roc_curve\nfrom sklearn.metrics import precision_recall_curve\nfrom sklearn.metrics import confusion_matrix","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.866869Z","iopub.execute_input":"2021-10-16T19:12:48.867613Z","iopub.status.idle":"2021-10-16T19:12:48.879604Z","shell.execute_reply.started":"2021-10-16T19:12:48.867574Z","shell.execute_reply":"2021-10-16T19:12:48.878810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# make a dictionary of the models to iterate through to train and test each one \nmodels = {\n    \"K-Nearest Neighbors\": KNeighborsClassifier(),\n    # this guy is currently throwing an error I can't figure out\n    # \"Logistic Regression\": LogisticRegression(),  \n    \"Stochastic Gradient Descent Classifier\": SGDClassifier(),\n    \"Support Vector Classifier\": SVC(),\n    \"Linear Support Vector Classifier\": LinearSVC(),\n    \"Decision Tree Classifier\": DecisionTreeClassifier(),\n    \"Random Forest Classifer\": RandomForestClassifier()         \n         }","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.880935Z","iopub.execute_input":"2021-10-16T19:12:48.881377Z","iopub.status.idle":"2021-10-16T19:12:48.890556Z","shell.execute_reply.started":"2021-10-16T19:12:48.881336Z","shell.execute_reply":"2021-10-16T19:12:48.889827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train\nfor name, model in models.items():\n    model.fit(k_X_train, k_y_train)\n    print(name + \" trained.\")","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:48.891717Z","iopub.execute_input":"2021-10-16T19:12:48.892480Z","iopub.status.idle":"2021-10-16T19:12:52.493511Z","shell.execute_reply.started":"2021-10-16T19:12:48.892440Z","shell.execute_reply":"2021-10-16T19:12:52.492622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for name, model in models.items():\n    print(name + \" R^2 Score: {:.5f}\".format(model.score(k_X_test, k_y_test)))\n    print(confusion_matrix(model.predict(k_X_test), k_y_test))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:52.494643Z","iopub.execute_input":"2021-10-16T19:12:52.495063Z","iopub.status.idle":"2021-10-16T19:12:56.019022Z","shell.execute_reply.started":"2021-10-16T19:12:52.495033Z","shell.execute_reply":"2021-10-16T19:12:56.018070Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Evaluation\nAll of the models achieved nearly 100% accuracy.\n","metadata":{}},{"cell_type":"markdown","source":"#### Closer look at the Decision Tree Classifier","metadata":{}},{"cell_type":"code","source":"from sklearn.tree import plot_tree","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:23:03.369261Z","iopub.execute_input":"2021-10-16T19:23:03.369522Z","iopub.status.idle":"2021-10-16T19:23:03.374363Z","shell.execute_reply.started":"2021-10-16T19:23:03.369495Z","shell.execute_reply":"2021-10-16T19:23:03.373238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dTree = DecisionTreeClassifier()\ndTree.fit(k_X_train, k_y_train)\npred_dt = dTree.predict(k_X_test)\nprint(\"Classification Report:\\n\")\nprint(classification_report(k_y_test, pred_dt))\nprint(\"\\nConfusion Matrix:\\n\")\nprint(confusion_matrix(k_y_test, pred_dt))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:56.697388Z","iopub.execute_input":"2021-10-16T19:12:56.697591Z","iopub.status.idle":"2021-10-16T19:12:56.770637Z","shell.execute_reply.started":"2021-10-16T19:12:56.697567Z","shell.execute_reply":"2021-10-16T19:12:56.769707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Classification Report\nThe exact numbers here will change every time this notebook is run, but here is a summary for right now:\n\nIn the negative class - the not-penalty class - 99% of the predictions were correct (precision = 0.99) and 97% of the plays that did not result in a penalty were predicted by the model (recall = 0.97).\n\nIn the positive class, 79% of plays with penalties detected actually did result in penalties, and 80% of the plays that did result in a penalty were predicted.\n\n##### Confusion Matrix\n\nThe first row is the negative class. 1478 plays were correctly predicted *not to have penalties*, and 16 were incorrectly predicted as having penalties. The second row is the positive class. 15 plays were incorrectly predicted to have penalties (i.e, most), and 60 were correctly predicted to have penalties.\n\nThis model is a lot better at making its predictions on the negative class.","metadata":{}},{"cell_type":"code","source":"#Create the tree plot\n\nplt.figure(figsize=(20,14))\nplot_tree(dTree,\n           feature_names = k_X_test.columns, #Feature names\n           class_names = [\"0\",\"1\"], #Class names\n           rounded = True,\n           filled = True)\n\n# leaving this in for now even though it's inelligible\nplt.savefig('dTree.png')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:37:42.488527Z","iopub.execute_input":"2021-10-16T19:37:42.488824Z","iopub.status.idle":"2021-10-16T19:38:02.402079Z","shell.execute_reply.started":"2021-10-16T19:37:42.488794Z","shell.execute_reply":"2021-10-16T19:38:02.401219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Punts","metadata":{}},{"cell_type":"code","source":"p_X_train, p_X_test, p_y_train, p_y_test = process_for_modelling(process_punts(punts), 'penaltyFlag')","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:56.796709Z","iopub.execute_input":"2021-10-16T19:12:56.797164Z","iopub.status.idle":"2021-10-16T19:12:56.892574Z","shell.execute_reply.started":"2021-10-16T19:12:56.797133Z","shell.execute_reply":"2021-10-16T19:12:56.891949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"p_X_train","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:56.893605Z","iopub.execute_input":"2021-10-16T19:12:56.893999Z","iopub.status.idle":"2021-10-16T19:12:56.974166Z","shell.execute_reply.started":"2021-10-16T19:12:56.893968Z","shell.execute_reply":"2021-10-16T19:12:56.973476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"p_X_train.describe()","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:56.975163Z","iopub.execute_input":"2021-10-16T19:12:56.975481Z","iopub.status.idle":"2021-10-16T19:12:57.232118Z","shell.execute_reply.started":"2021-10-16T19:12:56.975455Z","shell.execute_reply":"2021-10-16T19:12:57.231503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"models = {\n    \"K-Nearest Neighbors\": KNeighborsClassifier(),\n    #\"Logistic Regression\": LogisticRegression(),\n    \"Stochastic Gradient Descent Classifier\": SGDClassifier(),\n    \"Support Vector Classifier\": SVC(),\n    \"Linear Support Vector Classifier\": LinearSVC(),\n    \"Decision Tree Classifier\": DecisionTreeClassifier(),\n    \"Random Forest Classifer\": RandomForestClassifier()         \n         }","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:57.233085Z","iopub.execute_input":"2021-10-16T19:12:57.233396Z","iopub.status.idle":"2021-10-16T19:12:57.237019Z","shell.execute_reply.started":"2021-10-16T19:12:57.233369Z","shell.execute_reply":"2021-10-16T19:12:57.236477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train\nfor name, model in models.items():\n    model.fit(p_X_train, p_y_train)\n    print(name + \" trained.\")","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:12:57.237986Z","iopub.execute_input":"2021-10-16T19:12:57.238290Z","iopub.status.idle":"2021-10-16T19:13:01.589785Z","shell.execute_reply.started":"2021-10-16T19:12:57.238266Z","shell.execute_reply":"2021-10-16T19:13:01.588986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for name, model in models.items():\n    print(name + \" R^2 Score: {:.5f}\".format(model.score(p_X_test, p_y_test)))","metadata":{"execution":{"iopub.status.busy":"2021-10-16T19:13:01.590961Z","iopub.execute_input":"2021-10-16T19:13:01.591185Z","iopub.status.idle":"2021-10-16T19:13:03.077665Z","shell.execute_reply.started":"2021-10-16T19:13:01.591158Z","shell.execute_reply":"2021-10-16T19:13:03.075220Z"},"trusted":true},"execution_count":null,"outputs":[]}]}