{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# NFL strategy prediction\n\nWe all know that one of the main purposes of an NFL game is to push your team to a touchdown. To realize a touchdown, various strategies are evaluated and executed.\nIn our codes, we combine the team data, the player data, strategy used to create a model that helps to evaluate how far can a team push with specific strategies.\nBy using this model, we hope that teams can make their strategy with the assistance of our system to get a better performance in the coming season.","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-01-06T18:47:14.543638Z","iopub.execute_input":"2022-01-06T18:47:14.544056Z","iopub.status.idle":"2022-01-06T18:47:14.563091Z","shell.execute_reply.started":"2022-01-06T18:47:14.543941Z","shell.execute_reply":"2022-01-06T18:47:14.562117Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data pre-process\n\nData we use:\n* plays.csv (mainly used to create dataset)\n    * yardsToGo\n    * kickerId (for referencing)\n    * specialTeamsPlayType(One-Hot Encoded)\n    * yardlineNumber\n    * yardlineSidePossessionTeam\n    * playResult (main target)\n* players.csv (reference by kickerId)\n    * height (change format to numeric type)\n    * weight\n\nAfter all, we drop NAs.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport math\n\nimport os\n\nimport random\n\nfrom imblearn.over_sampling import BorderlineSMOTE\nfrom imblearn.under_sampling import RandomUnderSampler\n\ndef to_one_hot(df, additional_name, column_name):\n\tall_types = np.unique(df[column_name].to_numpy())\n\tfor per_type in all_types:\n\t\tdf[f\"{additional_name}_{per_type}\"] = (df[column_name] == per_type) * 1.0\n\tdf = df.drop(column_name, axis = 1)\n\treturn df\n\ndef label_to_number(df, column_name):\n\tcolumn_list = np.unique(df[column_name].to_numpy())\n\tcolumn_dict = {x:i for i, x in enumerate(column_list)}\n\tdf = df.copy()\n\tdf[f\"{column_name}_number\"] = df[column_name].map(column_dict)\n\treturn column_list, column_dict, df.drop(column_name, axis = 1)\n\ndef number_to_label(df, column_name, column_dict):\n\tcolumn_dict2 = {i:x for i, x in enumerate(column_dict)}\n\tdf = df.copy()\n\tdf[f\"{column_name}\"] = df[f\"{column_name}_number\"].map(column_dict2)\n\treturn df.drop(f\"{column_name}_number\", axis = 1)\n\ndef combine_label(df, column_names, new_name):\n\tdf[new_name] = df.apply(lambda _: 0, axis = 1)\n\tmi = 1\n\tfor column_name in column_names:\n\t\tdf[new_name] = df[new_name] + df[column_name] * mi\n\t\tdf = df.drop(column_name, axis = 1)\n\t\tmi *= 1000\n\treturn df\n\ndef split_label(df, column_names, new_name):\n\tfor column_name in column_names:\n\t\tdf[column_name] = df.apply(lambda _: 0, axis = 1)\n\t\tdf[column_name] = df[new_name]%1000\n\t\tdf[new_name] = df[new_name]//1000\n\tdf = df.drop(new_name, axis = 1)\n\treturn df\n\ndef get_players_dataframe(csv_path):\n\tdf = pd.read_csv(csv_path, keep_default_na=False)\n\ttry:\n\t\tdf = df.drop(\"birthDate\", axis = 1)\n\t\tdf = df.drop(\"collegeName\", axis = 1)\n\t\tdf = df.drop(\"Position\", axis = 1)\n\t\tdf = df.drop(\"displayName\", axis = 1)\n\n\t\tdf = df.reset_index().drop(\"index\", axis = 1)\n\t\tdf['height_number'] = df.apply(lambda _: 0, axis = 1)\n\t\tfor i in range(len(df)):\n\t\t\th = df.iloc[i][\"height\"]\n\t\t\tdf.at[i, \"height_number\"] = float(h) if \"-\" not in h else float(h.split(\"-\")[0]) * 12 + float(h.split(\"-\")[1])\n\t\tdf = df.drop(\"height\", axis = 1)\n\n\texcept Exception as e:\n\t\tprint(e)\n\treturn df\n\n\ndef get_play_dataframe(csv_path):\n\tdf = pd.read_csv(csv_path, keep_default_na=False)\n\ttry:\n\t\tdf = df.drop(\"gameId\", axis = 1)\n\t\tdf = df.drop(\"playDescription\", axis = 1)\n\t\tdf = df.drop(\"playId\", axis = 1)\n\t\tdf = df.drop(\"passResult\", axis = 1)\n\n\t\tdf = df.iloc[(df[\"penaltyCodes\"] == \"NA\").to_numpy()]\n\t\tdf = df.drop(\"penaltyCodes\", axis = 1)\n\n\t\tdf = df.drop(\"penaltyJerseyNumbers\", axis = 1)\n\t\tdf = df.drop(\"penaltyYards\", axis = 1)\n\t\tdf = df.drop(\"returnerId\", axis = 1)\n\t\tdf = df.drop(\"kickBlockerId\", axis = 1)\n\n\t\tdf = df.iloc[(df[\"kickerId\"] != \"NA\").to_numpy()]\n\n\t\tdf[\"yardlineSidePossessionTeam\"] = (df[\"yardlineSide\"] == df[\"possessionTeam\"]) * 1.0\n\t\tdf = df.drop(\"yardlineSide\", axis = 1)\n\t\tdf = df.drop(\"possessionTeam\", axis = 1)\n\n\t\tdf = df.drop(\"kickLength\", axis = 1)\n\t\tdf = df.drop(\"kickReturnYardage\", axis = 1)\n\t\tdf = df.drop(\"absoluteYardlineNumber\", axis = 1)\n\n\t\tdf[\"playResult\"] = df[\"playResult\"].astype(float)\n\n\t\ttemp_removes = [\"quarter\", \"gameClock\", \"preSnapHomeScore\", \"preSnapVisitorScore\", \"down\"]\n\t\tfor temp_remove in temp_removes:\n\t\t\tdf = df.drop(temp_remove, axis = 1)\n\n\n\texcept Exception as e:\n\t\tprint(e)\n\treturn df\n\ndef get_dataframe(csvs_path):\n\tplayer_df = get_players_dataframe(os.path.join(csvs_path, \"players.csv\"))\n\tplayer_dict_height = {}\n\tplayer_dict_weight = {}\n\tfor i in range(len(player_df)):\n\t\tplayer_dict_height[str(player_df[\"nflId\"][i])] = player_df[\"height_number\"][i]\n\t\tplayer_dict_weight[str(player_df[\"nflId\"][i])] = float(player_df[\"weight\"][i])\n\n\tplays_df = get_play_dataframe(os.path.join(csvs_path, \"plays.csv\"))\n\t\n\tplays_df[\"kicker_height\"] = plays_df[\"kickerId\"].map(player_dict_height)\n\tplays_df[\"kicker_weight\"] = plays_df[\"kickerId\"].map(player_dict_weight)\n\tplays_df = plays_df.dropna()\n\n\tplays_df = plays_df.drop(\"kickerId\", axis = 1)\n\n\treturn plays_df\n\ndef undersample(df, column_name, random_state):\n\trus = RandomUnderSampler(random_state=random_state)\n\tx, y = rus.fit_resample(df.drop(column_name, axis = 1), df[column_name])\n\tx[column_name] = y\n\treturn x\n\ndef undersample_with_multiple_labels(df, column_names, random_state):\n\tls = []\n\tds = []\n\tfor column_name in column_names:\n\t\tl1, d1, df = label_to_number(df, column_name)\n\t\tls.append(l1)\n\t\tds.append(d1)\n\n\tdf = combine_label(df, [f\"{x}_number\" for x in column_names], \"new_label\")\n\tdf = undersample(df, \"new_label\", random_state)\n\tdf = split_label(df, [f\"{x}_number\" for x in column_names], \"new_label\")\n\n\tfor i in range(len(column_names)):\n\t\tdf = number_to_label(df, column_names[i], ds[i])\n\treturn df\n\ndef oversample(df, column_name, k_neighbors=5):\n\tsm = BorderlineSMOTE(k_neighbors = k_neighbors)\n\tx, y = sm.fit_resample(df.drop(column_name, axis = 1), df[column_name])\n\tx[column_name] = y\n\treturn x\n\ndef oversample_with_multiple_labels(df, column_names, k_neighbors=5):\n\tls = []\n\tds = []\n\tfor column_name in column_names:\n\t\tl1, d1, df = label_to_number(df, column_name)\n\t\tls.append(l1)\n\t\tds.append(d1)\n\n\tdf = combine_label(df, [f\"{x}_number\" for x in column_names], \"new_label\")\n\tdf = oversample(df, \"new_label\", k_neighbors)\n\tdf = split_label(df, [f\"{x}_number\" for x in column_names], \"new_label\")\n\n\tfor i in range(len(column_names)):\n\t\tdf = number_to_label(df, column_names[i], ds[i])\n\treturn df","metadata":{"execution":{"iopub.status.busy":"2022-01-06T18:47:14.565319Z","iopub.execute_input":"2022-01-06T18:47:14.566095Z","iopub.status.idle":"2022-01-06T18:47:15.117419Z","shell.execute_reply.started":"2022-01-06T18:47:14.566049Z","shell.execute_reply":"2022-01-06T18:47:15.116557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Dataset Helper\n\nIn this helper class, we define dataset to simplify the process of getting it's information.","metadata":{}},{"cell_type":"code","source":"import torch\nimport numpy as np\nimport pandas as pd\n\nclass Dataset(torch.utils.data.Dataset):\n\tdef __init__(self, df, df_ans):\n\t\tself.df = df\n\t\tself.df_ans = df_ans\n\t\t\n\n\tdef __getitem__(self, idx):\n\t\treturn {\"data\": torch.as_tensor(np.array(self.df.iloc[idx]).astype(\"float32\")), \n\t\t\t\t\"label\": torch.as_tensor(np.array(self.df_ans.iloc[idx]))}\n\n\tdef __len__(self):\n\t\treturn len(self.df)","metadata":{"execution":{"iopub.status.busy":"2022-01-06T18:47:15.118618Z","iopub.execute_input":"2022-01-06T18:47:15.11925Z","iopub.status.idle":"2022-01-06T18:47:15.878665Z","shell.execute_reply.started":"2022-01-06T18:47:15.119215Z","shell.execute_reply":"2022-01-06T18:47:15.877891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We propose a neural network to predict the playResult with different specialTeamsPlayTypes.","metadata":{}},{"cell_type":"code","source":"import torch\n\nclass Model(torch.nn.Module):\n\tdef __init__(self, in_channels, out_channels):\n\t\tsuper().__init__()\n\t\tmid_channels = int(in_channels * 3)\n\t\tself.process = torch.nn.Sequential(\n\t\t\ttorch.nn.Linear(in_channels, mid_channels),\n\t\t\ttorch.nn.PReLU(),\n\t\t\ttorch.nn.Linear(mid_channels, mid_channels),\n\t\t\ttorch.nn.PReLU(),\n\t\t\ttorch.nn.Linear(mid_channels, mid_channels),\n\t\t\ttorch.nn.PReLU(),\n\t\t\ttorch.nn.Linear(mid_channels, out_channels)\n\t\t)\n\n\tdef forward(self, x):\n\t\treturn self.process(x)","metadata":{"execution":{"iopub.status.busy":"2022-01-06T18:47:15.881311Z","iopub.execute_input":"2022-01-06T18:47:15.881636Z","iopub.status.idle":"2022-01-06T18:47:15.889316Z","shell.execute_reply.started":"2022-01-06T18:47:15.881595Z","shell.execute_reply":"2022-01-06T18:47:15.888392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Main Code\n\nIn our code, serveral process are down to help evaluate the strategy.\n* Data Pre-Process\n* Model Build\n* Environment Training\n* Model Training","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport torch\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport math\nanwers = []\nlabels = []\ndef main(csvs_path, model_path, learning_rate = 0.005, num_epochs = 3, num_traning = 30):\n\tdf = get_dataframe(csvs_path)\n\n\tdf = df.drop(\"specialTeamsResult\", axis = 1)\n\n\t#df = oversample_with_multiple_labels(df, ['specialTeamsPlayType'], 5)\n\n\tlabel_df = df[[\"playResult\"]]\n\n\tdf = df.drop(\"playResult\", axis = 1)\n\tdf = to_one_hot(df, \"specialTeamsPlayType\", \"specialTeamsPlayType\")\n\n\tprint(df.columns)\n\tprint(df)\n\n\t# training model\n\tmodel = Model(len(df.columns), len(label_df.columns))\n\ttry:\n\t\tmodel = torch.load(model_path)\n\t\tmodel.eval()\n\t\tprint(\"Read model from previous model data\")\n\texcept:\n\t\tprint(model_path)\n\t\tprint(\"No previous model data\")\n\t\tpass\n\n\t# training environment\n\tif torch.cuda.is_available():\n\t\tdevice = torch.device('cuda')\n\t\tprint(\"cuda\")\n\telse :\n\t\tdevice = torch.device('cpu')\n\t\tprint(\"cpu\")\n\n\ttest_len = len(df) // 8\n\tdf_training = df.iloc[:-test_len]\n\tlabel_df_training = label_df.iloc[:-test_len]\n\t\n\tdf_test = df.iloc[-test_len:]\n\tlabel_df_test = label_df.iloc[-test_len:]\n\n\tfor i in range(num_traning):\n\n\t\tprint(\"new Round!!\")\n\t\t\n\t\t# dataset\n\t\tdataset = Dataset(df_training, label_df_training)\n\n\t\tdata_loader = torch.utils.data.DataLoader(\n\t\t\tdataset, batch_size=1, shuffle=True, num_workers=4)\n\n\t\tdataset_test = Dataset(df_test, label_df_test)\n\n\t\tdata_loader_test = torch.utils.data.DataLoader(\n\t\t\tdataset_test, batch_size=1, shuffle=True, num_workers=4)\n\n\n\t\t# training setup\n\t\tmodel.to(device)\n\n\t\tcriterion = torch.nn.MSELoss()\n\t\toptimizer = optim.SGD(model.parameters(), lr=learning_rate, momentum=0.9)\n\t\tfor epoch in range(num_epochs):  # loop over the dataset multiple times\n\t\t\tprint(\"train:\")\n\t\t\trunning_loss = 0.0\n\t\t\tfor i, data_pairs in enumerate(data_loader, 0):\n\t\t\t\t# get the inputs; data is a list of [inputs, labels]\n\t\t\t\t\n\t\t\t\tdata = data_pairs[\"data\"].to(device)\n\t\t\t\tlabel = data_pairs[\"label\"].to(device)\n\n\t\t\t\t# zero the parameter gradients\n\t\t\t\toptimizer.zero_grad()\n\n\t\t\t\t# forward + backward + optimize\n\t\t\t\toutput = model(data)\n\t\t\t\t\n\t\t\t\tloss = criterion(output, label.float())\n\t\t\t\tloss.backward()\n\t\t\t\toptimizer.step()\n\n\t\t\t\t# print statistics\n\t\t\t\trunning_loss += loss.item()\n\t\t\t\tif i % 2000 == 1999:    # print every 2000 mini-batches\n\t\t\t\t\tprint('[%d, %5d] loss: %.3f' %\n\t\t\t\t\t\t(epoch + 1, i + 1, running_loss / 2000))\n\t\t\t\t\trunning_loss = 0.0\n\n\t\t\tprint(\"test:\")\n\t\t\tglobal answers\n\t\t\tanswers = []\n\t\t\tglobal labels\n\t\t\tlabels = []\n\t\t\twith torch.no_grad():\n\t\t\t\tfor i, data_pairs in enumerate(data_loader_test, 0):\n\t\t\t\t\t# get the inputs; data is a list of [inputs, labels]\n\t\t\t\t\t\n\t\t\t\t\tdata = data_pairs[\"data\"].to(device)\n\t\t\t\t\tlabel = data_pairs[\"label\"].to(device)\n\n\t\t\t\t\t# forward + backward + optimize\n\t\t\t\t\toutput = model(data)\n\t\t\t\t\tanswers.append(output.cpu().numpy()[0])\n\t\t\t\t\tlabels.append(label[:, 0].cpu().numpy()[0])\n\t\t\t\t\tloss = criterion(output, label.float())\n\t\t\t\t\t# print statistics\n\t\t\t\t\trunning_loss += loss.item()\n\t\t\t\t\tif i % 2000 == 1999:    # print every 2000 mini-batches\n\t\t\t\t\t\tprint(output)\n\t\t\t\t\t\tprint(label)\n\t\t\t\t\t\tprint('[%d, %5d] loss: %.3f' %\n\t\t\t\t\t\t\t(epoch + 1, i + 1, running_loss / 2000))\n\t\t\t\t\t\trunning_loss = 0.0\n\t\t\tfor i in range(10):\n\t\t\t\tprint(answers[i], labels[i])\n\n\t\ttorch.save(model, model_path)\n\n\tprint('Finished Training')","metadata":{"execution":{"iopub.status.busy":"2022-01-06T18:47:15.891154Z","iopub.execute_input":"2022-01-06T18:47:15.89146Z","iopub.status.idle":"2022-01-06T18:47:15.918791Z","shell.execute_reply.started":"2022-01-06T18:47:15.891421Z","shell.execute_reply":"2022-01-06T18:47:15.918088Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"main(\"/kaggle/input/nfl-big-data-bowl-2022\", model_path = \"playResult_model.pth\", learning_rate = 0.0000001, num_epochs = 3, num_traning = 1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Results","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nanswers_df = pd.DataFrame(answers).clip(-100, 100)\nlabels_df = pd.DataFrame(labels)\ndiff_df = labels_df - answers_df\ndiff_df = round(diff_df / 5)*5\ncounts = diff_df.value_counts().sort_index()\nplt.plot(list(counts.index), counts)\nplt.title(\"Error counts\")\nplt.show()\nprint(f\"Accuracy: {(counts[-10] + counts[-5] + counts[0] + counts[5] + counts[10]) / sum(counts)* 100} %\")","metadata":{"execution":{"iopub.status.busy":"2022-01-06T19:36:59.240781Z","iopub.execute_input":"2022-01-06T19:36:59.241511Z","iopub.status.idle":"2022-01-06T19:36:59.464501Z","shell.execute_reply.started":"2022-01-06T19:36:59.241389Z","shell.execute_reply":"2022-01-06T19:36:59.463827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In our test dataset, the prediction error can be reduced to $\\pm 10$. It means we can input the game conditions, and the neural network will predict how many yards the player can gain when he choose a specialTeamsPlayType. Although it is not allowed to input the status in the game, we expect the predicting results can help the players to choose which strategies to use when training.","metadata":{}}]}