{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

MLB In-Depth EDA

\n

One Stop for all your needs!

\n \n
","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"### **You can connect with me on [LinkedIn](https://www.linkedin.com/in/ishandutta0098)**","metadata":{}},{"cell_type":"markdown","source":"
\n

Contents

","metadata":{"execution":{"iopub.status.busy":"2021-06-11T07:04:51.921752Z","iopub.execute_input":"2021-06-11T07:04:51.922065Z","iopub.status.idle":"2021-06-11T07:04:51.928086Z","shell.execute_reply.started":"2021-06-11T07:04:51.922035Z","shell.execute_reply":"2021-06-11T07:04:51.926762Z"}}},{"cell_type":"markdown","source":"1. [Competition Overview](#competition-overview) \n2. [Libraries](#libraries) \n3. [Global Config](#global-config)\n4. [Weights and Biases](#weights-and-biases)\n5. [Load Datasets](#load-datasets) \n6. [Awards Exploration](#awards-exploration) \n7. [Model](#model)","metadata":{}},{"cell_type":"markdown","source":"
\n

If you find this notebook useful, do give me an upvote, it helps to keep up my motivation. This notebook will be updated frequently so keep checking for furthur developments.

","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Competition Overview

","metadata":{}},{"cell_type":"markdown","source":"Let's begin by understanding a brief overview of the competition. I like to code more than to write bulky paragraphs, so we will move straight ahead to the code part in the next step!","metadata":{}},{"cell_type":"markdown","source":"## Description\n- In this competition, you’ll predict how fans engage with MLB players’ digital content on a daily basis for a future date range. \n- You’ll have access to player performance data, social media data, and team factors like market size. Successful models will provide new insights into what signals most strongly correlate with and influence engagement.\n\n## Evaluation Criteria\n- Submissions are evaluated on the mean column-wise mean absolute error (MCMAE). \n- A mean absolute error is calculated for each of the four target variables and the score is the average of those four MAE values.","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Libraries

","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"We will import all the necessary modules at one place to make our life easier :-)","metadata":{}},{"cell_type":"code","source":"!pip install -q --upgrade wandb","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-11-22T05:59:52.976158Z","iopub.execute_input":"2021-11-22T05:59:52.976514Z","iopub.status.idle":"2021-11-22T06:00:03.919985Z","shell.execute_reply.started":"2021-11-22T05:59:52.976439Z","shell.execute_reply":"2021-11-22T06:00:03.919054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport os\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport cv2\nfrom wordcloud import WordCloud, STOPWORDS\n\n#Text Color\nfrom termcolor import colored\n\n#Data Preprocessing\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nfrom sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV\n\n#NLP\nfrom sklearn.feature_extraction.text import CountVectorizer\n\n#WordCloud\nfrom wordcloud import WordCloud, STOPWORDS\n\n#Text Processing\nimport re\nimport nltk\nnltk.download('popular')\n\n#Language Detection\n!pip install langdetect\nimport langdetect\n\n#Sentiment\nfrom textblob import TextBlob\n\n#ner\nimport spacy\n\n#Vectorizer\nfrom sklearn import feature_extraction, manifold\n\n#Word Embedding\nimport gensim.downloader as gensim_api\n\n#Topic Modeling\nimport gensim\n\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import DataLoader, Dataset\nimport torch.nn.functional as F\n\nimport random\n\nimport wandb\n\nfrom tqdm import tqdm\nimport os\nfrom sklearn.model_selection import StratifiedKFold, TimeSeriesSplit\n\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Login to wandb\nwandb.login()","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:00:18.543577Z","iopub.execute_input":"2021-11-22T06:00:18.543905Z","iopub.status.idle":"2021-11-22T06:00:34.341677Z","shell.execute_reply.started":"2021-11-22T06:00:18.543873Z","shell.execute_reply":"2021-11-22T06:00:34.340732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Global Config

","metadata":{}},{"cell_type":"code","source":"target_stats = pd.read_csv('../input/player-target-stats/player_target_stats.csv')\n\nclass CFG:\n nfolds = 5\n batch_size = 1024\n val_batch_size = batch_size * 8\n nepochs = 3\n lr = 0.001\n device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n seed = 42\n targets = ['target1', 'target2', 'target3', 'target4']\n engineered_cols = ['player_t1_mean', 'player_t2_mean', 'player_t3_mean', 'player_t4_mean']\n cols = ['month', 'week', 'weekday'] + engineered_cols + [col for col in target_stats.columns if col not in ['playerId']]\n debug = False","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:00:52.499736Z","iopub.execute_input":"2021-11-22T06:00:52.500117Z","iopub.status.idle":"2021-11-22T06:00:52.537677Z","shell.execute_reply.started":"2021-11-22T06:00:52.500086Z","shell.execute_reply":"2021-11-22T06:00:52.536911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# wandb config\nWANDB_CONFIG = {\n 'competition': 'MLB', \n '_wandb_kernel': 'neuracort'\n }","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:00:55.118516Z","iopub.execute_input":"2021-11-22T06:00:55.118851Z","iopub.status.idle":"2021-11-22T06:00:55.123117Z","shell.execute_reply.started":"2021-11-22T06:00:55.118803Z","shell.execute_reply":"2021-11-22T06:00:55.122032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def set_seed(seed=42):\n random.seed(seed)\n os.environ[\"PYTHONHASHSEED\"] = str(seed)\n np.random.seed(seed)\n \n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = False\n \nset_seed()","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:00:55.889046Z","iopub.execute_input":"2021-11-22T06:00:55.889446Z","iopub.status.idle":"2021-11-22T06:00:55.898930Z","shell.execute_reply.started":"2021-11-22T06:00:55.889410Z","shell.execute_reply":"2021-11-22T06:00:55.897973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Weights and Biases

","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"**Weights & Biases** is the machine learning platform for developers to build better models faster.\n\nYou can use W&B's lightweight, interoperable tools to\n\n- quickly track experiments,\n- version and iterate on datasets,\n- evaluate model performance,\n- reproduce models,\n- visualize results and spot regressions,\n- and share findings with colleagues.\n \nSet up W&B in 5 minutes, then quickly iterate on your machine learning pipeline with the confidence that your datasets and models are tracked and versioned in a reliable system of record.\n\nIn this notebook I will use Weights and Biases's amazing features to perform wonderful visualizations and logging seamlessly.","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Load Datasets

","metadata":{}},{"cell_type":"markdown","source":"Now that we have imported all we need let's begin by using the toughest Python function `read_csv` ;-)","metadata":{}},{"cell_type":"code","source":"awards = pd.read_csv(\"../input/c/mlb-player-digital-engagement-forecasting/awards.csv\")\ntrain = pd.read_csv('../input/unnest-train/train_nextDayPlayerEngagement.csv')\n\nif CFG.debug:\n train = train[:10000]","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:01:01.939326Z","iopub.execute_input":"2021-11-22T06:01:01.939665Z","iopub.status.idle":"2021-11-22T06:01:05.548671Z","shell.execute_reply.started":"2021-11-22T06:01:01.939635Z","shell.execute_reply":"2021-11-22T06:01:05.547814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next we will do exploration on each of the files separately and as mentioned \"InDepth\".","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Awards Exploration

","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"This file has awards won by players in the training set prior to the beginning of the daily data (i.e. before 2018).\n\n- `awardDate` - Date award was given.\n- `awardSeason` - Season award was from.\n- `awardId`\n- `awardName`\n- `playerId` - Unique identifier for a player.\n- `playerName`\n- `awardPlayerTeamId`","metadata":{}},{"cell_type":"markdown","source":"Let's take a quick peek into the dataset.","metadata":{}},{"cell_type":"code","source":"awards.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:27.184129Z","iopub.execute_input":"2021-11-22T05:55:27.184689Z","iopub.status.idle":"2021-11-22T05:55:27.206957Z","shell.execute_reply.started":"2021-11-22T05:55:27.184640Z","shell.execute_reply":"2021-11-22T05:55:27.206105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Dataset Size** ","metadata":{}},{"cell_type":"code","source":"print(f\"Dataset Shape: {colored(awards.shape, 'yellow')}\")","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:28.110790Z","iopub.execute_input":"2021-11-22T05:55:28.111192Z","iopub.status.idle":"2021-11-22T05:55:28.116193Z","shell.execute_reply.started":"2021-11-22T05:55:28.111158Z","shell.execute_reply":"2021-11-22T05:55:28.115207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Dataset Info** ","metadata":{}},{"cell_type":"markdown","source":"Now we shall check if there are any null values in the dataset and if the datatypes are correct.","metadata":{}},{"cell_type":"code","source":"awards.info()","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:30.562328Z","iopub.execute_input":"2021-11-22T05:55:30.562889Z","iopub.status.idle":"2021-11-22T05:55:30.589113Z","shell.execute_reply.started":"2021-11-22T05:55:30.562838Z","shell.execute_reply":"2021-11-22T05:55:30.587906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**OBSERVATIONS**\n- `awardDate`: No Null Values but data type is incorrect it should be `datetime`\n- `awardSeason`: No Null Values and data type is correct.\n- `awardId`: No Null Values and data type is correct.\n- `awardName`: No Null Values and data type is correct.\n- `playerId`: No Null Values and data type is correct.\n- `playerName`: No Null Values and data type is correct.\n- `awardPlayerTeamId`: 13 Null Values and data type is correct.","metadata":{}},{"cell_type":"markdown","source":"Since only 0.1% of the data has Null Values we can drop those rows and analyse with the rest of the dataset.","metadata":{}},{"cell_type":"markdown","source":"## **Cleaning Dataset** ","metadata":{}},{"cell_type":"code","source":"# Dropping Null Values\nawards.dropna(inplace = True)\n\n# Correcting DataType of `awardDate`\nawards[\"awardDate\"] = pd.to_datetime(awards[\"awardDate\"])","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:32.249661Z","iopub.execute_input":"2021-11-22T05:55:32.250304Z","iopub.status.idle":"2021-11-22T05:55:32.284629Z","shell.execute_reply.started":"2021-11-22T05:55:32.250270Z","shell.execute_reply":"2021-11-22T05:55:32.283874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Continuing further, we would look for unique values in the data to gain an intuition about repeated values.","metadata":{}},{"cell_type":"code","source":"for col in awards.columns:\n print(col + \":\" + colored(str(len(awards[col].unique())), 'yellow'))","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:32.953577Z","iopub.execute_input":"2021-11-22T05:55:32.953938Z","iopub.status.idle":"2021-11-22T05:55:32.966486Z","shell.execute_reply.started":"2021-11-22T05:55:32.953908Z","shell.execute_reply":"2021-11-22T05:55:32.965285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can infer that all columns have repeated values.","metadata":{}},{"cell_type":"markdown","source":"## **Text Column Preprocessing** ","metadata":{}},{"cell_type":"markdown","source":"We shall preprocess the Text Columns to analyse them in a better way.","metadata":{}},{"cell_type":"code","source":"def preprocess_text(text, flg_stemm=False, flg_lemm=True):\n\n lst_stopwords = nltk.corpus.stopwords.words(\"english\")\n \n ## clean (convert to lowercase and remove punctuations and characters and then strip)\n text = re.sub(r'[^\\w\\s]', '', str(text).lower().strip())\n \n ## Tokenize (convert from string to list)\n lst_text = text.split()\n ## remove Stopwords\n if lst_stopwords is not None:\n lst_text = [word for word in lst_text if word not in \n lst_stopwords]\n \n ## Stemming (remove -ing, -ly, ...)\n if flg_stemm == True:\n ps = nltk.stem.porter.PorterStemmer()\n lst_text = [ps.stem(word) for word in lst_text]\n \n ## Lemmatisation (convert the word into root word)\n if flg_lemm == True:\n lem = nltk.stem.wordnet.WordNetLemmatizer() \n lst_text = [lem.lemmatize(word) for word in lst_text]\n \n ## back to string from list\n text = \" \".join(lst_text)\n return text\n\ndef apply_preprocess_text(df, column):\n\n #Clean text\n clean_column = \"clean_\" + column\n df[clean_column] = df[column].apply(lambda x: preprocess_text(x, flg_stemm=False, flg_lemm=True, ))\n\n #Length of text\n clean_column_len = \"clean_\" + column + \"_len\"\n df[clean_column_len] = df[clean_column].apply(lambda x: len(x))\n\n #Word Count\n clean_column_word_count = \"clean_\" + column + \"_word_count\"\n df[clean_column_word_count] =df[clean_column].apply(lambda x: len(str(x).split(\" \")))\n\n #Character Count\n clean_column_char_count = \"clean_\" + column + \"_char_count\"\n df[clean_column_char_count] = df[clean_column].apply(lambda x: sum(len(word) for word in str(x).split(\" \")))\n\n #Average Word Length\n clean_column_avg_word_length = \"clean_\" + column + \"_avg_word_length\"\n df[clean_column_avg_word_length] = df[clean_column_char_count] / df[clean_column_word_count]","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:35.011035Z","iopub.execute_input":"2021-11-22T05:55:35.011380Z","iopub.status.idle":"2021-11-22T05:55:35.020139Z","shell.execute_reply.started":"2021-11-22T05:55:35.011352Z","shell.execute_reply":"2021-11-22T05:55:35.019095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"apply_preprocess_text(awards, \"awardName\")\napply_preprocess_text(awards, \"playerName\")","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:49.482036Z","iopub.execute_input":"2021-11-22T05:55:49.482381Z","iopub.status.idle":"2021-11-22T05:55:55.353117Z","shell.execute_reply.started":"2021-11-22T05:55:49.482352Z","shell.execute_reply":"2021-11-22T05:55:55.351882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Distribution Plot** ","metadata":{}},{"cell_type":"code","source":"import plotly.express as px\ndef plot_distribution(df, x, title):\n\n fig = px.histogram(\n df, \n x = x,\n width = 800,\n height = 500,\n title = title\n )\n \n fig.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:55.354917Z","iopub.execute_input":"2021-11-22T05:55:55.355249Z","iopub.status.idle":"2021-11-22T05:55:56.744745Z","shell.execute_reply.started":"2021-11-22T05:55:55.355212Z","shell.execute_reply":"2021-11-22T05:55:56.744009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 1. Award Name Distributions","metadata":{}},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_awardName_len', title = 'Award Name Length Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:58.014472Z","iopub.execute_input":"2021-11-22T05:55:58.015016Z","iopub.status.idle":"2021-11-22T05:55:59.029081Z","shell.execute_reply.started":"2021-11-22T05:55:58.014965Z","shell.execute_reply":"2021-11-22T05:55:59.027913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_awardName_word_count', title = 'Word Count Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:59.030822Z","iopub.execute_input":"2021-11-22T05:55:59.031219Z","iopub.status.idle":"2021-11-22T05:55:59.130817Z","shell.execute_reply.started":"2021-11-22T05:55:59.031176Z","shell.execute_reply":"2021-11-22T05:55:59.129863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_awardName_char_count', title = 'Character Count Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:59.132378Z","iopub.execute_input":"2021-11-22T05:55:59.132669Z","iopub.status.idle":"2021-11-22T05:55:59.230049Z","shell.execute_reply.started":"2021-11-22T05:55:59.132639Z","shell.execute_reply":"2021-11-22T05:55:59.229348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_awardName_avg_word_length', title = 'Average Word Length Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:55:59.231181Z","iopub.execute_input":"2021-11-22T05:55:59.231616Z","iopub.status.idle":"2021-11-22T05:55:59.341302Z","shell.execute_reply.started":"2021-11-22T05:55:59.231577Z","shell.execute_reply":"2021-11-22T05:55:59.340548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2. Player Name Distributions","metadata":{}},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_playerName_len', title = 'Award Name Length Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:56:04.294619Z","iopub.execute_input":"2021-11-22T05:56:04.295270Z","iopub.status.idle":"2021-11-22T05:56:04.395192Z","shell.execute_reply.started":"2021-11-22T05:56:04.295219Z","shell.execute_reply":"2021-11-22T05:56:04.394183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_playerName_word_count', title = 'Word Count Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:56:04.584767Z","iopub.execute_input":"2021-11-22T05:56:04.585168Z","iopub.status.idle":"2021-11-22T05:56:04.682930Z","shell.execute_reply.started":"2021-11-22T05:56:04.585135Z","shell.execute_reply":"2021-11-22T05:56:04.682028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_playerName_char_count', title = 'Character Count Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:56:04.848926Z","iopub.execute_input":"2021-11-22T05:56:04.849315Z","iopub.status.idle":"2021-11-22T05:56:04.947052Z","shell.execute_reply.started":"2021-11-22T05:56:04.849278Z","shell.execute_reply":"2021-11-22T05:56:04.946043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_distribution(df = awards, x = 'clean_playerName_char_count', title = 'Character Count Distribution')","metadata":{"execution":{"iopub.status.busy":"2021-11-22T05:56:05.106384Z","iopub.execute_input":"2021-11-22T05:56:05.106962Z","iopub.status.idle":"2021-11-22T05:56:05.203521Z","shell.execute_reply.started":"2021-11-22T05:56:05.106928Z","shell.execute_reply":"2021-11-22T05:56:05.202589Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Model

","metadata":{}},{"cell_type":"code","source":"class MLBDataset(Dataset):\n def __init__(self, df, targets=None, mode='train'):\n self.mode = mode\n self.data = df\n if mode == 'train':\n self.targets = targets\n \n def __len__(self):\n return len(self.data)\n \n def __getitem__(self, idx):\n if self.mode == 'train':\n x = self.data[idx]\n y = np.array(self.targets[idx])\n return torch.from_numpy(x).float(), torch.from_numpy(y).float()\n elif self.mode == 'test':\n return torch.from_numpy(self.data[idx]).float()\n \nclass MLBModel(nn.Module):\n def __init__(self, num_cols):\n super(MLBModel, self).__init__()\n self.dense1 = nn.Linear(num_cols, 100)\n self.dense2 = nn.Linear(100, 100)\n self.dense3 = nn.Linear(100, len(CFG.targets))\n\n def forward(self, x):\n x = F.relu(self.dense1(x))\n x = F.relu(self.dense2(x))\n x = self.dense3(x).squeeze()\n\n return x","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:01:22.584302Z","iopub.execute_input":"2021-11-22T06:01:22.584630Z","iopub.status.idle":"2021-11-22T06:01:22.594326Z","shell.execute_reply.started":"2021-11-22T06:01:22.584599Z","shell.execute_reply":"2021-11-22T06:01:22.593157Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_player_features(row, player_dict, n_feats, id_col, nan_value=-1):\n features = np.full((n_feats), nan_value, dtype=np.float32)\n \n pid = row[id_col]\n \n if pid in player_dict:\n # overall player means\n p_count = player_dict[pid]['count']\n features[0] = player_dict[pid]['mean_t1'] / p_count\n features[1] = player_dict[pid]['mean_t2'] / p_count\n features[2] = player_dict[pid]['mean_t3'] / p_count\n features[3] = player_dict[pid]['mean_t4'] / p_count\n \n return features\n\ndef update_player_features(row, player_dict, nan_value=-1):\n assert player_dict is not None\n \n pid = row[2]\n t1, t2, t3, t4 = row[3], row[4], row[5], row[6]\n \n if pid in player_dict:\n # update target lags\n player_dict[pid]['mean_t1'] += t1\n player_dict[pid]['mean_t2'] += t2\n player_dict[pid]['mean_t3'] += t3\n player_dict[pid]['mean_t4'] += t4\n player_dict[pid]['count'] += 1\n else:\n # init the feature dict for this player\n player_dict[pid] = {'mean_t1': t1, 'mean_t2': t2, 'mean_t3': t3, 'mean_t4': t4, 'count': 1}\n \n return player_dict\n\ndef do_feature_engineering(df, id_col=2):\n player_features = {}\n nfeats = len(CFG.engineered_cols)\n features = np.zeros((df.shape[0], nfeats), dtype=np.float32)\n \n for idx, row in enumerate(tqdm(df.values)):\n row_features = add_player_features(row, player_features, nfeats, id_col=id_col)\n player_features = update_player_features(row, player_features)\n features[idx] = row_features\n \n features = pd.DataFrame(features, columns=CFG.engineered_cols)\n df = pd.concat([df, features], axis=1)\n \n return df, player_features\n\ndef do_feature_engineering_test(df, player_dict, id_col=2):\n features = np.zeros((df.shape[0], len(CFG.engineered_cols)), dtype=np.float32)\n \n for idx, row in enumerate(df.values):\n row_features = add_player_features(row, player_dict, len(CFG.engineered_cols), id_col=id_col)\n features[idx] = row_features\n \n features = pd.DataFrame(features, columns=CFG.engineered_cols, index=df.index)\n df = pd.concat([df, features], axis=1)\n \n return df\n\ndef train_epoch(model, loader, optimizer, device, criterion):\n model.train()\n train_loss = 0.0\n\n for i, (sample, target) in enumerate(tqdm(loader)):\n sample, target = sample.to(device), target.to(device)\n optimizer.zero_grad()\n\n preds = model(sample)\n\n loss = criterion(preds, target)\n loss.backward()\n optimizer.step()\n\n train_loss += loss.item() / len(loader)\n\n return model, train_loss\n\ndef validate(model, loader, device, criterion):\n model.eval()\n val_loss = 0.0\n val_preds = []\n val_targets = []\n\n with torch.no_grad():\n for i, (sample, target) in enumerate(tqdm(loader)):\n sample, target = sample.to(device), target.to(device)\n\n preds = model(sample)\n loss = criterion(preds, target)\n val_loss += loss.item() / len(loader)\n\n val_preds.append(preds.cpu())\n val_targets.append(target.cpu())\n\n val_preds = np.concatenate(val_preds)\n val_targets = np.concatenate(val_targets)\n\n return val_loss, val_preds\n\nclass ColL1Loss(nn.Module):\n def __init__(self, num_targets):\n super().__init__()\n self.mae = nn.L1Loss()\n assert num_targets != 0\n self.num_targets = num_targets\n \n def forward(self, preds, targets):\n l1 = 0.0\n \n for i in range(self.num_targets):\n l1 += self.mae(preds[:, i], targets[:, i])\n \n return l1 / self.num_targets\n \ndef train_fold(xtrn, ytrn, xval, yval, fold):\n print(f\"Train fold {fold}\")\n train_set = MLBDataset(xtrn, ytrn)\n val_set = MLBDataset(xval, yval)\n \n train_loader = DataLoader(train_set, batch_size=CFG.batch_size, shuffle=True)\n val_loader = DataLoader(val_set, batch_size=CFG.val_batch_size, shuffle=False)\n \n model = MLBModel(xtrn.shape[1]).to(CFG.device)\n criterion = ColL1Loss(len(CFG.targets))\n optimizer = optim.Adam(model.parameters(), lr=CFG.lr)\n \n best_loss = {'train': np.inf, 'val': np.inf}\n best_val_preds = None\n \n # Initialize W&B\n run = wandb.init(project='MLB', config= WANDB_CONFIG)\n \n for epoch in range(1, CFG.nepochs + 1):\n print(f\"Train epoch {epoch}\")\n model, loss = train_epoch(model, train_loader, optimizer, CFG.device, criterion)\n val_loss, val_preds = validate(model, val_loader, CFG.device, criterion)\n \n # save best model\n if val_loss < best_loss['val']:\n best_loss = {'train': loss, 'val': val_loss}\n torch.save(model.state_dict(), f'fold{fold}.pt')\n \n # save best oof predictions\n best_val_preds = val_preds\n np.save(f'oof_fold{fold}.npy', val_preds)\n \n print(\"Train loss: {:5.5f}\".format(loss))\n print(\"Val loss: {:5.5f}\".format(val_loss))\n \n wandb.log({\n 'train_loss': loss,\n 'valid_loss': val_loss\n })\n \n # Close W&B run\n wandb.finish()\n \n return best_val_preds\n\ndef extract_date_feats(df):\n df['year'] = pd.DatetimeIndex(df['engagementMetricsDate']).year\n df['month'] = pd.DatetimeIndex(df['engagementMetricsDate']).month\n df['week'] = pd.DatetimeIndex(df['engagementMetricsDate']).week\n df['weekday'] = pd.DatetimeIndex(df['engagementMetricsDate']).weekday\n \n return df\n\ndef train_kfolds(df):\n set_seed(CFG.seed)\n ts = TimeSeriesSplit(n_splits=CFG.nfolds)\n df = extract_date_feats(df)\n \n oof_preds = []\n oof_targets = []\n \n for fold_idx, (trn_idx, val_idx) in enumerate(ts.split(df)):\n trn_df, val_df = df.iloc[trn_idx], df.iloc[val_idx]\n \n trn_df, player_dict = do_feature_engineering(trn_df)\n val_df = do_feature_engineering_test(val_df, player_dict)\n trn_df = pd.merge(trn_df, target_stats, how='left', on='playerId')\n val_df = pd.merge(val_df, target_stats, how='left', on='playerId')\n \n xtrn, ytrn = trn_df[CFG.cols].values, trn_df[CFG.targets].values\n xval, yval = val_df[CFG.cols].values, val_df[CFG.targets].values\n \n val_preds = train_fold(xtrn, ytrn, xval, yval, fold_idx)\n \n oof_preds.append(val_preds)\n oof_targets.append(yval)\n del trn_df, val_df, player_dict\n del xtrn, ytrn, xval, yval\n \n oof_preds = torch.from_numpy(np.clip(np.concatenate(oof_preds), 0, 100)).float()\n oof_targets = torch.from_numpy(np.concatenate(oof_targets)).float()\n \n eval_metric = ColL1Loss(len(CFG.targets))\n oof_loss = eval_metric(oof_preds, oof_targets)\n df, player_dict = do_feature_engineering(df)\n del oof_preds, oof_targets\n del df\n \n return oof_loss, player_dict","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:01:27.154073Z","iopub.execute_input":"2021-11-22T06:01:27.154435Z","iopub.status.idle":"2021-11-22T06:01:27.194056Z","shell.execute_reply.started":"2021-11-22T06:01:27.154405Z","shell.execute_reply":"2021-11-22T06:01:27.193109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss, player_dict = train_kfolds(train)\nprint(\"OOF loss: {:5.5f}\".format(loss))\ndel train","metadata":{"execution":{"iopub.status.busy":"2021-11-22T06:01:29.759684Z","iopub.execute_input":"2021-11-22T06:01:29.760048Z","iopub.status.idle":"2021-11-22T06:14:42.103129Z","shell.execute_reply.started":"2021-11-22T06:01:29.760016Z","shell.execute_reply":"2021-11-22T06:14:42.102194Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"--- \n\n## **Let's have a Talk!**\n> ### Reach out to me on [LinkedIn](https://www.linkedin.com/in/ishandutta0098)\n\n---","metadata":{}}]}