{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**What are you trying to do in this notebook?**\n\nIn this notebook, I'm building a neural networks using fastai. This may be one of the most challenging Tabular Playground competitions to date! It just so happens that one of Kaggle's software engineers is an avid Rocket League player and assembled a dataset of Rocket League gameplay for this month's TPS.\n\nThis month's challenge is to predict the probability of each team scoring within the next 10 seconds of the game given a snapshot from a Rocket League match. \nSounds awesome, right?\n\nWell, it's not that simple. The training data is fairly large; trying to read and model it in a single go might pose some challenges.\n\n**Why are you trying it?**\n\nThe purpose of this month's competition is for to explore ways that can take a big dataset and make it manageable within the time and resources I have. For most people, typical brute force approaches aren't going to work well.\n\n- Can I scale down the dataset?\n- Can I use, e.g., online learning methods that allow me to train from the data one row at a time? (FTLR is a great place to start e.g., this notebook)\n- Can I figure out a nice set of features to reduce the dataset down to?\n\nIn addition, while my predictions must be made pointwise, the training data is made up of timeseries—maybe I can use that temporal information to improve my model? \n\nThis competition also has plenty of opportunity for data visualizations.\n\n\nFor every id in the test data, I will be predicting the probability for whether each of the two teams will score a goal within the next 10 seconds of gameplay. Submissions are scored by the log loss:\n\nscore=−12∑𝑚=1𝑀1𝑁∑𝑖=1𝑁[𝑦𝑖,𝑚log(𝑦̂ 𝑖,𝑚)+(1−𝑦𝑖,𝑚)log(1−𝑦̂ 𝑖,𝑚)]\n\nwhere:\n\n- 𝑁 is the number of id observations in the test data\n- 𝑀 is the number of scored targets (here 𝑀=2, one for each team)\n- 𝑦̂ 𝑖,𝑚 is the predicted scoring probability of team 𝑚 (Team A or Team B in the dataset)\n- 𝑦𝑖,𝑚 is the ground truth for team 𝑚, 1 for a goal within 10 seconds, 0 otherwise\n- 𝑙𝑜𝑔() is the natural (base e) logarithm\n\n**Note:** the actual submitted predicted probabilities are replaced with 𝑚𝑎𝑥(𝑚𝑖𝑛(𝑝,1−10−15),10−15). A smaller log loss is better.\n\n","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.369009Z","iopub.execute_input":"2022-10-21T05:19:05.369498Z","iopub.status.idle":"2022-10-21T05:19:05.384668Z","shell.execute_reply.started":"2022-10-21T05:19:05.369455Z","shell.execute_reply":"2022-10-21T05:19:05.381306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import gc\nimport random","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.389129Z","iopub.execute_input":"2022-10-21T05:19:05.389825Z","iopub.status.idle":"2022-10-21T05:19:05.400371Z","shell.execute_reply.started":"2022-10-21T05:19:05.389788Z","shell.execute_reply":"2022-10-21T05:19:05.399310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pathlib import Path\nfrom fastai.tabular.all import *\nimport fastai.losses as loss","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.402002Z","iopub.execute_input":"2022-10-21T05:19:05.402669Z","iopub.status.idle":"2022-10-21T05:19:05.412224Z","shell.execute_reply.started":"2022-10-21T05:19:05.402635Z","shell.execute_reply":"2022-10-21T05:19:05.410706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features = [\n    'ball_pos_x', 'ball_pos_y','ball_pos_z', 'ball_vel_x', 'ball_vel_y', 'ball_vel_z', \n    'p0_pos_x', 'p0_pos_y', 'p0_pos_z', 'p0_vel_x', 'p0_vel_y', 'p0_vel_z', 'p0_boost', 'p0_na',\n    'p1_pos_x', 'p1_pos_y', 'p1_pos_z', 'p1_vel_x', 'p1_vel_y', 'p1_vel_z', 'p1_boost', 'p1_na',\n    'p2_pos_x', 'p2_pos_y', 'p2_pos_z', 'p2_vel_x', 'p2_vel_y', 'p2_vel_z', 'p2_boost', 'p2_na',\n    'p3_pos_x', 'p3_pos_y', 'p3_pos_z', 'p3_vel_x', 'p3_vel_y', 'p3_vel_z', 'p3_boost', 'p3_na',\n    'p4_pos_x', 'p4_pos_y', 'p4_pos_z', 'p4_vel_x', 'p4_vel_y', 'p4_vel_z', 'p4_boost', 'p4_na',\n    'p5_pos_x', 'p5_pos_y', 'p5_pos_z', 'p5_vel_x', 'p5_vel_y', 'p5_vel_z', 'p5_boost', 'p5_na',\n    'boost0_timer', 'boost1_timer', \n    'boost2_timer', 'boost3_timer',\n    'boost4_timer', 'boost5_timer']","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.413797Z","iopub.execute_input":"2022-10-21T05:19:05.415023Z","iopub.status.idle":"2022-10-21T05:19:05.427574Z","shell.execute_reply.started":"2022-10-21T05:19:05.414988Z","shell.execute_reply":"2022-10-21T05:19:05.426474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features_x_pos = [pos for pos, feature in enumerate(features) if feature.endswith('_x')]\nfeatures_y_pos = [pos for pos, feature in enumerate(features) if feature.endswith('_y')]","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.430734Z","iopub.execute_input":"2022-10-21T05:19:05.431419Z","iopub.status.idle":"2022-10-21T05:19:05.437552Z","shell.execute_reply.started":"2022-10-21T05:19:05.431386Z","shell.execute_reply":"2022-10-21T05:19:05.436503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"targets = [\n    'team_A_scoring_within_10sec',\n    'team_B_scoring_within_10sec']","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.439388Z","iopub.execute_input":"2022-10-21T05:19:05.439983Z","iopub.status.idle":"2022-10-21T05:19:05.448209Z","shell.execute_reply.started":"2022-10-21T05:19:05.439951Z","shell.execute_reply":"2022-10-21T05:19:05.447349Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nDEBUG = False\ninput_path = Path('../input/fast-loading-high-compression-with-feather/feather_data')","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.450005Z","iopub.execute_input":"2022-10-21T05:19:05.451015Z","iopub.status.idle":"2022-10-21T05:19:05.463025Z","shell.execute_reply.started":"2022-10-21T05:19:05.450943Z","shell.execute_reply":"2022-10-21T05:19:05.462109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fe(x):\n    x['p0_na'] = x['p0_pos_x'].isna().astype('int8')\n    x['p1_na'] = x['p1_pos_x'].isna().astype('int8')\n    x['p2_na'] = x['p2_pos_x'].isna().astype('int8')\n    x['p3_na'] = x['p3_pos_x'].isna().astype('int8')\n    x['p4_na'] = x['p4_pos_x'].isna().astype('int8')\n    x['p5_na'] = x['p5_pos_x'].isna().astype('int8')\n    for feature in features:\n        if feature.endswith('_na'):\n            continue\n        if feature.endswith('_x'):\n            x[feature] = (x[feature] / 82).fillna(0).astype('float16')\n        if feature.endswith('_y'):\n            x[feature] = (x[feature] / 120).fillna(0).astype('float16')\n        if feature.endswith('_z'):\n            x[feature] = (x[feature] / 40).fillna(0).astype('float16')\n        if feature.endswith('_boost'):\n            x[feature] = (x[feature] / 100).fillna(0).astype('float16')\n        if feature.endswith('_timer'):\n            x[feature] = (-x[feature] / 100).astype('float16')\n    return x","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.464775Z","iopub.execute_input":"2022-10-21T05:19:05.465500Z","iopub.status.idle":"2022-10-21T05:19:05.478986Z","shell.execute_reply.started":"2022-10-21T05:19:05.465463Z","shell.execute_reply":"2022-10-21T05:19:05.478085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def read_train():\n    dfs = []\n    for i in range(10):\n        dfs.append(fe(pd.read_feather(input_path / f'train_{i}_compressed.ftr')))\n    result = pd.concat(dfs)\n    if DEBUG:\n        result = result.sample(frac=0.05)\n    return result","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.480585Z","iopub.execute_input":"2022-10-21T05:19:05.481328Z","iopub.status.idle":"2022-10-21T05:19:05.489732Z","shell.execute_reply.started":"2022-10-21T05:19:05.481292Z","shell.execute_reply":"2022-10-21T05:19:05.488580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def read_test():\n    return fe(pd.read_feather(input_path / 'test_compressed.ftr'))","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.620817Z","iopub.execute_input":"2022-10-21T05:19:05.621147Z","iopub.status.idle":"2022-10-21T05:19:05.626054Z","shell.execute_reply.started":"2022-10-21T05:19:05.621120Z","shell.execute_reply":"2022-10-21T05:19:05.625053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = read_train()\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:05.628129Z","iopub.execute_input":"2022-10-21T05:19:05.629619Z","iopub.status.idle":"2022-10-21T05:19:44.993948Z","shell.execute_reply.started":"2022-10-21T05:19:05.629580Z","shell.execute_reply":"2022-10-21T05:19:44.992988Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Train Rows = {len(df_train):,}  ' \n      f'Memory Usage = {df_train.memory_usage(deep=True).sum() / (1024 * 1024):4.1f} Mb' '\\n')","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:44.995294Z","iopub.execute_input":"2022-10-21T05:19:44.995629Z","iopub.status.idle":"2022-10-21T05:19:45.006472Z","shell.execute_reply.started":"2022-10-21T05:19:44.995603Z","shell.execute_reply":"2022-10-21T05:19:45.005360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"game_nums = df_train['game_num'].unique()\ntrain_game_nums = random.sample(list(game_nums), int(len(game_nums) * 0.80))","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:45.009851Z","iopub.execute_input":"2022-10-21T05:19:45.010653Z","iopub.status.idle":"2022-10-21T05:19:45.185739Z","shell.execute_reply.started":"2022-10-21T05:19:45.010612Z","shell.execute_reply":"2022-10-21T05:19:45.184587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_feature_tensor = torch.tensor(\n    df_train.query(\"game_num in @train_game_nums\")[features].to_numpy())\ntrain_target_tensor  = torch.tensor(\n    df_train.query(\"game_num in @train_game_nums\")[targets].to_numpy())","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:19:45.187436Z","iopub.execute_input":"2022-10-21T05:19:45.187855Z","iopub.status.idle":"2022-10-21T05:20:09.289156Z","shell.execute_reply.started":"2022-10-21T05:19:45.187814Z","shell.execute_reply":"2022-10-21T05:20:09.288088Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"valid_feature_tensor = torch.tensor(\n    df_train.query(\"game_num not in @train_game_nums\")[features].to_numpy())\nvalid_target_tensor  = torch.tensor(\n    df_train.query(\"game_num not in @train_game_nums\")[targets].to_numpy())","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:09.291003Z","iopub.execute_input":"2022-10-21T05:20:09.291404Z","iopub.status.idle":"2022-10-21T05:20:15.448122Z","shell.execute_reply.started":"2022-10-21T05:20:09.291368Z","shell.execute_reply":"2022-10-21T05:20:15.446984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gc.collect()","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:15.449799Z","iopub.execute_input":"2022-10-21T05:20:15.450233Z","iopub.status.idle":"2022-10-21T05:20:15.568053Z","shell.execute_reply.started":"2022-10-21T05:20:15.450194Z","shell.execute_reply":"2022-10-21T05:20:15.566997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if torch.cuda.is_available():\n    train_feature_tensor = train_feature_tensor.cuda()\n    train_target_tensor  = train_target_tensor.cuda()\n    valid_feature_tensor = valid_feature_tensor.cuda()\n    valid_target_tensor  = valid_target_tensor.cuda()","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:15.569973Z","iopub.execute_input":"2022-10-21T05:20:15.570720Z","iopub.status.idle":"2022-10-21T05:20:19.689678Z","shell.execute_reply.started":"2022-10-21T05:20:15.570682Z","shell.execute_reply":"2022-10-21T05:20:19.688680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del df_train\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.691071Z","iopub.execute_input":"2022-10-21T05:20:19.691540Z","iopub.status.idle":"2022-10-21T05:20:19.812345Z","shell.execute_reply.started":"2022-10-21T05:20:19.691503Z","shell.execute_reply":"2022-10-21T05:20:19.811269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fe_goal_distance(ball):\n    dist_a = ((ball[:,0:1] - 0) ** 2 + (ball[:,1:2] - 1) ** 2 + (ball[:,2:3] - 0) ** 2) ** 0.5 / 2\n    dist_b = ((ball[:,0:1] - 0) ** 2 + (ball[:,1:2] + 1) ** 2 + (ball[:,2:3] - 0) ** 2) ** 0.5 / 2 \n    return dist_a, dist_b\n    \ndef fe_dist_ball_player(ball, player):\n    dist = ((\n        (ball[:,0:1] - player[:,0:1]) ** 2 + \n        (ball[:,1:2] - player[:,1:2]) ** 2 + \n        (ball[:,2:3] - player[:,2:3]) ** 2) ** 0.5) / 12\n    return dist\n\ndef fe_speed_of_thing(thing):\n    return (thing[:, 3:4] ** 2 + thing[:, 4:5] ** 2 + thing[:, 5:6] ** 2) ** 0.5\n\ndef augment_fe(empty, X, Y):\n    ball = X[:, :6]\n    p0 = X[:,  6:14]\n    p1 = X[:, 14:22]\n    p2 = X[:, 22:30]\n    p3 = X[:, 30:38]\n    p4 = X[:, 38:46]\n    p5 = X[:, 46:54]\n    boosts = X[:, 54:]\n    \n    goal_a, goal_b = fe_goal_distance(ball)\n    \n    p0d = fe_dist_ball_player(ball, p0)\n    p1d = fe_dist_ball_player(ball, p1)\n    p2d = fe_dist_ball_player(ball, p2)\n    p3d = fe_dist_ball_player(ball, p3)\n    p4d = fe_dist_ball_player(ball, p4)\n    p5d = fe_dist_ball_player(ball, p5)\n    \n    ball_s = fe_speed_of_thing(ball)\n    p0s = fe_speed_of_thing(p0)\n    p1s = fe_speed_of_thing(p1)\n    p2s = fe_speed_of_thing(p2)\n    p3s = fe_speed_of_thing(p3)\n    p4s = fe_speed_of_thing(p4)\n    p5s = fe_speed_of_thing(p5)\n    \n    new_X = torch.cat([\n        ball, p0, p1, p2, p3, p4, p5, boosts,\n        goal_a, goal_b,\n        ball_s,\n        p0d, p1d, p2d, p3d, p4d, p5d,\n        p0s, p1s, p2s, p3s, p4s, p5s\n    ], dim=1)\n    \n    return empty, new_X, Y","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.816901Z","iopub.execute_input":"2022-10-21T05:20:19.817400Z","iopub.status.idle":"2022-10-21T05:20:19.832686Z","shell.execute_reply.started":"2022-10-21T05:20:19.817363Z","shell.execute_reply":"2022-10-21T05:20:19.831060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def augment_mirror(empty, X, Y):\n    positions = X[:,:54]\n    positions[:, features_x_pos] = -positions[:, features_x_pos]\n    positions[:, features_y_pos] = -positions[:, features_y_pos]\n    \n    ball = positions[:, :6]\n    p0 = positions[:,  6:14]\n    p1 = positions[:, 14:22]\n    p2 = positions[:, 22:30]\n    p3 = positions[:, 30:38]\n    p4 = positions[:, 38:46]\n    p5 = positions[:, 46:54]\n    \n    players = torch.cat([p3, p4, p5, p0, p1, p2], dim=1)\n    boosts = X[:, [59, 58, 57, 56, 55, 54]]\n    \n    flip_X = torch.cat([ball, players, boosts], dim=1)\n    flip_Y = Y[:, :, [1,0]]\n    \n    return empty, flip_X, flip_Y","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.834823Z","iopub.execute_input":"2022-10-21T05:20:19.835818Z","iopub.status.idle":"2022-10-21T05:20:19.846244Z","shell.execute_reply.started":"2022-10-21T05:20:19.835783Z","shell.execute_reply":"2022-10-21T05:20:19.844980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def augment_flip_x(empty, X, Y):\n    positions = X[:,:54]\n    positions[:, features_x_pos] = -positions[:, features_x_pos]\n    boosts = X[:, [55, 54, 57, 56, 59, 58]]\n    \n    flip_X = torch.cat([positions, boosts], dim=1)\n    \n    return empty, flip_X, Y","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.847645Z","iopub.execute_input":"2022-10-21T05:20:19.848130Z","iopub.status.idle":"2022-10-21T05:20:19.860577Z","shell.execute_reply.started":"2022-10-21T05:20:19.848095Z","shell.execute_reply":"2022-10-21T05:20:19.859623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def augment_shuffle(empty, X, Y):\n    ball = X[:, :6]\n    p0 = X[:,  6:14]\n    p1 = X[:, 14:22]\n    p2 = X[:, 22:30]\n    p3 = X[:, 30:38]\n    p4 = X[:, 38:46]\n    p5 = X[:, 46:54]\n    boosts = X[:, 54:]\n    \n    pA = torch.cat(random.sample([p0, p1, p2], 3), dim=1)\n    pB = torch.cat(random.sample([p3, p4, p5], 3), dim=1)\n    \n    shuffled_X = torch.cat([ball, pA, pB, boosts], dim=1)\n    \n    return empty, shuffled_X, Y\n \nclass BespokeDataset:\n    def __init__(self, feature_tensor, targets, augment=False):\n        store_attr()\n        self.n_inp = 2\n    def __getitem__(self, idx):\n        batch = torch.empty(0), self.feature_tensor[idx].float(), self.targets[idx, None]\n        if self.augment:\n            batch = augment_shuffle(*batch)\n            if random.random() > 0.5:\n                batch = augment_mirror(*batch)\n            if random.random() > 0.5:\n                batch = augment_flip_x(*batch)\n        batch = augment_fe(*batch)\n        return batch\n    \n    def __len__(self):\n        return len(self.feature_tensor)\n    \nclass BespokeDL(DataLoader):\n    def __iter__(self):\n        if self.shuffle:\n            self.__idxs = torch.tensor(np.random.permutation(range(0,self.n)))\n        else:\n            self.__idxs = torch.tensor(range(0,self.n))\n        for batch_start in range(0, self.n, self.bs):\n            if batch_start + self.bs > self.n and self.drop_last:\n                return \n            indices = self.__idxs[batch_start:batch_start+self.bs]\n            yield self.dataset[indices]","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.862158Z","iopub.execute_input":"2022-10-21T05:20:19.862569Z","iopub.status.idle":"2022-10-21T05:20:19.877074Z","shell.execute_reply.started":"2022-10-21T05:20:19.862536Z","shell.execute_reply":"2022-10-21T05:20:19.876074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train = BespokeDataset(train_feature_tensor, train_target_tensor, augment=True)\nds_val   = BespokeDataset(valid_feature_tensor, valid_target_tensor, augment=True)\n\ndls = DataLoaders.from_dsets(ds_train, ds_val, bs=4096, dl_type=BespokeDL, num_workers=0, shuffle=True)","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.878558Z","iopub.execute_input":"2022-10-21T05:20:19.879057Z","iopub.status.idle":"2022-10-21T05:20:19.889774Z","shell.execute_reply.started":"2022-10-21T05:20:19.879001Z","shell.execute_reply":"2022-10-21T05:20:19.888795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(ds_train), len(ds_val)","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.891289Z","iopub.execute_input":"2022-10-21T05:20:19.891707Z","iopub.status.idle":"2022-10-21T05:20:19.903916Z","shell.execute_reply.started":"2022-10-21T05:20:19.891674Z","shell.execute_reply":"2022-10-21T05:20:19.902966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = TabularModel(\n    emb_szs={}, \n    n_cont=len(features) + 15,\n    ps=0.3, out_sz=len(targets), \n    layers=[512, 256, 128, 64], y_range=(0,1))\nif torch.cuda.is_available():\n    model = model.cuda()\n    \nlearn = Learner(dls, model, loss_func=loss.BCELossFlat())\nmodel","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.905590Z","iopub.execute_input":"2022-10-21T05:20:19.906589Z","iopub.status.idle":"2022-10-21T05:20:19.930727Z","shell.execute_reply.started":"2022-10-21T05:20:19.906481Z","shell.execute_reply":"2022-10-21T05:20:19.929571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nfit = learn.fit(2 if DEBUG else 25, 1e-3)\nds_train = BespokeDataset(\n    torch.cat([train_feature_tensor, valid_feature_tensor]), \n    torch.cat([train_target_tensor, valid_target_tensor]), augment=True)\n\nds_val   = BespokeDataset(valid_feature_tensor, valid_target_tensor, augment=True)\n\ndls = DataLoaders.from_dsets(ds_train, ds_val, bs=4096, dl_type=BespokeDL, num_workers=0, shuffle=True)\n\nmodel = TabularModel(\n    emb_szs={}, n_cont=len(features) + 15, \n    ps=0.3, out_sz=len(targets), \n    layers=[512, 256, 128, 64], y_range=(0,1))\nif torch.cuda.is_available():\n    model = model.cuda()\n    \nlearn = Learner(dls, model, loss_func=loss.BCELossFlat())\n\nfit = learn.fit(2 if DEBUG else 75, 1e-3)","metadata":{"execution":{"iopub.status.busy":"2022-10-21T05:20:19.932487Z","iopub.execute_input":"2022-10-21T05:20:19.933068Z","iopub.status.idle":"2022-10-21T06:43:01.428400Z","shell.execute_reply.started":"2022-10-21T05:20:19.933016Z","shell.execute_reply":"2022-10-21T06:43:01.425395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test = read_test()\ngc.collect()\n\nds_test = BespokeDataset(torch.tensor(df_test[features].to_numpy()), torch.zeros(len(df_test), 2))\ntest_dl = learn.dls.test_dl(ds_test)\ntest_dl.augment = False\ntest_dl.shuffle = False \npreds, _ = learn.get_preds(dl=test_dl)","metadata":{"execution":{"iopub.status.busy":"2022-10-21T06:43:01.430094Z","iopub.execute_input":"2022-10-21T06:43:01.430470Z","iopub.status.idle":"2022-10-21T06:43:06.220945Z","shell.execute_reply.started":"2022-10-21T06:43:01.430436Z","shell.execute_reply":"2022-10-21T06:43:06.219856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.read_csv('../input/tabular-playground-series-oct-2022/sample_submission.csv')\nsubmission.iloc[:, 1:] = preds.numpy()\nsubmission.to_csv('model_fastai_v3.csv', index=False)\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-21T06:43:06.222710Z","iopub.execute_input":"2022-10-21T06:43:06.223165Z","iopub.status.idle":"2022-10-21T06:43:08.817443Z","shell.execute_reply.started":"2022-10-21T06:43:06.223124Z","shell.execute_reply":"2022-10-21T06:43:08.813483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Did it work?**\n\nThe dataset consists of sequences of snapshots of the state of a Rocket League match, including position and velocity of all players and the ball, as well as extra information. The goal of the competition is to predict from a given snapshot in the game for each team, the probability that will score within the next 10 seconds of game time.\n\nThe data was taken from professional Rocket League matches.\n\n\n**What did you not understand about it?**\n\nWell, everything provides in the competition data page. I've no problem while working on it. The data was taken from professional Rocket League matches. Each event consists of a chronological series of frames recorded at 10 frames per second. All events begin with a kickoff, and most end in one team scoring a goal, but some are truncated and end with no goal scored due to circumstances which can cause gameplay strategies to shift, for example :-\n\n- nearing end of regulation (where the game continues until the ball touches the ground)\n\n- becoming non-competitive, eg one team winning by 3+ goals with little time remaining.\n\n\n**Share your ideas about tackling this beast of a dataset and have a great time!**\n","metadata":{}}]}