I've been working on this problem for a while and I'd like to share my current approach. It does fairly well but I also don't know what I'm doing in a lot of aspects so I appreciate any suggestions. See my Notebook here.
\nI followed the ideas from The Zoo's solution in the 2020 NFL Big Data Bowl competition, which had a similar competition setup of predicting results from a snapshot of game state. I used their idea of applying a network-in-network (convolutional layer + pooling layer) over pairs of players, which I found to be a cool and effective way to funnel information to the model and take advantage of symmetry in the input data.
\nSince I'm using a CNN, I needed to handle the NaN cases where players are demoed. I added a p#_demoed boolean column for each player, and converted the NaN states to out-of-distribution states where they were unlikely to have an impact on the play: floating above the stadium, moving upwards with zero boost. There's probably better options here.
I also drop the boost#_timer columns since I'm currently not using them, but would like to.
def preprocess_features(df):\n df.drop([col for col in df.columns if col.startswith('boost')], axis=1, inplace=True)\n\n for p in range(6):\n # add demoed column\n df.insert(list(df.columns).index(f'p{p}_boost') + 1, f'p{p}_demoed', df[f'p{p}_boost'].isna())\n\n # Convert nan values from demoed state into dummy values, specifically sitting above the stadium, flying upwards with 0 boost\n x = -20 + 20 * (p % 3)\n y = -20 + 40 * (p // 3)\n cond = df[f'p{p}_boost'].isna()\n df.loc[cond, [f'p{p}_pos_x', f'p{p}_vel_x', f'p{p}_pos_y', f'p{p}_vel_y', f'p{p}_pos_z', f'p{p}_vel_z', f'p{p}_boost']] = \\\n [x, x, y, y, 100, 10, 0]\nSplit out input and reshape players
\ninputs = keras.Input(shape=(54,), name='inputs')\nball = inputs[:, :6]\n# batch_num, team_num, player_num, features (posX3, velX3, boost, demoed)\nplayers = keras.backend.reshape(inputs[:, 6:54], (-1, 2, 3, 8))\nFirst extract some ball features
\nball_preprocess = batched_denses(ball, [[48, 36], [36, 32]])\nThen concat player features with ball data, team number, and delta between player and ball
\n# Get vector diffs from each player pos/vel to ball pos/vel\nball_diffs = to_players_shape(ball) - players[:, :, :, :6]\nplayers_inflated = concat([players, teams, ball_diffs, to_players_shape(ball_preprocess)], axis=3)\nThen apply a network-in-network filter over each player vector
\nplayer_filter_widths = [[81, 64], [64, 48], 36]\np_conv = batched_conv2ds(players_inflated, player_filter_widths)\nFor each player, find each pair of teammates and each pair of opponents, and append their features together
\nteam_pairs = keras.backend.stack([tf.roll(layer, axis=2, shift=i) for i in range(3)], axis=3)\noppo_pairs = tf.roll(team_pairs, axis=2, shift=1)\nplayers_tiled = keras.backend.tile(keras.backend.reshape(layer, (-1, 2, 3, 1, layer.shape[-1])), [1, 1, 1, 3, 1])\nteammates = concat([team_pairs[:, :, :, 1:, :], players_tiled[:, :, :, 1:, :]], axis=4)\nopponents = concat([oppo_pairs, players_tiled], axis=4)\nThen apply a new network-in-network filter over each pair of players, one for teammate pairs and one for opponent pairs, and use Pooling to compress the most important informance per source player
\npair_filter_widths = player_filter_widths\nt_conv = convs_and_pool3d(teammates, pair_filter_widths)\no_conv = convs_and_pool3d(opponents, pair_filter_widths)\nLump all per-player information and apply one last network-in-network filter over each player, pooling again
\np_full = concat([p_conv, t_conv, o_conv], axis=3)\nfull_filter_widths = player_filter_widths\nfull_conv = convs_and_pool2d(p_full, full_filter_widths)\nAppend the original ball preprocessing, and just run some dense layers into our prediction layers
\nx = concat([ball_preprocess, keras.layers.Flatten()(full_conv)], axis=1)\nx = batched_denses(x, [[81, 64], [64, 48], 42])\noutputs = [keras.layers.Dense(3, activation=keras.activations.softmax)(x) for t in TARGETS]\nI use SparseCategoricalCrossentropy loss for each output, predicting between three classes: Team A scoring within Y seconds, Team B scoring within Y seconds, and neither team scoring within Y seconds, with each output having a different Y value (see below).
As others in the competition discovered, there are 144 ways to reshape the input vector: Flip over X axis, Flip over Y axis, or Reorder either team's players in one of 3! = 6 ways, for 2 * 2 * 6 * 6 = 144 permutations. I permuted and shuffled the dataset between every training pass.
\nOne big improvement I learned from the competitors in this competition was permuting the test features as well, making predictions over each of the permutations, and averaging the results. I was surprised at how much this improved my score, and it only takes a few extra minutes to do.
\nI thought it could be useful to have the model predict Pr[Team X scores within Y seconds] for multiple Y values and not just the main target of Y = 10. By doing so, we're able to give the model more information during training about scoring timeframes. I tried TARGETS = [1, 2, 3, ..., 10] and used event_time and team_scoring_next to construct the multi-output target vector, and the model took a bit longer to train but also seemed to perform marginally better. I also used loss_weights to still give more weight to the main Y = 10 target.
boost#_timer columns, for example building a mini-network to construct per-player features estimating their chance and timing of gaining a boost orb, and appending those values to the initial player features.I've been working on this problem for a while and I'd like to share my current approach. It does fairly well but I also don't know what I'm doing in a lot of aspects so I appreciate any suggestions. See my Notebook here.
\nI followed the ideas from The Zoo's solution in the 2020 NFL Big Data Bowl competition, which had a similar competition setup of predicting results from a snapshot of game state. I used their idea of applying a network-in-network (convolutional layer + pooling layer) over pairs of players, which I found to be a cool and effective way to funnel information to the model and take advantage of symmetry in the input data.
\nSince I'm using a CNN, I needed to handle the NaN cases where players are demoed. I added a p#_demoed boolean column for each player, and converted the NaN states to out-of-distribution states where they were unlikely to have an impact on the play: floating above the stadium, moving upwards with zero boost. There's probably better options here.
I also drop the boost#_timer columns since I'm currently not using them, but would like to.
def preprocess_features(df):\n df.drop([col for col in df.columns if col.startswith('boost')], axis=1, inplace=True)\n\n for p in range(6):\n # add demoed column\n df.insert(list(df.columns).index(f'p{p}_boost') + 1, f'p{p}_demoed', df[f'p{p}_boost'].isna())\n\n # Convert nan values from demoed state into dummy values, specifically sitting above the stadium, flying upwards with 0 boost\n x = -20 + 20 * (p % 3)\n y = -20 + 40 * (p // 3)\n cond = df[f'p{p}_boost'].isna()\n df.loc[cond, [f'p{p}_pos_x', f'p{p}_vel_x', f'p{p}_pos_y', f'p{p}_vel_y', f'p{p}_pos_z', f'p{p}_vel_z', f'p{p}_boost']] = \\\n [x, x, y, y, 100, 10, 0]\nSplit out input and reshape players
\ninputs = keras.Input(shape=(54,), name='inputs')\nball = inputs[:, :6]\n# batch_num, team_num, player_num, features (posX3, velX3, boost, demoed)\nplayers = keras.backend.reshape(inputs[:, 6:54], (-1, 2, 3, 8))\nFirst extract some ball features
\nball_preprocess = batched_denses(ball, [[48, 36], [36, 32]])\nThen concat player features with ball data, team number, and delta between player and ball
\n# Get vector diffs from each player pos/vel to ball pos/vel\nball_diffs = to_players_shape(ball) - players[:, :, :, :6]\nplayers_inflated = concat([players, teams, ball_diffs, to_players_shape(ball_preprocess)], axis=3)\nThen apply a network-in-network filter over each player vector
\nplayer_filter_widths = [[81, 64], [64, 48], 36]\np_conv = batched_conv2ds(players_inflated, player_filter_widths)\nFor each player, find each pair of teammates and each pair of opponents, and append their features together
\nteam_pairs = keras.backend.stack([tf.roll(layer, axis=2, shift=i) for i in range(3)], axis=3)\noppo_pairs = tf.roll(team_pairs, axis=2, shift=1)\nplayers_tiled = keras.backend.tile(keras.backend.reshape(layer, (-1, 2, 3, 1, layer.shape[-1])), [1, 1, 1, 3, 1])\nteammates = concat([team_pairs[:, :, :, 1:, :], players_tiled[:, :, :, 1:, :]], axis=4)\nopponents = concat([oppo_pairs, players_tiled], axis=4)\nThen apply a new network-in-network filter over each pair of players, one for teammate pairs and one for opponent pairs, and use Pooling to compress the most important informance per source player
\npair_filter_widths = player_filter_widths\nt_conv = convs_and_pool3d(teammates, pair_filter_widths)\no_conv = convs_and_pool3d(opponents, pair_filter_widths)\nLump all per-player information and apply one last network-in-network filter over each player, pooling again
\np_full = concat([p_conv, t_conv, o_conv], axis=3)\nfull_filter_widths = player_filter_widths\nfull_conv = convs_and_pool2d(p_full, full_filter_widths)\nAppend the original ball preprocessing, and just run some dense layers into our prediction layers
\nx = concat([ball_preprocess, keras.layers.Flatten()(full_conv)], axis=1)\nx = batched_denses(x, [[81, 64], [64, 48], 42])\noutputs = [keras.layers.Dense(3, activation=keras.activations.softmax)(x) for t in TARGETS]\nI use SparseCategoricalCrossentropy loss for each output, predicting between three classes: Team A scoring within Y seconds, Team B scoring within Y seconds, and neither team scoring within Y seconds, with each output having a different Y value (see below).
As others in the competition discovered, there are 144 ways to reshape the input vector: Flip over X axis, Flip over Y axis, or Reorder either team's players in one of 3! = 6 ways, for 2 * 2 * 6 * 6 = 144 permutations. I permuted and shuffled the dataset between every training pass.
\nOne big improvement I learned from the competitors in this competition was permuting the test features as well, making predictions over each of the permutations, and averaging the results. I was surprised at how much this improved my score, and it only takes a few extra minutes to do.
\nI thought it could be useful to have the model predict Pr[Team X scores within Y seconds] for multiple Y values and not just the main target of Y = 10. By doing so, we're able to give the model more information during training about scoring timeframes. I tried TARGETS = [1, 2, 3, ..., 10] and used event_time and team_scoring_next to construct the multi-output target vector, and the model took a bit longer to train but also seemed to perform marginally better. I also used loss_weights to still give more weight to the main Y = 10 target.
boost#_timer columns, for example building a mini-network to construct per-player features estimating their chance and timing of gaining a boost orb, and appending those values to the initial player features.Hi DJ,
\nHow many Kagglers read a solution from the previous Tab Playground Oct 2022 when they are already on November 09? And the winners were announced one week ago.
\nI think the majority has already \"jumped\" or better say \"flew\" (many are birds) for another competition.
\nMy tip for Kaggle team: Hosts Solutions could be a Pinned topic on the next competiton (in this case the current November Tab 22)
\nAnyway, thank you for sharing your valuable solution and your both \"Rocket League TPS Host solution\" and \"Rocket League TPS Preprocessing\" Notebooks.
\nI made a topic reminder (with your links) on November TPS just to check if my assumption that this topic would have more views where kagglers are really \"in action\".
\nBest regards,
\nMarília.
Hey Marília,
\nYes, I wanted to post this much sooner after the competition closed, and apologize that I was unable to do so. I agree that would have been more useful, and thank you for sharing in the November competition.
\nUnfortunately I do not expect Host Solutions to become a common practice for our TPS competitions, usually we set them up and build simple solutions to make sure they are tractable and fun, but we don't spend a lot of time developing a good solution. This competition is an outlier since I have been working on this problem for a while and already had some work prepared.
", "rawMarkdown": "Hey Marília,\n\nYes, I wanted to post this much sooner after the competition closed, and apologize that I was unable to do so. I agree that would have been more useful, and thank you for sharing in the November competition.\n\nUnfortunately I do not expect Host Solutions to become a common practice for our TPS competitions, usually we set them up and build simple solutions to make sure they are tractable and fun, but we don't spend a lot of time developing a good solution. This competition is an outlier since I have been working on this problem for a while and already had some work prepared.", "votes": null }, { "id": "2023549", "postDate": "11/09/2022 20:51:56", "content": "It's a pity that Host Solutions won't become a common practice for Kaggle TPS.
\nAs a beginner (what I really am) we (I) need to read code. Like you've published yours.
\nDo I vote winners solutions? Yes.
\nDo I get them? Of course not 😄😄😄 Though I'll always support users that are making their best to make Kaggle grow with their contributions it's very hard for any beginner to understand what winners are talking about.
We' ll be waiting for the next part since you've still some work prepared on the oven,
", "rawMarkdown": "It's a pity that Host Solutions won't become a common practice for Kaggle TPS.\n\nAs a beginner (what I really am) we (I) need to read code. Like you've published yours. \n\nDo I vote winners solutions? Yes. \nDo I get them? Of course not 😄😄😄 Though I'll always support users that are making their best to make Kaggle grow with their contributions it's very hard for any beginner to understand what winners are talking about.\n\nWe' ll be waiting for the next part since you've still some work prepared on the oven,", "votes": null }, { "id": "2024136", "postDate": "11/10/2022 09:34:05", "content": "It's really cool to see how you've solved this, an how much improvement it was over my own solution. In the end I left out many of the player/ball engineered features since they didn't seem to add that much and one has to make judgment call about which rabbit hole to go down into. Aside from the data augmentation, my 'best' added feature seemed to be a simple boolean based on which team had a numerical player advantage. Another improvement came from having a model with a fairly large first layer and big batch size, combined with long training (100 epochs).
\nThen, finally, my best model turned out to be a linear, unweighted, ensemble of 7 'decent' models. I was quite surprised how much ensembling improved the final score.
A huge congrats for your 1st place Thomas.
\nThat's wonderful to read someone that really understand code changing solutions experiences.
", "rawMarkdown": "A huge congrats for your 1st place Thomas.\n\nThat's wonderful to read someone that really understand code changing solutions experiences.", "votes": null }, { "id": "2024640", "postDate": "11/10/2022 16:42:40", "content": "Thank you Marília, yes we are aware that winners' solutions are some of the best content available on Kaggle, and we're looking into ways to better encourage and publicize high-quality solution posts :)
", "rawMarkdown": "Thank you Marília, yes we are aware that winners' solutions are some of the best content available on Kaggle, and we're looking into ways to better encourage and publicize high-quality solution posts :)", "votes": null }, { "id": "2024642", "postDate": "11/10/2022 16:44:11", "content": "Thank you for sharing Tomas, I will have to try ensembling! And congratulations on your 1st place finish :)
", "rawMarkdown": "Thank you for sharing Tomas, I will have to try ensembling! And congratulations on your 1st place finish :)", "votes": null }, { "id": "2024735", "postDate": "11/10/2022 18:33:08", "content": "Solutions could be shown after the pinned topics. Before the other topics on the competition.
\nSince the other topics only follow a publication time line.
\nWe would have: Pinned topics (=Host topics) then Solutions.
\nAfter (pinned and solutions) it would come the other topics . It' d be easier for anyone to read that.
Hey Marília, here's our first announcement regarding our efforts to improve the culture around solution writeups: https://www.kaggle.com/competitions/AI4Code/discussion/365712
", "rawMarkdown": "Hey Marília, here's our first announcement regarding our efforts to improve the culture around solution writeups: https://www.kaggle.com/competitions/AI4Code/discussion/365712", "votes": null }, { "id": "2030281", "postDate": "11/15/2022 10:55:36", "content": "I read it and noted DJ.
", "rawMarkdown": "I read it and noted DJ.", "votes": null } ], "comments": [ { "id": 2023508, "author_name": "mpwolke", "author_url": "", "post_date": "11/09/2022 19:35:37", "content": "Hi DJ,
\nHow many Kagglers read a solution from the previous Tab Playground Oct 2022 when they are already on November 09? And the winners were announced one week ago.
\nI think the majority has already \"jumped\" or better say \"flew\" (many are birds) for another competition.
\nMy tip for Kaggle team: Hosts Solutions could be a Pinned topic on the next competiton (in this case the current November Tab 22)
\nAnyway, thank you for sharing your valuable solution and your both \"Rocket League TPS Host solution\" and \"Rocket League TPS Preprocessing\" Notebooks.
\nI made a topic reminder (with your links) on November TPS just to check if my assumption that this topic would have more views where kagglers are really \"in action\".
\nBest regards,
\nMarília.
Hey Marília,
\nYes, I wanted to post this much sooner after the competition closed, and apologize that I was unable to do so. I agree that would have been more useful, and thank you for sharing in the November competition.
\nUnfortunately I do not expect Host Solutions to become a common practice for our TPS competitions, usually we set them up and build simple solutions to make sure they are tractable and fun, but we don't spend a lot of time developing a good solution. This competition is an outlier since I have been working on this problem for a while and already had some work prepared.
", "votes": null, "replies": [] }, { "id": 2023549, "author_name": "mpwolke", "author_url": "", "post_date": "11/09/2022 20:51:56", "content": "It's a pity that Host Solutions won't become a common practice for Kaggle TPS.
\nAs a beginner (what I really am) we (I) need to read code. Like you've published yours.
\nDo I vote winners solutions? Yes.
\nDo I get them? Of course not 😄😄😄 Though I'll always support users that are making their best to make Kaggle grow with their contributions it's very hard for any beginner to understand what winners are talking about.
We' ll be waiting for the next part since you've still some work prepared on the oven,
", "votes": null, "replies": [] }, { "id": 2024640, "author_name": "dster", "author_url": "", "post_date": "11/10/2022 16:42:40", "content": "Thank you Marília, yes we are aware that winners' solutions are some of the best content available on Kaggle, and we're looking into ways to better encourage and publicize high-quality solution posts :)
", "votes": null, "replies": [] }, { "id": 2024735, "author_name": "mpwolke", "author_url": "", "post_date": "11/10/2022 18:33:08", "content": "Solutions could be shown after the pinned topics. Before the other topics on the competition.
\nSince the other topics only follow a publication time line.
\nWe would have: Pinned topics (=Host topics) then Solutions.
\nAfter (pinned and solutions) it would come the other topics . It' d be easier for anyone to read that.
Hey Marília, here's our first announcement regarding our efforts to improve the culture around solution writeups: https://www.kaggle.com/competitions/AI4Code/discussion/365712
", "votes": null, "replies": [] }, { "id": 2030281, "author_name": "mpwolke", "author_url": "", "post_date": "11/15/2022 10:55:36", "content": "I read it and noted DJ.
", "votes": null, "replies": [] } ] }, { "id": 2024136, "author_name": "tomasvdb", "author_url": "", "post_date": "11/10/2022 09:34:05", "content": "It's really cool to see how you've solved this, an how much improvement it was over my own solution. In the end I left out many of the player/ball engineered features since they didn't seem to add that much and one has to make judgment call about which rabbit hole to go down into. Aside from the data augmentation, my 'best' added feature seemed to be a simple boolean based on which team had a numerical player advantage. Another improvement came from having a model with a fairly large first layer and big batch size, combined with long training (100 epochs).
\nThen, finally, my best model turned out to be a linear, unweighted, ensemble of 7 'decent' models. I was quite surprised how much ensembling improved the final score.
A huge congrats for your 1st place Thomas.
\nThat's wonderful to read someone that really understand code changing solutions experiences.
", "votes": null, "replies": [] }, { "id": 2024642, "author_name": "dster", "author_url": "", "post_date": "11/10/2022 16:44:11", "content": "Thank you for sharing Tomas, I will have to try ensembling! And congratulations on your 1st place finish :)
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2022322": "I've been working on this problem for a while and I'd like to share my current approach. It does fairly well but I also don't know what I'm doing in a lot of aspects so I appreciate any suggestions. See my Notebook [here](https://www.kaggle.com/code/dster/rocket-league-tps-host-solution).\n\n# Overview\n\nI followed the ideas from [The Zoo's solution](https://www.kaggle.com/competitions/nfl-big-data-bowl-2020/discussion/119400) in the 2020 NFL Big Data Bowl competition, which had a similar competition setup of predicting results from a snapshot of game state. I used their idea of applying a network-in-network (convolutional layer + pooling layer) over pairs of players, which I found to be a cool and effective way to funnel information to the model and take advantage of symmetry in the input data.\n\n# Preprocessing\n\nSince I'm using a CNN, I needed to handle the `NaN` cases where players are demoed. I added a `p#_demoed` boolean column for each player, and converted the `NaN` states to out-of-distribution states where they were unlikely to have an impact on the play: floating above the stadium, moving upwards with zero boost. There's probably better options here.\n\nI also drop the `boost#_timer` columns since I'm currently not using them, but would like to.\n\n```\ndef preprocess_features(df):\n df.drop([col for col in df.columns if col.startswith('boost')], axis=1, inplace=True)\n \n for p in range(6):\n # add demoed column\n df.insert(list(df.columns).index(f'p{p}_boost') + 1, f'p{p}_demoed', df[f'p{p}_boost'].isna())\n \n # Convert nan values from demoed state into dummy values, specifically sitting above the stadium, flying upwards with 0 boost\n x = -20 + 20 * (p % 3)\n y = -20 + 40 * (p // 3)\n cond = df[f'p{p}_boost'].isna()\n df.loc[cond, [f'p{p}_pos_x', f'p{p}_vel_x', f'p{p}_pos_y', f'p{p}_vel_y', f'p{p}_pos_z', f'p{p}_vel_z', f'p{p}_boost']] = \\\n [x, x, y, y, 100, 10, 0]\n```\n\n# Model\n\nSplit out input and reshape players\n\n```\ninputs = keras.Input(shape=(54,), name='inputs')\nball = inputs[:, :6]\n# batch_num, team_num, player_num, features (posX3, velX3, boost, demoed)\nplayers = keras.backend.reshape(inputs[:, 6:54], (-1, 2, 3, 8))\n```\n\nFirst extract some ball features\n```\nball_preprocess = batched_denses(ball, [[48, 36], [36, 32]])\n```\n\nThen concat player features with ball data, team number, and delta between player and ball\n```\n# Get vector diffs from each player pos/vel to ball pos/vel\nball_diffs = to_players_shape(ball) - players[:, :, :, :6]\nplayers_inflated = concat([players, teams, ball_diffs, to_players_shape(ball_preprocess)], axis=3)\n```\n\nThen apply a network-in-network filter over each player vector\n```\nplayer_filter_widths = [[81, 64], [64, 48], 36]\np_conv = batched_conv2ds(players_inflated, player_filter_widths)\n```\n\nFor each player, find each pair of teammates and each pair of opponents, and append their features together\n```\nteam_pairs = keras.backend.stack([tf.roll(layer, axis=2, shift=i) for i in range(3)], axis=3)\noppo_pairs = tf.roll(team_pairs, axis=2, shift=1)\nplayers_tiled = keras.backend.tile(keras.backend.reshape(layer, (-1, 2, 3, 1, layer.shape[-1])), [1, 1, 1, 3, 1])\nteammates = concat([team_pairs[:, :, :, 1:, :], players_tiled[:, :, :, 1:, :]], axis=4)\nopponents = concat([oppo_pairs, players_tiled], axis=4)\n```\n\nThen apply a new network-in-network filter over each pair of players, one for teammate pairs and one for opponent pairs, and use Pooling to compress the most important informance per source player\n```\npair_filter_widths = player_filter_widths\nt_conv = convs_and_pool3d(teammates, pair_filter_widths)\no_conv = convs_and_pool3d(opponents, pair_filter_widths)\n```\n\nLump all per-player information and apply one last network-in-network filter over each player, pooling again\n```\np_full = concat([p_conv, t_conv, o_conv], axis=3)\nfull_filter_widths = player_filter_widths\nfull_conv = convs_and_pool2d(p_full, full_filter_widths)\n```\n\nAppend the original ball preprocessing, and just run some dense layers into our prediction layers\n```\nx = concat([ball_preprocess, keras.layers.Flatten()(full_conv)], axis=1)\nx = batched_denses(x, [[81, 64], [64, 48], 42])\noutputs = [keras.layers.Dense(3, activation=keras.activations.softmax)(x) for t in TARGETS]\n```\n\nI use `SparseCategoricalCrossentropy` loss for each output, predicting between three classes: Team A scoring within Y seconds, Team B scoring within Y seconds, and neither team scoring within Y seconds, with each output having a different `Y` value (see below).\n\n# Data augmentation\n\nAs others in the competition discovered, there are 144 ways to reshape the input vector: Flip over X axis, Flip over Y axis, or Reorder either team's players in one of 3! = 6 ways, for 2 * 2 * 6 * 6 = 144 permutations. I permuted and shuffled the dataset between every training pass.\n\nOne big improvement I learned from the competitors in this competition was permuting the test features as well, making predictions over each of the permutations, and averaging the results. I was surprised at how much this improved my score, and it only takes a few extra minutes to do.\n\n# Predicting multiple timeframes\n\nI thought it could be useful to have the model predict `Pr[Team X scores within Y seconds]` for multiple `Y` values and not just the main target of `Y = 10`. By doing so, we're able to give the model more information during training about scoring timeframes. I tried `TARGETS = [1, 2, 3, ..., 10]` and used `event_time` and `team_scoring_next` to construct the multi-output target vector, and the model took a bit longer to train but also seemed to perform marginally better. I also used `loss_weights` to still give more weight to the main `Y = 10` target.\n\n# Future improvements\n\n1. I could probably have better replacement values for demoed players, or even consider having special models just for states with less players like 2v3.\n2. I would like to utilize the `boost#_timer` columns, for example building a mini-network to construct per-player features estimating their chance and timing of gaining a boost orb, and appending those values to the initial player features.\n3. I would like to explore predicting future states of ball / players, as others suggested in the forums, and add those features to the model.\n4. Maybe similar to #3, I think we could use timeseries information from the training set as part of a RNN, LSTM, or similar setup, but I don't know a lot about the best practices there and need to do some research.\n5. I'm still figuring out a good learning rate strategy. I found that sometimes reducing the LR greatly improved my loss within just one epoch, but I'm finding that reducing it too early does not show the same effect.\n6. A meta-improvement to the competition setup would be to add each player's orientation vector to the dataset, which should add more signal for a model to learn.\n7. From watching prediction traces of my model side-by-side with a gameplay sequence, it's clear to me that there's still a lot of room for improvement, and it might be time to rethink my overall model structure instead of whittling off small increases by playing with hyperparams.", "2023508": "Hi DJ,\n\nHow many Kagglers read a solution from the previous Tab Playground Oct 2022 when they are already on November 09? And the winners were announced one week ago. \n\nI think the majority has already \"jumped\" or better say \"flew\" (many are birds) for another competition.\n\nMy tip for Kaggle team: Hosts Solutions could be a Pinned topic on the next competiton (in this case the current November Tab 22)\n\nAnyway, thank you for sharing your valuable solution and your both \"Rocket League TPS Host solution\" and \"Rocket League TPS Preprocessing\" Notebooks.\n\nI made a topic reminder (with your links) on November TPS just to check if my assumption that this topic would have more views where kagglers are really \"in action\". \n\nBest regards, \nMarília.", "2023544": "Hey Marília,\n\nYes, I wanted to post this much sooner after the competition closed, and apologize that I was unable to do so. I agree that would have been more useful, and thank you for sharing in the November competition.\n\nUnfortunately I do not expect Host Solutions to become a common practice for our TPS competitions, usually we set them up and build simple solutions to make sure they are tractable and fun, but we don't spend a lot of time developing a good solution. This competition is an outlier since I have been working on this problem for a while and already had some work prepared.", "2023549": "It's a pity that Host Solutions won't become a common practice for Kaggle TPS.\n\nAs a beginner (what I really am) we (I) need to read code. Like you've published yours. \n\nDo I vote winners solutions? Yes. \nDo I get them? Of course not 😄😄😄 Though I'll always support users that are making their best to make Kaggle grow with their contributions it's very hard for any beginner to understand what winners are talking about.\n\nWe' ll be waiting for the next part since you've still some work prepared on the oven,", "2024136": "It's really cool to see how you've solved this, an how much improvement it was over my own solution. In the end I left out many of the player/ball engineered features since they didn't seem to add that much and one has to make judgment call about which rabbit hole to go down into. Aside from the data augmentation, my 'best' added feature seemed to be a simple boolean based on which team had a numerical player advantage. Another improvement came from having a model with a fairly large first layer and big batch size, combined with long training (100 epochs). \nThen, finally, my best model turned out to be a linear, unweighted, ensemble of 7 'decent' models. I was quite surprised how much ensembling improved the final score.", "2024292": "A huge congrats for your 1st place Thomas.\n\nThat's wonderful to read someone that really understand code changing solutions experiences.", "2024640": "Thank you Marília, yes we are aware that winners' solutions are some of the best content available on Kaggle, and we're looking into ways to better encourage and publicize high-quality solution posts :)", "2024642": "Thank you for sharing Tomas, I will have to try ensembling! And congratulations on your 1st place finish :)", "2024735": "Solutions could be shown after the pinned topics. Before the other topics on the competition. \n\nSince the other topics only follow a publication time line. \n\nWe would have: Pinned topics (=Host topics) then Solutions. \nAfter (pinned and solutions) it would come the other topics . It' d be easier for anyone to read that.", "2029911": "Hey Marília, here's our first announcement regarding our efforts to improve the culture around solution writeups: https://www.kaggle.com/competitions/AI4Code/discussion/365712", "2030281": "I read it and noted DJ." }, "source": "meta" }