One would expect that in Rocketball, there is no inherent advantage of a team scoring left-to-right or right-to-left etc.
\nHowever, I found it strange that my Team A and Team B models were not interchangeable.
Team A won 4.5% more games than Team B, and
\nTeam A's outscored Team B's 13370-12780 overall, almost 5% more goals.
\nIf you filp a fair coin this many times, the chances of getting this type of result is 0.02459016
\nSee my working here
Does anyone know of any \"seeding\" or reason for this to happen?
", "messages": [ { "id": 1970347, "postDate": "2022-10-04T04:04:18.767Z", "content": "One would expect that in Rocketball, there is no inherent advantage of a team scoring left-to-right or right-to-left etc.
\nHowever, I found it strange that my Team A and Team B models were not interchangeable.
Team A won 4.5% more games than Team B, and
\nTeam A's outscored Team B's 13370-12780 overall, almost 5% more goals.
\nIf you filp a fair coin this many times, the chances of getting this type of result is 0.02459016
\nSee my working here
Does anyone know of any \"seeding\" or reason for this to happen?
", "rawMarkdown": "One would expect that in Rocketball, there is no inherent advantage of a team scoring left-to-right or right-to-left etc.\nHowever, I found it strange that my Team A and Team B models were not interchangeable.\n\nTeam A won 4.5% more games than Team B, and \nTeam A's outscored Team B's 13370-12780 overall, almost 5% more goals.\nIf you filp a fair coin this many times, the chances of getting this type of result is 0.02459016\n[See my working here](https://www.kaggle.com/code/nigelhenry/are-teams-a-and-b-symmetric?scriptVersionId=107144710)\n\nDoes anyone know of any \"seeding\" or reason for this to happen?", "votes": 7 }, { "id": 1971308, "postDate": "2022-10-04T14:18:09.953Z", "content": "Hey Nigel, good catch, thanks for pointing this out. The data is all from professional Rocket League matches, and I believe there are sometimes seeding effects where higher seeds play as a particular team, I imagine that's what's happening here.
\nThe test and train data are all pulled from the same source distribution of data, but, without saying too much, I can tell you that the test data does not suffer from this asymmetry.
", "rawMarkdown": "Hey Nigel, good catch, thanks for pointing this out. The data is all from professional Rocket League matches, and I believe there are sometimes seeding effects where higher seeds play as a particular team, I imagine that's what's happening here.\n\nThe test and train data are all pulled from the same source distribution of data, but, without saying too much, I can tell you that the test data does not suffer from this asymmetry.", "votes": 5, "replies": [ { "id": 1972066, "postDate": "2022-10-05T00:45:45.803Z", "content": "Very interesting and helpful!
", "rawMarkdown": "Very interesting and helpful!" } ] }, { "id": 1970485, "postDate": "2022-10-04T05:42:10.520Z", "content": "Which side of the field probably doesn't matter, but skill of players probably does? Based on the train_0 dataset, it would appear that player 3 and player 5 are the weak links.
np.unique(train_0.groupby('event_id')['player_scoring_next']\\\n .agg(lambda s: s.iloc[0]).to_numpy(),return_counts=True)\n(array([-1, 0, 1, 2, 3, 4, 5], dtype=int8),\n array([516, 449, 446, 442, 414, 457, 407]))\nYou might be onto something!
", "rawMarkdown": "You might be onto something!", "votes": 1 } ] }, { "id": 1977285, "postDate": "2022-10-07T23:16:14.027Z", "content": "Thank you for the finding. Very interesting indeed.
", "rawMarkdown": "Thank you for the finding. Very interesting indeed.", "votes": 1 }, { "id": 1970909, "postDate": "2022-10-04T11:04:26.180Z", "content": "Turning this into a question for the host: where does the data come from?
\nIs it a \"private\" setting where the training and test set are the same 6 players playing multiple games against each other? Or a \"public\" setting like a league/tournament?
", "rawMarkdown": "Turning this into a question for the host: where does the data come from?\n\nIs it a \"private\" setting where the training and test set are the same 6 players playing multiple games against each other? Or a \"public\" setting like a league/tournament?", "votes": 1 }, { "id": 1970984, "postDate": "2022-10-04T11:57:54.473Z", "content": "This kind of things can appear in tournaments, i.e. for exemple when the winner of a group stage will be given seed a on the next turn and second one will be given a seed b in another bracket. This might introduce some bias in skill / goal scoring.
\nMaybe some slight difference in skill should be taken into account between player A and B.
", "rawMarkdown": "This kind of things can appear in tournaments, i.e. for exemple when the winner of a group stage will be given seed a on the next turn and second one will be given a seed b in another bracket. This might introduce some bias in skill / goal scoring. \n\nMaybe some slight difference in skill should be taken into account between player A and B. ", "replies": [ { "id": 1971051, "postDate": "2022-10-04T12:46:45.367Z", "content": "Exactly.
\nAnd more importantly, if the training set is from a tournament, and the test set is from a different tournament, you may be in for a surprise. 😨
Hey Nigel, good catch, thanks for pointing this out. The data is all from professional Rocket League matches, and I believe there are sometimes seeding effects where higher seeds play as a particular team, I imagine that's what's happening here.
\nThe test and train data are all pulled from the same source distribution of data, but, without saying too much, I can tell you that the test data does not suffer from this asymmetry.
", "votes": 5, "replies": [ { "id": 1972066, "author_name": "Nigel A. R. Henry", "author_url": "", "post_date": "2022-10-05T00:45:45.803000", "content": "Very interesting and helpful!
", "votes": 0, "replies": [] } ] }, { "id": 1970485, "author_name": "broccoli beef", "author_url": "", "post_date": "2022-10-04T05:42:10.520000", "content": "Which side of the field probably doesn't matter, but skill of players probably does? Based on the train_0 dataset, it would appear that player 3 and player 5 are the weak links.
np.unique(train_0.groupby('event_id')['player_scoring_next']\\\n .agg(lambda s: s.iloc[0]).to_numpy(),return_counts=True)\n(array([-1, 0, 1, 2, 3, 4, 5], dtype=int8),\n array([516, 449, 446, 442, 414, 457, 407]))\nYou might be onto something!
", "votes": 1, "replies": [] } ] }, { "id": 1977285, "author_name": "Wei Xie", "author_url": "", "post_date": "2022-10-07T23:16:14.027000", "content": "Thank you for the finding. Very interesting indeed.
", "votes": 1, "replies": [] }, { "id": 1970909, "author_name": "Nigel A. R. Henry", "author_url": "", "post_date": "2022-10-04T11:04:26.180000", "content": "Turning this into a question for the host: where does the data come from?
\nIs it a \"private\" setting where the training and test set are the same 6 players playing multiple games against each other? Or a \"public\" setting like a league/tournament?
", "votes": 1, "replies": [] }, { "id": 1970984, "author_name": "Lucas Morin", "author_url": "", "post_date": "2022-10-04T11:57:54.473000", "content": "This kind of things can appear in tournaments, i.e. for exemple when the winner of a group stage will be given seed a on the next turn and second one will be given a seed b in another bracket. This might introduce some bias in skill / goal scoring.
\nMaybe some slight difference in skill should be taken into account between player A and B.
", "votes": 0, "replies": [ { "id": 1971051, "author_name": "Nigel A. R. Henry", "author_url": "", "post_date": "2022-10-04T12:46:45.367000", "content": "Exactly.
\nAnd more importantly, if the training set is from a tournament, and the test set is from a different tournament, you may be in for a surprise. 😨