The long drive is almost over.
\nHere's one of the classic discussion.
\nShake up or not shake up?
A similar competition, the indoor competition,
\nseemed to be a competition with relatively few shakes.
We don't have any unknown test data for this competition.
\nSo there may not be that big of a shake.
On the other hand,
\nthe 95th percentile error in the evaluation metric is a bit scary.
\nIf you are missing a lot of points in your private data due to post-processing or other reasons,
\nthis error may be very large and cause your overall score to worse.
Please let me know what you think!
", "messages": [ { "id": "1407057", "postDate": "08/01/2021 13:21:39", "content": "The long drive is almost over.
\nHere's one of the classic discussion.
\nShake up or not shake up?
A similar competition, the indoor competition,
\nseemed to be a competition with relatively few shakes.
We don't have any unknown test data for this competition.
\nSo there may not be that big of a shake.
On the other hand,
\nthe 95th percentile error in the evaluation metric is a bit scary.
\nIf you are missing a lot of points in your private data due to post-processing or other reasons,
\nthis error may be very large and cause your overall score to worse.
Please let me know what you think!
", "rawMarkdown": "The long drive is almost over.\nHere's one of the classic discussion.\n**Shake up or not shake up?**\n\nA similar competition, the indoor competition, \nseemed to be a competition with relatively few shakes.\n\nWe don't have any unknown test data for this competition. \nSo there may not be that big of a shake.\n\nOn the other hand, \nthe 95th percentile error in the evaluation metric is a bit scary. \nIf you are missing a lot of points in your private data due to post-processing or other reasons, \nthis error may be very large and cause your overall score to worse. \n\nPlease let me know what you think!", "votes": null }, { "id": "1407218", "postDate": "08/01/2021 16:24:26", "content": "Since there is only one downtown phone in private, teams that are improving by focusing on the downtown area are expected to shake down somewhat.
", "rawMarkdown": "Since there is only one downtown phone in private, teams that are improving by focusing on the downtown area are expected to shake down somewhat.", "votes": null }, { "id": "1407323", "postDate": "08/01/2021 18:14:16", "content": "Completely agree with the 95th quantile part, I dont submit things recently because of that reason. Never thought before that quantiles would make me that afraid lol
", "rawMarkdown": "Completely agree with the 95th quantile part, I dont submit things recently because of that reason. Never thought before that quantiles would make me that afraid lol", "votes": null }, { "id": "1407332", "postDate": "08/01/2021 18:20:18", "content": "In the recent thread about CV vs LB scores (https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/254574), many teams had similar results (that CV was lower than LB), so that makes me think that all teams will be similarly affected by the private LB (so no big shake ups).
\nYou never know though!
", "rawMarkdown": "In the recent thread about CV vs LB scores (https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/254574), many teams had similar results (that CV was lower than LB), so that makes me think that all teams will be similarly affected by the private LB (so no big shake ups). \n\nYou never know though!", "votes": null }, { "id": "1407402", "postDate": "08/01/2021 20:29:09", "content": "The team that has a good score in the \"highway\", which has a large impact on the private score, is going to shake up.
", "rawMarkdown": "The team that has a good score in the \"highway\", which has a large impact on the private score, is going to shake up.", "votes": null }, { "id": "1452121", "postDate": "08/05/2021 14:22:19", "content": "Shake up was surprisingly big. I see a lot of people has changed their position by 50+
", "rawMarkdown": "Shake up was surprisingly big. I see a lot of people has changed their position by 50+", "votes": null } ], "comments": [ { "id": 1407218, "author_name": "shimacos", "author_url": "", "post_date": "08/01/2021 16:24:26", "content": "Since there is only one downtown phone in private, teams that are improving by focusing on the downtown area are expected to shake down somewhat.
", "votes": null, "replies": [] }, { "id": 1407323, "author_name": "avtobusbratiev", "author_url": "", "post_date": "08/01/2021 18:14:16", "content": "Completely agree with the 95th quantile part, I dont submit things recently because of that reason. Never thought before that quantiles would make me that afraid lol
", "votes": null, "replies": [] }, { "id": 1407332, "author_name": "chris62", "author_url": "", "post_date": "08/01/2021 18:20:18", "content": "In the recent thread about CV vs LB scores (https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/254574), many teams had similar results (that CV was lower than LB), so that makes me think that all teams will be similarly affected by the private LB (so no big shake ups).
\nYou never know though!
", "votes": null, "replies": [] }, { "id": 1407402, "author_name": "dehokanta", "author_url": "", "post_date": "08/01/2021 20:29:09", "content": "The team that has a good score in the \"highway\", which has a large impact on the private score, is going to shake up.
", "votes": null, "replies": [] }, { "id": 1452121, "author_name": "avtobusbratiev", "author_url": "", "post_date": "08/05/2021 14:22:19", "content": "Shake up was surprisingly big. I see a lot of people has changed their position by 50+
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1407057": "The long drive is almost over.\nHere's one of the classic discussion.\n**Shake up or not shake up?**\n\nA similar competition, the indoor competition, \nseemed to be a competition with relatively few shakes.\n\nWe don't have any unknown test data for this competition. \nSo there may not be that big of a shake.\n\nOn the other hand, \nthe 95th percentile error in the evaluation metric is a bit scary. \nIf you are missing a lot of points in your private data due to post-processing or other reasons, \nthis error may be very large and cause your overall score to worse. \n\nPlease let me know what you think!", "1407218": "Since there is only one downtown phone in private, teams that are improving by focusing on the downtown area are expected to shake down somewhat.", "1407323": "Completely agree with the 95th quantile part, I dont submit things recently because of that reason. Never thought before that quantiles would make me that afraid lol", "1407332": "In the recent thread about CV vs LB scores (https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/254574), many teams had similar results (that CV was lower than LB), so that makes me think that all teams will be similarly affected by the private LB (so no big shake ups). \n\nYou never know though!", "1407402": "The team that has a good score in the \"highway\", which has a large impact on the private score, is going to shake up.", "1452121": "Shake up was surprisingly big. I see a lot of people has changed their position by 50+" }, "source": "meta" }