{ "id": 377894, "title": "Revealing the 4th digit of the score", "url": "/competitions/otto-recommender-system/discussion/377894", "author_name": "NikhilMishra", "post_date": "2023-01-13T10:35:58.396000", "votes": 13, "comment_count": 14, "views": 0, "content": "

Can kaggle reveal the 4th digit of the score for the competition ? With many people being stuck at the same score like 0.602, 0.6, 0.599, I think it will be helpful to better track our progress. Otherwise the crowd is bound to grow even more @maggiemd

", "messages": [ { "id": 2098223, "postDate": "2023-01-13T10:35:58.397Z", "content": "

Can kaggle reveal the 4th digit of the score for the competition ? With many people being stuck at the same score like 0.602, 0.6, 0.599, I think it will be helpful to better track our progress. Otherwise the crowd is bound to grow even more @maggiemd

", "rawMarkdown": "Can kaggle reveal the 4th digit of the score for the competition ? With many people being stuck at the same score like 0.602, 0.6, 0.599, I think it will be helpful to better track our progress. Otherwise the crowd is bound to grow even more @maggiemd ", "votes": 13 }, { "id": 2098465, "postDate": "2023-01-13T15:34:00.973Z", "content": "

In general, while the Public LB are useful, they are not always a good indicator of model accuracy. More decimal points are not done to avoid LB probing. Focus on improving your CV and choose the best CV.

", "rawMarkdown": "In general, while the Public LB are useful, they are not always a good indicator of model accuracy. More decimal points are not done to avoid LB probing. Focus on improving your CV and choose the best CV. ", "votes": 6, "replies": [ { "id": 2098577, "postDate": "2023-01-13T17:06:59.400Z", "content": "

this time will take this advice .. recently i lost 4 sliver medal coz i ignore CV and choice the based on LB

", "rawMarkdown": "this time will take this advice .. recently i lost 4 sliver medal coz i ignore CV and choice the based on LB ", "votes": -1, "replies": [ { "id": 2098897, "postDate": "2023-01-13T23:36:12.890Z", "content": "

Overfitting a leaderboard is much easier than finding a good solution

", "rawMarkdown": "Overfitting a leaderboard is much easier than finding a good solution" } ] } ] }, { "id": 2106716, "postDate": "2023-01-19T10:18:13.120Z", "content": "

Poke @pnormann !
\nI think LB is reliable enough for us to have a 4th digit without worrying about people overfitting.

\n

It's gonna look quite awkward when the top 10 teams have the same 0.602 score otherwise :)

", "rawMarkdown": "Poke @pnormann ! \nI think LB is reliable enough for us to have a 4th digit without worrying about people overfitting.\n\nIt's gonna look quite awkward when the top 10 teams have the same 0.602 score otherwise :)", "votes": 1, "replies": [ { "id": 2106759, "postDate": "2023-01-19T10:41:31.953Z", "content": "

or an or alternate solution, distribute prize money to all teams scoring 0.602 :D

", "rawMarkdown": "or an or alternate solution, distribute prize money to all teams scoring 0.602 :D " }, { "id": 2107180, "postDate": "2023-01-19T16:09:10.093Z", "content": "

Hi @theoviel, we do not plan on increasing leaderboard precision until the competition has concluded. Competitors should rely on their local validation for model selection.

", "rawMarkdown": "Hi @theoviel, we do not plan on increasing leaderboard precision until the competition has concluded. Competitors should rely on their local validation for model selection.", "votes": 6, "replies": [ { "id": 2107209, "postDate": "2023-01-19T16:25:07.017Z", "content": "

Makes sense!
\nThanks for the quick reply :)

", "rawMarkdown": "Makes sense!\nThanks for the quick reply :)", "votes": 1 } ] } ] }, { "id": 2104730, "postDate": "2023-01-18T02:05:20.530Z", "content": "

Can you take a loot into this? @maggiemd @inversion @sohier

", "rawMarkdown": "Can you take a loot into this? @maggiemd @inversion @sohier " }, { "id": 2099046, "postDate": "2023-01-14T05:34:23.847Z", "content": "

I agree with you. The difference of 4th digit score is important for top teams.
\nRevealing the 4th digit of the score is helpful to develop worthwhile solutions.

", "rawMarkdown": "I agree with you. The difference of 4th digit score is important for top teams.\nRevealing the 4th digit of the score is helpful to develop worthwhile solutions." }, { "id": 2098558, "postDate": "2023-01-13T16:49:07.843Z", "content": "

Agree, we have made many same-score submission but can only randomly guess their improvement.

", "rawMarkdown": "Agree, we have made many same-score submission but can only randomly guess their improvement.", "replies": [ { "id": 2098570, "postDate": "2023-01-13T17:02:29.597Z", "content": "

exactly all I see is juggling of the top 3 spots without any improvement in the leaderboard :D

", "rawMarkdown": "exactly all I see is juggling of the top 3 spots without any improvement in the leaderboard :D" } ] }, { "id": 2098255, "postDate": "2023-01-13T11:15:23.233Z", "content": "

I agree - that would be very helpful.

", "rawMarkdown": "I agree - that would be very helpful." }, { "id": 2106763, "postDate": "2023-01-19T10:45:47.810Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 2099181, "postDate": "2023-01-14T08:41:00.210Z", "rawMarkdown": "", "votes": -2, "isDeleted": true } ], "comments": [ { "id": 2098465, "author_name": "Kamal Das", "author_url": "", "post_date": "2023-01-13T15:34:00.973000", "content": "

In general, while the Public LB are useful, they are not always a good indicator of model accuracy. More decimal points are not done to avoid LB probing. Focus on improving your CV and choose the best CV.

", "votes": 6, "replies": [ { "id": 2098577, "author_name": "AlmaamoonAl-saedi", "author_url": "", "post_date": "2023-01-13T17:06:59.400000", "content": "

this time will take this advice .. recently i lost 4 sliver medal coz i ignore CV and choice the based on LB

", "votes": -1, "replies": [ { "id": 2098897, "author_name": "Thomas", "author_url": "", "post_date": "2023-01-13T23:36:12.890000", "content": "

Overfitting a leaderboard is much easier than finding a good solution

", "votes": 0, "replies": [] } ] } ] }, { "id": 2106716, "author_name": "Theo Viel", "author_url": "", "post_date": "2023-01-19T10:18:13.120000", "content": "

Poke @pnormann !
\nI think LB is reliable enough for us to have a 4th digit without worrying about people overfitting.

\n

It's gonna look quite awkward when the top 10 teams have the same 0.602 score otherwise :)

", "votes": 1, "replies": [ { "id": 2106759, "author_name": "NikhilMishra", "author_url": "", "post_date": "2023-01-19T10:41:31.953000", "content": "

or an or alternate solution, distribute prize money to all teams scoring 0.602 :D

", "votes": 0, "replies": [] }, { "id": 2107180, "author_name": "Philipp Normann", "author_url": "", "post_date": "2023-01-19T16:09:10.093000", "content": "

Hi @theoviel, we do not plan on increasing leaderboard precision until the competition has concluded. Competitors should rely on their local validation for model selection.

", "votes": 6, "replies": [ { "id": 2107209, "author_name": "Theo Viel", "author_url": "", "post_date": "2023-01-19T16:25:07.017000", "content": "

Makes sense!
\nThanks for the quick reply :)

", "votes": 1, "replies": [] } ] } ] }, { "id": 2104730, "author_name": "ONODERA", "author_url": "", "post_date": "2023-01-18T02:05:20.530000", "content": "

Can you take a loot into this? @maggiemd @inversion @sohier

", "votes": 0, "replies": [] }, { "id": 2099046, "author_name": "toshi_k", "author_url": "", "post_date": "2023-01-14T05:34:23.847000", "content": "

I agree with you. The difference of 4th digit score is important for top teams.
\nRevealing the 4th digit of the score is helpful to develop worthwhile solutions.

", "votes": 0, "replies": [] }, { "id": 2098558, "author_name": "Carno Zhao", "author_url": "", "post_date": "2023-01-13T16:49:07.843000", "content": "

Agree, we have made many same-score submission but can only randomly guess their improvement.

", "votes": 0, "replies": [ { "id": 2098570, "author_name": "NikhilMishra", "author_url": "", "post_date": "2023-01-13T17:02:29.597000", "content": "

exactly all I see is juggling of the top 3 spots without any improvement in the leaderboard :D

", "votes": 0, "replies": [] } ] }, { "id": 2098255, "author_name": "John Wakefield", "author_url": "", "post_date": "2023-01-13T11:15:23.233000", "content": "

I agree - that would be very helpful.

", "votes": 0, "replies": [] }, { "id": 2106763, "author_name": "", "author_url": "", "post_date": "2023-01-19T10:45:47.810000", "content": "", "votes": -1, "replies": [] }, { "id": 2099181, "author_name": "", "author_url": "", "post_date": "2023-01-14T08:41:00.210000", "content": "", "votes": -2, "replies": [] } ], "raw_markdown_by_id": { "2098223": "Can kaggle reveal the 4th digit of the score for the competition ? With many people being stuck at the same score like 0.602, 0.6, 0.599, I think it will be helpful to better track our progress. Otherwise the crowd is bound to grow even more @maggiemd ", "2098465": "In general, while the Public LB are useful, they are not always a good indicator of model accuracy. More decimal points are not done to avoid LB probing. Focus on improving your CV and choose the best CV. ", "2106716": "Poke @pnormann ! \nI think LB is reliable enough for us to have a 4th digit without worrying about people overfitting.\n\nIt's gonna look quite awkward when the top 10 teams have the same 0.602 score otherwise :)", "2104730": "Can you take a loot into this? @maggiemd @inversion @sohier ", "2099046": "I agree with you. The difference of 4th digit score is important for top teams.\nRevealing the 4th digit of the score is helpful to develop worthwhile solutions.", "2098558": "Agree, we have made many same-score submission but can only randomly guess their improvement.", "2098255": "I agree - that would be very helpful.", "2106763": "", "2099181": "" } }