![\"86\"](\"https://i.imgur.com/RliopAl.png\")
\n
![\"261\"](\"https://i.imgur.com/Wgq9qWT.png\")
This competition definitely needs 4th decimal sign for submission scoring
\nIt is absolutely non-obvious what progress does model have
\n0.799 results are from 19th place to 305th
This competition definitely needs 4th decimal sign for submission scoring
\nIt is absolutely non-obvious what progress does model have
\n0.799 results are from 19th place to 305th
This competition CV and LB didn’t match well.The 4 decimal only make the score overfit and overfit
", "rawMarkdown": "This competition CV and LB didn’t match well.The 4 decimal only make the score overfit and overfit", "votes": 5, "replies": [ { "id": 1852421, "postDate": "2022-07-12T04:14:12.787Z", "content": "OK, may be You are right
", "rawMarkdown": "OK, may be You are right" } ] }, { "id": 1851622, "postDate": "2022-07-11T11:53:42.060Z", "content": "Do we really need the fourth decimal? As the lb ordering takes the hidden decimals into account, we may assume that 19th place is 0.7999 and 305th place is 0.7990. And even without seeing the fourth decimal, we can guess that places 86 through 261 all submitted identical predictions (identical scores are sorted by submission datetime):
\n
\n
0.7995 and bigger must be rounded to 0.800
\n0.799 lies between 0.7995 and 0.7986
\nof course, we can assume that one position at the LB costs something near 0.001 / (305-18)
\nbut when I making submission all of them can be 0.799 with no progress
\nand I can’t understand what are real changes in result
\nmy submission score can be 0.7986 ibstead of 0.7991, but I can’t see it
and this particular competition seems very specific because of metric that changes really slow
", "rawMarkdown": "and this particular competition seems very specific because of metric that changes really slow" }, { "id": 1851702, "postDate": "2022-07-11T12:55:57.367Z", "content": "\n\n0.7995 and bigger must be rounded to 0.800
\n
No. Kaggle truncates. Kaggle does not round. So 0.7999 will display as 0.799.
", "rawMarkdown": ">0.7995 and bigger must be rounded to 0.800\n\nNo. Kaggle truncates. Kaggle does not round. So 0.7999 will display as 0.799.", "votes": 8 }, { "id": 1851704, "postDate": "2022-07-11T13:00:21.217Z", "content": "thanks!
\nbut I still think 4th sign would be usefull to see the actual results changes if submission does not improve the score
You can sort your own submissions by public score to see which one has improved or not.
", "rawMarkdown": "You can sort your own submissions by public score to see which one has improved or not. ", "votes": 3 } ] }, { "id": 1851593, "postDate": "2022-07-11T11:30:09.230Z", "content": "Well, don't forget that the best public solution has a score of 0.799. Therefore many of the solutions between 19 and 305 heavily depend on this solution, and may even be identical. I think one argument against more precision is that it doesn't really add much value, since after 3 decimals places it becomes mostly deviation by noise anyway. But overall I agree, one more digit would be nice to have :)
", "rawMarkdown": "Well, don't forget that the best public solution has a score of 0.799. Therefore many of the solutions between 19 and 305 heavily depend on this solution, and may even be identical. I think one argument against more precision is that it doesn't really add much value, since after 3 decimals places it becomes mostly deviation by noise anyway. But overall I agree, one more digit would be nice to have :)", "votes": 2, "replies": [ { "id": 1851672, "postDate": "2022-07-11T12:38:51.100Z", "content": "main reason for having 4th decimal sign is too see changes in submission scoring if it was not the best one
\nright now I can’t see if may model have no improvements or if it became even worse than may submissions are not better than my best try
As stated a couple of times - SORT your submissions list by Public Score - that will tell you if today's submission is better or worse than yesterdays. Of course, the big question would be - is 0.000x a true improvement.
", "rawMarkdown": "As stated a couple of times - SORT your submissions list by Public Score - that will tell you if today's submission is better or worse than yesterdays. Of course, the big question would be - is 0.000x a true improvement." } ] }, { "id": 1861009, "postDate": "2022-07-18T18:00:07.630Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1852282, "author_name": "Young for you", "author_url": "", "post_date": "2022-07-12T01:17:32.197000", "content": "This competition CV and LB didn’t match well.The 4 decimal only make the score overfit and overfit
", "votes": 5, "replies": [ { "id": 1852421, "author_name": "Oleg Khudyakov", "author_url": "", "post_date": "2022-07-12T04:14:12.787000", "content": "OK, may be You are right
", "votes": 0, "replies": [] } ] }, { "id": 1851622, "author_name": "AmbrosM", "author_url": "", "post_date": "2022-07-11T11:53:42.060000", "content": "Do we really need the fourth decimal? As the lb ordering takes the hidden decimals into account, we may assume that 19th place is 0.7999 and 305th place is 0.7990. And even without seeing the fourth decimal, we can guess that places 86 through 261 all submitted identical predictions (identical scores are sorted by submission datetime):
\n
\n
0.7995 and bigger must be rounded to 0.800
\n0.799 lies between 0.7995 and 0.7986
\nof course, we can assume that one position at the LB costs something near 0.001 / (305-18)
\nbut when I making submission all of them can be 0.799 with no progress
\nand I can’t understand what are real changes in result
\nmy submission score can be 0.7986 ibstead of 0.7991, but I can’t see it
and this particular competition seems very specific because of metric that changes really slow
", "votes": 0, "replies": [] }, { "id": 1851702, "author_name": "Chris Deotte", "author_url": "", "post_date": "2022-07-11T12:55:57.367000", "content": "\n\n0.7995 and bigger must be rounded to 0.800
\n
No. Kaggle truncates. Kaggle does not round. So 0.7999 will display as 0.799.
", "votes": 8, "replies": [] }, { "id": 1851704, "author_name": "Oleg Khudyakov", "author_url": "", "post_date": "2022-07-11T13:00:21.217000", "content": "thanks!
\nbut I still think 4th sign would be usefull to see the actual results changes if submission does not improve the score
You can sort your own submissions by public score to see which one has improved or not.
", "votes": 3, "replies": [] } ] }, { "id": 1851593, "author_name": "Fritz Cremer", "author_url": "", "post_date": "2022-07-11T11:30:09.230000", "content": "Well, don't forget that the best public solution has a score of 0.799. Therefore many of the solutions between 19 and 305 heavily depend on this solution, and may even be identical. I think one argument against more precision is that it doesn't really add much value, since after 3 decimals places it becomes mostly deviation by noise anyway. But overall I agree, one more digit would be nice to have :)
", "votes": 2, "replies": [ { "id": 1851672, "author_name": "Oleg Khudyakov", "author_url": "", "post_date": "2022-07-11T12:38:51.100000", "content": "main reason for having 4th decimal sign is too see changes in submission scoring if it was not the best one
\nright now I can’t see if may model have no improvements or if it became even worse than may submissions are not better than my best try
As stated a couple of times - SORT your submissions list by Public Score - that will tell you if today's submission is better or worse than yesterdays. Of course, the big question would be - is 0.000x a true improvement.
", "votes": 0, "replies": [] } ] }, { "id": 1861009, "author_name": "", "author_url": "", "post_date": "2022-07-18T18:00:07.630000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1851568": "This competition definitely needs 4th decimal sign for submission scoring\nIt is absolutely non-obvious what progress does model have\n0.799 results are from 19th place to 305th\n", "1852282": "This competition CV and LB didn’t match well.The 4 decimal only make the score overfit and overfit", "1851622": "Do we really need the fourth decimal? As the lb ordering takes the hidden decimals into account, we may assume that 19th place is 0.7999 and 305th place is 0.7990. And even without seeing the fourth decimal, we can guess that places 86 through 261 all submitted identical predictions (identical scores are sorted by submission datetime):\n\n\n\n", "1851593": "Well, don't forget that the best public solution has a score of 0.799. Therefore many of the solutions between 19 and 305 heavily depend on this solution, and may even be identical. I think one argument against more precision is that it doesn't really add much value, since after 3 decimals places it becomes mostly deviation by noise anyway. But overall I agree, one more digit would be nice to have :)", "1861009": "" } }