The first place 0.9931 is on the whole other level while all other participants are stuck within 0.97-0.96 range but mostly in the 0.96 range cause of the public kernel. Any takes or shakeup thoughts?
", "messages": [ { "id": "962715", "postDate": "08/08/2020 11:26:34", "content": "The first place 0.9931 is on the whole other level while all other participants are stuck within 0.97-0.96 range but mostly in the 0.96 range cause of the public kernel. Any takes or shakeup thoughts?
", "rawMarkdown": "The first place 0.9931 is on the whole other level while all other participants are stuck within 0.97-0.96 range but mostly in the 0.96 range cause of the public kernel. Any takes or shakeup thoughts?", "votes": null }, { "id": "962717", "postDate": "08/08/2020 11:28:18", "content": "This score is very likely due to intensive probing of public test data.
", "rawMarkdown": "This score is very likely due to intensive probing of public test data.", "votes": null }, { "id": "962722", "postDate": "08/08/2020 11:30:09", "content": "It is. He admitted it in a topic he dedicated to it :). If he also knows what the private and public images are, then he has a great advantage (pseudo-labeling or just extra data that is similar to the private LB).
", "rawMarkdown": "It is. He admitted it in a topic he dedicated to it :). If he also knows what the private and public images are, then he has a great advantage (pseudo-labeling or just extra data that is similar to the private LB).", "votes": null }, { "id": "962726", "postDate": "08/08/2020 11:34:30", "content": "Given hand labelling of test data is forbidden by rules, using ground truth derived from LB probing is forbidden…
", "rawMarkdown": "Given hand labelling of test data is forbidden by rules, using ground truth derived from LB probing is forbidden...", "votes": null }, { "id": "962807", "postDate": "08/08/2020 12:57:57", "content": "The question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.
", "rawMarkdown": "The question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.", "votes": null }, { "id": "962918", "postDate": "08/08/2020 14:39:38", "content": "If he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?
", "rawMarkdown": "If he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?", "votes": null }, { "id": "963040", "postDate": "08/08/2020 16:09:20", "content": "\n\nThe question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.
\n
They will check prize winners code for sure.
", "rawMarkdown": "> The question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.\n\nThey will check prize winners code for sure.", "votes": null }, { "id": "963041", "postDate": "08/08/2020 16:09:55", "content": "\n\nIf he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?
\n
Maybe not if he has a great solution. Which is very possible.
", "rawMarkdown": "> If he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?\n\nMaybe not if he has a great solution. Which is very possible.", "votes": null }, { "id": "963279", "postDate": "08/08/2020 20:51:40", "content": "There's a difference between hand-labelling test data and using ground truth derived by LB probing, as long as that's done algorithmically. Link1, Link2.
", "rawMarkdown": "There's a difference between hand-labelling test data and using ground truth derived by LB probing, as long as that's done algorithmically. [Link1](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/171801#959706), [Link2](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/170846#950967).", "votes": null }, { "id": "963330", "postDate": "08/08/2020 22:59:48", "content": "LB probing is an algorithm?
", "rawMarkdown": "LB probing is an algorithm?", "votes": null } ], "comments": [ { "id": 962717, "author_name": "cpmpml", "author_url": "", "post_date": "08/08/2020 11:28:18", "content": "This score is very likely due to intensive probing of public test data.
", "votes": null, "replies": [ { "id": 962722, "author_name": "group16", "author_url": "", "post_date": "08/08/2020 11:30:09", "content": "It is. He admitted it in a topic he dedicated to it :). If he also knows what the private and public images are, then he has a great advantage (pseudo-labeling or just extra data that is similar to the private LB).
", "votes": null, "replies": [] }, { "id": 962726, "author_name": "cpmpml", "author_url": "", "post_date": "08/08/2020 11:34:30", "content": "Given hand labelling of test data is forbidden by rules, using ground truth derived from LB probing is forbidden…
", "votes": null, "replies": [] }, { "id": 962807, "author_name": "group16", "author_url": "", "post_date": "08/08/2020 12:57:57", "content": "The question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.
", "votes": null, "replies": [] }, { "id": 962918, "author_name": "fahad7", "author_url": "", "post_date": "08/08/2020 14:39:38", "content": "If he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?
", "votes": null, "replies": [] }, { "id": 963040, "author_name": "cpmpml", "author_url": "", "post_date": "08/08/2020 16:09:20", "content": "\n\nThe question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.
\n
They will check prize winners code for sure.
", "votes": null, "replies": [] }, { "id": 963041, "author_name": "cpmpml", "author_url": "", "post_date": "08/08/2020 16:09:55", "content": "\n\nIf he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?
\n
Maybe not if he has a great solution. Which is very possible.
", "votes": null, "replies": [] }, { "id": 963279, "author_name": "rohitagarwal", "author_url": "", "post_date": "08/08/2020 20:51:40", "content": "There's a difference between hand-labelling test data and using ground truth derived by LB probing, as long as that's done algorithmically. Link1, Link2.
", "votes": null, "replies": [] }, { "id": 963330, "author_name": "cpmpml", "author_url": "", "post_date": "08/08/2020 22:59:48", "content": "LB probing is an algorithm?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "962715": "The first place 0.9931 is on the whole other level while all other participants are stuck within 0.97-0.96 range but mostly in the 0.96 range cause of the public kernel. Any takes or shakeup thoughts?", "962717": "This score is very likely due to intensive probing of public test data.", "962722": "It is. He admitted it in a topic he dedicated to it :). If he also knows what the private and public images are, then he has a great advantage (pseudo-labeling or just extra data that is similar to the private LB).", "962726": "Given hand labelling of test data is forbidden by rules, using ground truth derived from LB probing is forbidden...", "962807": "The question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.", "962918": "If he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?", "963040": "> The question is how they will check that. Kaggle is not that thorough or strict when it comes to reproduction/checking of winner's solutions.\n\nThey will check prize winners code for sure.", "963041": "> If he has intensively probed the public test data, doesn't that mean he would surely fall down the private leaderboard?\n\nMaybe not if he has a great solution. Which is very possible.", "963279": "There's a difference between hand-labelling test data and using ground truth derived by LB probing, as long as that's done algorithmically. [Link1](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/171801#959706), [Link2](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/170846#950967).", "963330": "LB probing is an algorithm?" }, "source": "meta" }