![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2006644%2F37ec4663ee99805a86fbc99d530d6f1a%2Fdownload.png?generation=1593524284516478&alt=media\")
I made an estimation of public/private LB.
\n\n\n\n\nThis leaderboard is calculated with approximately 20% of the test data.\n The final results will be based on the other 80%, so the final standings may be different.
\n
The result like below\n
.png?generation=1593524356349332&alt=media)
If this is correct, does it mean the public LB is too small. How do you think about it.
", "messages": [ { "id": 908303, "postDate": "2020-06-30T13:52:09.193Z", "content": "I made an estimation of public/private LB.
\n\n\n\n\nThis leaderboard is calculated with approximately 20% of the test data.\n The final results will be based on the other 80%, so the final standings may be different.
\n
The result like below\n
.png?generation=1593524356349332&alt=media)
If this is correct, does it mean the public LB is too small. How do you think about it.
", "rawMarkdown": "I made an estimation of public/private LB.\n\n> This leaderboard is calculated with approximately 20% of the test data.\nThe final results will be based on the other 80%, so the final standings may be different.\n\n1. Use a model to predict the train dataset (75000 x 4 = 300000 images).\n2. Random choice 5000 from 300000\n3. Random choice 1000 from 5000\n4. Calculate the wAUC of 1000 and other 4000\n5. Loop 2~4\n\nThe result like below\n\n\n.png?generation=1593524356349332&alt=media)\n\nIf this is correct, does it mean the public LB is too small. How do you think about it. ", "votes": 11 }, { "id": 909366, "postDate": "2020-06-30T14:42:55.137Z", "content": "Great approach! It should give a rough idea about the shake-up that might come.
\n\nI think you should choose the random 5000 from your validation or OOF set. That way there's a guarantee that the model will process only unseen data.\nThen you can again randomly choose 1k from the above 5k to simulate the public/private LB
", "rawMarkdown": "Great approach! It should give a rough idea about the shake-up that might come.\n\nI think you should choose the random 5000 from your validation or OOF set. That way there's a guarantee that the model will process only unseen data.\nThen you can again randomly choose 1k from the above 5k to simulate the public/private LB", "votes": 3 }, { "id": 921343, "postDate": "2020-07-09T08:17:02.413Z", "content": "Even with 60,000 - the size of most val sets during training, the range is significant at .004 - be careful even with local cv choice for final submission.
\n\nRandom 60000, from an example 300000 CV set, 10000 times below.\nCould the metric be tuned for less random impact?
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1582365%2F2d14539e28329ae503c41a5f4f5fec76%2FScreenshot%202020-07-09%2009.16.07.png?generation=1594282614100250&alt=media\")
Great approach! It should give a rough idea about the shake-up that might come.
\n\nI think you should choose the random 5000 from your validation or OOF set. That way there's a guarantee that the model will process only unseen data.\nThen you can again randomly choose 1k from the above 5k to simulate the public/private LB
", "votes": 3, "replies": [] }, { "id": 921343, "author_name": "robga", "author_url": "", "post_date": "2020-07-09T08:17:02.413000", "content": "Even with 60,000 - the size of most val sets during training, the range is significant at .004 - be careful even with local cv choice for final submission.
\n\nRandom 60000, from an example 300000 CV set, 10000 times below.\nCould the metric be tuned for less random impact?
\n\n