Just curious.
", "messages": [ { "id": 775804, "postDate": "2020-03-17T01:13:05.363Z", "content": "Just curious.
", "rawMarkdown": "Just curious.", "votes": 3 }, { "id": 775845, "postDate": "2020-03-17T01:43:56.690Z", "content": "One important reason may be that everyone was chasing the last few errors in the public test set and to do so they eventually overtuned the model for that. also, there was an incredible amount of information shared in the forum which can result in big shakeups.
", "rawMarkdown": "One important reason may be that everyone was chasing the last few errors in the public test set and to do so they eventually overtuned the model for that. also, there was an incredible amount of information shared in the forum which can result in big shakeups.", "votes": 1 }, { "id": 776016, "postDate": "2020-03-17T04:40:32.507Z", "content": "If you're curious what caused the shakeup, i provide some clues in my writeup here. Namely correcting macro recall for the rare classes increases private LB from 0.930 to 0.955
", "rawMarkdown": "If you're curious what caused the shakeup, i provide some clues in my writeup [here][1]. Namely correcting macro recall for the rare classes increases private LB from 0.930 to 0.955\n\n[1]: https://www.kaggle.com/c/bengaliai-cv19/discussion/136021", "votes": 2 }, { "id": 775848, "postDate": "2020-03-17T01:47:21.693Z", "content": "I dropped 1200 position... My best model is 0.9389 which is top 50, but I selected my lastest models which scored much higher in lb.. It doesn't really make sense.. the better lb score i have the worst final score i got. I think the lb should give a hint of what score we should get.. oh well i learned something today
", "rawMarkdown": "I dropped 1200 position... My best model is 0.9389 which is top 50, but I selected my lastest models which scored much higher in lb.. It doesn't really make sense.. the better lb score i have the worst final score i got. I think the lb should give a hint of what score we should get.. oh well i learned something today", "votes": 2, "replies": [ { "id": 775851, "postDate": "2020-03-17T01:52:07.083Z", "content": "@yannmajewski LB does not suggests final leaderboard since they use different datasets. But usually fina leaderboard score should correlate with local CV score, but in my experiement that is not the case. Also, since 54% of data is used for public LB, this significant shake is truly surprising.
", "rawMarkdown": "@yannmajewski LB does not suggests final leaderboard since they use different datasets. But usually fina leaderboard score should correlate with local CV score, but in my experiement that is not the case. Also, since 54% of data is used for public LB, this significant shake is truly surprising." }, { "id": 775860, "postDate": "2020-03-17T01:56:46.183Z", "content": "@tonychenxyz Yes this is what i mean, it should reflect a little bit what the final score should be! Oh well :(
", "rawMarkdown": "@tonychenxyz Yes this is what i mean, it should reflect a little bit what the final score should be! Oh well :(" } ] }, { "id": 775875, "postDate": "2020-03-17T02:12:18.580Z", "content": "My lb score doesn't correlate with private score much. I don't even know why my best private submission would be best. A little bit disappointed about the results
", "rawMarkdown": "My lb score doesn't correlate with private score much. I don't even know why my best private submission would be best. A little bit disappointed about the results" }, { "id": 775826, "postDate": "2020-03-17T01:28:36.750Z", "rawMarkdown": "", "votes": -1, "isDeleted": true } ], "comments": [ { "id": 775845, "author_name": "Dhananjay Raut", "author_url": "", "post_date": "2020-03-17T01:43:56.690000", "content": "One important reason may be that everyone was chasing the last few errors in the public test set and to do so they eventually overtuned the model for that. also, there was an incredible amount of information shared in the forum which can result in big shakeups.
", "votes": 1, "replies": [] }, { "id": 776016, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-03-17T04:40:32.507000", "content": "If you're curious what caused the shakeup, i provide some clues in my writeup here. Namely correcting macro recall for the rare classes increases private LB from 0.930 to 0.955
", "votes": 2, "replies": [] }, { "id": 775848, "author_name": "Yann Majewski", "author_url": "", "post_date": "2020-03-17T01:47:21.693000", "content": "I dropped 1200 position... My best model is 0.9389 which is top 50, but I selected my lastest models which scored much higher in lb.. It doesn't really make sense.. the better lb score i have the worst final score i got. I think the lb should give a hint of what score we should get.. oh well i learned something today
", "votes": 2, "replies": [ { "id": 775851, "author_name": "Gold Retriever", "author_url": "", "post_date": "2020-03-17T01:52:07.083000", "content": "@yannmajewski LB does not suggests final leaderboard since they use different datasets. But usually fina leaderboard score should correlate with local CV score, but in my experiement that is not the case. Also, since 54% of data is used for public LB, this significant shake is truly surprising.
", "votes": 0, "replies": [] }, { "id": 775860, "author_name": "Yann Majewski", "author_url": "", "post_date": "2020-03-17T01:56:46.183000", "content": "@tonychenxyz Yes this is what i mean, it should reflect a little bit what the final score should be! Oh well :(
", "votes": 0, "replies": [] } ] }, { "id": 775875, "author_name": "syoya", "author_url": "", "post_date": "2020-03-17T02:12:18.580000", "content": "My lb score doesn't correlate with private score much. I don't even know why my best private submission would be best. A little bit disappointed about the results
", "votes": 0, "replies": [] }, { "id": 775826, "author_name": "", "author_url": "", "post_date": "2020-03-17T01:28:36.750000", "content": "", "votes": -1, "replies": [] } ], "raw_markdown_by_id": { "775804": "Just curious.", "775845": "One important reason may be that everyone was chasing the last few errors in the public test set and to do so they eventually overtuned the model for that. also, there was an incredible amount of information shared in the forum which can result in big shakeups.", "776016": "If you're curious what caused the shakeup, i provide some clues in my writeup [here][1]. Namely correcting macro recall for the rare classes increases private LB from 0.930 to 0.955\n\n[1]: https://www.kaggle.com/c/bengaliai-cv19/discussion/136021", "775848": "I dropped 1200 position... My best model is 0.9389 which is top 50, but I selected my lastest models which scored much higher in lb.. It doesn't really make sense.. the better lb score i have the worst final score i got. I think the lb should give a hint of what score we should get.. oh well i learned something today", "775875": "My lb score doesn't correlate with private score much. I don't even know why my best private submission would be best. A little bit disappointed about the results", "775826": "" } }