Do you guys think this competition will have large shakeup?
\n\n.jpeg?generation=1595740831787579&alt=media)
Do you guys think this competition will have large shakeup?
\n\n.jpeg?generation=1595740831787579&alt=media)
@bopengiowa Are you seeing a big difference in your CV vs LB scores and also variation between folds? Is that why you think there will be a large shakeup?
\n\nBesides, isn't it too early to discuss shakeup when the deadline is more than 7-weeks away?
", "rawMarkdown": "bopengiowa Are you seeing a big difference in your CV vs LB scores and also variation between folds? Is that why you think there will be a large shakeup?\n\nBesides, isn't it too early to discuss shakeup when the deadline is more than 7-weeks away?", "votes": null }, { "id": "945746", "postDate": "07/26/2020 05:39:44", "content": "I'm not really playing this competition right now Sirish. I just submitted public notebook. Shakeup is not dependent on how far the deadline, but the distribution in data similarity between public train, public LB and private LB. Yea your right Sirish, I guess CV/LB correlations can be good thing. I'm just asking what people think about shakeup or not.
", "rawMarkdown": "I'm not really playing this competition right now Sirish. I just submitted public notebook. Shakeup is not dependent on how far the deadline, but the distribution in data similarity between public train, public LB and private LB. Yea your right Sirish, I guess CV/LB correlations can be good thing. I'm just asking what people think about shakeup or not.", "votes": null }, { "id": "946011", "postDate": "07/26/2020 09:31:06", "content": "I have never seen this level of discrepancy between train / test in Kaggle competitions, so my answer is yes.
", "rawMarkdown": "I have never seen this level of discrepancy between train / test in Kaggle competitions, so my answer is yes.", "votes": null }, { "id": "946071", "postDate": "07/26/2020 10:30:04", "content": "the results will be similar to CLEFBird2018,19,20 soundscape detection.
", "rawMarkdown": "the results will be similar to CLEFBird2018,19,20 soundscape detection.", "votes": null }, { "id": "947626", "postDate": "07/27/2020 11:35:40", "content": "Yes, definitely think so. Currently the public LB is calculated using only 27% of the test data. Furthermore, many of the recordings are nocalls. The 73% can massively shake things up depending on the species present
", "rawMarkdown": "Yes, definitely think so. Currently the public LB is calculated using only 27% of the test data. Furthermore, many of the recordings are nocalls. The 73% can massively shake things up depending on the species present", "votes": null }, { "id": "947724", "postDate": "07/27/2020 12:55:46", "content": "I think it can be constructive :). Discussing about how data is tested can help inform our model training better. Preparing too late for shakeup can be like setting up oneself for great disappointment
", "rawMarkdown": "I think it can be constructive :). Discussing about how data is tested can help inform our model training better. Preparing too late for shakeup can be like setting up oneself for great disappointment", "votes": null }, { "id": "948507", "postDate": "07/28/2020 02:57:19", "content": "Ok, then I'm going to try to be this guy for the competition.
\n\n.jpeg?generation=1595905021854468&alt=media)
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F385905%2Fd8837126b971dcded31f85fa79c85a67%2Fj84ci6q.jpg?generation=1595905258846824&alt=media\")
Hahaha, thank you for making my day!
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F443651%2F87fef28867b44d7cc792a6dc54853eb7%2FScreenshot%20from%202020-07-28%2014-43-50.png?generation=1595940314533270&alt=media\")
Reviving this post 3 days before the competition end…
", "rawMarkdown": "Reviving this post 3 days before the competition end...", "votes": null }, { "id": "1008400", "postDate": "09/13/2020 05:05:45", "content": "I think the top performing team (money and gold zone) will stay as they mention their CV is similar to LB with less than 0.01 discrepancy.
\nSilver and bronze zone will hv huge shake up becoz many of them (including me) is just a random guess / overfit (CV is very different from LB)
", "rawMarkdown": "I think the top performing team (money and gold zone) will stay as they mention their CV is similar to LB with less than 0.01 discrepancy. \n\nSilver and bronze zone will hv huge shake up becoz many of them (including me) is just a random guess / overfit (CV is very different from LB)", "votes": null }, { "id": "1008648", "postDate": "09/13/2020 09:48:59", "content": "If public/private test is not random then anything is possible. If it is random then the only source of shakeup would be the people who overfitted like crazy to public LB by only using LB feedback for guiding their model selection. In my experience with past competitions many do this, unfortunately for them. I guess some of the top teams did set proper CV hence there will be little shakeup at the top.
\nI will be very cautious for our team fate as we did not find a good CV setting that mimics what we know of test data. We entered too late probably and did not spend time on CV setting. A big mistake IMHO. We'll see.
\nMy 2 cents.
", "rawMarkdown": "If public/private test is not random then anything is possible. If it is random then the only source of shakeup would be the people who overfitted like crazy to public LB by only using LB feedback for guiding their model selection. In my experience with past competitions many do this, unfortunately for them. I guess some of the top teams did set proper CV hence there will be little shakeup at the top.\n\nI will be very cautious for our team fate as we did not find a good CV setting that mimics what we know of test data. We entered too late probably and did not spend time on CV setting. A big mistake IMHO. We'll see.\n\nMy 2 cents.", "votes": null } ], "comments": [ { "id": 1008322, "author_name": "tonychenxyz", "author_url": "", "post_date": "09/13/2020 02:46:07", "content": "Reviving this post 3 days before the competition end…
", "votes": null, "replies": [ { "id": 1008400, "author_name": "fiyeroleung", "author_url": "", "post_date": "09/13/2020 05:05:45", "content": "I think the top performing team (money and gold zone) will stay as they mention their CV is similar to LB with less than 0.01 discrepancy.
\nSilver and bronze zone will hv huge shake up becoz many of them (including me) is just a random guess / overfit (CV is very different from LB)
", "votes": null, "replies": [] } ] }, { "id": 1008648, "author_name": "cpmpml", "author_url": "", "post_date": "09/13/2020 09:48:59", "content": "If public/private test is not random then anything is possible. If it is random then the only source of shakeup would be the people who overfitted like crazy to public LB by only using LB feedback for guiding their model selection. In my experience with past competitions many do this, unfortunately for them. I guess some of the top teams did set proper CV hence there will be little shakeup at the top.
\nI will be very cautious for our team fate as we did not find a good CV setting that mimics what we know of test data. We entered too late probably and did not spend time on CV setting. A big mistake IMHO. We'll see.
\nMy 2 cents.
", "votes": null, "replies": [] }, { "id": 945735, "author_name": "sirishks", "author_url": "", "post_date": "07/26/2020 05:30:36", "content": "@bopengiowa Are you seeing a big difference in your CV vs LB scores and also variation between folds? Is that why you think there will be a large shakeup?
\n\nBesides, isn't it too early to discuss shakeup when the deadline is more than 7-weeks away?
", "votes": null, "replies": [ { "id": 945746, "author_name": "bopengiowa", "author_url": "", "post_date": "07/26/2020 05:39:44", "content": "I'm not really playing this competition right now Sirish. I just submitted public notebook. Shakeup is not dependent on how far the deadline, but the distribution in data similarity between public train, public LB and private LB. Yea your right Sirish, I guess CV/LB correlations can be good thing. I'm just asking what people think about shakeup or not.
", "votes": null, "replies": [] }, { "id": 947724, "author_name": "alanchn31", "author_url": "", "post_date": "07/27/2020 12:55:46", "content": "I think it can be constructive :). Discussing about how data is tested can help inform our model training better. Preparing too late for shakeup can be like setting up oneself for great disappointment
", "votes": null, "replies": [] } ] }, { "id": 946011, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "07/26/2020 09:31:06", "content": "I have never seen this level of discrepancy between train / test in Kaggle competitions, so my answer is yes.
", "votes": null, "replies": [] }, { "id": 946071, "author_name": "hengck23", "author_url": "", "post_date": "07/26/2020 10:30:04", "content": "the results will be similar to CLEFBird2018,19,20 soundscape detection.
", "votes": null, "replies": [] }, { "id": 947626, "author_name": "alanchn31", "author_url": "", "post_date": "07/27/2020 11:35:40", "content": "Yes, definitely think so. Currently the public LB is calculated using only 27% of the test data. Furthermore, many of the recordings are nocalls. The 73% can massively shake things up depending on the species present
", "votes": null, "replies": [] }, { "id": 948507, "author_name": "bopengiowa", "author_url": "", "post_date": "07/28/2020 02:57:19", "content": "Ok, then I'm going to try to be this guy for the competition.
\n\n.jpeg?generation=1595905021854468&alt=media)
Hahaha, thank you for making my day!
\n\n