\n
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16772839%2F53d2feed4488c1cc1f75f3d88945a195%2FScreenshot%202024-06-11%20at%2008.19.12.png?generation=1718069122670446&alt=media\")
It seems that many teams with a large number of submissions have overfitted to the public leaderboard (including my team), while several teams with fewer submissions have significantly improved their rankings.
\nWhat are your thoughts on this?
\n
It seems that many teams with a large number of submissions have overfitted to the public leaderboard (including my team), while several teams with fewer submissions have significantly improved their rankings.
\nWhat are your thoughts on this?
\n
I think you may have mis interpreted this data (only an opinion). I think that what this shows (using the first blue line) is that many quickly copy pasted the top public notebook as their submission and were not able to beat it, so they gave up. That public notebook ended up doing even better in the private data. Causing many teams to shake down, including the owner who created that notebook in the first place, as he unfortunately didn’t have it selected. This years data was fairly prone to shake up and many of us were hit by it this time around! Learning lessons!
", "rawMarkdown": "I think you may have mis interpreted this data (only an opinion). I think that what this shows (using the first blue line) is that many quickly copy pasted the top public notebook as their submission and were not able to beat it, so they gave up. That public notebook ended up doing even better in the private data. Causing many teams to shake down, including the owner who created that notebook in the first place, as he unfortunately didn’t have it selected. This years data was fairly prone to shake up and many of us were hit by it this time around! Learning lessons!", "votes": 4, "replies": [ { "id": 2865758, "postDate": "2024-06-11T01:09:49.990Z", "content": "Thanks for pointing it out. That's actually what I want to imply to the host.
", "rawMarkdown": "Thanks for pointing it out. That's actually what I want to imply to the host.", "votes": 1, "replies": [ { "id": 2865766, "postDate": "2024-06-11T01:16:03.053Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2865798, "postDate": "2024-06-11T02:01:35.413Z", "content": "I’m so sad experiencing such big shake occurred for many people including me.
\nI’m investigating the cause, but yet being unable to find the cause:)
", "rawMarkdown": "I’m so sad experiencing such big shake occurred for many people including me.\n\nI’m investigating the cause, but yet being unable to find the cause:)", "votes": 2 }, { "id": 2865754, "postDate": "2024-06-11T01:08:13.247Z", "content": "The notebook with a major shake up is one that employs only simple augmentation, just mixups within the data of the same acoustic file. This itself seemed to perform robustly against the week label. However, the augmentation for other private datasets did not work well, and seemed to overfit only the leader board.
", "rawMarkdown": "The notebook with a major shake up is one that employs only simple augmentation, just mixups within the data of the same acoustic file. This itself seemed to perform robustly against the week label. However, the augmentation for other private datasets did not work well, and seemed to overfit only the leader board.", "votes": 2, "replies": [ { "id": 2865771, "postDate": "2024-06-11T01:24:04.390Z", "content": "You mean mix that bird call with the other audio labeled the same.
", "rawMarkdown": "You mean mix that bird call with the other audio labeled the same.", "votes": 1, "replies": [ { "id": 2865775, "postDate": "2024-06-11T01:29:00.983Z", "content": "Not exactly. It is a mixup within the same LABEL. What makes this competition unique is that within a single audio file, there are times when the birds are singing and times when they are not. (This is called the week label.) So, we can make the mixup as robust as possible by using the same audio file without using the non-singing timing as much as possible.
", "rawMarkdown": "Not exactly. It is a mixup within the same LABEL. What makes this competition unique is that within a single audio file, there are times when the birds are singing and times when they are not. (This is called the week label.) So, we can make the mixup as robust as possible by using the same audio file without using the non-singing timing as much as possible.", "votes": 1 } ] } ] }, { "id": 2865750, "postDate": "2024-06-11T01:07:15.950Z", "content": "We all tuning to the hell. Should this call “tuning backprobagation” !?? :)
", "rawMarkdown": "We all tuning to the hell. Should this call “tuning backprobagation” !?? :)\n", "votes": 2, "replies": [ { "id": 2865753, "postDate": "2024-06-11T01:07:55.593Z", "content": "And our time is wasted
", "rawMarkdown": "And our time is wasted" }, { "id": 2865761, "postDate": "2024-06-11T01:11:23.677Z", "content": "“tuning backprobagation” !!! nice word. you should write a new paper for it, bro!!
", "rawMarkdown": "“tuning backprobagation” !!! nice word. you should write a new paper for it, bro!!" } ] }, { "id": 2865794, "postDate": "2024-06-11T01:55:08.127Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 2865752, "author_name": "Cody_Null", "author_url": "", "post_date": "2024-06-11T01:07:43.620000", "content": "I think you may have mis interpreted this data (only an opinion). I think that what this shows (using the first blue line) is that many quickly copy pasted the top public notebook as their submission and were not able to beat it, so they gave up. That public notebook ended up doing even better in the private data. Causing many teams to shake down, including the owner who created that notebook in the first place, as he unfortunately didn’t have it selected. This years data was fairly prone to shake up and many of us were hit by it this time around! Learning lessons!
", "votes": 4, "replies": [ { "id": 2865758, "author_name": "Levi", "author_url": "", "post_date": "2024-06-11T01:09:49.990000", "content": "Thanks for pointing it out. That's actually what I want to imply to the host.
", "votes": 1, "replies": [ { "id": 2865766, "author_name": "", "author_url": "", "post_date": "2024-06-11T01:16:03.053000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2865798, "author_name": "MBOOK", "author_url": "", "post_date": "2024-06-11T02:01:35.413000", "content": "I’m so sad experiencing such big shake occurred for many people including me.
\nI’m investigating the cause, but yet being unable to find the cause:)
", "votes": 2, "replies": [] }, { "id": 2865754, "author_name": "tc0000", "author_url": "", "post_date": "2024-06-11T01:08:13.247000", "content": "The notebook with a major shake up is one that employs only simple augmentation, just mixups within the data of the same acoustic file. This itself seemed to perform robustly against the week label. However, the augmentation for other private datasets did not work well, and seemed to overfit only the leader board.
", "votes": 2, "replies": [ { "id": 2865771, "author_name": "LHK", "author_url": "", "post_date": "2024-06-11T01:24:04.390000", "content": "You mean mix that bird call with the other audio labeled the same.
", "votes": 1, "replies": [ { "id": 2865775, "author_name": "tc0000", "author_url": "", "post_date": "2024-06-11T01:29:00.983000", "content": "Not exactly. It is a mixup within the same LABEL. What makes this competition unique is that within a single audio file, there are times when the birds are singing and times when they are not. (This is called the week label.) So, we can make the mixup as robust as possible by using the same audio file without using the non-singing timing as much as possible.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2865750, "author_name": "LHK", "author_url": "", "post_date": "2024-06-11T01:07:15.950000", "content": "We all tuning to the hell. Should this call “tuning backprobagation” !?? :)
", "votes": 2, "replies": [ { "id": 2865753, "author_name": "LHK", "author_url": "", "post_date": "2024-06-11T01:07:55.593000", "content": "And our time is wasted
", "votes": 0, "replies": [] }, { "id": 2865761, "author_name": "Levi", "author_url": "", "post_date": "2024-06-11T01:11:23.677000", "content": "“tuning backprobagation” !!! nice word. you should write a new paper for it, bro!!
", "votes": 0, "replies": [] } ] }, { "id": 2865794, "author_name": "", "author_url": "", "post_date": "2024-06-11T01:55:08.127000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2865741": "It seems that many teams with a large number of submissions have overfitted to the public leaderboard (including my team), while several teams with fewer submissions have significantly improved their rankings.\n\nWhat are your thoughts on this?\n", "2865752": "I think you may have mis interpreted this data (only an opinion). I think that what this shows (using the first blue line) is that many quickly copy pasted the top public notebook as their submission and were not able to beat it, so they gave up. That public notebook ended up doing even better in the private data. Causing many teams to shake down, including the owner who created that notebook in the first place, as he unfortunately didn’t have it selected. This years data was fairly prone to shake up and many of us were hit by it this time around! Learning lessons!", "2865798": "I’m so sad experiencing such big shake occurred for many people including me.\n\nI’m investigating the cause, but yet being unable to find the cause:)", "2865754": "The notebook with a major shake up is one that employs only simple augmentation, just mixups within the data of the same acoustic file. This itself seemed to perform robustly against the week label. However, the augmentation for other private datasets did not work well, and seemed to overfit only the leader board.", "2865750": "We all tuning to the hell. Should this call “tuning backprobagation” !?? :)\n", "2865794": "" } }