The BIRD competition has traditionally not been a big shakeup. I'm new to BIRD competitions, but this one feels a bit different than previous ones. I don't think it's all that unusual for most of my experiments to fail, but I've had at least one experiment that I expected to at least not decrease my score, if not increase it, but which resulted in a significant decrease in my score. I'm curious what you think about shakeups.
", "messages": [ { "id": 2857934, "postDate": "2024-06-06T07:20:36.813Z", "content": "The BIRD competition has traditionally not been a big shakeup. I'm new to BIRD competitions, but this one feels a bit different than previous ones. I don't think it's all that unusual for most of my experiments to fail, but I've had at least one experiment that I expected to at least not decrease my score, if not increase it, but which resulted in a significant decrease in my score. I'm curious what you think about shakeups.
", "rawMarkdown": "The BIRD competition has traditionally not been a big shakeup. I'm new to BIRD competitions, but this one feels a bit different than previous ones. I don't think it's all that unusual for most of my experiments to fail, but I've had at least one experiment that I expected to at least not decrease my score, if not increase it, but which resulted in a significant decrease in my score. I'm curious what you think about shakeups.", "votes": 10 }, { "id": 2859102, "postDate": "2024-06-06T19:33:38.350Z", "content": "Yes.
", "rawMarkdown": "Yes. ", "votes": 7, "replies": [ { "id": 2859442, "postDate": "2024-06-07T03:40:04.740Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2858676, "postDate": "2024-06-06T15:45:44.607Z", "content": "I think, the shakeup will be quite big. The reason is it's quite difficult to build a reliable CV scheme in this competition, and overfitting to the public LB is generally not good idea. At the same time, in this particular competition LB is very sensitive to noise. I've experienced a drastical score decrease when just random_seed was changed.
", "rawMarkdown": "I think, the shakeup will be quite big. The reason is it's quite difficult to build a reliable CV scheme in this competition, and overfitting to the public LB is generally not good idea. At the same time, in this particular competition LB is very sensitive to noise. I've experienced a drastical score decrease when just random_seed was changed.", "votes": 1, "replies": [ { "id": 2859066, "postDate": "2024-06-06T19:06:20.980Z", "content": "If it's a question of noise robustness and weakness, wouldn't there be samples with similar noise in the private data? The hosts will have obviously done a lot of testing, and given that there has traditionally been less shakeup, I would expect this competition to be a bit shakier than in the past, but not by much. However, I think the distribution of labels will be quite different between public and private data.
", "rawMarkdown": "If it's a question of noise robustness and weakness, wouldn't there be samples with similar noise in the private data? The hosts will have obviously done a lot of testing, and given that there has traditionally been less shakeup, I would expect this competition to be a bit shakier than in the past, but not by much. However, I think the distribution of labels will be quite different between public and private data.", "votes": 1 } ] }, { "id": 2864834, "postDate": "2024-06-10T10:57:23.537Z", "content": "I'll let you know in 11 hours.
", "rawMarkdown": "I'll let you know in 11 hours.", "replies": [ { "id": 2864857, "postDate": "2024-06-10T11:09:08.840Z", "content": "Not 13 hours?!🤪
", "rawMarkdown": "Not 13 hours?!🤪", "votes": 1, "replies": [ { "id": 2864938, "postDate": "2024-06-10T12:35:47.040Z", "content": "you're probably right. Time for this to finish if you ask me.
", "rawMarkdown": "you're probably right. Time for this to finish if you ask me." } ] } ] }, { "id": 2864706, "postDate": "2024-06-10T09:31:40.793Z", "content": "I checked BirdCLEF 2023 results - and there was no big shakeup in the medal zone.
", "rawMarkdown": "I checked BirdCLEF 2023 results - and there was no big shakeup in the medal zone." }, { "id": 2863961, "postDate": "2024-06-09T19:54:42.087Z", "content": "I think my diff from exp5 to exp100 was 0.01 (LB), some bug probably :D
", "rawMarkdown": "I think my diff from exp5 to exp100 was 0.01 (LB), some bug probably :D" }, { "id": 2863093, "postDate": "2024-06-09T08:14:30.903Z", "content": "Hi guys, as a rookie, I'm curious that how do you guys overfit the LB (any general methodology)? I use strat k-fold, argumentation like gauess noise with SNR, hflip and xy_mask and my CV is around 0.97-0.98, but my best LB is 0.64 and a small change in argumentation will decrease the LB a lot. I'm using EffNetB90 without ensemble.
\nAny help will be appreciated!
", "rawMarkdown": "Hi guys, as a rookie, I'm curious that how do you guys overfit the LB (any general methodology)? I use strat k-fold, argumentation like gauess noise with SNR, hflip and xy_mask and my CV is around 0.97-0.98, but my best LB is 0.64 and a small change in argumentation will decrease the LB a lot. I'm using EffNetB90 without ensemble.\n\nAny help will be appreciated!" }, { "id": 2859386, "postDate": "2024-06-07T02:36:40.077Z", "content": "I believe it will.
", "rawMarkdown": "I believe it will." }, { "id": 2858760, "postDate": "2024-06-06T16:07:03.230Z", "content": "I couldn't get a good CV, so I probably overfitted to the Public LB. I'm just hoping the Public and Private data distributions are similar.
", "rawMarkdown": "I couldn't get a good CV, so I probably overfitted to the Public LB. I'm just hoping the Public and Private data distributions are similar.", "replies": [ { "id": 2859052, "postDate": "2024-06-06T19:02:41.163Z", "content": "Thanks for the comment. I too would love it if public and private data were the same.
", "rawMarkdown": "Thanks for the comment. I too would love it if public and private data were the same." } ] }, { "id": 2863656, "postDate": "2024-06-09T15:04:26.173Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2859445, "postDate": "2024-06-07T03:41:22.550Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2859102, "author_name": "Ivan Panshin", "author_url": "", "post_date": "2024-06-06T19:33:38.350000", "content": "Yes.
", "votes": 7, "replies": [ { "id": 2859442, "author_name": "", "author_url": "", "post_date": "2024-06-07T03:40:04.740000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2858676, "author_name": "Konstantin Dmitriev", "author_url": "", "post_date": "2024-06-06T15:45:44.607000", "content": "I think, the shakeup will be quite big. The reason is it's quite difficult to build a reliable CV scheme in this competition, and overfitting to the public LB is generally not good idea. At the same time, in this particular competition LB is very sensitive to noise. I've experienced a drastical score decrease when just random_seed was changed.
", "votes": 1, "replies": [ { "id": 2859066, "author_name": "HB", "author_url": "", "post_date": "2024-06-06T19:06:20.980000", "content": "If it's a question of noise robustness and weakness, wouldn't there be samples with similar noise in the private data? The hosts will have obviously done a lot of testing, and given that there has traditionally been less shakeup, I would expect this competition to be a bit shakier than in the past, but not by much. However, I think the distribution of labels will be quite different between public and private data.
", "votes": 1, "replies": [] } ] }, { "id": 2864834, "author_name": "CPMP", "author_url": "", "post_date": "2024-06-10T10:57:23.537000", "content": "I'll let you know in 11 hours.
", "votes": 0, "replies": [ { "id": 2864857, "author_name": "HB", "author_url": "", "post_date": "2024-06-10T11:09:08.840000", "content": "Not 13 hours?!🤪
", "votes": 1, "replies": [ { "id": 2864938, "author_name": "CPMP", "author_url": "", "post_date": "2024-06-10T12:35:47.040000", "content": "you're probably right. Time for this to finish if you ask me.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2864706, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2024-06-10T09:31:40.793000", "content": "I checked BirdCLEF 2023 results - and there was no big shakeup in the medal zone.
", "votes": 0, "replies": [] }, { "id": 2863961, "author_name": "slime", "author_url": "", "post_date": "2024-06-09T19:54:42.087000", "content": "I think my diff from exp5 to exp100 was 0.01 (LB), some bug probably :D
", "votes": 0, "replies": [] }, { "id": 2863093, "author_name": "SakuraYuyuko", "author_url": "", "post_date": "2024-06-09T08:14:30.903000", "content": "Hi guys, as a rookie, I'm curious that how do you guys overfit the LB (any general methodology)? I use strat k-fold, argumentation like gauess noise with SNR, hflip and xy_mask and my CV is around 0.97-0.98, but my best LB is 0.64 and a small change in argumentation will decrease the LB a lot. I'm using EffNetB90 without ensemble.
\nAny help will be appreciated!
", "votes": 0, "replies": [] }, { "id": 2859386, "author_name": "Levi", "author_url": "", "post_date": "2024-06-07T02:36:40.077000", "content": "I believe it will.
", "votes": 0, "replies": [] }, { "id": 2858760, "author_name": "Moyashii", "author_url": "", "post_date": "2024-06-06T16:07:03.230000", "content": "I couldn't get a good CV, so I probably overfitted to the Public LB. I'm just hoping the Public and Private data distributions are similar.
", "votes": 0, "replies": [ { "id": 2859052, "author_name": "HB", "author_url": "", "post_date": "2024-06-06T19:02:41.163000", "content": "Thanks for the comment. I too would love it if public and private data were the same.
", "votes": 0, "replies": [] } ] }, { "id": 2863656, "author_name": "", "author_url": "", "post_date": "2024-06-09T15:04:26.173000", "content": "", "votes": 0, "replies": [] }, { "id": 2859445, "author_name": "", "author_url": "", "post_date": "2024-06-07T03:41:22.550000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2857934": "The BIRD competition has traditionally not been a big shakeup. I'm new to BIRD competitions, but this one feels a bit different than previous ones. I don't think it's all that unusual for most of my experiments to fail, but I've had at least one experiment that I expected to at least not decrease my score, if not increase it, but which resulted in a significant decrease in my score. I'm curious what you think about shakeups.", "2859102": "Yes. ", "2858676": "I think, the shakeup will be quite big. The reason is it's quite difficult to build a reliable CV scheme in this competition, and overfitting to the public LB is generally not good idea. At the same time, in this particular competition LB is very sensitive to noise. I've experienced a drastical score decrease when just random_seed was changed.", "2864834": "I'll let you know in 11 hours.", "2864706": "I checked BirdCLEF 2023 results - and there was no big shakeup in the medal zone.", "2863961": "I think my diff from exp5 to exp100 was 0.01 (LB), some bug probably :D", "2863093": "Hi guys, as a rookie, I'm curious that how do you guys overfit the LB (any general methodology)? I use strat k-fold, argumentation like gauess noise with SNR, hflip and xy_mask and my CV is around 0.97-0.98, but my best LB is 0.64 and a small change in argumentation will decrease the LB a lot. I'm using EffNetB90 without ensemble.\n\nAny help will be appreciated!", "2859386": "I believe it will.", "2858760": "I couldn't get a good CV, so I probably overfitted to the Public LB. I'm just hoping the Public and Private data distributions are similar.", "2863656": "", "2859445": "" } }