Hi all, I wanted to share and ask about your baseline results to this point. Despite finding reliable CV yet we have been able to establish some models that are likely to work once we figure out CV. Here is what I am seeing so far: efficientvit_b1.r224_in1k, efficientvit_b1.r256_in1k, efficientvit_b1.r288_in1k, efficientnet_b0, efficientnet_b1, and efficientnet_b2. Has anyone else found anything useful or any hints on what on earth has happened to cv in the birdclef comp?
", "messages": [ { "id": "2766823", "postDate": "04/21/2024 23:06:22", "content": "Hi all, I wanted to share and ask about your baseline results to this point. Despite finding reliable CV yet we have been able to establish some models that are likely to work once we figure out CV. Here is what I am seeing so far: efficientvit_b1.r224_in1k, efficientvit_b1.r256_in1k, efficientvit_b1.r288_in1k, efficientnet_b0, efficientnet_b1, and efficientnet_b2. Has anyone else found anything useful or any hints on what on earth has happened to cv in the birdclef comp?
", "rawMarkdown": "Hi all, I wanted to share and ask about your baseline results to this point. Despite finding reliable CV yet we have been able to establish some models that are likely to work once we figure out CV. Here is what I am seeing so far: efficientvit_b1.r224_in1k, efficientvit_b1.r256_in1k, efficientvit_b1.r288_in1k, efficientnet_b0, efficientnet_b1, and efficientnet_b2. Has anyone else found anything useful or any hints on what on earth has happened to cv in the birdclef comp?", "votes": null }, { "id": "2767507", "postDate": "04/22/2024 11:04:42", "content": "In the last year competition, my best models were seresnext26t_32x4d, eca_nfnet_l0, seresnext50_32x4d, and convnext_tiny. However, this comp seems a bit different and more challenging based on what I've read.
", "rawMarkdown": "In the last year competition, my best models were seresnext26t_32x4d, eca_nfnet_l0, seresnext50_32x4d, and convnext_tiny. However, this comp seems a bit different and more challenging based on what I've read.", "votes": null }, { "id": "2767570", "postDate": "04/22/2024 12:18:36", "content": "lats year test data was about 2000 minutes (200 x 10 mins). This year it is about 4400 (1100 x 4 mins). It means that the time to perform inference is 2.2 times smaller this year. It may preclude using models that were fast enough for last year competition.
", "rawMarkdown": "lats year test data was about 2000 minutes (200 x 10 mins). This year it is about 4400 (1100 x 4 mins). It means that the time to perform inference is 2.2 times smaller this year. It may preclude using models that were fast enough for last year competition.", "votes": null }, { "id": "2767997", "postDate": "04/22/2024 16:00:47", "content": "The lack of correlation is bizarre. For example, I trained 20 epochs and epoch 19 performed best. I used it to run inference on some test data I downloaded from Xeno-Canto, and it greatly outperformed epoch 3. But when I submitted them, epoch 3 scored .58 and epoch 19 scored .57! Are you able to reproduce an example similar to this? If there's a bug in the scoring code, it's hard to explain why some people are scoring pretty well, but the results make no sense to me, and I finished in the top 5% last year.
", "rawMarkdown": "The lack of correlation is bizarre. For example, I trained 20 epochs and epoch 19 performed best. I used it to run inference on some test data I downloaded from Xeno-Canto, and it greatly outperformed epoch 3. But when I submitted them, epoch 3 scored .58 and epoch 19 scored .57! Are you able to reproduce an example similar to this? If there's a bug in the scoring code, it's hard to explain why some people are scoring pretty well, but the results make no sense to me, and I finished in the top 5% last year.", "votes": null }, { "id": "2768097", "postDate": "04/22/2024 17:23:52", "content": "Actually, I wonder if it's because I haven't made good use of the unlabelled recordings. There could be significant domain shift going on there. I'll spend some time on that.
", "rawMarkdown": "Actually, I wonder if it's because I haven't made good use of the unlabelled recordings. There could be significant domain shift going on there. I'll spend some time on that.", "votes": null }, { "id": "2768168", "postDate": "04/22/2024 18:03:10", "content": "The comp hosts did warn us this time around of domain shift. Just a little disappointed myself in the lack of correlation on this one compared to previous years (from my understanding). We will get there though!
", "rawMarkdown": "The comp hosts did warn us this time around of domain shift. Just a little disappointed myself in the lack of correlation on this one compared to previous years (from my understanding). We will get there though!", "votes": null }, { "id": "2768169", "postDate": "04/22/2024 18:03:22", "content": "Good to note!
", "rawMarkdown": "Good to note!", "votes": null } ], "comments": [ { "id": 2767507, "author_name": "maxdiazbattan", "author_url": "", "post_date": "04/22/2024 11:04:42", "content": "In the last year competition, my best models were seresnext26t_32x4d, eca_nfnet_l0, seresnext50_32x4d, and convnext_tiny. However, this comp seems a bit different and more challenging based on what I've read.
", "votes": null, "replies": [ { "id": 2767570, "author_name": "cpmpml", "author_url": "", "post_date": "04/22/2024 12:18:36", "content": "lats year test data was about 2000 minutes (200 x 10 mins). This year it is about 4400 (1100 x 4 mins). It means that the time to perform inference is 2.2 times smaller this year. It may preclude using models that were fast enough for last year competition.
", "votes": null, "replies": [ { "id": 2768169, "author_name": "cody11null", "author_url": "", "post_date": "04/22/2024 18:03:22", "content": "Good to note!
", "votes": null, "replies": [] } ] } ] }, { "id": 2767997, "author_name": "janhuus", "author_url": "", "post_date": "04/22/2024 16:00:47", "content": "The lack of correlation is bizarre. For example, I trained 20 epochs and epoch 19 performed best. I used it to run inference on some test data I downloaded from Xeno-Canto, and it greatly outperformed epoch 3. But when I submitted them, epoch 3 scored .58 and epoch 19 scored .57! Are you able to reproduce an example similar to this? If there's a bug in the scoring code, it's hard to explain why some people are scoring pretty well, but the results make no sense to me, and I finished in the top 5% last year.
", "votes": null, "replies": [ { "id": 2768097, "author_name": "janhuus", "author_url": "", "post_date": "04/22/2024 17:23:52", "content": "Actually, I wonder if it's because I haven't made good use of the unlabelled recordings. There could be significant domain shift going on there. I'll spend some time on that.
", "votes": null, "replies": [ { "id": 2768168, "author_name": "cody11null", "author_url": "", "post_date": "04/22/2024 18:03:10", "content": "The comp hosts did warn us this time around of domain shift. Just a little disappointed myself in the lack of correlation on this one compared to previous years (from my understanding). We will get there though!
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2766823": "Hi all, I wanted to share and ask about your baseline results to this point. Despite finding reliable CV yet we have been able to establish some models that are likely to work once we figure out CV. Here is what I am seeing so far: efficientvit_b1.r224_in1k, efficientvit_b1.r256_in1k, efficientvit_b1.r288_in1k, efficientnet_b0, efficientnet_b1, and efficientnet_b2. Has anyone else found anything useful or any hints on what on earth has happened to cv in the birdclef comp?", "2767507": "In the last year competition, my best models were seresnext26t_32x4d, eca_nfnet_l0, seresnext50_32x4d, and convnext_tiny. However, this comp seems a bit different and more challenging based on what I've read.", "2767570": "lats year test data was about 2000 minutes (200 x 10 mins). This year it is about 4400 (1100 x 4 mins). It means that the time to perform inference is 2.2 times smaller this year. It may preclude using models that were fast enough for last year competition.", "2767997": "The lack of correlation is bizarre. For example, I trained 20 epochs and epoch 19 performed best. I used it to run inference on some test data I downloaded from Xeno-Canto, and it greatly outperformed epoch 3. But when I submitted them, epoch 3 scored .58 and epoch 19 scored .57! Are you able to reproduce an example similar to this? If there's a bug in the scoring code, it's hard to explain why some people are scoring pretty well, but the results make no sense to me, and I finished in the top 5% last year.", "2768097": "Actually, I wonder if it's because I haven't made good use of the unlabelled recordings. There could be significant domain shift going on there. I'll spend some time on that.", "2768168": "The comp hosts did warn us this time around of domain shift. Just a little disappointed myself in the lack of correlation on this one compared to previous years (from my understanding). We will get there though!", "2768169": "Good to note!" }, "source": "meta" }