First of all, thanks to the organizers for hosting this competition.
\nAnd congrats to all winners.
2024 data
\n| model | \nbackbone | \nweights | \npublic | \nprivate | \n
|---|---|---|---|---|
| raw signale | \nhgnetb0 | \n3/5 folds | \n0.720354 | \n0.666671 | \n
| spectrum | \nefficientnetb0 | \n1/5 folds | \n0.708198 | \n0.672360 | \n
| raw+spectrum | \nhgnetb0+effb0 | \n1/5 folds | \n0.698408 | \n0.660503 | \n
| ensemble | \n… | \n0.5+0.4+0.1 | \n0.743960 | \n0.687173 | \n
\n\n = x.view(bs, -, ) \n\n = torch.transpose(x, , ) \n\n = x.view(bs, , -, ) \n = self.backbone(x) \ntorchaudio.transforms.MelSpectrogram(\n 32000,\n =512,\n =0,\n =16000,\n =2048*2,\n =512,\n =,\n ),\ntorchaudio.transforms.AmplitudeToDB(=80)\n = self.raw_model(x)\n = self.spec_model(x)\n = torch.cat([raw_f,spec_f],dim=)\n=self.fc(feature)\n\n\nI make a post to discussion the method feeding raw signal to vision model. Here is the link good luck
\ninference notebook
First of all, thanks to the organizers for hosting this competition.
\nAnd congrats to all winners.
2024 data
\n| model | \nbackbone | \nweights | \npublic | \nprivate | \n
|---|---|---|---|---|
| raw signale | \nhgnetb0 | \n3/5 folds | \n0.720354 | \n0.666671 | \n
| spectrum | \nefficientnetb0 | \n1/5 folds | \n0.708198 | \n0.672360 | \n
| raw+spectrum | \nhgnetb0+effb0 | \n1/5 folds | \n0.698408 | \n0.660503 | \n
| ensemble | \n… | \n0.5+0.4+0.1 | \n0.743960 | \n0.687173 | \n
\n\n = x.view(bs, -, ) \n\n = torch.transpose(x, , ) \n\n = x.view(bs, , -, ) \n = self.backbone(x) \ntorchaudio.transforms.MelSpectrogram(\n 32000,\n =512,\n =0,\n =16000,\n =2048*2,\n =512,\n =,\n ),\ntorchaudio.transforms.AmplitudeToDB(=80)\n = self.raw_model(x)\n = self.spec_model(x)\n = torch.cat([raw_f,spec_f],dim=)\n=self.fc(feature)\n\n\nI make a post to discussion the method feeding raw signal to vision model. Here is the link good luck
\ninference notebook
Regarding the \"raw model\", how did you deal with it to achieve good results? I rarely see people using raw signals in Bird competitions.
", "rawMarkdown": "Regarding the \"raw model\", how did you deal with it to achieve good results? I rarely see people using raw signals in Bird competitions.", "votes": 3, "replies": [ { "id": 2866242, "postDate": "2024-06-11T08:08:36.480Z", "content": "I add some comments one the pse codes, i think you can get it now.
", "rawMarkdown": "I add some comments one the pse codes, i think you can get it now.", "votes": 1 } ] }, { "id": 2869294, "postDate": "2024-06-13T02:25:50.187Z", "content": "Good work. I was betting you'd use packed raw waveform as you did in HMS competition!
\nCongrats on becoming a GM. It was a matter of time given what you did in HMS.
\nDid you compare openvino accuracy with pytorch models? Asking because we found that it loses some accuracy and we didn't use it as a result.
", "rawMarkdown": "Good work. I was betting you'd use packed raw waveform as you did in HMS competition!\n\nCongrats on becoming a GM. It was a matter of time given what you did in HMS.\n\nDid you compare openvino accuracy with pytorch models? Asking because we found that it loses some accuracy and we didn't use it as a result.", "votes": 1, "replies": [ { "id": 2869391, "postDate": "2024-06-13T04:32:40.203Z", "content": "Thanks.
\nI noticed that you'd mentioned the precision loss with openvino.
\nI manully check some of the outputs of pytorch, onnxruntime, and openvino, and notice very small differences, but didn't try to sub with onnxruntime. By the way, as the the score is rank relavent, i use logits as final output.
I used logits too until we found that smoothing worked better when applied to probabilities.
", "rawMarkdown": "I used logits too until we found that smoothing worked better when applied to probabilities.", "votes": 1 }, { "id": 2871005, "postDate": "2024-06-14T02:23:33Z", "content": "Lesson learned, i should have checked previous competition carefully. I saw several solutions that use postprocess that worked. I didn't use post process and previous data, a little bit pity.
", "rawMarkdown": "Lesson learned, i should have checked previous competition carefully. I saw several solutions that use postprocess that worked. I didn't use post process and previous data, a little bit pity.", "votes": 1 } ] } ] }, { "id": 2867896, "postDate": "2024-06-12T06:09:15.733Z", "content": "Congrats for reaching Competition Grandmaster status 🎉 Huge achievement
", "rawMarkdown": "Congrats for reaching Competition Grandmaster status 🎉 Huge achievement", "votes": 1 }, { "id": 2866944, "postDate": "2024-06-11T15:25:39.187Z", "content": "Congrats for your medal!
\nQuestion about your signal approach, what is the intuition behind your reshaping? Why picking 2 channels instead of 3 for example?
", "rawMarkdown": "Congrats for your medal!\n\nQuestion about your signal approach, what is the intuition behind your reshaping? Why picking 2 channels instead of 3 for example?", "votes": 1, "replies": [ { "id": 2867029, "postDate": "2024-06-11T15:50:15.747Z", "content": "I want to make a 1 channel input ,but the input size is too big.
\nAnd that makes it too hard to do ensemble.
\nThen i reshape it as 2 channels( [123456]->[[135], [246]]). In some way, you can regard the input as 2 waves that with 16000hz sample rate.
Nice use of 1D-signals again! Also, congrats on becoming GM.
\nIs there a reason why you use n_mels=512 and then resize to (256, 256) rather than just setting n_mels=256?
", "rawMarkdown": "Nice use of 1D-signals again! Also, congrats on becoming GM.\n\nIs there a reason why you use n_mels=512 and then resize to (256, 256) rather than just setting n_mels=256?", "votes": 1, "replies": [ { "id": 2866923, "postDate": "2024-06-11T15:11:27.893Z", "content": "For the spectrum model i use raw spectrum, but for the mix model, i use resized 256x256 for inference speed reason, otherwise, it would be timeout.
", "rawMarkdown": "For the spectrum model i use raw spectrum, but for the mix model, i use resized 256x256 for inference speed reason, otherwise, it would be timeout.", "votes": 1, "replies": [ { "id": 2867030, "postDate": "2024-06-11T15:51:20.997Z", "content": "Hmm okay, thanks for the reply.
\nWhy you did you not set n_mels=256? This should make the height match the resize shape?
eg.
\nimport torch\nimport torch.nn as nn\nimport torchaudio\n\nl= nn.Sequential(\n torchaudio.transforms.MelSpectrogram(\n 32000,\n =256, # <- change this 256?\n =0,\n =16000,\n =2048*2,\n =512,\n =,\n ),\ntorchaudio.transforms.AmplitudeToDB(=80),\n)\nx= torch.rand(32_000)\n(l(x).shape)\n\n\nYeah， My models need raw data, spectrum512, spectrum256.
\nTherefor, the dataiter produce raw, spectrum512, spectrum256(resized by spectrum512). In this way i just need to load every ogg file one time to reduce the time cost.
\nI use n_mel=512 because i developed my spectrum model( input size 512,313) early and that is the best params for my model.
Great how you managed to train raw model that well, in my experiments it had bad CV so I didn't submit it
", "rawMarkdown": "Great how you managed to train raw model that well, in my experiments it had bad CV so I didn't submit it", "votes": 1 }, { "id": 2866141, "postDate": "2024-06-11T06:49:41.843Z", "content": "Hi, I had a doubt like taking 5 random seconds from sample during training wont guarantee the bird noise in it right? If yes then how the model works?
", "rawMarkdown": "Hi, I had a doubt like taking 5 random seconds from sample during training wont guarantee the bird noise in it right? If yes then how the model works?", "votes": 1, "replies": [ { "id": 2866174, "postDate": "2024-06-11T07:11:42.023Z", "content": "I see some people train with first 5 sec data. But in my experiments, random sample 5 sec data, perform better.
", "rawMarkdown": "I see some people train with first 5 sec data. But in my experiments, random sample 5 sec data, perform better.", "votes": 2 } ] }, { "id": 2866332, "postDate": "2024-06-11T09:15:34.257Z", "content": "Solution seems so simple! How did you get such powerful single models using this approach? Were there any other tricks you used or parameters you optimized to get such high scoring models? Im so curious how such basic settings score so high. And congrats on GM rank!
", "rawMarkdown": "Solution seems so simple! How did you get such powerful single models using this approach? Were there any other tricks you used or parameters you optimized to get such high scoring models? Im so curious how such basic settings score so high. And congrats on GM rank!", "votes": 2, "replies": [ { "id": 2866334, "postDate": "2024-06-11T09:16:23.080Z", "content": "Would you be willing to share training code?
", "rawMarkdown": "Would you be willing to share training code?" }, { "id": 2866398, "postDate": "2024-06-11T10:03:44.310Z", "content": "Thanks.
\nThe model is quiet simple, but it's sensitive to random seed. I use lb to check model performance. Single model is not that powerful. But the ensemble of raw signal and spectrum helps a lot.
What I’m asking is how did you get the spectrum only model to 0.70 public score when the baseline score given by most public notebook or discussion showed only 0.66 public score for spectrum only.
", "rawMarkdown": "What I’m asking is how did you get the spectrum only model to 0.70 public score when the baseline score given by most public notebook or discussion showed only 0.66 public score for spectrum only." } ] } ] }, { "id": 2866143, "postDate": "2024-06-11T06:52:31.407Z", "content": "Impressive!! Will you mind providing some more details about the raw signals model?
\nI think it is really a big breakthrough because current solutions like birdnet, bird-vocalization-classifier are all based on computer vision, thus solution with raw signal maybe cutting edge.
Like the code below,
\n = x.view(bs, -, )\n = torch.transpose(x, , )\n = x.view(bs, , -, )\nthen feeding to a 2d-CNN
\nAnd i add some comment one the pse codes, i think you can get it now.
", "rawMarkdown": "Like the code below,\n```\nx = x.view(bs, -1, 2)\nx = torch.transpose(x, 2, 1)\nx = x.view(bs, 2, -1, 80)\n```\n\nthen feeding to a 2d-CNN\n\nAnd i add some comment one the pse codes, i think you can get it now.", "votes": 4 }, { "id": 2867154, "postDate": "2024-06-11T16:53:38.527Z", "content": "I am not sure if it makes sense, but here is my thought since the model was trained on raw waves model might had learned to predict very well for the noisy data , it might have created some range pass that range it might be treating those features as an outlier, since the test data had too much noise it would had discarded all those unnecssary features upto some range based on training data, and since we normalize the value for mels we might not had those outliers , i am still confused tho :)
", "rawMarkdown": "I am not sure if it makes sense, but here is my thought since the model was trained on raw waves model might had learned to predict very well for the noisy data , it might have created some range pass that range it might be treating those features as an outlier, since the test data had too much noise it would had discarded all those unnecssary features upto some range based on training data, and since we normalize the value for mels we might not had those outliers , i am still confused tho :) " } ] }, { "id": 2871578, "postDate": "2024-06-14T09:23:09.860Z", "content": "Congratulations 👏
", "rawMarkdown": "Congratulations 👏" }, { "id": 2869286, "postDate": "2024-06-13T02:09:33.057Z", "content": "Congrats for becoming Competition Grandmaster!!!!!
", "rawMarkdown": "Congrats for becoming Competition Grandmaster!!!!!" }, { "id": 2868710, "postDate": "2024-06-12T15:13:20.170Z", "content": "Congrats for becoming a competition GM! Huge achievement! Also impressive to see the relatively simple solution can achieve the great final result.
\nHave the following questions:
\nWhy did you choose 5-sec clip during the training stage? In my experience, 10-sec clip from the middle of the train audio improves the cv score a bit, and also gives the model more samples to learn from my opinion.
Why didn't you use more image/audio augmentations?
Is there a reason for choosing n_fft=2048 * 2? That seems a lot more computation required than when n_fft=512, which some solutions adopt to use.
Thanks again for sharing the solution!
", "rawMarkdown": "Congrats for becoming a competition GM! Huge achievement! Also impressive to see the relatively simple solution can achieve the great final result. \n\nHave the following questions: \n\n1. Why did you choose 5-sec clip during the training stage? In my experience, 10-sec clip from the middle of the train audio improves the cv score a bit, and also gives the model more samples to learn from my opinion. \n\n2. Why didn't you use more image/audio augmentations? \n\n3. Is there a reason for choosing n_fft=2048 * 2? That seems a lot more computation required than when n_fft=512, which some solutions adopt to use.\n\nThanks again for sharing the solution!" }, { "id": 2868227, "postDate": "2024-06-12T08:52:23.450Z", "content": "Well done. The idea to mix raw and MEL spectrogram is great.
\nCould you give more details about the openvino part, thanks!
Like most people, convert pytorch model to onnx, then use openvino do inference. But spectrum is produced by torchaudio. And I will share my notebook later.
", "rawMarkdown": "Like most people, convert pytorch model to onnx, then use openvino do inference. But spectrum is produced by torchaudio. And I will share my notebook later.", "votes": 1 } ] }, { "id": 2868184, "postDate": "2024-06-12T08:31:25.813Z", "content": "Congratulations! I've just started studying I feel that I can grow quickly with the help of people like you, who are willing to discuss, share, and provide guidance. Thank you.
", "rawMarkdown": "Congratulations! I've just started studying I feel that I can grow quickly with the help of people like you, who are willing to discuss, share, and provide guidance. Thank you." }, { "id": 2867811, "postDate": "2024-06-12T05:15:59.517Z", "content": "Congratulations on securing 5th Place in this competition. Also, thanks on sharing the solution details.
", "rawMarkdown": "Congratulations on securing 5th Place in this competition. Also, thanks on sharing the solution details. " }, { "id": 2867231, "postDate": "2024-06-11T18:03:10.843Z", "content": "Thanks for sharing and congratulations on the 5th place and GM title!
\nI'm surprised that 1D models worked. We tried them early and discarded the idea because the performance was considerably worse than melspecs. Maybe the size of the test made a difference (only part of the data for 1 fold and 10 epochs). A reminder to revisit early decisions.
\nCould you please elaborate a bit on the approach you followed to evaluate models and define the weights for the ensemble? This was a struggle for us and I wonder how other teams approached it.
", "rawMarkdown": "Thanks for sharing and congratulations on the 5th place and GM title!\n\nI'm surprised that 1D models worked. We tried them early and discarded the idea because the performance was considerably worse than melspecs. Maybe the size of the test made a difference (only part of the data for 1 fold and 10 epochs). A reminder to revisit early decisions.\n\nCould you please elaborate a bit on the approach you followed to evaluate models and define the weights for the ensemble? This was a struggle for us and I wonder how other teams approached it.", "replies": [ { "id": 2868149, "postDate": "2024-06-12T08:00:23.920Z", "content": "I didn't find a good way to do local cv, and raw signal model cv is lower than spectrum model.
\nI use 0.5,0.4,0.1 as ensemble weights. Because raw signal model score is about the same with spectrum model, and the mix model is a combine of them, use a small weight here.
Congratulations ! Would you be publishing your inference pipeline? I'd be very interested to see how you optimised MelSpec layers with openvino, I failed at this.
", "rawMarkdown": "Congratulations ! Would you be publishing your inference pipeline? I'd be very interested to see how you optimised MelSpec layers with openvino, I failed at this. ", "replies": [ { "id": 2866495, "postDate": "2024-06-11T11:11:46.423Z", "content": "I will share it later. However I met the same problem, I use torchaudio to produce the spectrum in dataiter, the layer not included in the model.
", "rawMarkdown": "I will share it later. However I met the same problem, I use torchaudio to produce the spectrum in dataiter, the layer not included in the model." }, { "id": 2866574, "postDate": "2024-06-11T11:46:09.813Z", "content": "Try my last year notebook
", "rawMarkdown": "Try my last year [notebook](https://www.kaggle.com/code/aphysict/fork-of-birdclef-submit-rewrite-onnx)", "votes": 1, "replies": [ { "id": 2868130, "postDate": "2024-06-12T07:50:29.093Z", "content": "Thanks, I will go through your NB.
", "rawMarkdown": "Thanks, I will go through your NB. " } ] } ] }, { "id": 3146440, "postDate": "2025-03-10T21:39:57.853Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2875213, "postDate": "2024-06-17T00:28:55.100Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2866694, "postDate": "2024-06-11T13:02:35.197Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2866062, "author_name": "Donghui Zhang", "author_url": "", "post_date": "2024-06-11T06:04:18.460000", "content": "Regarding the \"raw model\", how did you deal with it to achieve good results? I rarely see people using raw signals in Bird competitions.
", "votes": 3, "replies": [ { "id": 2866242, "author_name": "coolz", "author_url": "", "post_date": "2024-06-11T08:08:36.480000", "content": "I add some comments one the pse codes, i think you can get it now.
", "votes": 1, "replies": [] } ] }, { "id": 2869294, "author_name": "CPMP", "author_url": "", "post_date": "2024-06-13T02:25:50.187000", "content": "Good work. I was betting you'd use packed raw waveform as you did in HMS competition!
\nCongrats on becoming a GM. It was a matter of time given what you did in HMS.
\nDid you compare openvino accuracy with pytorch models? Asking because we found that it loses some accuracy and we didn't use it as a result.
", "votes": 1, "replies": [ { "id": 2869391, "author_name": "coolz", "author_url": "", "post_date": "2024-06-13T04:32:40.203000", "content": "Thanks.
\nI noticed that you'd mentioned the precision loss with openvino.
\nI manully check some of the outputs of pytorch, onnxruntime, and openvino, and notice very small differences, but didn't try to sub with onnxruntime. By the way, as the the score is rank relavent, i use logits as final output.
I used logits too until we found that smoothing worked better when applied to probabilities.
", "votes": 1, "replies": [] }, { "id": 2871005, "author_name": "coolz", "author_url": "", "post_date": "2024-06-14T02:23:33", "content": "Lesson learned, i should have checked previous competition carefully. I saw several solutions that use postprocess that worked. I didn't use post process and previous data, a little bit pity.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2867896, "author_name": "Dieter", "author_url": "", "post_date": "2024-06-12T06:09:15.733000", "content": "Congrats for reaching Competition Grandmaster status 🎉 Huge achievement
", "votes": 1, "replies": [] }, { "id": 2866944, "author_name": "Shiro", "author_url": "", "post_date": "2024-06-11T15:25:39.187000", "content": "Congrats for your medal!
\nQuestion about your signal approach, what is the intuition behind your reshaping? Why picking 2 channels instead of 3 for example?
", "votes": 1, "replies": [ { "id": 2867029, "author_name": "coolz", "author_url": "", "post_date": "2024-06-11T15:50:15.747000", "content": "I want to make a 1 channel input ,but the input size is too big.
\nAnd that makes it too hard to do ensemble.
\nThen i reshape it as 2 channels( [123456]->[[135], [246]]). In some way, you can regard the input as 2 waves that with 16000hz sample rate.
Nice use of 1D-signals again! Also, congrats on becoming GM.
\nIs there a reason why you use n_mels=512 and then resize to (256, 256) rather than just setting n_mels=256?
", "votes": 1, "replies": [ { "id": 2866923, "author_name": "coolz", "author_url": "", "post_date": "2024-06-11T15:11:27.893000", "content": "For the spectrum model i use raw spectrum, but for the mix model, i use resized 256x256 for inference speed reason, otherwise, it would be timeout.
", "votes": 1, "replies": [ { "id": 2867030, "author_name": "Bartley", "author_url": "", "post_date": "2024-06-11T15:51:20.997000", "content": "Hmm okay, thanks for the reply.
\nWhy you did you not set n_mels=256? This should make the height match the resize shape?
eg.
\nimport torch\nimport torch.nn as nn\nimport torchaudio\n\nl= nn.Sequential(\n torchaudio.transforms.MelSpectrogram(\n 32000,\n =256, # <- change this 256?\n =0,\n =16000,\n =2048*2,\n =512,\n =,\n ),\ntorchaudio.transforms.AmplitudeToDB(=80),\n)\nx= torch.rand(32_000)\n(l(x).shape)\n\n\nYeah， My models need raw data, spectrum512, spectrum256.
\nTherefor, the dataiter produce raw, spectrum512, spectrum256(resized by spectrum512). In this way i just need to load every ogg file one time to reduce the time cost.
\nI use n_mel=512 because i developed my spectrum model( input size 512,313) early and that is the best params for my model.
Great how you managed to train raw model that well, in my experiments it had bad CV so I didn't submit it
", "votes": 1, "replies": [] }, { "id": 2866141, "author_name": "Ayush Solanki", "author_url": "", "post_date": "2024-06-11T06:49:41.843000", "content": "Hi, I had a doubt like taking 5 random seconds from sample during training wont guarantee the bird noise in it right? If yes then how the model works?
", "votes": 1, "replies": [ { "id": 2866174, "author_name": "coolz", "author_url": "", "post_date": "2024-06-11T07:11:42.023000", "content": "I see some people train with first 5 sec data. But in my experiments, random sample 5 sec data, perform better.
", "votes": 2, "replies": [] } ] }, { "id": 2866332, "author_name": "snehal", "author_url": "", "post_date": "2024-06-11T09:15:34.257000", "content": "Solution seems so simple! How did you get such powerful single models using this approach? Were there any other tricks you used or parameters you optimized to get such high scoring models? Im so curious how such basic settings score so high. And congrats on GM rank!
", "votes": 2, "replies": [ { "id": 2866334, "author_name": "snehal", "author_url": "", "post_date": "2024-06-11T09:16:23.080000", "content": "Would you be willing to share training code?
", "votes": 0, "replies": [] }, { "id": 2866398, "author_name": "coolz", "author_url": "", "post_date": "2024-06-11T10:03:44.310000", "content": "Thanks.
\nThe model is quiet simple, but it's sensitive to random seed. I use lb to check model performance. Single model is not that powerful. But the ensemble of raw signal and spectrum helps a lot.
What I’m asking is how did you get the spectrum only model to 0.70 public score when the baseline score given by most public notebook or discussion showed only 0.66 public score for spectrum only.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2866143, "author_name": "RihanPiggy", "author_url": "", "post_date": "2024-06-11T06:52:31.407000", "content": "Impressive!! Will you mind providing some more details about the raw signals model?
\nI think it is really a big breakthrough because current solutions like birdnet, bird-vocalization-classifier are all based on computer vision, thus solution with raw signal maybe cutting edge.
Like the code below,
\n = x.view(bs, -, )\n = torch.transpose(x, , )\n = x.view(bs, , -, )\nthen feeding to a 2d-CNN
\nAnd i add some comment one the pse codes, i think you can get it now.
", "votes": 4, "replies": [] }, { "id": 2867154, "author_name": "Shubham Thapa", "author_url": "", "post_date": "2024-06-11T16:53:38.527000", "content": "I am not sure if it makes sense, but here is my thought since the model was trained on raw waves model might had learned to predict very well for the noisy data , it might have created some range pass that range it might be treating those features as an outlier, since the test data had too much noise it would had discarded all those unnecssary features upto some range based on training data, and since we normalize the value for mels we might not had those outliers , i am still confused tho :)
", "votes": 0, "replies": [] } ] }, { "id": 2871578, "author_name": "Muhammad Azeem", "author_url": "", "post_date": "2024-06-14T09:23:09.860000", "content": "Congratulations 👏
", "votes": 0, "replies": [] }, { "id": 2869286, "author_name": "Aaditya Porwal", "author_url": "", "post_date": "2024-06-13T02:09:33.057000", "content": "Congrats for becoming Competition Grandmaster!!!!!
", "votes": 0, "replies": [] }, { "id": 2868710, "author_name": "faithk7u", "author_url": "", "post_date": "2024-06-12T15:13:20.170000", "content": "Congrats for becoming a competition GM! Huge achievement! Also impressive to see the relatively simple solution can achieve the great final result.
\nHave the following questions:
\nWhy did you choose 5-sec clip during the training stage? In my experience, 10-sec clip from the middle of the train audio improves the cv score a bit, and also gives the model more samples to learn from my opinion.
Why didn't you use more image/audio augmentations?
Is there a reason for choosing n_fft=2048 * 2? That seems a lot more computation required than when n_fft=512, which some solutions adopt to use.
Thanks again for sharing the solution!
", "votes": 0, "replies": [] }, { "id": 2868227, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2024-06-12T08:52:23.450000", "content": "Well done. The idea to mix raw and MEL spectrogram is great.
\nCould you give more details about the openvino part, thanks!
Like most people, convert pytorch model to onnx, then use openvino do inference. But spectrum is produced by torchaudio. And I will share my notebook later.
", "votes": 1, "replies": [] } ] }, { "id": 2868184, "author_name": "YoungJun_Player", "author_url": "", "post_date": "2024-06-12T08:31:25.813000", "content": "Congratulations! I've just started studying I feel that I can grow quickly with the help of people like you, who are willing to discuss, share, and provide guidance. Thank you.
", "votes": 0, "replies": [] }, { "id": 2867811, "author_name": "C R Suthikshn Kumar", "author_url": "", "post_date": "2024-06-12T05:15:59.517000", "content": "Congratulations on securing 5th Place in this competition. Also, thanks on sharing the solution details.
", "votes": 0, "replies": [] }, { "id": 2867231, "author_name": "vialactea", "author_url": "", "post_date": "2024-06-11T18:03:10.843000", "content": "Thanks for sharing and congratulations on the 5th place and GM title!
\nI'm surprised that 1D models worked. We tried them early and discarded the idea because the performance was considerably worse than melspecs. Maybe the size of the test made a difference (only part of the data for 1 fold and 10 epochs). A reminder to revisit early decisions.
\nCould you please elaborate a bit on the approach you followed to evaluate models and define the weights for the ensemble? This was a struggle for us and I wonder how other teams approached it.
", "votes": 0, "replies": [ { "id": 2868149, "author_name": "coolz", "author_url": "", "post_date": "2024-06-12T08:00:23.920000", "content": "I didn't find a good way to do local cv, and raw signal model cv is lower than spectrum model.
\nI use 0.5,0.4,0.1 as ensemble weights. Because raw signal model score is about the same with spectrum model, and the mix model is a combine of them, use a small weight here.
Congratulations ! Would you be publishing your inference pipeline? I'd be very interested to see how you optimised MelSpec layers with openvino, I failed at this.
", "votes": 0, "replies": [ { "id": 2866495, "author_name": "coolz", "author_url": "", "post_date": "2024-06-11T11:11:46.423000", "content": "I will share it later. However I met the same problem, I use torchaudio to produce the spectrum in dataiter, the layer not included in the model.
", "votes": 0, "replies": [] }, { "id": 2866574, "author_name": "Aphysict", "author_url": "", "post_date": "2024-06-11T11:46:09.813000", "content": "Try my last year notebook
", "votes": 1, "replies": [ { "id": 2868130, "author_name": "Phaedrus", "author_url": "", "post_date": "2024-06-12T07:50:29.093000", "content": "Thanks, I will go through your NB.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3146440, "author_name": "", "author_url": "", "post_date": "2025-03-10T21:39:57.853000", "content": "", "votes": 0, "replies": [] }, { "id": 2875213, "author_name": "", "author_url": "", "post_date": "2024-06-17T00:28:55.100000", "content": "", "votes": 0, "replies": [] }, { "id": 2866694, "author_name": "", "author_url": "", "post_date": "2024-06-11T13:02:35.197000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2866034": "\n\nFirst of all, thanks to the organizers for hosting this competition.\nAnd congrats to all winners.\n\n\n### Data\n2024 data\n\n### Model\n| model | backbone | weights | public | private |\n|--------------------------------|----------------|-------------|----------|----------|\n| raw signale | hgnetb0 | 3/5 folds | 0.720354 | 0.666671 |\n| spectrum | efficientnetb0 | 1/5 folds | 0.708198 | 0.672360 |\n| raw+spectrum | hgnetb0+effb0 | 1/5 folds | 0.698408 | 0.660503 |\n| ensemble | ... | 0.5+0.4+0.1 | 0.743960 | 0.687173 |\n\n+ 1. raw signal model\n```\n# x bsx160000\n\n#bsx80000x2\nx = x.view(bs, -1, 2) \n#bsx2x80000\nx = torch.transpose(x, 2, 1) \n#bsx2x1000x80\nx = x.view(bs, 2, -1, 80) \nfeature = self.backbone(x) \n```\n+ 2. spectrum model\n```\ntorchaudio.transforms.MelSpectrogram(\n 32000,\n n_mels=512,\n f_min=0,\n f_max=16000,\n n_fft=2048*2,\n hop_length=512,\n normalized=True,\n ),\n\ntorchaudio.transforms.AmplitudeToDB(top_db=80)\n```\n+ 3. mix model\n\n```commandline\nraw_f = self.raw_model(x)\nspec_f = self.spec_model(x)\nfeature = torch.cat([raw_f,spec_f],dim=1)\nx=self.fc(feature)\n\n## spectrum is resized to 256x256 for fast speed, \n## other params for mel spectrum is the same with spectrm model.\n```\n### Preprocess\n+ 1. wave/max(wave) if max(wave)>1\n### Augmentation\n+ XY cut out for spectrum model\n### Train\n+ 1. Random sample 5 seconds data for trainning, \nfirst 5 seconds data for validation.\n+ 2. Mixup p=1.\n+ 3. BCEWithLogitsLoss\n\n### inference\n+ openvino\n\n\n### extra\nI make a post to discussion the method feeding raw signal to vision model. Here is the link [good luck](https://www.kaggle.com/competitions/birdclef-2024/discussion/511763)\n\n\ninference [notebook](https://www.kaggle.com/code/cooolz/5nd-solution)\n", "2866062": "Regarding the \"raw model\", how did you deal with it to achieve good results? I rarely see people using raw signals in Bird competitions.", "2869294": "Good work. I was betting you'd use packed raw waveform as you did in HMS competition!\n\nCongrats on becoming a GM. It was a matter of time given what you did in HMS.\n\nDid you compare openvino accuracy with pytorch models? Asking because we found that it loses some accuracy and we didn't use it as a result.", "2867896": "Congrats for reaching Competition Grandmaster status 🎉 Huge achievement", "2866944": "Congrats for your medal!\n\nQuestion about your signal approach, what is the intuition behind your reshaping? Why picking 2 channels instead of 3 for example?", "2866892": "Nice use of 1D-signals again! Also, congrats on becoming GM.\n\nIs there a reason why you use n_mels=512 and then resize to (256, 256) rather than just setting n_mels=256?", "2866413": "Great how you managed to train raw model that well, in my experiments it had bad CV so I didn't submit it", "2866141": "Hi, I had a doubt like taking 5 random seconds from sample during training wont guarantee the bird noise in it right? If yes then how the model works?", "2866332": "Solution seems so simple! How did you get such powerful single models using this approach? Were there any other tricks you used or parameters you optimized to get such high scoring models? Im so curious how such basic settings score so high. And congrats on GM rank!", "2866143": "Impressive!! Will you mind providing some more details about the raw signals model?\nI think it is really a big breakthrough because current solutions like birdnet, bird-vocalization-classifier are all based on computer vision, thus solution with raw signal maybe cutting edge.", "2871578": "Congratulations 👏", "2869286": "Congrats for becoming Competition Grandmaster!!!!!", "2868710": "Congrats for becoming a competition GM! Huge achievement! Also impressive to see the relatively simple solution can achieve the great final result. \n\nHave the following questions: \n\n1. Why did you choose 5-sec clip during the training stage? In my experience, 10-sec clip from the middle of the train audio improves the cv score a bit, and also gives the model more samples to learn from my opinion. \n\n2. Why didn't you use more image/audio augmentations? \n\n3. Is there a reason for choosing n_fft=2048 * 2? That seems a lot more computation required than when n_fft=512, which some solutions adopt to use.\n\nThanks again for sharing the solution!", "2868227": "Well done. The idea to mix raw and MEL spectrogram is great.\nCould you give more details about the `openvino` part, thanks!", "2868184": "Congratulations! I've just started studying I feel that I can grow quickly with the help of people like you, who are willing to discuss, share, and provide guidance. Thank you.", "2867811": "Congratulations on securing 5th Place in this competition. Also, thanks on sharing the solution details. ", "2867231": "Thanks for sharing and congratulations on the 5th place and GM title!\n\nI'm surprised that 1D models worked. We tried them early and discarded the idea because the performance was considerably worse than melspecs. Maybe the size of the test made a difference (only part of the data for 1 fold and 10 epochs). A reminder to revisit early decisions.\n\nCould you please elaborate a bit on the approach you followed to evaluate models and define the weights for the ensemble? This was a struggle for us and I wonder how other teams approached it.", "2866485": "Congratulations ! Would you be publishing your inference pipeline? I'd be very interested to see how you optimised MelSpec layers with openvino, I failed at this. ", "3146440": "", "2875213": "", "2866694": "" } }