It said bagging works good in this competition.
\nWe thought so too from us experiment, so our strategy is training various model and they ensemble.
Our final submission is voting by 5 models that three rexnet_200, densenet161, and resnest50d.
\nmy aproach is cnn classifier model
\nI cut out 5 seconds of audio from train short audio.
\nThe training data was converted to logmel using torchlibrosa, and then increased to 3 channels.
\nI used delta to increase channels, referring to past competitions. There was not much difference from simple repeat.
Reference Cornell Birdcall Identification 4-th place solutinon
\nrexnet_200
\nThe rexnet 200 was unusually strong:)
\nI keep track of the f-1 for the train sound scape data and use the best f-1 weight.
for ensemble [densenet161, efficientnetv2_s, resnext50_32x4]
\nnot worked efficientnetb0-4,resnest50d
\nI mainly developed post-processing (And hand labeing).
\nOur post-processing improved LB very well (LB 0.65 -> 0.70 jump up).
\nOur post-processing is 7 ideas. But it's too long to write here.
If you want to know our post-processing, please comment me.
\nThen I will build another thread.
※ My model has not used in our team final submission, so this part is \"I try, but it was not working\".
\nI have used the ResNet18 based SED model.
\nMy ingenuity points are there three:
In order to make the audio of the training data as close as possible to the test data, I used the data of ESC50 to add noise to the training data. The selected audio are listed below.
\nairplane, rain, water_drops, crackling fire, engine, insects, crickets, frog, wind
\nI also adjusted Gain to give variation to the volume of the bird's voice, but this did not work.
\nBirds that are observed differ depending on the area, and it is expected that some birds, such as migratory birds, will be observed only for a specific period. I decided to make a specialized model.
\nSpecifically, the radius was within 1000 km from the observation point of the test data, and the period was narrowed down with a buffer of one month before and after the target period (for example, January to August for SSW).
\nAfter narrowing down, we sorted by rating of each audio file and extracted the first 5 seconds of the top 300.
\nThe dataset is here: SSW, SNE, COR, COL
\nI add nocall data that made from other site.
\nThe method making nocall datasets is reversed the method making datasets each site.
\nI extracted audio that is more than 1000km away from each site and is not in the target period.
nocall data is here
", "messages": [ { "id": "1332201", "postDate": "06/02/2021 01:02:44", "content": "It said bagging works good in this competition.
\nWe thought so too from us experiment, so our strategy is training various model and they ensemble.
Our final submission is voting by 5 models that three rexnet_200, densenet161, and resnest50d.
\nmy aproach is cnn classifier model
\nI cut out 5 seconds of audio from train short audio.
\nThe training data was converted to logmel using torchlibrosa, and then increased to 3 channels.
\nI used delta to increase channels, referring to past competitions. There was not much difference from simple repeat.
Reference Cornell Birdcall Identification 4-th place solutinon
\nrexnet_200
\nThe rexnet 200 was unusually strong:)
\nI keep track of the f-1 for the train sound scape data and use the best f-1 weight.
for ensemble [densenet161, efficientnetv2_s, resnext50_32x4]
\nnot worked efficientnetb0-4,resnest50d
\nI mainly developed post-processing (And hand labeing).
\nOur post-processing improved LB very well (LB 0.65 -> 0.70 jump up).
\nOur post-processing is 7 ideas. But it's too long to write here.
If you want to know our post-processing, please comment me.
\nThen I will build another thread.
※ My model has not used in our team final submission, so this part is \"I try, but it was not working\".
\nI have used the ResNet18 based SED model.
\nMy ingenuity points are there three:
In order to make the audio of the training data as close as possible to the test data, I used the data of ESC50 to add noise to the training data. The selected audio are listed below.
\nairplane, rain, water_drops, crackling fire, engine, insects, crickets, frog, wind
\nI also adjusted Gain to give variation to the volume of the bird's voice, but this did not work.
\nBirds that are observed differ depending on the area, and it is expected that some birds, such as migratory birds, will be observed only for a specific period. I decided to make a specialized model.
\nSpecifically, the radius was within 1000 km from the observation point of the test data, and the period was narrowed down with a buffer of one month before and after the target period (for example, January to August for SSW).
\nAfter narrowing down, we sorted by rating of each audio file and extracted the first 5 seconds of the top 300.
\nThe dataset is here: SSW, SNE, COR, COL
\nI add nocall data that made from other site.
\nThe method making nocall datasets is reversed the method making datasets each site.
\nI extracted audio that is more than 1000km away from each site and is not in the target period.
nocall data is here
", "rawMarkdown": "# [Interim] 68 place solution\n\n[It](https://www.kaggle.com/c/birdclef-2021/discussion/233454) said bagging works good in this competition.\nWe thought so too from us experiment, so our strategy is training various model and they ensemble.\n\nOur final submission is voting by 5 models that three rexnet_200, densenet161, and resnest50d.\n\n\n## teyo part\n\nmy aproach is cnn classifier model\n\n### Data Preparation\n\nI cut out 5 seconds of audio from train short audio.\nThe training data was converted to logmel using torchlibrosa, and then increased to 3 channels.\nI used delta to increase channels, referring to past competitions. There was not much difference from simple repeat.\n\nReference [Cornell Birdcall Identification 4-th place solutinon](https://www.kaggle.com/vladimirsydor/4-th-place-solution-inference-and-training-tips)\n\n### model\n\nrexnet_200\n\nThe rexnet 200 was unusually strong:)\nI keep track of the f-1 for the train sound scape data and use the best f-1 weight.\n\nfor ensemble [densenet161, efficientnetv2_s, resnext50_32x4]\n\nnot worked efficientnetb0-4,resnest50d\n\n### data augmentation\n\n+ NoiseInjection\n+ PinkNoise\n+ RandomVolume\n+ SpecAugmentation (do in model)\n+ [Modified Mixup](https://www.kaggle.com/c/birdsong-recognition/discussion/183199)\n + increase 0.02 in public LB\n\n### other settings\n\n+ split\n + StratifiedKFold\n + mainly 5 fold\n+ Scheduler\n + CosineAnnealingWarmRestarts\n + 30 epochs\n + lr=1e-4\n + min_lr=1e-6\n+ optimizer\n + Adam\n+ criterion\n + BCEWithLogitsLoss\n\n## shinmura part\nI mainly developed post-processing (And [hand labeing](https://www.kaggle.com/c/birdclef-2021/discussion/239911)). \n\nOur post-processing improved LB very well (**LB 0.65 -> 0.70** jump up). \nOur post-processing is 7 ideas. But it's too long to write here.\n\nIf you want to know our post-processing, please comment me.\nThen I will build another thread.\n\n## toda part\n\n**※ My model has not used in our team final submission, so this part is \"I try, but it was not working\".**\n\nI have used the ResNet18 based SED model.\nMy ingenuity points are there three:\n\n### 1. add background noise \n\nIn order to make the audio of the training data as close as possible to the test data, I used the data of ESC50 to add noise to the training data. The selected audio are listed below.\n\nairplane, rain, water_drops, crackling fire, engine, insects, crickets, frog, wind\n\nI also adjusted Gain to give variation to the volume of the bird's voice, but this did not work.\n\n### 2. make model each site\n\nBirds that are observed differ depending on the area, and it is expected that some birds, such as migratory birds, will be observed only for a specific period. I decided to make a specialized model.\n\nSpecifically, the radius was within 1000 km from the observation point of the test data, and the period was narrowed down with a buffer of one month before and after the target period (for example, January to August for SSW).\n\nAfter narrowing down, we sorted by rating of each audio file and extracted the first 5 seconds of the top 300.\n\nThe dataset is here: [SSW](https://www.kaggle.com/takamichitoda/birdclef-ssw-max300), [SNE](https://www.kaggle.com/takamichitoda/birdclef-sne-max300), [COR](https://www.kaggle.com/takamichitoda/birdclef-cor-max300), [COL](https://www.kaggle.com/takamichitoda/birdclef-col-max300)\n\n### 3. generate nocall data\n\nI add nocall data that made from other site.\nThe method making nocall datasets is reversed the method making datasets each site.\nI extracted audio that is more than 1000km away from each site and is not in the target period.\n\nnocall data is [here](https://www.kaggle.com/takamichitoda/birdclef-nocall-each-site)", "votes": null }, { "id": "1332214", "postDate": "06/02/2021 01:36:45", "content": "Congratulations @takamichitoda and team Overall a very good stable approach 👋 Keep it up!
", "rawMarkdown": "Congratulations @takamichitoda and team Overall a very good stable approach 👋 Keep it up!", "votes": null }, { "id": "1332218", "postDate": "06/02/2021 01:41:27", "content": "Our post-processing improved LB very well (LB 0.65 -> 0.70 jump up).\nOur post-processing is 7 ideas. But it's too long to write here.\n\nIf you want to know our post-processing, please comment me.\nThen I will build another thread.\ni definitely want to know about your post processing, this 0.5 improvement is also present on the private LB?
", "rawMarkdown": "```\nOur post-processing improved LB very well (LB 0.65 -> 0.70 jump up).\nOur post-processing is 7 ideas. But it's too long to write here.\n\nIf you want to know our post-processing, please comment me.\nThen I will build another thread.\n```\n\ni definitely want to know about your post processing, this 0.5 improvement is also present on the private LB?", "votes": null }, { "id": "1332222", "postDate": "06/02/2021 01:46:28", "content": "My post-processing improves CV by 0.055. In ~2 hours i would know if it improves private LB by the same amount, and publish it, if it does.
", "rawMarkdown": "My post-processing improves CV by 0.055. In ~2 hours i would know if it improves private LB by the same amount, and publish it, if it does.", "votes": null }, { "id": "1332226", "postDate": "06/02/2021 01:52:21", "content": "On mine, i did tried several things, like bird-wise thresholds, determining nocalls/calls ratios with different models, combining bagging and mean with many models, nothing worked better than the good&old weights and thresholds forcebreak, in the end it was my best score
", "rawMarkdown": "On mine, i did tried several things, like bird-wise thresholds, determining nocalls/calls ratios with different models, combining bagging and mean with many models, nothing worked better than the good&old weights and thresholds forcebreak, in the end it was my best score", "votes": null }, { "id": "1332266", "postDate": "06/02/2021 02:34:22", "content": "Thanks for the prompt write-up and congrat!
\nThe gain by post-processing is impressive, would love to know the tricks that you did
I'm a team member.
\nIn detail, our post-processing improvement was here.
Thanks for sharing your team solution. I look forward to you and your team's gold medals next time.
", "rawMarkdown": "Thanks for sharing your team solution. I look forward to you and your team's gold medals next time.", "votes": null } ], "comments": [ { "id": 1332214, "author_name": "lplenka", "author_url": "", "post_date": "06/02/2021 01:36:45", "content": "Congratulations @takamichitoda and team Overall a very good stable approach 👋 Keep it up!
", "votes": null, "replies": [] }, { "id": 1332218, "author_name": "victorasso", "author_url": "", "post_date": "06/02/2021 01:41:27", "content": "Our post-processing improved LB very well (LB 0.65 -> 0.70 jump up).\nOur post-processing is 7 ideas. But it's too long to write here.\n\nIf you want to know our post-processing, please comment me.\nThen I will build another thread.\ni definitely want to know about your post processing, this 0.5 improvement is also present on the private LB?
", "votes": null, "replies": [ { "id": 1332222, "author_name": "fffrrt", "author_url": "", "post_date": "06/02/2021 01:46:28", "content": "My post-processing improves CV by 0.055. In ~2 hours i would know if it improves private LB by the same amount, and publish it, if it does.
", "votes": null, "replies": [] }, { "id": 1332226, "author_name": "victorasso", "author_url": "", "post_date": "06/02/2021 01:52:21", "content": "On mine, i did tried several things, like bird-wise thresholds, determining nocalls/calls ratios with different models, combining bagging and mean with many models, nothing worked better than the good&old weights and thresholds forcebreak, in the end it was my best score
", "votes": null, "replies": [] }, { "id": 1332282, "author_name": "shinmurashinmura", "author_url": "", "post_date": "06/02/2021 03:01:32", "content": "I'm a team member.
\nIn detail, our post-processing improvement was here.
Thanks for the prompt write-up and congrat!
\nThe gain by post-processing is impressive, would love to know the tricks that you did
Thanks for sharing your team solution. I look forward to you and your team's gold medals next time.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1332201": "# [Interim] 68 place solution\n\n[It](https://www.kaggle.com/c/birdclef-2021/discussion/233454) said bagging works good in this competition.\nWe thought so too from us experiment, so our strategy is training various model and they ensemble.\n\nOur final submission is voting by 5 models that three rexnet_200, densenet161, and resnest50d.\n\n\n## teyo part\n\nmy aproach is cnn classifier model\n\n### Data Preparation\n\nI cut out 5 seconds of audio from train short audio.\nThe training data was converted to logmel using torchlibrosa, and then increased to 3 channels.\nI used delta to increase channels, referring to past competitions. There was not much difference from simple repeat.\n\nReference [Cornell Birdcall Identification 4-th place solutinon](https://www.kaggle.com/vladimirsydor/4-th-place-solution-inference-and-training-tips)\n\n### model\n\nrexnet_200\n\nThe rexnet 200 was unusually strong:)\nI keep track of the f-1 for the train sound scape data and use the best f-1 weight.\n\nfor ensemble [densenet161, efficientnetv2_s, resnext50_32x4]\n\nnot worked efficientnetb0-4,resnest50d\n\n### data augmentation\n\n+ NoiseInjection\n+ PinkNoise\n+ RandomVolume\n+ SpecAugmentation (do in model)\n+ [Modified Mixup](https://www.kaggle.com/c/birdsong-recognition/discussion/183199)\n + increase 0.02 in public LB\n\n### other settings\n\n+ split\n + StratifiedKFold\n + mainly 5 fold\n+ Scheduler\n + CosineAnnealingWarmRestarts\n + 30 epochs\n + lr=1e-4\n + min_lr=1e-6\n+ optimizer\n + Adam\n+ criterion\n + BCEWithLogitsLoss\n\n## shinmura part\nI mainly developed post-processing (And [hand labeing](https://www.kaggle.com/c/birdclef-2021/discussion/239911)). \n\nOur post-processing improved LB very well (**LB 0.65 -> 0.70** jump up). \nOur post-processing is 7 ideas. But it's too long to write here.\n\nIf you want to know our post-processing, please comment me.\nThen I will build another thread.\n\n## toda part\n\n**※ My model has not used in our team final submission, so this part is \"I try, but it was not working\".**\n\nI have used the ResNet18 based SED model.\nMy ingenuity points are there three:\n\n### 1. add background noise \n\nIn order to make the audio of the training data as close as possible to the test data, I used the data of ESC50 to add noise to the training data. The selected audio are listed below.\n\nairplane, rain, water_drops, crackling fire, engine, insects, crickets, frog, wind\n\nI also adjusted Gain to give variation to the volume of the bird's voice, but this did not work.\n\n### 2. make model each site\n\nBirds that are observed differ depending on the area, and it is expected that some birds, such as migratory birds, will be observed only for a specific period. I decided to make a specialized model.\n\nSpecifically, the radius was within 1000 km from the observation point of the test data, and the period was narrowed down with a buffer of one month before and after the target period (for example, January to August for SSW).\n\nAfter narrowing down, we sorted by rating of each audio file and extracted the first 5 seconds of the top 300.\n\nThe dataset is here: [SSW](https://www.kaggle.com/takamichitoda/birdclef-ssw-max300), [SNE](https://www.kaggle.com/takamichitoda/birdclef-sne-max300), [COR](https://www.kaggle.com/takamichitoda/birdclef-cor-max300), [COL](https://www.kaggle.com/takamichitoda/birdclef-col-max300)\n\n### 3. generate nocall data\n\nI add nocall data that made from other site.\nThe method making nocall datasets is reversed the method making datasets each site.\nI extracted audio that is more than 1000km away from each site and is not in the target period.\n\nnocall data is [here](https://www.kaggle.com/takamichitoda/birdclef-nocall-each-site)", "1332214": "Congratulations @takamichitoda and team Overall a very good stable approach 👋 Keep it up!", "1332218": "```\nOur post-processing improved LB very well (LB 0.65 -> 0.70 jump up).\nOur post-processing is 7 ideas. But it's too long to write here.\n\nIf you want to know our post-processing, please comment me.\nThen I will build another thread.\n```\n\ni definitely want to know about your post processing, this 0.5 improvement is also present on the private LB?", "1332222": "My post-processing improves CV by 0.055. In ~2 hours i would know if it improves private LB by the same amount, and publish it, if it does.", "1332226": "On mine, i did tried several things, like bird-wise thresholds, determining nocalls/calls ratios with different models, combining bagging and mean with many models, nothing worked better than the good&old weights and thresholds forcebreak, in the end it was my best score", "1332266": "Thanks for the prompt write-up and congrat!\nThe gain by post-processing is impressive, would love to know the tricks that you did", "1332282": "I'm a team member.\nIn detail, our post-processing improvement was here.\n+ public LB 0.65->0.70\n+ private LB 0.58->0.60", "1333357": "Thanks for sharing your team solution. I look forward to you and your team's gold medals next time." }, "source": "meta" }