Thanks to the competition host and kaggle teams for holding this competition and congratulations to all winners. And thanks @ttahara for providing resampled train data and @rohanrao for external data. Also thanks @hidehisaarai1213 for providing a good baseline and introduction to SED. I am excited to get my solo gold.
\nThe labels provided by competition host is super noisy. There is even a 40 minute audio with a single label. I think how to use the secondary label is vital in this competition. In my test, using only primary labels will only predict very view bird calls and get many nocalls.
\nMix sound
\nIn order to generate more samples and get clips with more than one bird(with good labels), I choose to mix the clips from audios with no secondary labels. I use an or operation to generate the labels for mixed clips and mix sound like this:
\nmixed_sound=sound1*uniform(0.8,1.2)+sound2*uniform(0.8,1.2)+sound3*uniform(0.8,1.2)\nGenerate pseudo strong labels
\nI tried to generate pseudo strong labels with a SED method. I train SED models with 10s clips and make prediction on whole audios. However, the generated pseudo labels are still noisy and unreliable,there is too many false positives. Finally, I used them to fix labels for audios with secondary labels.
\nTrain Detail
\nI used logmel spectrogram as input and randomly crop 5s clips from that. Then mixed sound augmentations will be applied to audios with no secondary labels, labels for audio with secondary labels will be adjusted according to pseudo strong labels.
\nMaking a good validation is very difficult in this competition, since there are no reliable nocall samples and a single clip can contain multiple bird calls. I chose to use a similar mechanism with site_3 audios in testset and calculated record-wise F1 scores.
\nPreprocessing: MelSpectrogram-> ToDB -> Normalize -> Resize
\nDataAugmentation: GaussianNoise, BackgroundNoise, Shift, Drop, Clipping,
\nBackbones: Resnest50, Regnety_040
\nSingle fold Score: 0.654/0.591/CV 0.75
\nIn my best submission, the models are 5fold resnest50d(256x512) + 5fold resnest50d(320x768)+ 4fold regnety_040 (224x512).
\nI used two shifted versions of sound clip as TTA, both with half hop length. However, they make no difference on LB as well as on my CV.
\nUsing secondary labels, record-wise F1 increases, but clip-wise F1 decreases.
\nTrain longer make result stable. Train with 40 epochs can also get decent scores, but the score is varying a lot. Training 80 epochs results in higher CV and stable scores in LB.
\nSimulate nocall sound and train a binary classifier. There is a huge domain gap between simulated ones and real ones. The classifier just finds a shortcut. My classifier prediction is nearly all positive on competition data.
\nIt seems there is an additional mp3 compression for testset. I tried to finetune on compressed audios. It improves some score on public LB but not on private LB.
\nMixup make my result worse.
\nUsing more mel bins can improve local CV, but hurts LB.
", "messages": [ { "id": "1012271", "postDate": "09/16/2020 02:20:35", "content": "Thanks to the competition host and kaggle teams for holding this competition and congratulations to all winners. And thanks @ttahara for providing resampled train data and @rohanrao for external data. Also thanks @hidehisaarai1213 for providing a good baseline and introduction to SED. I am excited to get my solo gold.
\nThe labels provided by competition host is super noisy. There is even a 40 minute audio with a single label. I think how to use the secondary label is vital in this competition. In my test, using only primary labels will only predict very view bird calls and get many nocalls.
\nMix sound
\nIn order to generate more samples and get clips with more than one bird(with good labels), I choose to mix the clips from audios with no secondary labels. I use an or operation to generate the labels for mixed clips and mix sound like this:
\nmixed_sound=sound1*uniform(0.8,1.2)+sound2*uniform(0.8,1.2)+sound3*uniform(0.8,1.2)\nGenerate pseudo strong labels
\nI tried to generate pseudo strong labels with a SED method. I train SED models with 10s clips and make prediction on whole audios. However, the generated pseudo labels are still noisy and unreliable,there is too many false positives. Finally, I used them to fix labels for audios with secondary labels.
\nTrain Detail
\nI used logmel spectrogram as input and randomly crop 5s clips from that. Then mixed sound augmentations will be applied to audios with no secondary labels, labels for audio with secondary labels will be adjusted according to pseudo strong labels.
\nMaking a good validation is very difficult in this competition, since there are no reliable nocall samples and a single clip can contain multiple bird calls. I chose to use a similar mechanism with site_3 audios in testset and calculated record-wise F1 scores.
\nPreprocessing: MelSpectrogram-> ToDB -> Normalize -> Resize
\nDataAugmentation: GaussianNoise, BackgroundNoise, Shift, Drop, Clipping,
\nBackbones: Resnest50, Regnety_040
\nSingle fold Score: 0.654/0.591/CV 0.75
\nIn my best submission, the models are 5fold resnest50d(256x512) + 5fold resnest50d(320x768)+ 4fold regnety_040 (224x512).
\nI used two shifted versions of sound clip as TTA, both with half hop length. However, they make no difference on LB as well as on my CV.
\nUsing secondary labels, record-wise F1 increases, but clip-wise F1 decreases.
\nTrain longer make result stable. Train with 40 epochs can also get decent scores, but the score is varying a lot. Training 80 epochs results in higher CV and stable scores in LB.
\nSimulate nocall sound and train a binary classifier. There is a huge domain gap between simulated ones and real ones. The classifier just finds a shortcut. My classifier prediction is nearly all positive on competition data.
\nIt seems there is an additional mp3 compression for testset. I tried to finetune on compressed audios. It improves some score on public LB but not on private LB.
\nMixup make my result worse.
\nUsing more mel bins can improve local CV, but hurts LB.
", "rawMarkdown": "Thanks to the competition host and kaggle teams for holding this competition and congratulations to all winners. And thanks @ttahara for providing resampled train data and @rohanrao for external data. Also thanks @hidehisaarai1213 for providing a good baseline and introduction to SED. I am excited to get my solo gold.\n\n#### Train\n\nThe labels provided by competition host is super noisy. There is even a 40 minute audio with a single label. I think how to use the secondary label is vital in this competition. In my test, using only primary labels will only predict very view bird calls and get many nocalls. \n\n**Mix sound**\n\nIn order to generate more samples and get clips with more than one bird(with good labels), I choose to mix the clips from audios with no secondary labels. I use an or operation to generate the labels for mixed clips and mix sound like this:\n\n```\nmixed_sound=sound1*uniform(0.8,1.2)+sound2*uniform(0.8,1.2)+sound3*uniform(0.8,1.2)\n```\n\n**Generate pseudo strong labels**\n\nI tried to generate pseudo strong labels with a SED method. I train SED models with 10s clips and make prediction on whole audios. However, the generated pseudo labels are still noisy and unreliable,there is too many false positives. Finally, I used them to fix labels for audios with secondary labels. \n\n**Train Detail**\n\nI used logmel spectrogram as input and randomly crop 5s clips from that. Then mixed sound augmentations will be applied to audios with no secondary labels, labels for audio with secondary labels will be adjusted according to pseudo strong labels. \n\nMaking a good validation is very difficult in this competition, since there are no reliable nocall samples and a single clip can contain multiple bird calls. I chose to use a similar mechanism with site_3 audios in testset and calculated record-wise F1 scores. \n\nPreprocessing: MelSpectrogram-> ToDB -> Normalize -> Resize\n\nDataAugmentation: GaussianNoise, BackgroundNoise, Shift, Drop, Clipping,\n\nBackbones: Resnest50, Regnety_040\n\nSingle fold Score: 0.654/0.591/CV 0.75\n\n#### Ensemble and TTA\n\nIn my best submission, the models are 5fold resnest50d(256x512) + 5fold resnest50d(320x768)+ 4fold regnety_040 (224x512).\n\nI used two shifted versions of sound clip as TTA, both with half hop length. However, they make no difference on LB as well as on my CV.\n\n#### What works\n\nUsing secondary labels, record-wise F1 increases, but clip-wise F1 decreases. \n\nTrain longer make result stable. Train with 40 epochs can also get decent scores, but the score is varying a lot. Training 80 epochs results in higher CV and stable scores in LB.\n\n#### What doesn't work\n\nSimulate nocall sound and train a binary classifier. There is a huge domain gap between simulated ones and real ones. The classifier just finds a shortcut. My classifier prediction is nearly all positive on competition data. \n\nIt seems there is an additional mp3 compression for testset. I tried to finetune on compressed audios. It improves some score on public LB but not on private LB.\n\nMixup make my result worse.\n\nUsing more mel bins can improve local CV, but hurts LB.", "votes": null }, { "id": "1012290", "postDate": "09/16/2020 02:37:11", "content": "Congrats on 8th place and solo gold medal @rguo97
", "rawMarkdown": "Congrats on 8th place and solo gold medal @rguo97", "votes": null }, { "id": "1012340", "postDate": "09/16/2020 03:15:09", "content": "Thank you!
", "rawMarkdown": "Thank you!", "votes": null }, { "id": "1019693", "postDate": "09/20/2020 15:51:35", "content": "Congratulations!!
", "rawMarkdown": "Congratulations!!", "votes": null } ], "comments": [ { "id": 1012290, "author_name": "duykhanh99", "author_url": "", "post_date": "09/16/2020 02:37:11", "content": "Congrats on 8th place and solo gold medal @rguo97
", "votes": null, "replies": [ { "id": 1012340, "author_name": "rguo97", "author_url": "", "post_date": "09/16/2020 03:15:09", "content": "Thank you!
", "votes": null, "replies": [] } ] }, { "id": 1019693, "author_name": "shivambhardwaj0101", "author_url": "", "post_date": "09/20/2020 15:51:35", "content": "Congratulations!!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1012271": "Thanks to the competition host and kaggle teams for holding this competition and congratulations to all winners. And thanks @ttahara for providing resampled train data and @rohanrao for external data. Also thanks @hidehisaarai1213 for providing a good baseline and introduction to SED. I am excited to get my solo gold.\n\n#### Train\n\nThe labels provided by competition host is super noisy. There is even a 40 minute audio with a single label. I think how to use the secondary label is vital in this competition. In my test, using only primary labels will only predict very view bird calls and get many nocalls. \n\n**Mix sound**\n\nIn order to generate more samples and get clips with more than one bird(with good labels), I choose to mix the clips from audios with no secondary labels. I use an or operation to generate the labels for mixed clips and mix sound like this:\n\n```\nmixed_sound=sound1*uniform(0.8,1.2)+sound2*uniform(0.8,1.2)+sound3*uniform(0.8,1.2)\n```\n\n**Generate pseudo strong labels**\n\nI tried to generate pseudo strong labels with a SED method. I train SED models with 10s clips and make prediction on whole audios. However, the generated pseudo labels are still noisy and unreliable,there is too many false positives. Finally, I used them to fix labels for audios with secondary labels. \n\n**Train Detail**\n\nI used logmel spectrogram as input and randomly crop 5s clips from that. Then mixed sound augmentations will be applied to audios with no secondary labels, labels for audio with secondary labels will be adjusted according to pseudo strong labels. \n\nMaking a good validation is very difficult in this competition, since there are no reliable nocall samples and a single clip can contain multiple bird calls. I chose to use a similar mechanism with site_3 audios in testset and calculated record-wise F1 scores. \n\nPreprocessing: MelSpectrogram-> ToDB -> Normalize -> Resize\n\nDataAugmentation: GaussianNoise, BackgroundNoise, Shift, Drop, Clipping,\n\nBackbones: Resnest50, Regnety_040\n\nSingle fold Score: 0.654/0.591/CV 0.75\n\n#### Ensemble and TTA\n\nIn my best submission, the models are 5fold resnest50d(256x512) + 5fold resnest50d(320x768)+ 4fold regnety_040 (224x512).\n\nI used two shifted versions of sound clip as TTA, both with half hop length. However, they make no difference on LB as well as on my CV.\n\n#### What works\n\nUsing secondary labels, record-wise F1 increases, but clip-wise F1 decreases. \n\nTrain longer make result stable. Train with 40 epochs can also get decent scores, but the score is varying a lot. Training 80 epochs results in higher CV and stable scores in LB.\n\n#### What doesn't work\n\nSimulate nocall sound and train a binary classifier. There is a huge domain gap between simulated ones and real ones. The classifier just finds a shortcut. My classifier prediction is nearly all positive on competition data. \n\nIt seems there is an additional mp3 compression for testset. I tried to finetune on compressed audios. It improves some score on public LB but not on private LB.\n\nMixup make my result worse.\n\nUsing more mel bins can improve local CV, but hurts LB.", "1012290": "Congrats on 8th place and solo gold medal @rguo97", "1012340": "Thank you!", "1019693": "Congratulations!!" }, "source": "meta" }