Congratulations to all the winners and gold getters, I guess those teams that broke the 0.950 wall have found the essence of this competition, which we couldn't.
\n
\nFirstly, thanks to the host for holding quite an interesting competition. Partly labeled classification is a challenging task, which made this competition more interesting than a simple bioacoustics audio tagging competition.
\n
\nOur solution is a ranking average of image classification models and SED models. Y.Nakama, kaerururu, and I worked a lot on SED models but couldn't break 0.90 until we merge with Taku Hiraiwa. His model was based on image classification similar to @cpmpml's model. We found that quite good , so we focused on improving image classification models in the remained days.
It was Taku Hiraiwa's idea to only use the annotated part of the train data. To do so, we crop the image patches from log-melspectrogram of train data based on the t_min, t_max, f_min, f_max information of train_tp.csv and train_fp.csv, and resized the patch to fixed shape (say, 320 x 320 or so). The cropping is performed on the fly through training, so the part we crop out is randomized along with time axis.
\nWith these image patches we trained EfficientNet models, and monitored F1 score with threshold 0.5.
\nHere's the other details of image classification models.
timm and used tf_efficientnet_b<0-5>_ns weights).tf_efficientnet_b0_ns image size 244tf_efficientnet_b0_ns image size 320tf_efficientnet_b0_ns image size 456tf_efficientnet_b1_ns image size 456tf_efficientnet_b2_ns image size 456tf_efficientnet_b3_ns image size 456tf_efficientnet_b4_ns image size 456tf_efficientnet_b5_ns image size 456All of our SED models use the head architecture introduced in PANNs repository. The CNN encoder is either EfficientNet or ResNeSt, and they are trained with weak/strong supervision. We tried a lot of things on this, but couldn't find factors that consistently work well - which means the LB scores varied quite randomly w.r.t CV score.
\nOur best SED model is rank average of 11 models(public: 0.901, privte: 0.911) below - each of them differs slightly so we describe the difference briefly.
Congratulations to all the winners and gold getters, I guess those teams that broke the 0.950 wall have found the essence of this competition, which we couldn't.
\n
\nFirstly, thanks to the host for holding quite an interesting competition. Partly labeled classification is a challenging task, which made this competition more interesting than a simple bioacoustics audio tagging competition.
\n
\nOur solution is a ranking average of image classification models and SED models. Y.Nakama, kaerururu, and I worked a lot on SED models but couldn't break 0.90 until we merge with Taku Hiraiwa. His model was based on image classification similar to @cpmpml's model. We found that quite good , so we focused on improving image classification models in the remained days.
It was Taku Hiraiwa's idea to only use the annotated part of the train data. To do so, we crop the image patches from log-melspectrogram of train data based on the t_min, t_max, f_min, f_max information of train_tp.csv and train_fp.csv, and resized the patch to fixed shape (say, 320 x 320 or so). The cropping is performed on the fly through training, so the part we crop out is randomized along with time axis.
\nWith these image patches we trained EfficientNet models, and monitored F1 score with threshold 0.5.
\nHere's the other details of image classification models.
timm and used tf_efficientnet_b<0-5>_ns weights).tf_efficientnet_b0_ns image size 244tf_efficientnet_b0_ns image size 320tf_efficientnet_b0_ns image size 456tf_efficientnet_b1_ns image size 456tf_efficientnet_b2_ns image size 456tf_efficientnet_b3_ns image size 456tf_efficientnet_b4_ns image size 456tf_efficientnet_b5_ns image size 456All of our SED models use the head architecture introduced in PANNs repository. The CNN encoder is either EfficientNet or ResNeSt, and they are trained with weak/strong supervision. We tried a lot of things on this, but couldn't find factors that consistently work well - which means the LB scores varied quite randomly w.r.t CV score.
\nOur best SED model is rank average of 11 models(public: 0.901, privte: 0.911) below - each of them differs slightly so we describe the difference briefly.
Congrats on strong finish @hidehisaarai1213 and team and thanks for sharing details solution
", "rawMarkdown": "Congrats on strong finish @hidehisaarai1213 and team and thanks for sharing details solution", "votes": null }, { "id": "1208635", "postDate": "02/18/2021 11:45:20", "content": "Congratz ! Good to see you at the top of another audio competition :)
", "rawMarkdown": "Congratz ! Good to see you at the top of another audio competition :)", "votes": null }, { "id": "1208655", "postDate": "02/18/2021 12:06:57", "content": "Congratulations🎉🎉
", "rawMarkdown": "Congratulations🎉🎉", "votes": null }, { "id": "1208690", "postDate": "02/18/2021 12:44:04", "content": "Your solution (Mixing Image classification models and SED) is interesting!
\nI have a question.
\nIn Image classification models, can models detect multi label?
Wow, you pushed SED quite far! I am glad you mentioned my Cornell model :)
\nCongrats on the strong ending, I was afraid you'd pass me.
", "rawMarkdown": "Wow, you pushed SED quite far! I am glad you mentioned my Cornell model :)\n\nCongrats on the strong ending, I was afraid you'd pass me.", "votes": null }, { "id": "1209645", "postDate": "02/19/2021 01:49:16", "content": "is there any experimental results on COLA from \"Contrastive learning of general purpose audio representations\"?
", "rawMarkdown": "is there any experimental results on COLA from \"Contrastive learning of general purpose audio representations\"?", "votes": null }, { "id": "1209689", "postDate": "02/19/2021 02:30:25", "content": "Thank you! You've also finished in good place!
", "rawMarkdown": "Thank you! You've also finished in good place!", "votes": null }, { "id": "1209693", "postDate": "02/19/2021 02:33:26", "content": "Same to you! Also good to see many competitors of Birdcall comes to this competition. At first it seemed similar to birdcall but it turned out it wasn't - both are interesting in different ways
", "rawMarkdown": "Same to you! Also good to see many competitors of Birdcall comes to this competition. At first it seemed similar to birdcall but it turned out it wasn't - both are interesting in different ways", "votes": null }, { "id": "1209696", "postDate": "02/19/2021 02:33:51", "content": "Thank you😊😊😊
", "rawMarkdown": "Thank you😊😊😊", "votes": null }, { "id": "1209712", "postDate": "02/19/2021 02:43:49", "content": "Thanks!
\n\n\nIn Image classification models, can models detect multi label?
\n
No, it cannot. To be precise, image classification model only takes the region around where a specific species exist. We created dicts that tell the frequency range of species, which was calculated based on train data. When we crop image patch, we only take the region inside that frequency range. Of course, the frequency range is different between species, so the result of cropping have different shape each other, so we resize that to a fixed shape.
\nWhen on inference time, we crop 24 patches (each of them corresponds to each species) from one time window, and put that 24 patches into CNN, so it takes 24x longer than usual CNN forward path.
Thank you!
\n\n\nI am glad you mentioned my Cornell model :)
\n
Oh, I meant it turned out that our image classification model was similar to your model in this competition (I found that after we read you solution.), but yes we were also helped a lot from your solution in Cornell's competition.
\nI saw many top performing teams used loss masking technique, which was also used in your Cornell's solution. It's not sure whether they came up with that idea alone or inspired from yours, but I'm quite sure your solution had big impact on this competition.
Not unfortunately.
\nAt first I thought it is worth trying because I thought the key of this competition is Few-shot learning, but after a while I found this competition is more like a PU learning competition, so I changed the priority of COLA or other audio representation learning methods.
Thank you for your reply. If that is so, how do you detect multi label( secodary label)?
\nIs it SED? If it is SED, It's a amazing ensemble result!
I also tried Image classification with ArcFace. But score was too low.
\nBut mixing SED and ArcFace (simple prediction average ) improved a littel.
\nThis might be the same as your team.
That is addressed by this
\n\n", "rawMarkdown": "That is addressed by this\n> When on inference time, we crop 24 patches (each of them corresponds to each species) from one time window, and put that 24 patches into CNN, so it takes 24x longer than usual CNN forward path.", "votes": null }, { "id": "1211583", "postDate": "02/20/2021 10:40:00", "content": "When on inference time, we crop 24 patches (each of them corresponds to each species) from one time window, and put that 24 patches into CNN, so it takes 24x longer than usual CNN forward path.
\n
\n\n. It's not sure whether they came up with that idea alone or inspired from yours, but I'm quite sure your solution had big impact on this competition.
\n
I am not sure it was influential given it was almost never cited in writeups. I was mentioned only a couple of times unless mistaken, and you are the only one to mention explicitly my Cornell solution.
", "rawMarkdown": "> . It's not sure whether they came up with that idea alone or inspired from yours, but I'm quite sure your solution had big impact on this competition.\n\nI am not sure it was influential given it was almost never cited in writeups. I was mentioned only a couple of times unless mistaken, and you are the only one to mention explicitly my Cornell solution.", "votes": null } ], "comments": [ { "id": 1208562, "author_name": "duykhanh99", "author_url": "", "post_date": "02/18/2021 10:41:36", "content": "Congrats on strong finish @hidehisaarai1213 and team and thanks for sharing details solution
", "votes": null, "replies": [ { "id": 1209689, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/19/2021 02:30:25", "content": "Thank you! You've also finished in good place!
", "votes": null, "replies": [] } ] }, { "id": 1208635, "author_name": "theoviel", "author_url": "", "post_date": "02/18/2021 11:45:20", "content": "Congratz ! Good to see you at the top of another audio competition :)
", "votes": null, "replies": [ { "id": 1209693, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/19/2021 02:33:26", "content": "Same to you! Also good to see many competitors of Birdcall comes to this competition. At first it seemed similar to birdcall but it turned out it wasn't - both are interesting in different ways
", "votes": null, "replies": [] } ] }, { "id": 1208655, "author_name": "riadalmadani", "author_url": "", "post_date": "02/18/2021 12:06:57", "content": "Congratulations🎉🎉
", "votes": null, "replies": [ { "id": 1209696, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/19/2021 02:33:51", "content": "Thank you😊😊😊
", "votes": null, "replies": [] } ] }, { "id": 1208690, "author_name": "shinmurashinmura", "author_url": "", "post_date": "02/18/2021 12:44:04", "content": "Your solution (Mixing Image classification models and SED) is interesting!
\nI have a question.
\nIn Image classification models, can models detect multi label?
Thanks!
\n\n\nIn Image classification models, can models detect multi label?
\n
No, it cannot. To be precise, image classification model only takes the region around where a specific species exist. We created dicts that tell the frequency range of species, which was calculated based on train data. When we crop image patch, we only take the region inside that frequency range. Of course, the frequency range is different between species, so the result of cropping have different shape each other, so we resize that to a fixed shape.
\nWhen on inference time, we crop 24 patches (each of them corresponds to each species) from one time window, and put that 24 patches into CNN, so it takes 24x longer than usual CNN forward path.
Thank you for your reply. If that is so, how do you detect multi label( secodary label)?
\nIs it SED? If it is SED, It's a amazing ensemble result!
I also tried Image classification with ArcFace. But score was too low.
\nBut mixing SED and ArcFace (simple prediction average ) improved a littel.
\nThis might be the same as your team.
That is addressed by this
\n\n", "votes": null, "replies": [] } ] }, { "id": 1209458, "author_name": "cpmpml", "author_url": "", "post_date": "02/18/2021 22:59:16", "content": "When on inference time, we crop 24 patches (each of them corresponds to each species) from one time window, and put that 24 patches into CNN, so it takes 24x longer than usual CNN forward path.
\n
Wow, you pushed SED quite far! I am glad you mentioned my Cornell model :)
\nCongrats on the strong ending, I was afraid you'd pass me.
", "votes": null, "replies": [ { "id": 1209716, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/19/2021 02:49:32", "content": "Thank you!
\n\n\nI am glad you mentioned my Cornell model :)
\n
Oh, I meant it turned out that our image classification model was similar to your model in this competition (I found that after we read you solution.), but yes we were also helped a lot from your solution in Cornell's competition.
\nI saw many top performing teams used loss masking technique, which was also used in your Cornell's solution. It's not sure whether they came up with that idea alone or inspired from yours, but I'm quite sure your solution had big impact on this competition.
\n\n. It's not sure whether they came up with that idea alone or inspired from yours, but I'm quite sure your solution had big impact on this competition.
\n
I am not sure it was influential given it was almost never cited in writeups. I was mentioned only a couple of times unless mistaken, and you are the only one to mention explicitly my Cornell solution.
", "votes": null, "replies": [] } ] }, { "id": 1209645, "author_name": "hengck23", "author_url": "", "post_date": "02/19/2021 01:49:16", "content": "is there any experimental results on COLA from \"Contrastive learning of general purpose audio representations\"?
", "votes": null, "replies": [ { "id": 1209729, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "02/19/2021 02:53:18", "content": "Not unfortunately.
\nAt first I thought it is worth trying because I thought the key of this competition is Few-shot learning, but after a while I found this competition is more like a PU learning competition, so I changed the priority of COLA or other audio representation learning methods.