{ "id": 243349, "title": "Thanks . We all did Best. Solution Outline (Public LB 9th place)", "url": "/competitions/birdclef-2021/discussion/243349", "author_name": "", "post_date": "2021-06-02T06:41:33.688506600Z", "votes": 17, "comment_count": 5, "views": 0, "content": "

Congratulations to Winners !
\n@cpmpml you stood 1st/2nd throughout competition moved quite a places down to still end in Gold zone on solo , that is really a commendable effort put by you in competition . Well Done.

\n

.I was expecting scaling down of all scores to the the level of previous version of competition. Some thing that might disappoint Host . I think Public/Private split was not quite stratified so people kept feeling motivated with high score on just 15-20 percent of classes that were there in public lb/Train sc set . Probably stratified split of classes might have given realistic score to participants and they would have tried some thing more. But nevertheless this was a good competition that my team joined 3 weeks ago.

\n

Thanks to @rohitsingh9990 for putting his very hard efforts during last days and also all Team member @tomohiroh @rsinda @imeintanis for their contributions.

\n

Below is summary of our solution that stood in Gold till last on Public LB ,but stayed bit far from same on Private lb

\n

Our solution was nothing rocket science. Below are highlights
\n1)Base Line Framework kkiller @kneroma for your public kernel

\n

2) Extended this baseline framework to fit to the last version competition Second place solution that was based on Noise/Mixup set up https://www.kaggle.com/vlomme/surfin-bird-2nd-place
\nWe gave 0.33 to 0.2 weights to Secondary labels and also in one of my model i reduced the label strength based on rating of Recording 2,3,4.

\n

3) Models used : Timms Densenet121,Resnest50d, Effnetb0, b1/b2 Mobile NetV3, Sed (backed b0/b1 with and without noise)

\n

4) Ensemble of above models different versions which would not have went far if we had not used Post processing that gave boost to ensembles from mere 0.72 to hoping 0.78 check below public kernel which is not our best score but please look for post processing method towards the end where we see scores thanks to @tomohiroh for bringing it in .
\nhttps://www.kaggle.com/jaideepvalani/lb2-pipeline-master-ensemble-noise-tta-new

\n

5) Out best private score was 0.66 that was result of Differential threshold of classes that gave boost to train sc CV uptill F1-82 , but it gave F1 score of just 77 on leaderboard less than our best public score so we thought probable it overfit and we dint select :(

\n

6) Our understanding was that the ensemble model having right mix of precision/recall friendly models should perform fairly on both LB but i feel recall friendly ensembled model and fine tuning of ensemble weights of models to increase recall performed better than ones those were rightly balanced .

\n

Overall i still feel every one in range of (0.64 to 0.67) did their best but at end of competition you have to reward team scoring highest on Leader board otherwise just because your model couldnt classify some bird correctly dosent means solution is bad.

\n

Things that I wanted to try and explore but could not because of Time and resource

\n

1) Training on No Call classes
\n2) Training using More audiomentations of Torch on Raw Sound file to address issue of Domain Shift between Train site and Test site recordings
\n3) Explore audio features using Libross like removing silences and other dozens of features one can extract based on this notebook
\nhttps://www.kaggle.com/andradaolteanu/birdcall-recognition-eda-and-audio-fe

\n

Thanks to Host and all kagglers for their discussions and contribuions

\n

Regards
\nJaideep

", "messages": [ { "id": "1332496", "postDate": "06/02/2021 06:41:33", "content": "

Congratulations to Winners !
\n@cpmpml you stood 1st/2nd throughout competition moved quite a places down to still end in Gold zone on solo , that is really a commendable effort put by you in competition . Well Done.

\n

.I was expecting scaling down of all scores to the the level of previous version of competition. Some thing that might disappoint Host . I think Public/Private split was not quite stratified so people kept feeling motivated with high score on just 15-20 percent of classes that were there in public lb/Train sc set . Probably stratified split of classes might have given realistic score to participants and they would have tried some thing more. But nevertheless this was a good competition that my team joined 3 weeks ago.

\n

Thanks to @rohitsingh9990 for putting his very hard efforts during last days and also all Team member @tomohiroh @rsinda @imeintanis for their contributions.

\n

Below is summary of our solution that stood in Gold till last on Public LB ,but stayed bit far from same on Private lb

\n

Our solution was nothing rocket science. Below are highlights
\n1)Base Line Framework kkiller @kneroma for your public kernel

\n

2) Extended this baseline framework to fit to the last version competition Second place solution that was based on Noise/Mixup set up https://www.kaggle.com/vlomme/surfin-bird-2nd-place
\nWe gave 0.33 to 0.2 weights to Secondary labels and also in one of my model i reduced the label strength based on rating of Recording 2,3,4.

\n

3) Models used : Timms Densenet121,Resnest50d, Effnetb0, b1/b2 Mobile NetV3, Sed (backed b0/b1 with and without noise)

\n

4) Ensemble of above models different versions which would not have went far if we had not used Post processing that gave boost to ensembles from mere 0.72 to hoping 0.78 check below public kernel which is not our best score but please look for post processing method towards the end where we see scores thanks to @tomohiroh for bringing it in .
\nhttps://www.kaggle.com/jaideepvalani/lb2-pipeline-master-ensemble-noise-tta-new

\n

5) Out best private score was 0.66 that was result of Differential threshold of classes that gave boost to train sc CV uptill F1-82 , but it gave F1 score of just 77 on leaderboard less than our best public score so we thought probable it overfit and we dint select :(

\n

6) Our understanding was that the ensemble model having right mix of precision/recall friendly models should perform fairly on both LB but i feel recall friendly ensembled model and fine tuning of ensemble weights of models to increase recall performed better than ones those were rightly balanced .

\n

Overall i still feel every one in range of (0.64 to 0.67) did their best but at end of competition you have to reward team scoring highest on Leader board otherwise just because your model couldnt classify some bird correctly dosent means solution is bad.

\n

Things that I wanted to try and explore but could not because of Time and resource

\n

1) Training on No Call classes
\n2) Training using More audiomentations of Torch on Raw Sound file to address issue of Domain Shift between Train site and Test site recordings
\n3) Explore audio features using Libross like removing silences and other dozens of features one can extract based on this notebook
\nhttps://www.kaggle.com/andradaolteanu/birdcall-recognition-eda-and-audio-fe

\n

Thanks to Host and all kagglers for their discussions and contribuions

\n

Regards
\nJaideep

", "rawMarkdown": "Congratulations to Winners ! \n@cpmpml you stood 1st/2nd throughout competition moved quite a places down to still end in Gold zone on solo , that is really a commendable effort put by you in competition . Well Done. \n\n\n.I was expecting scaling down of all scores to the the level of previous version of competition. Some thing that might disappoint Host . I think Public/Private split was not quite stratified so people kept feeling motivated with high score on just 15-20 percent of classes that were there in public lb/Train sc set . Probably stratified split of classes might have given realistic score to participants and they would have tried some thing more. But nevertheless this was a good competition that my team joined 3 weeks ago. \n\nThanks to @rohitsingh9990 **for putting his very hard efforts** during last days and also all Team member @tomohiroh @rsinda @imeintanis for their contributions.\n\n**Below is summary of our solution that stood in Gold till last on Public LB ,but stayed bit far from same on Private lb**\n\n Our solution was nothing rocket science. Below are highlights\n1)Base Line Framework kkiller @kneroma for your public kernel\n\n2) Extended this baseline framework to fit to the last version competition Second place solution that was based on Noise/Mixup set up https://www.kaggle.com/vlomme/surfin-bird-2nd-place\nWe gave 0.33 to 0.2 weights to Secondary labels and also in one of my model i reduced the label strength based on rating of Recording 2,3,4. \n\n3) Models used : Timms Densenet121,Resnest50d, Effnetb0, b1/b2 Mobile NetV3, Sed (backed b0/b1 with and without noise)\n\n4) Ensemble of above models different versions which would not have went far if we had not used Post processing that gave boost to ensembles from mere 0.72 to hoping 0.78 check below public kernel which is not our best score but please look for post processing method towards the end where we see scores thanks to @tomohiroh for bringing it in .\nhttps://www.kaggle.com/jaideepvalani/lb2-pipeline-master-ensemble-noise-tta-new\n\n5) Out best private score was 0.66 that was result of Differential threshold of classes that gave boost to train sc CV uptill F1-82 , but it gave F1 score of just 77 on leaderboard less than our best public score so we thought probable it overfit and we dint select :( \n\n6) Our understanding was that the ensemble model having right mix of precision/recall friendly models should perform fairly on both LB but i feel recall friendly ensembled model and fine tuning of ensemble weights of models to increase recall performed better than ones those were rightly balanced .\n\nOverall i still feel every one in range of (0.64 to 0.67) did their best but at end of competition you have to reward team scoring highest on Leader board otherwise just because your model couldnt classify some bird correctly dosent means solution is bad. \n\nThings that I wanted to try and explore but could not because of Time and resource\n\n1) Training on No Call classes \n2) Training using More audiomentations of Torch on Raw Sound file to address issue of Domain Shift between Train site and Test site recordings\n3) Explore audio features using Libross like removing silences and other dozens of features one can extract based on this notebook\nhttps://www.kaggle.com/andradaolteanu/birdcall-recognition-eda-and-audio-fe\n\nThanks to Host and all kagglers for their discussions and contribuions\n\nRegards\nJaideep", "votes": null }, { "id": "1332523", "postDate": "06/02/2021 06:55:56", "content": "

I'm very glad you found my solution helpful

", "rawMarkdown": "I'm very glad you found my solution helpful", "votes": null }, { "id": "1332528", "postDate": "06/02/2021 06:59:12", "content": "

but we coudlnt use it in way you might have used @vlomme congrats for standing in top 5 yet another time. Congrats for another solo gold. and may be now a level upgrade to CM..

", "rawMarkdown": "but we coudlnt use it in way you might have used @vlomme congrats for standing in top 5 yet another time. Congrats for another solo gold. and may be now a level upgrade to CM..", "votes": null }, { "id": "1333339", "postDate": "06/02/2021 16:37:21", "content": "

Thanks for sharing good solution and I look forward to seeing your team get gold medals in the next competition. :)

", "rawMarkdown": "Thanks for sharing good solution and I look forward to seeing your team get gold medals in the next competition. :)", "votes": null }, { "id": "1333351", "postDate": "06/02/2021 16:45:00", "content": "

Thanks for the kind words, and congrats on your final result!

\n
\n

5) Out best private score was 0.66 …

\n
\n

You meant 0.68 instead of 0.66 maybe?

", "rawMarkdown": "Thanks for the kind words, and congrats on your final result!\n\n> 5) Out best private score was 0.66 ...\n\nYou meant 0.68 instead of 0.66 maybe?", "votes": null }, { "id": "1336643", "postDate": "06/05/2021 05:34:17", "content": "

Its 0.6628 to be precise now .

", "rawMarkdown": "Its 0.6628 to be precise now .", "votes": null } ], "comments": [ { "id": 1332523, "author_name": "vlomme", "author_url": "", "post_date": "06/02/2021 06:55:56", "content": "

I'm very glad you found my solution helpful

", "votes": null, "replies": [ { "id": 1332528, "author_name": "jaideepvalani", "author_url": "", "post_date": "06/02/2021 06:59:12", "content": "

but we coudlnt use it in way you might have used @vlomme congrats for standing in top 5 yet another time. Congrats for another solo gold. and may be now a level upgrade to CM..

", "votes": null, "replies": [] } ] }, { "id": 1333339, "author_name": "piantic", "author_url": "", "post_date": "06/02/2021 16:37:21", "content": "

Thanks for sharing good solution and I look forward to seeing your team get gold medals in the next competition. :)

", "votes": null, "replies": [] }, { "id": 1333351, "author_name": "cpmpml", "author_url": "", "post_date": "06/02/2021 16:45:00", "content": "

Thanks for the kind words, and congrats on your final result!

\n
\n

5) Out best private score was 0.66 …

\n
\n

You meant 0.68 instead of 0.66 maybe?

", "votes": null, "replies": [ { "id": 1336643, "author_name": "jaideepvalani", "author_url": "", "post_date": "06/05/2021 05:34:17", "content": "

Its 0.6628 to be precise now .

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1332496": "Congratulations to Winners ! \n@cpmpml you stood 1st/2nd throughout competition moved quite a places down to still end in Gold zone on solo , that is really a commendable effort put by you in competition . Well Done. \n\n\n.I was expecting scaling down of all scores to the the level of previous version of competition. Some thing that might disappoint Host . I think Public/Private split was not quite stratified so people kept feeling motivated with high score on just 15-20 percent of classes that were there in public lb/Train sc set . Probably stratified split of classes might have given realistic score to participants and they would have tried some thing more. But nevertheless this was a good competition that my team joined 3 weeks ago. \n\nThanks to @rohitsingh9990 **for putting his very hard efforts** during last days and also all Team member @tomohiroh @rsinda @imeintanis for their contributions.\n\n**Below is summary of our solution that stood in Gold till last on Public LB ,but stayed bit far from same on Private lb**\n\n Our solution was nothing rocket science. Below are highlights\n1)Base Line Framework kkiller @kneroma for your public kernel\n\n2) Extended this baseline framework to fit to the last version competition Second place solution that was based on Noise/Mixup set up https://www.kaggle.com/vlomme/surfin-bird-2nd-place\nWe gave 0.33 to 0.2 weights to Secondary labels and also in one of my model i reduced the label strength based on rating of Recording 2,3,4. \n\n3) Models used : Timms Densenet121,Resnest50d, Effnetb0, b1/b2 Mobile NetV3, Sed (backed b0/b1 with and without noise)\n\n4) Ensemble of above models different versions which would not have went far if we had not used Post processing that gave boost to ensembles from mere 0.72 to hoping 0.78 check below public kernel which is not our best score but please look for post processing method towards the end where we see scores thanks to @tomohiroh for bringing it in .\nhttps://www.kaggle.com/jaideepvalani/lb2-pipeline-master-ensemble-noise-tta-new\n\n5) Out best private score was 0.66 that was result of Differential threshold of classes that gave boost to train sc CV uptill F1-82 , but it gave F1 score of just 77 on leaderboard less than our best public score so we thought probable it overfit and we dint select :( \n\n6) Our understanding was that the ensemble model having right mix of precision/recall friendly models should perform fairly on both LB but i feel recall friendly ensembled model and fine tuning of ensemble weights of models to increase recall performed better than ones those were rightly balanced .\n\nOverall i still feel every one in range of (0.64 to 0.67) did their best but at end of competition you have to reward team scoring highest on Leader board otherwise just because your model couldnt classify some bird correctly dosent means solution is bad. \n\nThings that I wanted to try and explore but could not because of Time and resource\n\n1) Training on No Call classes \n2) Training using More audiomentations of Torch on Raw Sound file to address issue of Domain Shift between Train site and Test site recordings\n3) Explore audio features using Libross like removing silences and other dozens of features one can extract based on this notebook\nhttps://www.kaggle.com/andradaolteanu/birdcall-recognition-eda-and-audio-fe\n\nThanks to Host and all kagglers for their discussions and contribuions\n\nRegards\nJaideep", "1332523": "I'm very glad you found my solution helpful", "1332528": "but we coudlnt use it in way you might have used @vlomme congrats for standing in top 5 yet another time. Congrats for another solo gold. and may be now a level upgrade to CM..", "1333339": "Thanks for sharing good solution and I look forward to seeing your team get gold medals in the next competition. :)", "1333351": "Thanks for the kind words, and congrats on your final result!\n\n> 5) Out best private score was 0.66 ...\n\nYou meant 0.68 instead of 0.66 maybe?", "1336643": "Its 0.6628 to be precise now ." }, "source": "meta" }