Thanks to the organizers for another sound competition. My journey to kaggle started with your bird competition and I am becoming a Grandmaster in your bird competition as well. It's sad that a model trained on non-public data won, because of which, it just couldn't be beaten.
\nI didn't have much time to participate, so my solution is overfitting last year's model 2020
\n 2021
Differences:
\nOtherwise, my decision repeats my public decision of previous years
\n1 fold = 0.78 private lb
Thanks to the organizers for another sound competition. My journey to kaggle started with your bird competition and I am becoming a Grandmaster in your bird competition as well. It's sad that a model trained on non-public data won, because of which, it just couldn't be beaten.
\nI didn't have much time to participate, so my solution is overfitting last year's model 2020
\n 2021
Differences:
\nOtherwise, my decision repeats my public decision of previous years
\n1 fold = 0.78 private lb
Кратко и понятно! Уже представляю вас в золотом кольце 😄👍!
", "rawMarkdown": "Кратко и понятно! Уже представляю вас в золотом кольце 😄👍!", "votes": null }, { "id": "1800874", "postDate": "05/25/2022 09:32:21", "content": "Have you used pre-trained weights from your 2021 models (trained with 2021 data),
\nor does the weights come from ImageNet pre training?
If so, did it have any effect or not?
", "rawMarkdown": "Have you used pre-trained weights from your 2021 models (trained with 2021 data), \nor does the weights come from ImageNet pre training?\n\nIf so, did it have any effect or not?", "votes": null }, { "id": "1800885", "postDate": "05/25/2022 09:40:03", "content": "I didn't use weights since I removed them)
\nBut in the first step, I trained on all birds, which is the same
\nFor me, the pretrain on the ImageNet allowed the model to converge faster, and the pretrain on all birds reduced overfitting because of the small dataset of the Hawaiian birds
Make sure to save them, maybe they'll come in handy for the BirdCLEF 2023 competition :)
", "rawMarkdown": "Make sure to save them, maybe they'll come in handy for the BirdCLEF 2023 competition :)", "votes": null }, { "id": "1800986", "postDate": "05/25/2022 10:42:01", "content": "First I train on all birds, then I finish training on 21
Thanks for strong solo finish
\nDo you freeze all layers except last one? Or you created a new model just for training on 21?
", "rawMarkdown": "`First I train on all birds, then I finish training on 21`\n\nThanks for strong solo finish\n\nDo you freeze all layers except last one? Or you created a new model just for training on 21?", "votes": null }, { "id": "1801000", "postDate": "05/25/2022 10:52:57", "content": "No, I trained all the layers. I tried replacing the head with a 21 class, but it didn't help much.
\nI also balanced the number of examples in each epoch
How did you balanced the number of examples in each epoch? I used imbalanced-dataset-sampler but it didn't work for me.
", "rawMarkdown": "How did you balanced the number of examples in each epoch? I used [imbalanced-dataset-sampler](https://github.com/ufoym/imbalanced-dataset-sampler) but it didn't work for me.", "votes": null }, { "id": "1801007", "postDate": "05/25/2022 10:59:26", "content": "congrats on your solo gold!
", "rawMarkdown": "congrats on your solo gold!", "votes": null }, { "id": "1801010", "postDate": "05/25/2022 11:05:16", "content": "I wrote simple code
\nfor bird in birds:\n tmp = all_bird_list[all_bird_list.bird == bird]\n if len(tmp) < 10:\n a = 10\n elif len(tmp) < 30:\n a = 30 \n else:\n a = 60\n bird_lists.append(tmp.sample(a, replace=True))\nbird_list = pd.concat(bird_lists).reset_index(drop=True) \n`
", "rawMarkdown": "I wrote simple code\n\n\n for bird in birds:\n tmp = all_bird_list[all_bird_list.bird == bird]\n if len(tmp) < 10:\n a = 10\n elif len(tmp) < 30:\n a = 30 \n else:\n a = 60\n bird_lists.append(tmp.sample(a, replace=True))\n bird_list = pd.concat(bird_lists).reset_index(drop=True) \n`", "votes": null }, { "id": "1801016", "postDate": "05/25/2022 11:09:28", "content": "Thank you. This victory was very important, because I became a solo Grandmaster)
", "rawMarkdown": "Thank you. This victory was very important, because I became a solo Grandmaster)", "votes": null }, { "id": "1801031", "postDate": "05/25/2022 11:18:00", "content": "Thanks, and congrats on your solo Grandmaster!
", "rawMarkdown": "Thanks, and congrats on your solo Grandmaster!", "votes": null }, { "id": "1801556", "postDate": "05/25/2022 21:38:13", "content": "Amazing work @vlomme. I kept an eye on your astonishing performances and It's just awesome. Keep it up !
", "rawMarkdown": "Amazing work @vlomme. I kept an eye on your astonishing performances and It's just awesome. Keep it up !", "votes": null }, { "id": "1801571", "postDate": "05/25/2022 22:25:49", "content": "Great work
", "rawMarkdown": "Great work", "votes": null }, { "id": "1803374", "postDate": "05/27/2022 18:27:49", "content": "Nice. I was planning to do something similar with my last year models but lacked time/motivation. Well done!
\nCongrats on this result and your GM title, well deserved!
", "rawMarkdown": "Nice. I was planning to do something similar with my last year models but lacked time/motivation. Well done!\n\nCongrats on this result and your GM title, well deserved!", "votes": null }, { "id": "1803759", "postDate": "05/28/2022 06:37:56", "content": "Great work!
", "rawMarkdown": "Great work!", "votes": null }, { "id": "1804668", "postDate": "05/29/2022 10:36:45", "content": "Congrats Kramarenko Vladislav!!😊
", "rawMarkdown": "Congrats Kramarenko Vladislav!!😊", "votes": null } ], "comments": [ { "id": 1800754, "author_name": "vad13irt", "author_url": "", "post_date": "05/25/2022 07:40:27", "content": "Кратко и понятно! Уже представляю вас в золотом кольце 😄👍!
", "votes": null, "replies": [] }, { "id": 1800874, "author_name": "martynoveduard", "author_url": "", "post_date": "05/25/2022 09:32:21", "content": "Have you used pre-trained weights from your 2021 models (trained with 2021 data),
\nor does the weights come from ImageNet pre training?
If so, did it have any effect or not?
", "votes": null, "replies": [ { "id": 1800885, "author_name": "vlomme", "author_url": "", "post_date": "05/25/2022 09:40:03", "content": "I didn't use weights since I removed them)
\nBut in the first step, I trained on all birds, which is the same
\nFor me, the pretrain on the ImageNet allowed the model to converge faster, and the pretrain on all birds reduced overfitting because of the small dataset of the Hawaiian birds
Make sure to save them, maybe they'll come in handy for the BirdCLEF 2023 competition :)
", "votes": null, "replies": [] } ] }, { "id": 1800986, "author_name": "allohvk", "author_url": "", "post_date": "05/25/2022 10:42:01", "content": "First I train on all birds, then I finish training on 21
Thanks for strong solo finish
\nDo you freeze all layers except last one? Or you created a new model just for training on 21?
", "votes": null, "replies": [ { "id": 1801000, "author_name": "vlomme", "author_url": "", "post_date": "05/25/2022 10:52:57", "content": "No, I trained all the layers. I tried replacing the head with a 21 class, but it didn't help much.
\nI also balanced the number of examples in each epoch
How did you balanced the number of examples in each epoch? I used imbalanced-dataset-sampler but it didn't work for me.
", "votes": null, "replies": [] }, { "id": 1801010, "author_name": "vlomme", "author_url": "", "post_date": "05/25/2022 11:05:16", "content": "I wrote simple code
\nfor bird in birds:\n tmp = all_bird_list[all_bird_list.bird == bird]\n if len(tmp) < 10:\n a = 10\n elif len(tmp) < 30:\n a = 30 \n else:\n a = 60\n bird_lists.append(tmp.sample(a, replace=True))\nbird_list = pd.concat(bird_lists).reset_index(drop=True) \n`
", "votes": null, "replies": [] }, { "id": 1801031, "author_name": "shigemitsutomizawa", "author_url": "", "post_date": "05/25/2022 11:18:00", "content": "Thanks, and congrats on your solo Grandmaster!
", "votes": null, "replies": [] } ] }, { "id": 1801007, "author_name": "titericz", "author_url": "", "post_date": "05/25/2022 10:59:26", "content": "congrats on your solo gold!
", "votes": null, "replies": [ { "id": 1801016, "author_name": "vlomme", "author_url": "", "post_date": "05/25/2022 11:09:28", "content": "Thank you. This victory was very important, because I became a solo Grandmaster)
", "votes": null, "replies": [] } ] }, { "id": 1801556, "author_name": "kneroma", "author_url": "", "post_date": "05/25/2022 21:38:13", "content": "Amazing work @vlomme. I kept an eye on your astonishing performances and It's just awesome. Keep it up !
", "votes": null, "replies": [] }, { "id": 1801571, "author_name": "osamahussien", "author_url": "", "post_date": "05/25/2022 22:25:49", "content": "Great work
", "votes": null, "replies": [] }, { "id": 1803374, "author_name": "cpmpml", "author_url": "", "post_date": "05/27/2022 18:27:49", "content": "Nice. I was planning to do something similar with my last year models but lacked time/motivation. Well done!
\nCongrats on this result and your GM title, well deserved!
", "votes": null, "replies": [] }, { "id": 1803759, "author_name": "zinkink", "author_url": "", "post_date": "05/28/2022 06:37:56", "content": "Great work!
", "votes": null, "replies": [] }, { "id": 1804668, "author_name": "paritoshmahto", "author_url": "", "post_date": "05/29/2022 10:36:45", "content": "Congrats Kramarenko Vladislav!!😊
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1800738": "Thanks to the organizers for another sound competition. My journey to kaggle started with your bird competition and I am becoming a Grandmaster in your bird competition as well. It's sad that a model trained on non-public data won, because of which, it just couldn't be beaten. \n\nI didn't have much time to participate, so my solution is overfitting last year's model [2020](https://www.kaggle.com/competitions/birdsong-recognition/discussion/183269)\n [2021](https://www.kaggle.com/competitions/birdclef-2021/discussion/243351)\n\nDifferences:\n1. First I train on all birds, then I finish training on 21\n2. I select a different lb threshold for each species.\n\nOtherwise, my decision repeats my public decision of previous years\n1 fold = 0.78 private lb", "1800754": "Кратко и понятно! Уже представляю вас в золотом кольце 😄👍!", "1800874": "Have you used pre-trained weights from your 2021 models (trained with 2021 data), \nor does the weights come from ImageNet pre training?\n\nIf so, did it have any effect or not?", "1800885": "I didn't use weights since I removed them)\nBut in the first step, I trained on all birds, which is the same\nFor me, the pretrain on the ImageNet allowed the model to converge faster, and the pretrain on all birds reduced overfitting because of the small dataset of the Hawaiian birds", "1800957": "Make sure to save them, maybe they'll come in handy for the BirdCLEF 2023 competition :)", "1800986": "`First I train on all birds, then I finish training on 21`\n\nThanks for strong solo finish\n\nDo you freeze all layers except last one? Or you created a new model just for training on 21?", "1801000": "No, I trained all the layers. I tried replacing the head with a 21 class, but it didn't help much.\nI also balanced the number of examples in each epoch", "1801006": "How did you balanced the number of examples in each epoch? I used [imbalanced-dataset-sampler](https://github.com/ufoym/imbalanced-dataset-sampler) but it didn't work for me.", "1801007": "congrats on your solo gold!", "1801010": "I wrote simple code\n\n\n for bird in birds:\n tmp = all_bird_list[all_bird_list.bird == bird]\n if len(tmp) < 10:\n a = 10\n elif len(tmp) < 30:\n a = 30 \n else:\n a = 60\n bird_lists.append(tmp.sample(a, replace=True))\n bird_list = pd.concat(bird_lists).reset_index(drop=True) \n`", "1801016": "Thank you. This victory was very important, because I became a solo Grandmaster)", "1801031": "Thanks, and congrats on your solo Grandmaster!", "1801556": "Amazing work @vlomme. I kept an eye on your astonishing performances and It's just awesome. Keep it up !", "1801571": "Great work", "1803374": "Nice. I was planning to do something similar with my last year models but lacked time/motivation. Well done!\n\nCongrats on this result and your GM title, well deserved!", "1803759": "Great work!", "1804668": "Congrats Kramarenko Vladislav!!😊" }, "source": "meta" }