Congrats to all the winners, and thank you to Kaggle and Cornell Lab of Ornithology for organizing such an interesting competition. I will mainly explain the method from my solution that I believe helped improve my LB position.
\nI trained the model in three stages. First, I used all the data from 2021 to 2024. Next, I trained on either just the 2024 data or on the data from 2021 to 2024, focusing on the target of this competition. Finally, I trained on the second set of data, adding the unlabeled data from 2024. The final stage of training improved the model's performance on both the public and private LB.
\n2021,2022,2023,2024
\ntf_efficientnetv2_b0, b3 (timm)
\ninput: single-channel spectrogram
I created pseudo-labels for the unlabeled data and used them by adding them to the labeled data with a 50% probability. This method was inspired by the 7th solution in 2023 and implemented with reference to GitHub. tgt_wav was the unlabeled data, add_wav was the labeled data, and db was a random positive value(~50). The labels were simply combined.
\ndef __add_noise(self, tgt_wav, add_wav, db):\n tgt_rms = .(.(.square(tgt_wav), axis=-))\n add_rms = .(.(.square(add_wav), axis=-))\n\n noise_rms = tgt_rms / (**((db) / )) \n new_wav = tgt_wav + add_wav * (noise_rms / (add_rms + ))\n new_wav = .clip(new_wav, .(new_wav)*, .(new_wav)*)\n new_wav\nI divided each 4-minute unlabeled data file into 5-second segments, creating 48 files from one unlabeled file. After generating pseudo-labels for each segmented file, I used the top 10% of those with the lowest class label entropy for training. Labels below the top 90% for each class's pseudo-label were set to 0. The labels were used as soft labels. This method improved the LB by ~0.04.
", "messages": [ { "id": "2866527", "postDate": "06/11/2024 11:31:52", "content": "Congrats to all the winners, and thank you to Kaggle and Cornell Lab of Ornithology for organizing such an interesting competition. I will mainly explain the method from my solution that I believe helped improve my LB position.
\nI trained the model in three stages. First, I used all the data from 2021 to 2024. Next, I trained on either just the 2024 data or on the data from 2021 to 2024, focusing on the target of this competition. Finally, I trained on the second set of data, adding the unlabeled data from 2024. The final stage of training improved the model's performance on both the public and private LB.
\n2021,2022,2023,2024
\ntf_efficientnetv2_b0, b3 (timm)
\ninput: single-channel spectrogram
I created pseudo-labels for the unlabeled data and used them by adding them to the labeled data with a 50% probability. This method was inspired by the 7th solution in 2023 and implemented with reference to GitHub. tgt_wav was the unlabeled data, add_wav was the labeled data, and db was a random positive value(~50). The labels were simply combined.
\ndef __add_noise(self, tgt_wav, add_wav, db):\n tgt_rms = .(.(.square(tgt_wav), axis=-))\n add_rms = .(.(.square(add_wav), axis=-))\n\n noise_rms = tgt_rms / (**((db) / )) \n new_wav = tgt_wav + add_wav * (noise_rms / (add_rms + ))\n new_wav = .clip(new_wav, .(new_wav)*, .(new_wav)*)\n new_wav\nI divided each 4-minute unlabeled data file into 5-second segments, creating 48 files from one unlabeled file. After generating pseudo-labels for each segmented file, I used the top 10% of those with the lowest class label entropy for training. Labels below the top 90% for each class's pseudo-label were set to 0. The labels were used as soft labels. This method improved the LB by ~0.04.
", "rawMarkdown": "Congrats to all the winners, and thank you to Kaggle and Cornell Lab of Ornithology for organizing such an interesting competition. I will mainly explain the method from my solution that I believe helped improve my LB position.\n\n### Summary\nI trained the model in three stages. First, I used all the data from 2021 to 2024. Next, I trained on either just the 2024 data or on the data from 2021 to 2024, focusing on the target of this competition. Finally, I trained on the second set of data, adding the unlabeled data from 2024. The final stage of training improved the model's performance on both the public and private LB.\n\n### Data\n2021,2022,2023,2024\n\n### Model\ntf_efficientnetv2_b0, b3 (timm)\ninput: single-channel spectrogram\n\n### Use of Unlabeled Data\nI created pseudo-labels for the unlabeled data and used them by adding them to the labeled data with a 50% probability. This method was inspired by the [7th solution in 2023](https://www.kaggle.com/competitions/birdclef-2023/discussion/412922) and implemented with reference to [GitHub](https://github.com/Sato-Kunihiko/audio-SNR). tgt_wav was the unlabeled data, add_wav was the labeled data, and db was a random positive value(~50). The labels were simply combined.\n\n```\ndef __add_noise(self, tgt_wav, add_wav, db):\n tgt_rms = np.sqrt(np.mean(np.square(tgt_wav), axis=-1))\n add_rms = np.sqrt(np.mean(np.square(add_wav), axis=-1))\n \n noise_rms = tgt_rms / (10**(float(db) / 20)) \n new_wav = tgt_wav + add_wav * (noise_rms / (add_rms + 1e-6))\n new_wav = np.clip(new_wav, np.min(new_wav)*2, np.max(new_wav)*2)\n return new_wav\n```\n\nI divided each 4-minute unlabeled data file into 5-second segments, creating 48 files from one unlabeled file. After generating pseudo-labels for each segmented file, I used the top 10% of those with the lowest class label entropy for training. Labels below the top 90% for each class's pseudo-label were set to 0. The labels were used as soft labels. This method improved the LB by ~0.04.", "votes": null }, { "id": "2866614", "postDate": "06/11/2024 12:04:37", "content": "Congratulations solo gold medal.
", "rawMarkdown": "Congratulations solo gold medal.", "votes": null }, { "id": "2867834", "postDate": "06/12/2024 05:24:52", "content": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.
", "rawMarkdown": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.", "votes": null }, { "id": "2868746", "postDate": "06/12/2024 15:40:04", "content": "Congratulations on securing 10th place in this competition, @yuyuki11235
", "rawMarkdown": "Congratulations on securing 10th place in this competition, @yuyuki11235", "votes": null }, { "id": "2870366", "postDate": "06/13/2024 14:45:24", "content": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.
", "rawMarkdown": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.", "votes": null } ], "comments": [ { "id": 2866614, "author_name": "clora16", "author_url": "", "post_date": "06/11/2024 12:04:37", "content": "Congratulations solo gold medal.
", "votes": null, "replies": [] }, { "id": 2867834, "author_name": "crsuthikshnkumar", "author_url": "", "post_date": "06/12/2024 05:24:52", "content": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.
", "votes": null, "replies": [] }, { "id": 2868746, "author_name": "mubashirsidiki", "author_url": "", "post_date": "06/12/2024 15:40:04", "content": "Congratulations on securing 10th place in this competition, @yuyuki11235
", "votes": null, "replies": [] }, { "id": 2870366, "author_name": "germanarley", "author_url": "", "post_date": "06/13/2024 14:45:24", "content": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2866527": "Congrats to all the winners, and thank you to Kaggle and Cornell Lab of Ornithology for organizing such an interesting competition. I will mainly explain the method from my solution that I believe helped improve my LB position.\n\n### Summary\nI trained the model in three stages. First, I used all the data from 2021 to 2024. Next, I trained on either just the 2024 data or on the data from 2021 to 2024, focusing on the target of this competition. Finally, I trained on the second set of data, adding the unlabeled data from 2024. The final stage of training improved the model's performance on both the public and private LB.\n\n### Data\n2021,2022,2023,2024\n\n### Model\ntf_efficientnetv2_b0, b3 (timm)\ninput: single-channel spectrogram\n\n### Use of Unlabeled Data\nI created pseudo-labels for the unlabeled data and used them by adding them to the labeled data with a 50% probability. This method was inspired by the [7th solution in 2023](https://www.kaggle.com/competitions/birdclef-2023/discussion/412922) and implemented with reference to [GitHub](https://github.com/Sato-Kunihiko/audio-SNR). tgt_wav was the unlabeled data, add_wav was the labeled data, and db was a random positive value(~50). The labels were simply combined.\n\n```\ndef __add_noise(self, tgt_wav, add_wav, db):\n tgt_rms = np.sqrt(np.mean(np.square(tgt_wav), axis=-1))\n add_rms = np.sqrt(np.mean(np.square(add_wav), axis=-1))\n \n noise_rms = tgt_rms / (10**(float(db) / 20)) \n new_wav = tgt_wav + add_wav * (noise_rms / (add_rms + 1e-6))\n new_wav = np.clip(new_wav, np.min(new_wav)*2, np.max(new_wav)*2)\n return new_wav\n```\n\nI divided each 4-minute unlabeled data file into 5-second segments, creating 48 files from one unlabeled file. After generating pseudo-labels for each segmented file, I used the top 10% of those with the lowest class label entropy for training. Labels below the top 90% for each class's pseudo-label were set to 0. The labels were used as soft labels. This method improved the LB by ~0.04.", "2866614": "Congratulations solo gold medal.", "2867834": "Congratulations on securing 10th Place in this competition. Thanks for sharing details.", "2868746": "Congratulations on securing 10th place in this competition, @yuyuki11235", "2870366": "Congratulations on securing 10th Place in this competition. Thanks for sharing details." }, "source": "meta" }