First of all, we’d like to thank the hosts and Kaggle for organizing this competition.
\nCongratulations to @xyzdivergence on achieving Kaggle Competitions Grandmaster—well deserved!
\nWe used a SED architecture and trained on random 10-second audio segments. For the final ensemble, we relied solely on tf_efficientnetv2_m.in21k, trained with slightly varied configurations.
\nThe best private score for a single SED model was 0.922 (not the selected submission), and 0.894 for a CNN model using the same backbone.
\nOur training pipeline consisted of several stages:
\nStage 1: Pretraining on the training audio with true labels.
Stage 2: Knowledge distillation using both train audio and train soundscapes. We combined average pseudo labels across the full audio (with a 1-second stride, weighted at 0.3) and chunk-level pseudo labels from the teacher model (10-second chunks, weighted at 0.7).
We performed several rounds of distillation, selecting the best-performing teacher from the previous round based on leaderboard improvements.
Melspectrogram settings:
\n \n \n \n \n \n \nAugmentations:
\nOn waveform: sumix (=1)\nOn spectrogram: mixup (=1), 3 time/frequency masks (=0.5), horizontal flip (=0.5), random erasing (=0.5)\nThe final submission was a simple average of three tf_efficientnetv2_m.in21k checkpoints, followed by smoothing using neighboring clips with weights of 0.1, 0.8, and 0.1.
\nTo speed up inference, all models were converted to OpenVINO.
", "messages": [ { "id": "3218405", "postDate": "06/06/2025 06:26:22", "content": "First of all, we’d like to thank the hosts and Kaggle for organizing this competition.
\nCongratulations to @xyzdivergence on achieving Kaggle Competitions Grandmaster—well deserved!
\nWe used a SED architecture and trained on random 10-second audio segments. For the final ensemble, we relied solely on tf_efficientnetv2_m.in21k, trained with slightly varied configurations.
\nThe best private score for a single SED model was 0.922 (not the selected submission), and 0.894 for a CNN model using the same backbone.
\nOur training pipeline consisted of several stages:
\nStage 1: Pretraining on the training audio with true labels.
Stage 2: Knowledge distillation using both train audio and train soundscapes. We combined average pseudo labels across the full audio (with a 1-second stride, weighted at 0.3) and chunk-level pseudo labels from the teacher model (10-second chunks, weighted at 0.7).
We performed several rounds of distillation, selecting the best-performing teacher from the previous round based on leaderboard improvements.
Melspectrogram settings:
\n \n \n \n \n \n \nAugmentations:
\nOn waveform: sumix (=1)\nOn spectrogram: mixup (=1), 3 time/frequency masks (=0.5), horizontal flip (=0.5), random erasing (=0.5)\nThe final submission was a simple average of three tf_efficientnetv2_m.in21k checkpoints, followed by smoothing using neighboring clips with weights of 0.1, 0.8, and 0.1.
\nTo speed up inference, all models were converted to OpenVINO.
", "rawMarkdown": "First of all, we’d like to thank the hosts and Kaggle for organizing this competition.\n\nCongratulations to @xyzdivergence on achieving Kaggle Competitions Grandmaster—well deserved!\n\n### Model & Training\n\nWe used a SED architecture and trained on random 10-second audio segments. For the final ensemble, we relied solely on tf_efficientnetv2_m.in21k, trained with slightly varied configurations.\n\nThe best private score for a single SED model was 0.922 (not the selected submission), and 0.894 for a CNN model using the same backbone.\n\nOur training pipeline consisted of several stages:\n\n* Stage 1: Pretraining on the training audio with true labels.\n\n* Stage 2: Knowledge distillation using both train audio and train soundscapes. We combined average pseudo labels across the full audio (with a 1-second stride, weighted at 0.3) and chunk-level pseudo labels from the teacher model (10-second chunks, weighted at 0.7).\n\n* We performed several rounds of distillation, selecting the best-performing teacher from the previous round based on leaderboard improvements.\n\n### Features\n\nMelspectrogram settings:\n\n```\nsample_rate: 32000\nmel_bins: 128\nfmin: 40\nfmax: 15000\nnfft: 1024\nhop_length: 512\n```\n\nAugmentations:\n```\nOn waveform: sumix (p=1)\nOn spectrogram: mixup (p=1), 3 time/frequency masks (p=0.5), horizontal flip (p=0.5), and random erasing (p=0.5)\n```\n\n### Final Submission\n\nThe final submission was a simple average of three tf_efficientnetv2_m.in21k checkpoints, followed by smoothing using neighboring clips with weights of 0.1, 0.8, and 0.1.\n\nTo speed up inference, all models were converted to OpenVINO.", "votes": null }, { "id": "3218445", "postDate": "06/06/2025 07:37:10", "content": "Congrats!
\nHow many hours do you spend for training models in stage 1 and 2 and final?
\nAnd did you use only GPU from Kaggle? If from the outside how much did it cost approximately?
\nThank you.
Congratulations, my old friend — new GrandMaster!🎉
", "rawMarkdown": "Congratulations, my old friend — new GrandMaster!🎉", "votes": null }, { "id": "3218749", "postDate": "06/06/2025 16:50:46", "content": "Congrats, and thanks for sharing! Do you plan to share the model training code?
\nFor step 1, did you use secondary labels?
\nFor step 2, is this right (no true labels):
\nTake random 10s crop -> target is (teacher probs on 10s crop) * 0.7 + (averaged teacher probs over entire file) * 0.3
\nIn the end, you selected the “best-performing teacher” — meaning you did not select the student model, but instead the teacher model, and chose the teacher based on the student’s LB performance?
", "rawMarkdown": "Congrats, and thanks for sharing! Do you plan to share the model training code?\n\nFor step 1, did you use secondary labels?\n\nFor step 2, is this right (no true labels):\n\nTake random 10s crop -> target is (teacher probs on 10s crop) * 0.7 + (averaged teacher probs over entire file) * 0.3\n\nIn the end, you selected the “best-performing teacher” — meaning you did not select the student model, but instead the teacher model, and chose the teacher based on the student’s LB performance?", "votes": null } ], "comments": [ { "id": 3218445, "author_name": "overvalueawareness", "author_url": "", "post_date": "06/06/2025 07:37:10", "content": "Congrats!
\nHow many hours do you spend for training models in stage 1 and 2 and final?
\nAnd did you use only GPU from Kaggle? If from the outside how much did it cost approximately?
\nThank you.
Congratulations, my old friend — new GrandMaster!🎉
", "votes": null, "replies": [] }, { "id": 3218749, "author_name": "robbynevels", "author_url": "", "post_date": "06/06/2025 16:50:46", "content": "Congrats, and thanks for sharing! Do you plan to share the model training code?
\nFor step 1, did you use secondary labels?
\nFor step 2, is this right (no true labels):
\nTake random 10s crop -> target is (teacher probs on 10s crop) * 0.7 + (averaged teacher probs over entire file) * 0.3
\nIn the end, you selected the “best-performing teacher” — meaning you did not select the student model, but instead the teacher model, and chose the teacher based on the student’s LB performance?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3218405": "First of all, we’d like to thank the hosts and Kaggle for organizing this competition.\n\nCongratulations to @xyzdivergence on achieving Kaggle Competitions Grandmaster—well deserved!\n\n### Model & Training\n\nWe used a SED architecture and trained on random 10-second audio segments. For the final ensemble, we relied solely on tf_efficientnetv2_m.in21k, trained with slightly varied configurations.\n\nThe best private score for a single SED model was 0.922 (not the selected submission), and 0.894 for a CNN model using the same backbone.\n\nOur training pipeline consisted of several stages:\n\n* Stage 1: Pretraining on the training audio with true labels.\n\n* Stage 2: Knowledge distillation using both train audio and train soundscapes. We combined average pseudo labels across the full audio (with a 1-second stride, weighted at 0.3) and chunk-level pseudo labels from the teacher model (10-second chunks, weighted at 0.7).\n\n* We performed several rounds of distillation, selecting the best-performing teacher from the previous round based on leaderboard improvements.\n\n### Features\n\nMelspectrogram settings:\n\n```\nsample_rate: 32000\nmel_bins: 128\nfmin: 40\nfmax: 15000\nnfft: 1024\nhop_length: 512\n```\n\nAugmentations:\n```\nOn waveform: sumix (p=1)\nOn spectrogram: mixup (p=1), 3 time/frequency masks (p=0.5), horizontal flip (p=0.5), and random erasing (p=0.5)\n```\n\n### Final Submission\n\nThe final submission was a simple average of three tf_efficientnetv2_m.in21k checkpoints, followed by smoothing using neighboring clips with weights of 0.1, 0.8, and 0.1.\n\nTo speed up inference, all models were converted to OpenVINO.", "3218445": "Congrats!\nHow many hours do you spend for training models in stage 1 and 2 and final?\nAnd did you use only GPU from Kaggle? If from the outside how much did it cost approximately?\nThank you.", "3218449": "Congratulations, my old friend — new GrandMaster!🎉", "3218749": "Congrats, and thanks for sharing! Do you plan to share the model training code?\n\nFor step 1, did you use secondary labels?\n\nFor step 2, is this right (no true labels):\n\nTake random 10s crop -> target is (teacher probs on 10s crop) * 0.7 + (averaged teacher probs over entire file) * 0.3\n\nIn the end, you selected the “best-performing teacher” — meaning you did not select the student model, but instead the teacher model, and chose the teacher based on the student’s LB performance?" }, "source": "meta" }