Here is the logic that I am currently using:
\nDataset Preparation
\nFor each training epoch:
\n-A random 5-second chunk is selected from the cleaned waveform.
\n-If the waveform is shorter than 5s post-cleaning, it's zero-padded.
Input Representation
\n-generate 3-channel mel spectrograms per input:Original audio, Noise-augmented version ,Time-stretched version
\n-These are stacked to form an input with shape [3, 128, T] (T = time frames).
Model
\nBackbone: tf_efficientnet_b0 from timm, with in_chans=3.
\nFinal layer: Linear(1280 → num_classes)
\nDropout: p=0.3 before the classifier head.
Augmentation
\nSpecAugment is applied to each spectrogram (time & frequency masking).
\nMixup is optionally applied between random training pairs with α=0.5.
Loss & Optimization
\nLoss: BCEWithLogitsLoss for multi-label classification.
\nOptimizer: Adam with weight decay and learning rate scheduling.
\nMetric: Validation AUC is monitored for early stopping and model checkpointing.
Here is the logic that I am currently using:
\nDataset Preparation
\nFor each training epoch:
\n-A random 5-second chunk is selected from the cleaned waveform.
\n-If the waveform is shorter than 5s post-cleaning, it's zero-padded.
Input Representation
\n-generate 3-channel mel spectrograms per input:Original audio, Noise-augmented version ,Time-stretched version
\n-These are stacked to form an input with shape [3, 128, T] (T = time frames).
Model
\nBackbone: tf_efficientnet_b0 from timm, with in_chans=3.
\nFinal layer: Linear(1280 → num_classes)
\nDropout: p=0.3 before the classifier head.
Augmentation
\nSpecAugment is applied to each spectrogram (time & frequency masking).
\nMixup is optionally applied between random training pairs with α=0.5.
Loss & Optimization
\nLoss: BCEWithLogitsLoss for multi-label classification.
\nOptimizer: Adam with weight decay and learning rate scheduling.
\nMetric: Validation AUC is monitored for early stopping and model checkpointing.
The simplest option is to change tf_efficientnet_b0 to tf_efficientnet_b2 or tf_efficientnetv2_b0.
", "rawMarkdown": "The simplest option is to change tf_efficientnet_b0 to tf_efficientnet_b2 or tf_efficientnetv2_b0.", "votes": null }, { "id": "3202810", "postDate": "05/16/2025 00:20:11", "content": "Maybe try not deleting human voice, in recent discussion people say removing humvoice somehow worse their LB.
\nI didn’t try humvoice yet, working with pseudo)
There is a discussion with the ideas from previous birdclef and it is stated in there that birdclef 2024 1st place used loop padding. Maybe trying that might help.
\nOr combining the approaches, at least it might make the train data more diverse.
\nLink: https://www.kaggle.com/competitions/birdclef-2025/discussion/572928
", "rawMarkdown": "There is a discussion with the ideas from previous birdclef and it is stated in there that birdclef 2024 1st place used loop padding. Maybe trying that might help.\n\nOr combining the approaches, at least it might make the train data more diverse.\n\nLink: https://www.kaggle.com/competitions/birdclef-2025/discussion/572928", "votes": null }, { "id": "3203303", "postDate": "05/16/2025 14:46:28", "content": "Based on my observations, ensembling cross-validated models might be a promising direction. While a single b0 model provides a strong baseline, my experiments showed that mean-ensembling multiple instances tended to yield a modest improvement in validation consistency. Would appreciate hearing others' experiences with model stacking here!
", "rawMarkdown": "Based on my observations, ensembling cross-validated models might be a promising direction. While a single b0 model provides a strong baseline, my experiments showed that mean-ensembling multiple instances tended to yield a modest improvement in validation consistency. Would appreciate hearing others' experiences with model stacking here!", "votes": null }, { "id": "3203346", "postDate": "05/16/2025 15:39:59", "content": "I'm doing similar things. But for me, different versions of three channel inputs did not improve the results. Therefore, I went back to a single spectrogram as an input. Also, I'm using a different classification head, which consists of two layers and has a residual connection. With that I was able to reach 0.819 (IIRC) for the efficient net b0
", "rawMarkdown": "I'm doing similar things. But for me, different versions of three channel inputs did not improve the results. Therefore, I went back to a single spectrogram as an input. Also, I'm using a different classification head, which consists of two layers and has a residual connection. With that I was able to reach 0.819 (IIRC) for the efficient net b0", "votes": null }, { "id": "3203351", "postDate": "05/16/2025 15:45:40", "content": "And I used soft labels in conjunction with a symmetric BCE loss.
", "rawMarkdown": "And I used soft labels in conjunction with a symmetric BCE loss.", "votes": null }, { "id": "3203537", "postDate": "05/16/2025 22:20:51", "content": "Do you try thome other think?
", "rawMarkdown": "Do you try thome other think?", "votes": null }, { "id": "3210534", "postDate": "05/27/2025 10:45:51", "content": "halo,for your work,the other two models improve how much LB?
", "rawMarkdown": "halo,for your work,the other two models improve how much LB?", "votes": null }, { "id": "3210551", "postDate": "05/27/2025 11:04:00", "content": "0.8 to 0.818
", "rawMarkdown": "0.8 to 0.818", "votes": null }, { "id": "3231086", "postDate": "06/23/2025 21:51:42", "content": "Loss & Optimization what mean ?
", "rawMarkdown": "Loss & Optimization what mean ?", "votes": null } ], "comments": [ { "id": 3202562, "author_name": "junhanzangai", "author_url": "", "post_date": "05/15/2025 14:57:45", "content": "The simplest option is to change tf_efficientnet_b0 to tf_efficientnet_b2 or tf_efficientnetv2_b0.
", "votes": null, "replies": [ { "id": 3210534, "author_name": "xiayuxuan", "author_url": "", "post_date": "05/27/2025 10:45:51", "content": "halo,for your work,the other two models improve how much LB?
", "votes": null, "replies": [ { "id": 3210551, "author_name": "junhanzangai", "author_url": "", "post_date": "05/27/2025 11:04:00", "content": "0.8 to 0.818
", "votes": null, "replies": [] } ] } ] }, { "id": 3202810, "author_name": "player77", "author_url": "", "post_date": "05/16/2025 00:20:11", "content": "Maybe try not deleting human voice, in recent discussion people say removing humvoice somehow worse their LB.
\nI didn’t try humvoice yet, working with pseudo)
There is a discussion with the ideas from previous birdclef and it is stated in there that birdclef 2024 1st place used loop padding. Maybe trying that might help.
\nOr combining the approaches, at least it might make the train data more diverse.
\nLink: https://www.kaggle.com/competitions/birdclef-2025/discussion/572928
", "votes": null, "replies": [] }, { "id": 3203303, "author_name": "alexandergremyakov", "author_url": "", "post_date": "05/16/2025 14:46:28", "content": "Based on my observations, ensembling cross-validated models might be a promising direction. While a single b0 model provides a strong baseline, my experiments showed that mean-ensembling multiple instances tended to yield a modest improvement in validation consistency. Would appreciate hearing others' experiences with model stacking here!
", "votes": null, "replies": [] }, { "id": 3203346, "author_name": "tim6502", "author_url": "", "post_date": "05/16/2025 15:39:59", "content": "I'm doing similar things. But for me, different versions of three channel inputs did not improve the results. Therefore, I went back to a single spectrogram as an input. Also, I'm using a different classification head, which consists of two layers and has a residual connection. With that I was able to reach 0.819 (IIRC) for the efficient net b0
", "votes": null, "replies": [ { "id": 3203351, "author_name": "tim6502", "author_url": "", "post_date": "05/16/2025 15:45:40", "content": "And I used soft labels in conjunction with a symmetric BCE loss.
", "votes": null, "replies": [] } ] }, { "id": 3203537, "author_name": "matishb", "author_url": "", "post_date": "05/16/2025 22:20:51", "content": "Do you try thome other think?
", "votes": null, "replies": [ { "id": 3231086, "author_name": "ernestgarciaphd", "author_url": "", "post_date": "06/23/2025 21:51:42", "content": "Loss & Optimization what mean ?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3202399": "Here is the logic that I am currently using: \n\n\n1. Human Voice Detection & Removal\n- use Silero VAD to detect and remove human speech segments from each training audio file.\n-Detected timestamps are completely removed from the waveform, not masked.\n\n2. Dataset Preparation\nFor each training epoch:\n-A random 5-second chunk is selected from the cleaned waveform.\n-If the waveform is shorter than 5s post-cleaning, it's zero-padded.\n\n3. Input Representation\n-generate 3-channel mel spectrograms per input:Original audio, Noise-augmented version ,Time-stretched version\n-These are stacked to form an input with shape [3, 128, T] (T = time frames).\n\n4. Model\nBackbone: tf_efficientnet_b0 from timm, with in_chans=3.\nFinal layer: Linear(1280 → num_classes)\nDropout: p=0.3 before the classifier head.\n\n5. Augmentation\nSpecAugment is applied to each spectrogram (time & frequency masking).\nMixup is optionally applied between random training pairs with α=0.5.\n\n6. Loss & Optimization\nLoss: BCEWithLogitsLoss for multi-label classification.\nOptimizer: Adam with weight decay and learning rate scheduling.\nMetric: Validation AUC is monitored for early stopping and model checkpointing.", "3202562": "The simplest option is to change tf_efficientnet_b0 to tf_efficientnet_b2 or tf_efficientnetv2_b0.", "3202810": "Maybe try not deleting human voice, in recent discussion people say removing humvoice somehow worse their LB. \nI didn’t try humvoice yet, working with pseudo)", "3203296": "There is a discussion with the ideas from previous birdclef and it is stated in there that birdclef 2024 1st place used loop padding. Maybe trying that might help.\n\nOr combining the approaches, at least it might make the train data more diverse.\n\nLink: https://www.kaggle.com/competitions/birdclef-2025/discussion/572928", "3203303": "Based on my observations, ensembling cross-validated models might be a promising direction. While a single b0 model provides a strong baseline, my experiments showed that mean-ensembling multiple instances tended to yield a modest improvement in validation consistency. Would appreciate hearing others' experiences with model stacking here!", "3203346": "I'm doing similar things. But for me, different versions of three channel inputs did not improve the results. Therefore, I went back to a single spectrogram as an input. Also, I'm using a different classification head, which consists of two layers and has a residual connection. With that I was able to reach 0.819 (IIRC) for the efficient net b0", "3203351": "And I used soft labels in conjunction with a symmetric BCE loss.", "3203537": "Do you try thome other think?", "3210534": "halo,for your work,the other two models improve how much LB?", "3210551": "0.8 to 0.818", "3231086": "Loss & Optimization what mean ?" }, "source": "meta" }