My best single model can only reach0.78 pb score.
\nNow:
\nThanks all for all yours suggestion. Now I Already reach 0.8.
\nThis post will record my improvement progress.
My best single model can only reach0.78 pb score.
\nNow:
\nThanks all for all yours suggestion. Now I Already reach 0.8.
\nThis post will record my improvement progress.
Some trying in those days:
\n 1.filter top 3% loss in backward propgation. 0.81x ~0.824 (not stable, cannot repeat)
\n 2.focal loss without data clean. 0.824->0.81~
\n 3.cut off humvoice segments. 0.824-> 0.79~
\n 4.replace humvoice segments as zeros. 0.824->0.81x
\n 5.limit the longgest segment portion of hum voice to below 50%. 0.816->0.821
\n 6.average 1.and 5. ->0.838
The 5. really works. Single model 0.816 -> 0.831.
", "rawMarkdown": "The 5. really works. Single model 0.816 -> 0.831.", "votes": 2, "replies": [ { "id": 3188310, "postDate": "2025-04-27T11:40:03.687Z", "content": "Can you tell me what types of augmentations you used?
", "rawMarkdown": "Can you tell me what types of augmentations you used?" }, { "id": 3188389, "postDate": "2025-04-27T13:52:58.080Z", "content": "Only freq/time mask+mixup+time moise
", "rawMarkdown": "Only freq/time mask+mixup+time moise", "votes": 2 } ] }, { "id": 3188396, "postDate": "2025-04-27T14:05:39.993Z", "content": "Hey, dude!
\nWhich architecture do you use?
In one of the other comments, he mentioned, that he's using an efficientnet_b0. This architecture has been proven to work well in the previous competitions and has quite a good inference speed, so you could go for ensemble methods later.
", "rawMarkdown": "In one of the other comments, he mentioned, that he's using an efficientnet_b0. This architecture has been proven to work well in the previous competitions and has quite a good inference speed, so you could go for ensemble methods later." }, { "id": 3188604, "postDate": "2025-04-27T23:54:04.890Z", "content": "only efficientnet_b0 now.
", "rawMarkdown": "only efficientnet_b0 now." }, { "id": 3188981, "postDate": "2025-04-28T15:13:35.530Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3194988, "postDate": "2025-05-06T14:02:58.820Z", "content": "Hello, Have you ever tried using the SED architecture?
", "rawMarkdown": "Hello, Have you ever tried using the SED architecture?" } ] }, { "id": 3188625, "postDate": "2025-04-28T00:46:04.810Z", "content": "I have a question regarding point 5 (\"limit the longest segment portion of the human voice to below 50%\").
\nFrom my understanding:
Is my understanding correct? Could you kindly elaborate a bit more on how you implemented this?
", "rawMarkdown": "I have a question regarding point 5 (\"limit the longest segment portion of the human voice to below 50%\").\nFrom my understanding:\n- You randomly cut a segment (e.g., 5 seconds),\n- Then you detect the human voice regions within that segment,\n- If the human voice portions exceed 50% of the segment, you adjust it (e.g., mark, mask, or modify) so that the segment contains at most 50% human voice.\n\nIs my understanding correct? Could you kindly elaborate a bit more on how you implemented this?", "replies": [ { "id": 3188713, "postDate": "2025-04-28T04:55:53.430Z", "content": "Just discard \"the human voice portions exceed 50% of the segment\".
", "rawMarkdown": "Just discard \"the human voice portions exceed 50% of the segment\".", "votes": 2 } ] } ] }, { "id": 3183938, "postDate": "2025-04-21T13:11:24.643Z", "content": "So my current model is not too good either (0.819 pb), but it's at least >0.8. Most of the improvements came from optimizing the spectrogram parameters and adding augmentations. I augmented both, the waveform data (change speed, pitch shift, scale volume, ring shift and gaussian noise) and the spectrograms (time and frequency masking). I also used mixup (linear on the spectrograms and stft mixup).
", "rawMarkdown": "So my current model is not too good either (0.819 pb), but it's at least >0.8. Most of the improvements came from optimizing the spectrogram parameters and adding augmentations. I augmented both, the waveform data (change speed, pitch shift, scale volume, ring shift and gaussian noise) and the spectrograms (time and frequency masking). I also used mixup (linear on the spectrograms and stft mixup). ", "votes": 3, "replies": [ { "id": 3183994, "postDate": "2025-04-21T14:14:45.793Z", "content": "I tried the same waveform augs but it hurts my model's performance, I am still checking…
", "rawMarkdown": "I tried the same waveform augs but it hurts my model's performance, I am still checking... " } ] }, { "id": 3183767, "postDate": "2025-04-21T09:00:42.473Z", "content": "You can try some data augmentation (audio, image), and also try using mixup, and use bce_loss to do multilabel tasks, and then tweak some hyperparameters like batch size, n_mel, nfft, lr, epoch, etc.
\nAnd then there's the more esoteric stuff, namely the random seed, who might make my model crash in the first few rounds of training.
\nGood luck !
", "rawMarkdown": "You can try some data augmentation (audio, image), and also try using mixup, and use bce_loss to do multilabel tasks, and then tweak some hyperparameters like batch size, n_mel, nfft, lr, epoch, etc.\n\nAnd then there's the more esoteric stuff, namely the random seed, who might make my model crash in the first few rounds of training.\n\nGood luck !", "votes": 4, "replies": [ { "id": 3183778, "postDate": "2025-04-21T09:16:53.877Z", "content": "Thanks a lot my friend! I tried the methods you mentioned above besides random seed/bce_loss. I think i should start with bce loss becuase i use ce loss now.
", "rawMarkdown": "Thanks a lot my friend! I tried the methods you mentioned above besides random seed/bce_loss. I think i should start with bce loss becuase i use ce loss now." } ] }, { "id": 3189972, "postDate": "2025-04-30T02:24:42.687Z", "content": "It's so weird, so unstable, we need more luckiness!
", "rawMarkdown": "It's so weird, so unstable, we need more luckiness!", "votes": 1 }, { "id": 3185322, "postDate": "2025-04-23T06:52:20.690Z", "content": "Update. I use some loss trick to prevent overfitting in dirty data, Now i get pb score 0.824
", "rawMarkdown": "Update. I use some loss trick to prevent overfitting in dirty data, Now i get pb score 0.824", "votes": 1, "replies": [ { "id": 3188775, "postDate": "2025-04-28T07:15:55.490Z", "content": "What do you mean by \"dirty data\"?
", "rawMarkdown": "What do you mean by \"dirty data\"?", "replies": [ { "id": 3188921, "postDate": "2025-04-28T13:04:28.487Z", "content": "Means samples with top loss in a batch.
", "rawMarkdown": "Means samples with top loss in a batch. " } ] }, { "id": 3188777, "postDate": "2025-04-28T07:18:40.043Z", "content": "What methods were used?
", "rawMarkdown": "What methods were used?", "replies": [ { "id": 3188927, "postDate": "2025-04-28T13:16:51.450Z", "content": "Just filter large loss…
", "rawMarkdown": "Just filter large loss..." }, { "id": 3188968, "postDate": "2025-04-28T14:51:09.500Z", "content": "@xukongji whats your local cv? before filter large loss and after?
", "rawMarkdown": "@xukongji whats your local cv? before filter large loss and after?" }, { "id": 3188976, "postDate": "2025-04-28T15:04:16.137Z", "content": "I train full data. without local cv
", "rawMarkdown": "I train full data. without local cv" } ] }, { "id": 3199686, "postDate": "2025-05-11T10:27:48.927Z", "content": "Looks like it's time to change the 0.8 in the topic title to 0.9?
", "rawMarkdown": "Looks like it's time to change the 0.8 in the topic title to 0.9?", "replies": [ { "id": 3200900, "postDate": "2025-05-13T07:49:44.190Z", "content": "Hard to move one forward step😂
", "rawMarkdown": "Hard to move one forward step😂" } ] } ] }, { "id": 3185187, "postDate": "2025-04-23T01:28:41.717Z", "content": "Update. I used all training data to train a model and my pb score improves to 0.816. It's time to tune mel spec params/random seeds/different models. So happy hhh…😃
", "rawMarkdown": "Update. I used all training data to train a model and my pb score improves to 0.816. It's time to tune mel spec params/random seeds/different models. So happy hhh...😃", "votes": 2, "replies": [ { "id": 3185230, "postDate": "2025-04-23T03:11:15.283Z", "content": "Do you have some kind of metric for early stopping or are you just using a fixed number of epochs/iterations? I wouldn't know when to stop when using the whole data for training.
", "rawMarkdown": "Do you have some kind of metric for early stopping or are you just using a fixed number of epochs/iterations? I wouldn't know when to stop when using the whole data for training.", "replies": [ { "id": 3185321, "postDate": "2025-04-23T06:49:37.833Z", "content": "Without early stopping, I train for 30 epochs and submit the last epoch model.
", "rawMarkdown": "Without early stopping, I train for 30 epochs and submit the last epoch model." }, { "id": 3185612, "postDate": "2025-04-23T15:22:36.017Z", "content": "I'll try that as well. Thanks.
", "rawMarkdown": "I'll try that as well. Thanks." }, { "id": 3186736, "postDate": "2025-04-25T05:31:40.433Z", "content": "I tried that too but it hurt my model and got only .801
\ncompared to 0.806 on normal 10 epochs
", "rawMarkdown": "I tried that too but it hurt my model and got only .801\n\ncompared to 0.806 on normal 10 epochs " }, { "id": 3187762, "postDate": "2025-04-26T13:36:57.273Z", "content": "I never tune epochs using 30epochs all of the way. Maybe you can try EMA or SWA to reduce randomness between epochs.
", "rawMarkdown": "I never tune epochs using 30epochs all of the way. Maybe you can try EMA or SWA to reduce randomness between epochs." }, { "id": 3200828, "postDate": "2025-05-13T05:40:29.240Z", "content": "@xukongji What is the values of the learning rate, weight decay, and scheduler settings for the optimizer?
", "rawMarkdown": "@xukongji What is the values of the learning rate, weight decay, and scheduler settings for the optimizer?" }, { "id": 3200901, "postDate": "2025-05-13T07:51:23.730Z", "content": "I use lr=5e-4, weight_decay=1e-5, cosine_schedule_with_warmup
", "rawMarkdown": "I use lr=5e-4, weight_decay=1e-5, cosine_schedule_with_warmup", "votes": 1 }, { "id": 3200918, "postDate": "2025-05-13T08:21:15.853Z", "content": "thank you for your quick reply!!
", "rawMarkdown": "thank you for your quick reply!!" } ] }, { "id": 3186737, "postDate": "2025-04-25T05:33:45.670Z", "content": "All training data as in ?
\nDoes “used all training data” mean that there is no validation and all the labeled data are used for training?
", "rawMarkdown": "Does “used all training data” mean that there is no validation and all the labeled data are used for training?", "replies": [ { "id": 3187738, "postDate": "2025-04-26T13:03:50.337Z", "content": "That's right.
", "rawMarkdown": "That's right. " } ] } ] }, { "id": 3184098, "postDate": "2025-04-21T16:35:26.203Z", "content": "According to what I have read, you might get some boost for a single model if you:
\nBest of luck!
", "rawMarkdown": "According to what I have read, you might get some boost for a single model if you:\n\n- tweak a bit the melspectrogram base parameters\n- try various backbones \n- tweak training hyperparameters\n\nBest of luck!", "votes": 2, "replies": [ { "id": 3184337, "postDate": "2025-04-22T02:38:50.190Z", "content": "Thanks a lot and good suggestion! I still use efficientnet_b0. I will not change model until it reach 0.8 pb score because many public notebook reach this score with this model.
", "rawMarkdown": "Thanks a lot and good suggestion! I still use efficientnet_b0. I will not change model until it reach 0.8 pb score because many public notebook reach this score with this model.", "votes": 3, "replies": [ { "id": 3184574, "postDate": "2025-04-22T08:56:20.920Z", "content": "do you use tta on predict? maybe you should try label smooth.
", "rawMarkdown": "do you use tta on predict? maybe you should try label smooth." }, { "id": 3185181, "postDate": "2025-04-23T01:12:25.607Z", "content": "Not yet. Now I switch to bce loss and use prediction smoth
", "rawMarkdown": "Not yet. Now I switch to bce loss and use prediction smoth", "votes": 1 }, { "id": 3186070, "postDate": "2025-04-24T07:01:41.037Z", "content": "Yes, changing the loss is a good option as well. Best of luck @xukongji. 🫡
", "rawMarkdown": "Yes, changing the loss is a good option as well. Best of luck @xukongji. 🫡" } ] } ] }, { "id": 3199108, "postDate": "2025-05-10T13:23:51.750Z", "content": "AM I doing right? Is there any way I can do it more effeciently ?
\nTraining time is close to me.
\nMy submission takes almost 13mins(efficient_b0, single model)
Hey man,
\nCan I ask 3 chanels are mel spec, delta, delta-delta or sth else?
\nThank you.
just melspec
", "rawMarkdown": "just melspec" } ] }, { "id": 3188999, "postDate": "2025-04-28T15:39:55.493Z", "content": "Hey, Dude!
\nI was working with 1 input channel, and I had an 82.2 LB score (single-fold).
\nWhen I adapted my code to the 3 channels, sadly, it dropped to 79.8. This is because I need to refine the parameters to suit the 3 channels, right?
In my exps, 3channel is better than 1channel. If you train 3channel you should submit with 3channel.
", "rawMarkdown": "In my exps, 3channel is better than 1channel. If you train 3channel you should submit with 3channel.", "replies": [ { "id": 3189259, "postDate": "2025-04-29T03:06:20.013Z", "content": "Are you using different spectrograms for each channel, e.g., each with a different power?
", "rawMarkdown": "Are you using different spectrograms for each channel, e.g., each with a different power?" }, { "id": 3189316, "postDate": "2025-04-29T04:53:00.493Z", "content": "Not yet.
", "rawMarkdown": "Not yet. " } ] } ] }, { "id": 3185137, "postDate": "2025-04-22T22:26:17.403Z", "content": "Good Post!! Very Nice.
", "rawMarkdown": "Good Post!! Very Nice.", "replies": [ { "id": 3185403, "postDate": "2025-04-23T09:12:45.870Z", "content": "Thanks a lot my friend
", "rawMarkdown": "Thanks a lot my friend" } ] }, { "id": 3184344, "postDate": "2025-04-22T02:42:21.967Z", "content": "Update. I switch celoss -> bceloss + batch128->64 + Adam->AdamW + train with secondary label. And I get single model(one of the best val score model in 5fold models) pb score changes from 0.78->0.786
", "rawMarkdown": "Update. I switch celoss -> bceloss + batch128->64 + Adam->AdamW + train with secondary label. And I get single model(one of the best val score model in 5fold models) pb score changes from 0.78->0.786", "replies": [ { "id": 3184355, "postDate": "2025-04-22T02:48:03.207Z", "content": "Update. Refer to public notebook:https://www.kaggle.com/code/tomkkk/change-secondary-labels-in-train-csv , I add prediction smoth code.
\nI get pb score: 0.786->0.797，quite close to 0.80😁
Update.Change batch 64->32, now I reach pb score 0.80
", "rawMarkdown": "Update.Change batch 64->32, now I reach pb score 0.80 ", "votes": 3 }, { "id": 3184474, "postDate": "2025-04-22T06:12:39.073Z", "content": "wow, it seems like small batch helps.
", "rawMarkdown": "wow, it seems like small batch helps." }, { "id": 3184524, "postDate": "2025-04-22T08:00:42.007Z", "content": "Maybe model with small batch can jump out local optimal point. or train a longer steps … It's hard to say
", "rawMarkdown": "Maybe model with small batch can jump out local optimal point. or train a longer steps ... It's hard to say", "votes": 1 }, { "id": 3184677, "postDate": "2025-04-22T11:08:31.247Z", "content": "Update. I tried focal loss, but it hurts model performance in my validation set, I guess I should clean dataset first.
", "rawMarkdown": "Update. I tried focal loss, but it hurts model performance in my validation set, I guess I should clean dataset first.", "votes": 1 }, { "id": 3184719, "postDate": "2025-04-22T12:10:40.170Z", "content": "Update. Use vanilla efficientnet_b0+dropout0.2 ->0.804
", "rawMarkdown": "Update. Use vanilla efficientnet_b0+dropout0.2 ->0.804", "votes": 1 }, { "id": 3185228, "postDate": "2025-04-23T03:07:06.637Z", "content": "I thought about cleaning as well, but I'm not sure how to do so. I'll also try a smaller batch size, because currently I'm using 256.
", "rawMarkdown": "I thought about cleaning as well, but I'm not sure how to do so. I'll also try a smaller batch size, because currently I'm using 256. " }, { "id": 3185405, "postDate": "2025-04-23T09:16:11.333Z", "content": "You can cut off human voice part in the data. This notebook provides a good method for detecting human voice segment: https://www.kaggle.com/code/kdmitrie/bc25-separation-voice-from-data/notebook#Some-recordings-contain-human-voice
", "rawMarkdown": "You can cut off human voice part in the data. This notebook provides a good method for detecting human voice segment: https://www.kaggle.com/code/kdmitrie/bc25-separation-voice-from-data/notebook#Some-recordings-contain-human-voice", "votes": 1 }, { "id": 3186735, "postDate": "2025-04-25T05:30:16.043Z", "content": "how do you cut those out, I'm using silero-vad timestamps and running a sliding window.
", "rawMarkdown": "how do you cut those out, I'm using silero-vad timestamps and running a sliding window." }, { "id": 3186927, "postDate": "2025-04-25T09:55:54.210Z", "content": "I am still trying.
", "rawMarkdown": "I am still trying." } ] } ] } ], "comments": [ { "id": 3187752, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-26T13:25:01.880000", "content": "Some trying in those days:
\n 1.filter top 3% loss in backward propgation. 0.81x ~0.824 (not stable, cannot repeat)
\n 2.focal loss without data clean. 0.824->0.81~
\n 3.cut off humvoice segments. 0.824-> 0.79~
\n 4.replace humvoice segments as zeros. 0.824->0.81x
\n 5.limit the longgest segment portion of hum voice to below 50%. 0.816->0.821
\n 6.average 1.and 5. ->0.838
The 5. really works. Single model 0.816 -> 0.831.
", "votes": 2, "replies": [ { "id": 3188310, "author_name": "Swayam Singh", "author_url": "", "post_date": "2025-04-27T11:40:03.687000", "content": "Can you tell me what types of augmentations you used?
", "votes": 0, "replies": [] }, { "id": 3188389, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-27T13:52:58.080000", "content": "Only freq/time mask+mixup+time moise
", "votes": 2, "replies": [] } ] }, { "id": 3188396, "author_name": "Ahmed Elsayed", "author_url": "", "post_date": "2025-04-27T14:05:39.993000", "content": "Hey, dude!
\nWhich architecture do you use?
In one of the other comments, he mentioned, that he's using an efficientnet_b0. This architecture has been proven to work well in the previous competitions and has quite a good inference speed, so you could go for ensemble methods later.
", "votes": 0, "replies": [] }, { "id": 3188604, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-27T23:54:04.890000", "content": "only efficientnet_b0 now.
", "votes": 0, "replies": [] }, { "id": 3188981, "author_name": "", "author_url": "", "post_date": "2025-04-28T15:13:35.530000", "content": "", "votes": 0, "replies": [] }, { "id": 3194988, "author_name": "MagicL", "author_url": "", "post_date": "2025-05-06T14:02:58.820000", "content": "Hello, Have you ever tried using the SED architecture?
", "votes": 0, "replies": [] } ] }, { "id": 3188625, "author_name": "datnt114", "author_url": "", "post_date": "2025-04-28T00:46:04.810000", "content": "I have a question regarding point 5 (\"limit the longest segment portion of the human voice to below 50%\").
\nFrom my understanding:
Is my understanding correct? Could you kindly elaborate a bit more on how you implemented this?
", "votes": 0, "replies": [ { "id": 3188713, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-28T04:55:53.430000", "content": "Just discard \"the human voice portions exceed 50% of the segment\".
", "votes": 2, "replies": [] } ] } ] }, { "id": 3183938, "author_name": "Tim", "author_url": "", "post_date": "2025-04-21T13:11:24.643000", "content": "So my current model is not too good either (0.819 pb), but it's at least >0.8. Most of the improvements came from optimizing the spectrogram parameters and adding augmentations. I augmented both, the waveform data (change speed, pitch shift, scale volume, ring shift and gaussian noise) and the spectrograms (time and frequency masking). I also used mixup (linear on the spectrograms and stft mixup).
", "votes": 3, "replies": [ { "id": 3183994, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-21T14:14:45.793000", "content": "I tried the same waveform augs but it hurts my model's performance, I am still checking…
", "votes": 0, "replies": [] } ] }, { "id": 3183767, "author_name": "agcsdedf", "author_url": "", "post_date": "2025-04-21T09:00:42.473000", "content": "You can try some data augmentation (audio, image), and also try using mixup, and use bce_loss to do multilabel tasks, and then tweak some hyperparameters like batch size, n_mel, nfft, lr, epoch, etc.
\nAnd then there's the more esoteric stuff, namely the random seed, who might make my model crash in the first few rounds of training.
\nGood luck !
", "votes": 4, "replies": [ { "id": 3183778, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-21T09:16:53.877000", "content": "Thanks a lot my friend! I tried the methods you mentioned above besides random seed/bce_loss. I think i should start with bce loss becuase i use ce loss now.
", "votes": 0, "replies": [] } ] }, { "id": 3189972, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-30T02:24:42.687000", "content": "It's so weird, so unstable, we need more luckiness!
", "votes": 1, "replies": [] }, { "id": 3185322, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-23T06:52:20.690000", "content": "Update. I use some loss trick to prevent overfitting in dirty data, Now i get pb score 0.824
", "votes": 1, "replies": [ { "id": 3188775, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2025-04-28T07:15:55.490000", "content": "What do you mean by \"dirty data\"?
", "votes": 0, "replies": [ { "id": 3188921, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-28T13:04:28.487000", "content": "Means samples with top loss in a batch.
", "votes": 0, "replies": [] } ] }, { "id": 3188777, "author_name": "New bee", "author_url": "", "post_date": "2025-04-28T07:18:40.043000", "content": "What methods were used?
", "votes": 0, "replies": [ { "id": 3188927, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-28T13:16:51.450000", "content": "Just filter large loss…
", "votes": 0, "replies": [] }, { "id": 3188968, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2025-04-28T14:51:09.500000", "content": "@xukongji whats your local cv? before filter large loss and after?
", "votes": 0, "replies": [] }, { "id": 3188976, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-28T15:04:16.137000", "content": "I train full data. without local cv
", "votes": 0, "replies": [] } ] }, { "id": 3199686, "author_name": "Pavel Orlov", "author_url": "", "post_date": "2025-05-11T10:27:48.927000", "content": "Looks like it's time to change the 0.8 in the topic title to 0.9?
", "votes": 0, "replies": [ { "id": 3200900, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-05-13T07:49:44.190000", "content": "Hard to move one forward step😂
", "votes": 0, "replies": [] } ] } ] }, { "id": 3185187, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-23T01:28:41.717000", "content": "Update. I used all training data to train a model and my pb score improves to 0.816. It's time to tune mel spec params/random seeds/different models. So happy hhh…😃
", "votes": 2, "replies": [ { "id": 3185230, "author_name": "Tim", "author_url": "", "post_date": "2025-04-23T03:11:15.283000", "content": "Do you have some kind of metric for early stopping or are you just using a fixed number of epochs/iterations? I wouldn't know when to stop when using the whole data for training.
", "votes": 0, "replies": [ { "id": 3185321, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-23T06:49:37.833000", "content": "Without early stopping, I train for 30 epochs and submit the last epoch model.
", "votes": 0, "replies": [] }, { "id": 3185612, "author_name": "Tim", "author_url": "", "post_date": "2025-04-23T15:22:36.017000", "content": "I'll try that as well. Thanks.
", "votes": 0, "replies": [] }, { "id": 3186736, "author_name": "Devasy Patel23", "author_url": "", "post_date": "2025-04-25T05:31:40.433000", "content": "I tried that too but it hurt my model and got only .801
\ncompared to 0.806 on normal 10 epochs
", "votes": 0, "replies": [] }, { "id": 3187762, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-26T13:36:57.273000", "content": "I never tune epochs using 30epochs all of the way. Maybe you can try EMA or SWA to reduce randomness between epochs.
", "votes": 0, "replies": [] }, { "id": 3200828, "author_name": "Meisa0", "author_url": "", "post_date": "2025-05-13T05:40:29.240000", "content": "@xukongji What is the values of the learning rate, weight decay, and scheduler settings for the optimizer?
", "votes": 0, "replies": [] }, { "id": 3200901, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-05-13T07:51:23.730000", "content": "I use lr=5e-4, weight_decay=1e-5, cosine_schedule_with_warmup
", "votes": 1, "replies": [] }, { "id": 3200918, "author_name": "Meisa0", "author_url": "", "post_date": "2025-05-13T08:21:15.853000", "content": "thank you for your quick reply!!
", "votes": 0, "replies": [] } ] }, { "id": 3186737, "author_name": "Devasy Patel23", "author_url": "", "post_date": "2025-04-25T05:33:45.670000", "content": "All training data as in ?
\nDoes “used all training data” mean that there is no validation and all the labeled data are used for training?
", "votes": 0, "replies": [ { "id": 3187738, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-26T13:03:50.337000", "content": "That's right.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3184098, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2025-04-21T16:35:26.203000", "content": "According to what I have read, you might get some boost for a single model if you:
\nBest of luck!
", "votes": 2, "replies": [ { "id": 3184337, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-22T02:38:50.190000", "content": "Thanks a lot and good suggestion! I still use efficientnet_b0. I will not change model until it reach 0.8 pb score because many public notebook reach this score with this model.
", "votes": 3, "replies": [ { "id": 3184574, "author_name": "Yang Wei Hao", "author_url": "", "post_date": "2025-04-22T08:56:20.920000", "content": "do you use tta on predict? maybe you should try label smooth.
", "votes": 0, "replies": [] }, { "id": 3185181, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-23T01:12:25.607000", "content": "Not yet. Now I switch to bce loss and use prediction smoth
", "votes": 1, "replies": [] }, { "id": 3186070, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2025-04-24T07:01:41.037000", "content": "Yes, changing the loss is a good option as well. Best of luck @xukongji. 🫡
", "votes": 0, "replies": [] } ] } ] }, { "id": 3199108, "author_name": "Sonu Jha", "author_url": "", "post_date": "2025-05-10T13:23:51.750000", "content": "AM I doing right? Is there any way I can do it more effeciently ?
\nTraining time is close to me.
\nMy submission takes almost 13mins(efficient_b0, single model)
Hey man,
\nCan I ask 3 chanels are mel spec, delta, delta-delta or sth else?
\nThank you.
just melspec
", "votes": 0, "replies": [] } ] }, { "id": 3188999, "author_name": "Ahmed Elsayed", "author_url": "", "post_date": "2025-04-28T15:39:55.493000", "content": "Hey, Dude!
\nI was working with 1 input channel, and I had an 82.2 LB score (single-fold).
\nWhen I adapted my code to the 3 channels, sadly, it dropped to 79.8. This is because I need to refine the parameters to suit the 3 channels, right?
In my exps, 3channel is better than 1channel. If you train 3channel you should submit with 3channel.
", "votes": 0, "replies": [ { "id": 3189259, "author_name": "Tim", "author_url": "", "post_date": "2025-04-29T03:06:20.013000", "content": "Are you using different spectrograms for each channel, e.g., each with a different power?
", "votes": 0, "replies": [] }, { "id": 3189316, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-29T04:53:00.493000", "content": "Not yet.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3185137, "author_name": "VampireDataNerd", "author_url": "", "post_date": "2025-04-22T22:26:17.403000", "content": "Good Post!! Very Nice.
", "votes": 0, "replies": [ { "id": 3185403, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-23T09:12:45.870000", "content": "Thanks a lot my friend
", "votes": 0, "replies": [] } ] }, { "id": 3184344, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-22T02:42:21.967000", "content": "Update. I switch celoss -> bceloss + batch128->64 + Adam->AdamW + train with secondary label. And I get single model(one of the best val score model in 5fold models) pb score changes from 0.78->0.786
", "votes": 0, "replies": [ { "id": 3184355, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-22T02:48:03.207000", "content": "Update. Refer to public notebook:https://www.kaggle.com/code/tomkkk/change-secondary-labels-in-train-csv , I add prediction smoth code.
\nI get pb score: 0.786->0.797，quite close to 0.80😁
Update.Change batch 64->32, now I reach pb score 0.80
", "votes": 3, "replies": [] }, { "id": 3184474, "author_name": "Whisper Last", "author_url": "", "post_date": "2025-04-22T06:12:39.073000", "content": "wow, it seems like small batch helps.
", "votes": 0, "replies": [] }, { "id": 3184524, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-22T08:00:42.007000", "content": "Maybe model with small batch can jump out local optimal point. or train a longer steps … It's hard to say
", "votes": 1, "replies": [] }, { "id": 3184677, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-22T11:08:31.247000", "content": "Update. I tried focal loss, but it hurts model performance in my validation set, I guess I should clean dataset first.
", "votes": 1, "replies": [] }, { "id": 3184719, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-22T12:10:40.170000", "content": "Update. Use vanilla efficientnet_b0+dropout0.2 ->0.804
", "votes": 1, "replies": [] }, { "id": 3185228, "author_name": "Tim", "author_url": "", "post_date": "2025-04-23T03:07:06.637000", "content": "I thought about cleaning as well, but I'm not sure how to do so. I'll also try a smaller batch size, because currently I'm using 256.
", "votes": 0, "replies": [] }, { "id": 3185405, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-23T09:16:11.333000", "content": "You can cut off human voice part in the data. This notebook provides a good method for detecting human voice segment: https://www.kaggle.com/code/kdmitrie/bc25-separation-voice-from-data/notebook#Some-recordings-contain-human-voice
", "votes": 1, "replies": [] }, { "id": 3186735, "author_name": "Devasy Patel23", "author_url": "", "post_date": "2025-04-25T05:30:16.043000", "content": "how do you cut those out, I'm using silero-vad timestamps and running a sliding window.
", "votes": 0, "replies": [] }, { "id": 3186927, "author_name": "XuKong Ji", "author_url": "", "post_date": "2025-04-25T09:55:54.210000", "content": "I am still trying.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3183742": "My best single model can only reach0.78 pb score.\nNow:\nThanks all for all yours suggestion. Now I Already reach 0.8.\nThis post will record my improvement progress.", "3187752": "Some trying in those days:\n 1.filter top 3% loss in backward propgation. 0.81x ~0.824 (not stable, cannot repeat)\n 2.focal loss without data clean. 0.824->0.81~\n 3.cut off humvoice segments. 0.824-> 0.79~\n 4.replace humvoice segments as zeros. 0.824->0.81x\n 5.limit the longgest segment portion of hum voice to below 50%. 0.816->0.821\n 6.average 1.and 5. ->0.838", "3183938": "So my current model is not too good either (0.819 pb), but it's at least >0.8. Most of the improvements came from optimizing the spectrogram parameters and adding augmentations. I augmented both, the waveform data (change speed, pitch shift, scale volume, ring shift and gaussian noise) and the spectrograms (time and frequency masking). I also used mixup (linear on the spectrograms and stft mixup). ", "3183767": "You can try some data augmentation (audio, image), and also try using mixup, and use bce_loss to do multilabel tasks, and then tweak some hyperparameters like batch size, n_mel, nfft, lr, epoch, etc.\n\nAnd then there's the more esoteric stuff, namely the random seed, who might make my model crash in the first few rounds of training.\n\nGood luck !", "3189972": "It's so weird, so unstable, we need more luckiness!", "3185322": "Update. I use some loss trick to prevent overfitting in dirty data, Now i get pb score 0.824", "3185187": "Update. I used all training data to train a model and my pb score improves to 0.816. It's time to tune mel spec params/random seeds/different models. So happy hhh...😃", "3184098": "According to what I have read, you might get some boost for a single model if you:\n\n- tweak a bit the melspectrogram base parameters\n- try various backbones \n- tweak training hyperparameters\n\nBest of luck!", "3199108": "AM I doing right? Is there any way I can do it more effeciently ?\n\n- For data preprocessing change, we update the melspec parameter and run preprocessing pipeline. ~35 minutes to run \n- Train whole model - 2:30 hours \n- Make submission - 1 hours approx \n- For every experiment, it is taking 4 hours. ", "3190770": "Hey man,\nCan I ask 3 chanels are mel spec, delta, delta-delta or sth else?\nThank you.", "3188999": "Hey, Dude!\nI was working with 1 input channel, and I had an 82.2 LB score (single-fold).\nWhen I adapted my code to the 3 channels, sadly, it dropped to 79.8. This is because I need to refine the parameters to suit the 3 channels, right?", "3185137": "Good Post!! Very Nice.", "3184344": "Update. I switch celoss -> bceloss + batch128->64 + Adam->AdamW + train with secondary label. And I get single model(one of the best val score model in 5fold models) pb score changes from 0.78->0.786" } }