{ "id": 482775, "title": "Single Model Best LB Discussion", "url": "/competitions/hms-harmful-brain-activity-classification/discussion/482775", "author_name": "fireflysentinel", "post_date": "2024-03-09T11:14:32.441000", "votes": 49, "comment_count": 65, "views": 0, "content": "

Hello everyone, I'm a Grade 12 student from Shanghai World Foreign Language Academy, and I'm participating in a Kaggle competition for the first time. After experimenting with various models based on spectrograms, I've documented the best LB I achieved with each model. Here's a summary of my findings:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelLB
EfficientNetB70.36
EfficientNetB60.36
EfficientNetB50.37
EfficientNetB40.36
EfficientNetB30.38
EfficientNetB20.40
EfficientNetB10.41
EfficientNetB00.43
EfficientNetB2V20.38
EfficientNetB1V20.38
Resnet152d0.38
Resnet101d0.38
Resnet50d0.38
Resnet34d0.39
\n

I've also experimented with different variants of ResNet50, ViT-tiny, ViT-small, MobileNet, ConvNeXt, and DenseNet, but unfortunately, they didn't yield as promising results. In addition, I'm exploring models based on raw EEG signals, although my current best with WaveNet is at LB 0.47.

\n

I'm curious to know about the best LB that single models have achieved in your experiments. Thank you for your time and assistance🙏

", "messages": [ { "id": 2688648, "postDate": "2024-03-09T11:14:32.443Z", "content": "

Hello everyone, I'm a Grade 12 student from Shanghai World Foreign Language Academy, and I'm participating in a Kaggle competition for the first time. After experimenting with various models based on spectrograms, I've documented the best LB I achieved with each model. Here's a summary of my findings:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelLB
EfficientNetB70.36
EfficientNetB60.36
EfficientNetB50.37
EfficientNetB40.36
EfficientNetB30.38
EfficientNetB20.40
EfficientNetB10.41
EfficientNetB00.43
EfficientNetB2V20.38
EfficientNetB1V20.38
Resnet152d0.38
Resnet101d0.38
Resnet50d0.38
Resnet34d0.39
\n

I've also experimented with different variants of ResNet50, ViT-tiny, ViT-small, MobileNet, ConvNeXt, and DenseNet, but unfortunately, they didn't yield as promising results. In addition, I'm exploring models based on raw EEG signals, although my current best with WaveNet is at LB 0.47.

\n

I'm curious to know about the best LB that single models have achieved in your experiments. Thank you for your time and assistance🙏

", "rawMarkdown": "Hello everyone, I'm a Grade 12 student from Shanghai World Foreign Language Academy, and I'm participating in a Kaggle competition for the first time. After experimenting with various models based on spectrograms, I've documented the best LB I achieved with each model. Here's a summary of my findings:\n\n| Model | LB |\n|:-----------------------:|:--------:|\n| EfficientNetB7 | 0.36 |\n| EfficientNetB6 | 0.36 |\n| EfficientNetB5 | 0.37 |\n| EfficientNetB4 | **0.36** |\n| EfficientNetB3 | 0.38 |\n| EfficientNetB2 | 0.40 |\n| EfficientNetB1 | 0.41 |\n| EfficientNetB0 | 0.43 |\n| EfficientNetB2V2 | 0.38 |\n| EfficientNetB1V2 | **0.38** |\n| Resnet152d | 0.38 |\n| Resnet101d | 0.38 | \n| Resnet50d | **0.38** |\n| Resnet34d | 0.39 |\n\nI've also experimented with different variants of ResNet50, ViT-tiny, ViT-small, MobileNet, ConvNeXt, and DenseNet, but unfortunately, they didn't yield as promising results. In addition, I'm exploring models based on raw EEG signals, although my current best with WaveNet is at LB 0.47.\n\nI'm curious to know about the best LB that single models have achieved in your experiments. Thank you for your time and assistance🙏", "votes": 48 }, { "id": 2696315, "postDate": "2024-03-14T08:29:08.717Z", "content": "

Single model 4 folds with stage2
\n
\n
\n
\n
\nupdate LB 0.23 with CV 0.232
\nBasicly stage2 cv and LB match well, my suggestion is to trust your vote >= 10 cv only. vote < 10 data is noise, label unconfident. \nFor myself I might choose 1 submission base on best LB and another on best local cv for votes >= 10.

", "rawMarkdown": "Single model 4 folds with stage2 \n~~cv 0.265 and LB 0.27~~ \n~~update cv 0.265 LB 0.26~~ \n~~update cv 0.25 LB 0.25~~ \n~~update LB 0.24 with CV 0.244~~\nupdate LB 0.23 with CV 0.232\nBasicly stage2 cv and LB match well, my suggestion is to trust your vote >= 10 cv only. vote < 10 data is noise, label unconfident. \nFor myself I might choose 1 submission base on best LB and another on best local cv for votes >= 10.", "votes": 25, "replies": [ { "id": 2720450, "postDate": "2024-03-28T11:05:21.400Z", "content": "

by choosing best LB, best cv as your final submission, does it mean your best CV and best LB models are different?

", "rawMarkdown": "by choosing best LB, best cv as your final submission, does it mean your best CV and best LB models are different?", "replies": [ { "id": 2720591, "postDate": "2024-03-28T13:19:23.810Z", "content": "

They match mostly with a little difference, I would bet on using votes >=10, so I will choose 2 only from metrics perform well on votes >= 10.
\nIf you do cv locally you might still see a little model performance diff on different folds, as well as on LB, though not much gap.

", "rawMarkdown": "They match mostly with a little difference, I would bet on using votes >=10, so I will choose 2 only from metrics perform well on votes >= 10.\nIf you do cv locally you might still see a little model performance diff on different folds, as well as on LB, though not much gap.", "votes": 3 } ] } ] }, { "id": 2689661, "postDate": "2024-03-10T03:45:07.440Z", "content": "

Raw EEG -> 0.27
\nEEG Spectrogram -> 0.26
\nKaggle Spectrogram -> 0.35

\n

Edit: EEG spectrogram is 0.24 now.
\nEdit: Best single raw EEG model is 0.26 now.

", "rawMarkdown": "Raw EEG -> 0.27\nEEG Spectrogram -> 0.26\nKaggle Spectrogram -> 0.35\n\nEdit: EEG spectrogram is ~~0.25~~ 0.24 now.\nEdit: Best single raw EEG model is 0.26 now. ", "votes": 25, "replies": [ { "id": 2690330, "postDate": "2024-03-10T13:06:37.370Z", "content": "

It's pretty interesting to compare the kaggle spectrograms vs \"homemade\" spectrograms. Excellent raw EEG model! I'm still tuning mine and I've found a couple augmentations that get me to about 0.44, but i still have a lot of work to do.

", "rawMarkdown": "It's pretty interesting to compare the kaggle spectrograms vs \"homemade\" spectrograms. Excellent raw EEG model! I'm still tuning mine and I've found a couple augmentations that get me to about 0.44, but i still have a lot of work to do.", "votes": 1, "replies": [ { "id": 2690505, "postDate": "2024-03-10T15:39:30.633Z", "content": "

Thanks. It actually makes lot of sense since center 10 seconds is 5 pixels on kaggle spectrograms, but 1/5 of pixels on eeg spectrograms. No wonder why kaggle spectrograms aren't performing good.

", "rawMarkdown": "Thanks. It actually makes lot of sense since center 10 seconds is 5 pixels on kaggle spectrograms, but 1/5 of pixels on eeg spectrograms. No wonder why kaggle spectrograms aren't performing good.", "votes": 2 }, { "id": 2690518, "postDate": "2024-03-10T16:01:09.230Z", "content": "

Well now that you put it that way, it seems obvious in hindsight haha. Very nice! I'm spending the rest of the competition working directly with EEG data since my last two competitions were image competitions and this is a tad different. I'm a little surprised transformers haven't been more effective in this context, but so far I've failed to get any decent ones. Only Conv1D based models have been fruitful so far (haven't tried any graph based networks yet).

", "rawMarkdown": "Well now that you put it that way, it seems obvious in hindsight haha. Very nice! I'm spending the rest of the competition working directly with EEG data since my last two competitions were image competitions and this is a tad different. I'm a little surprised transformers haven't been more effective in this context, but so far I've failed to get any decent ones. Only Conv1D based models have been fruitful so far (haven't tried any graph based networks yet).", "votes": 1 } ] }, { "id": 2690408, "postDate": "2024-03-10T14:10:17.847Z", "content": "

Thanks @gunesevitan for sharing, Raw/Spectrograms achieve 0.27/0.26 motivate to explore data more then model choice. is my assumption right?

", "rawMarkdown": "Thanks @gunesevitan for sharing, Raw/Spectrograms achieve 0.27/0.26 motivate to explore data more then model choice. is my assumption right?", "votes": 1, "replies": [ { "id": 2690507, "postDate": "2024-03-10T15:43:05.100Z", "content": "

You can explore both models and preprocessing.

", "rawMarkdown": "You can explore both models and preprocessing.", "votes": 1 }, { "id": 2690715, "postDate": "2024-03-10T18:32:19.573Z", "content": "

@gunesevitan what your cv for Raw/Spectrograms achieve 0.27/0.26

", "rawMarkdown": "@gunesevitan what your cv for Raw/Spectrograms achieve 0.27/0.26" } ] }, { "id": 2691525, "postDate": "2024-03-11T09:42:05.927Z", "content": "

great result! looking forward to seeing the pipeline.

", "rawMarkdown": "great result! looking forward to seeing the pipeline. " }, { "id": 2694326, "postDate": "2024-03-13T02:03:15.760Z", "content": "

fascinating @gunesevitan , I believe preprocessing matters greatly.

", "rawMarkdown": "fascinating @gunesevitan , I believe preprocessing matters greatly." }, { "id": 2695803, "postDate": "2024-03-13T21:33:07.803Z", "content": "

Great single model @gunesevitan and team! 🔥 My best single model is LB=0.25, then ensemble to LB=0.24.

", "rawMarkdown": "Great single model @gunesevitan and team! 🔥 My best single model is `LB=0.25`, then ensemble to `LB=0.24`. ", "votes": 4 } ] }, { "id": 2695805, "postDate": "2024-03-13T21:34:20.680Z", "content": "

I'm excited to achieve single model LB = 0.25 using all data, both Spectrogram and EEG.
\nUPDATE Single model LB = 0.24, woohoo!

", "rawMarkdown": "I'm excited to achieve single model `LB = 0.25` using all data, both Spectrogram and EEG.\n**UPDATE** Single model `LB = 0.24`, woohoo!", "votes": 18, "replies": [ { "id": 2695844, "postDate": "2024-03-13T21:57:07.990Z", "content": "

do you use 2-stages model if you don't mind me asking?

", "rawMarkdown": "do you use 2-stages model if you don't mind me asking?", "votes": 2, "replies": [ { "id": 2695905, "postDate": "2024-03-14T00:08:42.360Z", "content": "

I use many stages such as pretrain, knowledge distilitation, pseudo label, finetune, etc.

", "rawMarkdown": "I use **many** stages such as pretrain, knowledge distilitation, pseudo label, finetune, etc.", "votes": 27 }, { "id": 2696084, "postDate": "2024-03-14T04:12:15.027Z", "content": "

Thanks for sharing @cdeotte We haven't started pseudo label yet. Does it work for you?

", "rawMarkdown": "Thanks for sharing @cdeotte We haven't started pseudo label yet. Does it work for you?", "votes": 1 }, { "id": 2696088, "postDate": "2024-03-14T04:16:43.233Z", "content": "
\n

Does it work for you?

\n
\n

In some situations, yes (boost CV LB). In some situations no (hurt CV LB). Note we must be careful to compute the correct (leak free) CV score when using pseudo label (i.e. be careful to maintain different pseudo label for each fold).

", "rawMarkdown": ">Does it work for you?\n\nIn some situations, yes (boost CV LB). In some situations no (hurt CV LB). Note we must be careful to compute the correct (leak free) CV score when using pseudo label (i.e. be careful to maintain different pseudo label for each fold).", "votes": 9 }, { "id": 2696313, "postDate": "2024-03-14T08:27:39.630Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2699264, "postDate": "2024-03-15T20:58:24.760Z", "content": "

I have tried the pseudo-labelling..it is improve the local CV. but LB gets worse.

\n
    \n
  1. I have stage1(previously trained) model to predict the satge2 data(votes > 10). then combined data, retrained the model the stage 1 model.
    2. \n
  2. Then I have the stage 2 model to predict the stage 1 data(votes < 10). then I have retrained the model(stage2).
    \nMy local CV 0.28 but LB is 0.33
    \nIs there anything am I missing?…if leak is there in the CV…how to avoid it…could you pls suggest.
    3. \n
", "rawMarkdown": "I have tried the pseudo-labelling..it is improve the local CV. but LB gets worse. \n\n1. I have stage1(previously trained) model to predict the satge2 data(votes > 10). then combined data, retrained the model the stage 1 model. \n2. Then I have the stage 2 model to predict the stage 1 data(votes < 10). then I have retrained the model(stage2).\nMy local CV 0.28 but LB is 0.33 \nIs there anything am I missing?...if leak is there in the CV...how to avoid it...could you pls suggest. " }, { "id": 2699899, "postDate": "2024-03-16T08:15:30.493Z", "content": "

@xaviertce I met the same issue, you could refer to Chris's comment. Note in this competion which highgly related to label distribution, normal OOF will leak. Because your OOF label, for example fold 1 label in stage 2 to predict on fold 0, leaked because fold1 OOF label train data has fold 0 so this label has label distribution info of fold0 already, you could use this ps label only and see local cv improve a lot but online a bit worse then before, no improvment.
\nI have not tried yet but seems a bit complex, we might need to use more folds, if you are using 4 folds, now you might try 8 folds and for each fold you will get 7 OOF label like for fold 0 have 7 OOF 1-7, where 1 means training on Folds without (0,1), 2 means training on Folds without(0,2).
\nI am not sure if ps will help a lot here, it will help only if our model strong enough, it could predict on votes<10 dataset more accurate then the labelers.

", "rawMarkdown": "@xaviertce I met the same issue, you could refer to Chris's comment. Note in this competion which highgly related to label distribution, normal OOF will leak. Because your OOF label, for example fold 1 label in stage 2 to predict on fold 0, leaked because fold1 OOF label train data has fold 0 so this label has label distribution info of fold0 already, you could use this ps label only and see local cv improve a lot but online a bit worse then before, no improvment.\nI have not tried yet but seems a bit complex, we might need to use more folds, if you are using 4 folds, now you might try 8 folds and for each fold you will get 7 OOF label like for fold 0 have 7 OOF 1-7, where 1 means training on Folds without (0,1), 2 means training on Folds without(0,2).\nI am not sure if ps will help a lot here, it will help only if our model strong enough, it could predict on votes<10 dataset more accurate then the labelers.", "votes": 2 } ] }, { "id": 2699132, "postDate": "2024-03-15T19:20:57.563Z", "content": "

@cdeotte how could you apply pseudo-labelling here? I tried and did awfully bad. My idea was training a model for some epochs on total_voters >= 10 then infer on total_voters < 10. Most pseudo-labelling techniques I know (usually for binary classification) imply converting soft labels into hard labels at some point which kinda defeats the purpose since here we want a model that makes predictions on a particular distribution.

", "rawMarkdown": "@cdeotte how could you apply pseudo-labelling here? I tried and did awfully bad. My idea was training a model for some epochs on `total_voters >= 10` then infer on `total_voters < 10`. Most pseudo-labelling techniques I know (usually for binary classification) imply converting soft labels into hard labels at some point which kinda defeats the purpose since here we want a model that makes predictions on a particular distribution." } ] }, { "id": 2697515, "postDate": "2024-03-14T23:59:22.407Z", "content": "

Current my best result of 3/5 fold & 3 seed ensemble(x9 models):

\n", "rawMarkdown": "Current my best result of 3/5 fold & 3 seed ensemble(x9 models):\n\n* 1D (Raw EEG) -> ~~LB=0.28~~ LB=0.26\n* 2D (EEG Spectrogram w/o Kaggle spectrogram) -> LB=~~0.31~~ LB=0.27\n* Ensemble(1D + 2D) -> ~~LB=0.27~~ LB=0.24", "votes": 11, "replies": [ { "id": 2703207, "postDate": "2024-03-18T04:21:23.897Z", "content": "

Great job. It's interesting how much you are gaining from the ensemble. Is the improvement same on your cv score?

", "rawMarkdown": "Great job. It's interesting how much you are gaining from the ensemble. Is the improvement same on your cv score?", "replies": [ { "id": 2703319, "postDate": "2024-03-18T05:22:04.603Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2704690, "postDate": "2024-03-18T21:36:53.027Z", "content": "

You mean difference between 1D/2D and their ensemble? Yes. CVs and LBs are highly correlated (R=0.924 for 5 submissions).

", "rawMarkdown": "You mean difference between 1D/2D and their ensemble? Yes. CVs and LBs are highly correlated (R=0.924 for 5 submissions). " } ] }, { "id": 2720598, "postDate": "2024-03-28T13:21:43.667Z", "content": "

Great job! Especially good score for raw eeg model. I could not make 1d model work…🤣.

", "rawMarkdown": "Great job! Especially good score for raw eeg model. I could not make 1d model work...🤣. ", "votes": 2, "replies": [ { "id": 2720998, "postDate": "2024-03-28T17:45:41.093Z", "content": "

@goldenlock Thanks. I can't believe you don't even use 1d model with ensemble and placed 9th on LB. I'm looking forward to see your solution.

", "rawMarkdown": "@goldenlock Thanks. I can't believe you don't even use 1d model with ensemble and placed 9th on LB. I'm looking forward to see your solution.", "votes": 1 }, { "id": 2726614, "postDate": "2024-04-01T09:36:31.757Z", "content": "

@goldenlock Are you willing to team up with us?

\n

However, you currently seems to not open up DM setting.
\nIf you are interested, please open up DM setting. And we will discuss by there.

\n

We are very sorry for late invitation, but the final merge deadline is April 1 23:59 GMT, so quick reply is needed.

\n

Looking forward to your reply.

", "rawMarkdown": "@goldenlock Are you willing to team up with us? \n\nHowever, you currently seems to not open up DM setting.\nIf you are interested, please open up DM setting. And we will discuss by there.\n\nWe are very sorry for late invitation, but the final merge deadline is April 1 23:59 GMT, so quick reply is needed.\n\nLooking forward to your reply." }, { "id": 2726652, "postDate": "2024-04-01T10:17:06.003Z", "content": "

@tatamikenn Thanks for inviting me, I would not teamup in this competion, and also seems we might exeed total submissions:) Good luck!

", "rawMarkdown": "@tatamikenn Thanks for inviting me, I would not teamup in this competion, and also seems we might exeed total submissions:) Good luck!" }, { "id": 2726664, "postDate": "2024-04-01T10:22:41.027Z", "content": "

@goldenlock Sure. Thanks for quick reply. Good luck, too!

", "rawMarkdown": "@goldenlock Sure. Thanks for quick reply. Good luck, too!" } ] } ] }, { "id": 2690595, "postDate": "2024-03-10T16:59:06.050Z", "content": "

Raw eeg custom network LB: 0.340X.

\n

Update: 0.32x (single stage)

", "rawMarkdown": "Raw eeg custom network LB: 0.340X.\n\nUpdate: 0.32x (single stage)", "votes": 11, "replies": [ { "id": 2720225, "postDate": "2024-03-28T07:24:03.637Z", "content": "

Is it because of some advanced tricks you have employed in preprocessing or superior architecture. It's perfectly fine if you don't want to give away some hints

", "rawMarkdown": "Is it because of some advanced tricks you have employed in preprocessing or superior architecture. It's perfectly fine if you don't want to give away some hints" } ] }, { "id": 2720582, "postDate": "2024-03-28T13:16:01.530Z", "content": "

A bit late, but for 5-fold GKF on total_votes >= 10.

\n

Single Model: CV: 0.23, LB: 0.24
\nEnsemble: CV: 0.21, LB: 0.22

", "rawMarkdown": "A bit late, but for 5-fold GKF on total_votes >= 10.\n\nSingle Model: CV: 0.23, LB: 0.24\nEnsemble: CV: 0.21, LB: ~~0.23~~ 0.22", "votes": 7, "replies": [ { "id": 2722532, "postDate": "2024-03-29T16:21:13.780Z", "content": "

Thanks for sharing.
\nHow do you evaluate ensemble cv score ?
\nJust average each fold score for all model?

", "rawMarkdown": "Thanks for sharing.\nHow do you evaluate ensemble cv score ?\nJust average each fold score for all model?", "votes": 1, "replies": [ { "id": 2723038, "postDate": "2024-03-29T23:51:30.210Z", "content": "

All OOF predictions are saved, and then KLdiv is taken over the complete validation set. The result is almost identical to the average across folds.

", "rawMarkdown": "All OOF predictions are saved, and then KLdiv is taken over the complete validation set. The result is almost identical to the average across folds.", "votes": 1 }, { "id": 2723081, "postDate": "2024-03-30T00:37:40.867Z", "content": "

Do you use custom CV scheme or just filtering vote >9？

", "rawMarkdown": "Do you use custom CV scheme or just filtering vote >9？" } ] }, { "id": 2723222, "postDate": "2024-03-30T03:58:12.183Z", "content": "

Thank you for sharing good result.
\nHow many epochs have you set on 2stage?
\nAnd your CV score is calculated using only data of total vote >=10?

", "rawMarkdown": "Thank you for sharing good result.\nHow many epochs have you set on 2stage?\nAnd your CV score is calculated using only data of total vote >=10?", "replies": [ { "id": 2724030, "postDate": "2024-03-30T16:45:05.517Z", "content": "

Yes, 5-fold GKF on total_votes >= 10.

", "rawMarkdown": "Yes, 5-fold GKF on total_votes >= 10.", "votes": 1 } ] } ] }, { "id": 2689229, "postDate": "2024-03-09T18:50:30.573Z", "content": "

Update: Image 0.31, raw 0.29, combo 0.28

", "rawMarkdown": "~~Image 0.33, raw 0.32, combo 0.32\n~~Update: Image 0.31, raw 0.29, combo 0.28", "votes": 7 }, { "id": 2689794, "postDate": "2024-03-10T06:01:12.190Z", "content": "

Single model, no two-stage, Group kfold by patient id, CV 0.48-LB 0.33.

", "rawMarkdown": "Single model, no two-stage, Group kfold by patient id, CV 0.48-LB 0.33.", "votes": 6 }, { "id": 2689400, "postDate": "2024-03-09T20:19:25.247Z", "content": "

Signal model (two-stage train)
\nEfficientNetB2 -> CV 0.3186 - LB 0.32

", "rawMarkdown": "Signal model (two-stage train)\nEfficientNetB2 -> CV 0.3186 - LB 0.32 ", "votes": 3, "replies": [ { "id": 2689420, "postDate": "2024-03-09T20:34:56.940Z", "content": "

How are you doing CV? Not sure I have seen any 2 stage models with good CV yet! Great work!

", "rawMarkdown": "How are you doing CV? Not sure I have seen any 2 stage models with good CV yet! Great work! ", "votes": 1, "replies": [ { "id": 2689647, "postDate": "2024-03-10T03:22:01.683Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2689699, "postDate": "2024-03-10T04:38:04.673Z", "content": "

Cv only for stage 2 data.. Not for full training data..since the public lb is similar to stage 2 training data it seems. So LB is good.. Not sure about private LB

", "rawMarkdown": "Cv only for stage 2 data.. Not for full training data..since the public lb is similar to stage 2 training data it seems. So LB is good.. Not sure about private LB", "votes": 1 }, { "id": 2691019, "postDate": "2024-03-11T00:30:34.143Z", "content": "

I would guess PB is the same as LB😃

", "rawMarkdown": "I would guess PB is the same as LB😃", "votes": 2 }, { "id": 2691101, "postDate": "2024-03-11T03:56:28.610Z", "content": "

So chances of shake up is less, since most of the good score public LB is two stage training.

", "rawMarkdown": "So chances of shake up is less, since most of the good score public LB is two stage training.", "votes": 1 }, { "id": 2691905, "postDate": "2024-03-11T14:22:54.233Z", "content": "

Unless data is closer to the training data than the public data. In which case the top would fall behind significantly

", "rawMarkdown": "Unless data is closer to the training data than the public data. In which case the top would fall behind significantly ", "votes": 2 } ] } ] }, { "id": 2688837, "postDate": "2024-03-09T13:56:50.203Z", "content": "

Thanks for sharing! I have actually been seeing different things in my testing which has me curious. In many of the examples I have worked with the smaller models preform the best. I think my best solo model is 0.33 and then 0.36 with the efficientnet models

", "rawMarkdown": "Thanks for sharing! I have actually been seeing different things in my testing which has me curious. In many of the examples I have worked with the smaller models preform the best. I think my best solo model is 0.33 and then 0.36 with the efficientnet models ", "votes": 3 }, { "id": 2688789, "postDate": "2024-03-09T13:16:13.987Z", "content": "

The best LB depends on the method of preprocessing of signals/eeg graghs, feature engineering, augments and finetuning hyperparameters and layers, etc, more than default sota itself. What you list are far from the pure/default pretrain sota itself. Different neural networks, GBDTs or others focus on different tasks, and all of them can reach good LB as I tried (I'm trying reaching better LB).

\n

For example, I can reach 0.34 using lightgbm. In the beginning of the competition, I tried RegNet and got 0.4, but it cost too much time.

\n

I can tell that best single model is below 0.3, but the limit I don't know.

", "rawMarkdown": "The best LB depends on the method of preprocessing of signals/eeg graghs, feature engineering, augments and finetuning hyperparameters and layers, etc, more than default sota itself. What you list are far from the pure/default pretrain sota itself. Different neural networks, GBDTs or others focus on different tasks, and all of them can reach good LB as I tried (I'm trying reaching better LB).\n\nFor example, I can reach 0.34 using lightgbm. In the beginning of the competition, I tried RegNet and got 0.4, but it cost too much time.\n\nI can tell that best single model is below 0.3, but the limit I don't know.", "votes": 3, "replies": [ { "id": 2688848, "postDate": "2024-03-09T14:02:41.960Z", "content": "

Signal model (two-stage train):
\nKaggle spec. -> LB 0.34 (EfficientNetB0)
\nKaggle raw eeg -> LB 0.36 (EfficientNetB0)

", "rawMarkdown": "Signal model (two-stage train):\nKaggle spec. -> LB 0.34 (EfficientNetB0)\nKaggle raw eeg -> LB 0.36 (EfficientNetB0)", "votes": 4, "replies": [ { "id": 2695071, "postDate": "2024-03-13T13:07:54.477Z", "content": "

great results, so your best score is ensemble of various models @sunyuri ?

", "rawMarkdown": "great results, so your best score is ensemble of various models @sunyuri ?" }, { "id": 2702198, "postDate": "2024-03-17T13:21:25.080Z", "content": "

Single model with multiple inputs (spec., raw eeg, and stft)

", "rawMarkdown": "Single model with multiple inputs (spec., raw eeg, and stft)" } ] }, { "id": 2688867, "postDate": "2024-03-09T14:24:01.133Z", "content": "

Thanks for the advice! I'm still learning and sorry for any confusion caused by my post🫡

", "rawMarkdown": "Thanks for the advice! I'm still learning and sorry for any confusion caused by my post🫡", "replies": [ { "id": 2689787, "postDate": "2024-03-10T05:48:58.967Z", "content": "

Not at all and It’s not an issue. I just think most sota can reach good score if using proper strategies.

", "rawMarkdown": "Not at all and It’s not an issue. I just think most sota can reach good score if using proper strategies." } ] } ] }, { "id": 2693610, "postDate": "2024-03-12T14:56:08.757Z", "content": "

CV=0.30 LB = 0.30.
\nCurrently my best score is single model.

", "rawMarkdown": "CV=0.30 LB = 0.30.\nCurrently my best score is single model.", "votes": 1, "replies": [ { "id": 2694048, "postDate": "2024-03-12T20:14:03.057Z", "content": "

The CV , LB correlation is awesome, congratulations

", "rawMarkdown": "The CV , LB correlation is awesome, congratulations", "votes": 2 } ] }, { "id": 2691233, "postDate": "2024-03-11T06:06:26.767Z", "content": "

happy to see your sharing😀

", "rawMarkdown": "happy to see your sharing😀", "votes": 1 }, { "id": 2689659, "postDate": "2024-03-10T03:39:16.517Z", "content": "

spectrograms: LB 0.34 (EfficientNetB0)>0.34 (Convnext-atto).
\nSignal: LB 0.36(1DResnet_GRU).

", "rawMarkdown": "spectrograms: LB 0.34 (EfficientNetB0)>0.34 (Convnext-atto).\nSignal: LB 0.36(1DResnet_GRU).", "votes": 1, "replies": [ { "id": 2697620, "postDate": "2024-03-15T03:00:01.630Z", "content": "

mel + STFT spectrograms (Convnext-atto) ->0.32

", "rawMarkdown": "mel + STFT spectrograms (Convnext-atto) ->0.32", "votes": 2 } ] }, { "id": 2705526, "postDate": "2024-03-19T11:52:50.727Z", "content": "

inspiring post😛

", "rawMarkdown": "inspiring post😛" }, { "id": 2702112, "postDate": "2024-03-17T11:55:02.557Z", "content": "

So far my single best model is EffecientNetB2 trained on kaggle's and EGG's spectrograms.
\nIt achieves 0.36 LB, it can be inferenced here

", "rawMarkdown": "So far my single best model is EffecientNetB2 trained on kaggle's and EGG's spectrograms.\nIt achieves 0.36 LB, it can be inferenced [here](https://www.kaggle.com/code/nartaa/inference-features-head-starter-k-e-ke)" }, { "id": 2694312, "postDate": "2024-03-13T01:52:27.247Z", "content": "

mixnet eeg spectrogram LB:0.34

", "rawMarkdown": "mixnet eeg spectrogram LB:0.34" }, { "id": 2689751, "postDate": "2024-03-10T05:37:16.693Z", "content": "

Thanks for the info, so does EfficientNetB7 performs better then EfficientNetB0 or have you done any preprocessing and modifications in EfficientNetB7 which lead to its better score?

", "rawMarkdown": "Thanks for the info, so does EfficientNetB7 performs better then EfficientNetB0 or have you done any preprocessing and modifications in EfficientNetB7 which lead to its better score?", "replies": [ { "id": 2689892, "postDate": "2024-03-10T07:26:11.567Z", "content": "

I think mine is just a baseline B7 model created from timm. I used Adam optimizer, lr 1e-3, 5 epoch for the first stage of training, lr 1e-5 , 3 epoch for the second stage. For the training data, I used Kaggle's spectrograms + spectrograms created using Chris Deotte's code and only used xy masking for data augmentation. B7 actually has a much worse CV than B4, so I'm a bit surprised to see it runs 0.36 on LB.

", "rawMarkdown": "I think mine is just a baseline B7 model created from timm. I used Adam optimizer, lr 1e-3, 5 epoch for the first stage of training, lr 1e-5 , 3 epoch for the second stage. For the training data, I used Kaggle's spectrograms + spectrograms created using Chris Deotte's code and only used xy masking for data augmentation. B7 actually has a much worse CV than B4, so I'm a bit surprised to see it runs 0.36 on LB." } ] } ], "comments": [ { "id": 2696315, "author_name": "gezi", "author_url": "", "post_date": "2024-03-14T08:29:08.717000", "content": "

Single model 4 folds with stage2
\n
\n
\n
\n
\nupdate LB 0.23 with CV 0.232
\nBasicly stage2 cv and LB match well, my suggestion is to trust your vote >= 10 cv only. vote < 10 data is noise, label unconfident. \nFor myself I might choose 1 submission base on best LB and another on best local cv for votes >= 10.

", "votes": 25, "replies": [ { "id": 2720450, "author_name": "Priyanshu Chaudhary", "author_url": "", "post_date": "2024-03-28T11:05:21.400000", "content": "

by choosing best LB, best cv as your final submission, does it mean your best CV and best LB models are different?

", "votes": 0, "replies": [ { "id": 2720591, "author_name": "gezi", "author_url": "", "post_date": "2024-03-28T13:19:23.810000", "content": "

They match mostly with a little difference, I would bet on using votes >=10, so I will choose 2 only from metrics perform well on votes >= 10.
\nIf you do cv locally you might still see a little model performance diff on different folds, as well as on LB, though not much gap.

", "votes": 3, "replies": [] } ] } ] }, { "id": 2689661, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-03-10T03:45:07.440000", "content": "

Raw EEG -> 0.27
\nEEG Spectrogram -> 0.26
\nKaggle Spectrogram -> 0.35

\n

Edit: EEG spectrogram is 0.24 now.
\nEdit: Best single raw EEG model is 0.26 now.

", "votes": 25, "replies": [ { "id": 2690330, "author_name": "chemdatafarmer", "author_url": "", "post_date": "2024-03-10T13:06:37.370000", "content": "

It's pretty interesting to compare the kaggle spectrograms vs \"homemade\" spectrograms. Excellent raw EEG model! I'm still tuning mine and I've found a couple augmentations that get me to about 0.44, but i still have a lot of work to do.

", "votes": 1, "replies": [ { "id": 2690505, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-03-10T15:39:30.633000", "content": "

Thanks. It actually makes lot of sense since center 10 seconds is 5 pixels on kaggle spectrograms, but 1/5 of pixels on eeg spectrograms. No wonder why kaggle spectrograms aren't performing good.

", "votes": 2, "replies": [] }, { "id": 2690518, "author_name": "chemdatafarmer", "author_url": "", "post_date": "2024-03-10T16:01:09.230000", "content": "

Well now that you put it that way, it seems obvious in hindsight haha. Very nice! I'm spending the rest of the competition working directly with EEG data since my last two competitions were image competitions and this is a tad different. I'm a little surprised transformers haven't been more effective in this context, but so far I've failed to get any decent ones. Only Conv1D based models have been fruitful so far (haven't tried any graph based networks yet).

", "votes": 1, "replies": [] } ] }, { "id": 2690408, "author_name": "SeshuRaju 🧘‍♂️", "author_url": "", "post_date": "2024-03-10T14:10:17.847000", "content": "

Thanks @gunesevitan for sharing, Raw/Spectrograms achieve 0.27/0.26 motivate to explore data more then model choice. is my assumption right?

", "votes": 1, "replies": [ { "id": 2690507, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-03-10T15:43:05.100000", "content": "

You can explore both models and preprocessing.

", "votes": 1, "replies": [] }, { "id": 2690715, "author_name": "SeshuRaju 🧘‍♂️", "author_url": "", "post_date": "2024-03-10T18:32:19.573000", "content": "

@gunesevitan what your cv for Raw/Spectrograms achieve 0.27/0.26

", "votes": 0, "replies": [] } ] }, { "id": 2691525, "author_name": "Aindriú", "author_url": "", "post_date": "2024-03-11T09:42:05.927000", "content": "

great result! looking forward to seeing the pipeline.

", "votes": 0, "replies": [] }, { "id": 2694326, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2024-03-13T02:03:15.760000", "content": "

fascinating @gunesevitan , I believe preprocessing matters greatly.

", "votes": 0, "replies": [] }, { "id": 2695803, "author_name": "Chris Deotte", "author_url": "", "post_date": "2024-03-13T21:33:07.803000", "content": "

Great single model @gunesevitan and team! 🔥 My best single model is LB=0.25, then ensemble to LB=0.24.

", "votes": 4, "replies": [] } ] }, { "id": 2695805, "author_name": "Chris Deotte", "author_url": "", "post_date": "2024-03-13T21:34:20.680000", "content": "

I'm excited to achieve single model LB = 0.25 using all data, both Spectrogram and EEG.
\nUPDATE Single model LB = 0.24, woohoo!

", "votes": 18, "replies": [ { "id": 2695844, "author_name": "Mohamed Eltayeb", "author_url": "", "post_date": "2024-03-13T21:57:07.990000", "content": "

do you use 2-stages model if you don't mind me asking?

", "votes": 2, "replies": [ { "id": 2695905, "author_name": "Chris Deotte", "author_url": "", "post_date": "2024-03-14T00:08:42.360000", "content": "

I use many stages such as pretrain, knowledge distilitation, pseudo label, finetune, etc.

", "votes": 27, "replies": [] }, { "id": 2696084, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-03-14T04:12:15.027000", "content": "

Thanks for sharing @cdeotte We haven't started pseudo label yet. Does it work for you?

", "votes": 1, "replies": [] }, { "id": 2696088, "author_name": "Chris Deotte", "author_url": "", "post_date": "2024-03-14T04:16:43.233000", "content": "
\n

Does it work for you?

\n
\n

In some situations, yes (boost CV LB). In some situations no (hurt CV LB). Note we must be careful to compute the correct (leak free) CV score when using pseudo label (i.e. be careful to maintain different pseudo label for each fold).

", "votes": 9, "replies": [] }, { "id": 2696313, "author_name": "", "author_url": "", "post_date": "2024-03-14T08:27:39.630000", "content": "", "votes": 0, "replies": [] }, { "id": 2699264, "author_name": "xavier fernando", "author_url": "", "post_date": "2024-03-15T20:58:24.760000", "content": "

I have tried the pseudo-labelling..it is improve the local CV. but LB gets worse.

\n
    \n
  1. I have stage1(previously trained) model to predict the satge2 data(votes > 10). then combined data, retrained the model the stage 1 model.
    2. \n
  2. Then I have the stage 2 model to predict the stage 1 data(votes < 10). then I have retrained the model(stage2).
    \nMy local CV 0.28 but LB is 0.33
    \nIs there anything am I missing?…if leak is there in the CV…how to avoid it…could you pls suggest.
    3. \n
", "votes": 0, "replies": [] }, { "id": 2699899, "author_name": "gezi", "author_url": "", "post_date": "2024-03-16T08:15:30.493000", "content": "

@xaviertce I met the same issue, you could refer to Chris's comment. Note in this competion which highgly related to label distribution, normal OOF will leak. Because your OOF label, for example fold 1 label in stage 2 to predict on fold 0, leaked because fold1 OOF label train data has fold 0 so this label has label distribution info of fold0 already, you could use this ps label only and see local cv improve a lot but online a bit worse then before, no improvment.
\nI have not tried yet but seems a bit complex, we might need to use more folds, if you are using 4 folds, now you might try 8 folds and for each fold you will get 7 OOF label like for fold 0 have 7 OOF 1-7, where 1 means training on Folds without (0,1), 2 means training on Folds without(0,2).
\nI am not sure if ps will help a lot here, it will help only if our model strong enough, it could predict on votes<10 dataset more accurate then the labelers.

", "votes": 2, "replies": [] } ] }, { "id": 2699132, "author_name": "moth", "author_url": "", "post_date": "2024-03-15T19:20:57.563000", "content": "

@cdeotte how could you apply pseudo-labelling here? I tried and did awfully bad. My idea was training a model for some epochs on total_voters >= 10 then infer on total_voters < 10. Most pseudo-labelling techniques I know (usually for binary classification) imply converting soft labels into hard labels at some point which kinda defeats the purpose since here we want a model that makes predictions on a particular distribution.

", "votes": 0, "replies": [] } ] }, { "id": 2697515, "author_name": "Bilzard", "author_url": "", "post_date": "2024-03-14T23:59:22.407000", "content": "

Current my best result of 3/5 fold & 3 seed ensemble(x9 models):

\n", "votes": 11, "replies": [ { "id": 2703207, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-03-18T04:21:23.897000", "content": "

Great job. It's interesting how much you are gaining from the ensemble. Is the improvement same on your cv score?

", "votes": 0, "replies": [ { "id": 2703319, "author_name": "", "author_url": "", "post_date": "2024-03-18T05:22:04.603000", "content": "", "votes": 0, "replies": [] }, { "id": 2704690, "author_name": "Bilzard", "author_url": "", "post_date": "2024-03-18T21:36:53.027000", "content": "

You mean difference between 1D/2D and their ensemble? Yes. CVs and LBs are highly correlated (R=0.924 for 5 submissions).

", "votes": 0, "replies": [] } ] }, { "id": 2720598, "author_name": "gezi", "author_url": "", "post_date": "2024-03-28T13:21:43.667000", "content": "

Great job! Especially good score for raw eeg model. I could not make 1d model work…🤣.

", "votes": 2, "replies": [ { "id": 2720998, "author_name": "Bilzard", "author_url": "", "post_date": "2024-03-28T17:45:41.093000", "content": "

@goldenlock Thanks. I can't believe you don't even use 1d model with ensemble and placed 9th on LB. I'm looking forward to see your solution.

", "votes": 1, "replies": [] }, { "id": 2726614, "author_name": "Bilzard", "author_url": "", "post_date": "2024-04-01T09:36:31.757000", "content": "

@goldenlock Are you willing to team up with us?

\n

However, you currently seems to not open up DM setting.
\nIf you are interested, please open up DM setting. And we will discuss by there.

\n

We are very sorry for late invitation, but the final merge deadline is April 1 23:59 GMT, so quick reply is needed.

\n

Looking forward to your reply.

", "votes": 0, "replies": [] }, { "id": 2726652, "author_name": "gezi", "author_url": "", "post_date": "2024-04-01T10:17:06.003000", "content": "

@tatamikenn Thanks for inviting me, I would not teamup in this competion, and also seems we might exeed total submissions:) Good luck!

", "votes": 0, "replies": [] }, { "id": 2726664, "author_name": "Bilzard", "author_url": "", "post_date": "2024-04-01T10:22:41.027000", "content": "

@goldenlock Sure. Thanks for quick reply. Good luck, too!

", "votes": 0, "replies": [] } ] } ] }, { "id": 2690595, "author_name": "Giba", "author_url": "", "post_date": "2024-03-10T16:59:06.050000", "content": "

Raw eeg custom network LB: 0.340X.

\n

Update: 0.32x (single stage)

", "votes": 11, "replies": [ { "id": 2720225, "author_name": "Roy Wei", "author_url": "", "post_date": "2024-03-28T07:24:03.637000", "content": "

Is it because of some advanced tricks you have employed in preprocessing or superior architecture. It's perfectly fine if you don't want to give away some hints

", "votes": 0, "replies": [] } ] }, { "id": 2720582, "author_name": "Bartley", "author_url": "", "post_date": "2024-03-28T13:16:01.530000", "content": "

A bit late, but for 5-fold GKF on total_votes >= 10.

\n

Single Model: CV: 0.23, LB: 0.24
\nEnsemble: CV: 0.21, LB: 0.22

", "votes": 7, "replies": [ { "id": 2722532, "author_name": "Aurora_blue", "author_url": "", "post_date": "2024-03-29T16:21:13.780000", "content": "

Thanks for sharing.
\nHow do you evaluate ensemble cv score ?
\nJust average each fold score for all model?

", "votes": 1, "replies": [ { "id": 2723038, "author_name": "Bartley", "author_url": "", "post_date": "2024-03-29T23:51:30.210000", "content": "

All OOF predictions are saved, and then KLdiv is taken over the complete validation set. The result is almost identical to the average across folds.

", "votes": 1, "replies": [] }, { "id": 2723081, "author_name": "Aurora_blue", "author_url": "", "post_date": "2024-03-30T00:37:40.867000", "content": "

Do you use custom CV scheme or just filtering vote >9？

", "votes": 0, "replies": [] } ] }, { "id": 2723222, "author_name": "Haru", "author_url": "", "post_date": "2024-03-30T03:58:12.183000", "content": "

Thank you for sharing good result.
\nHow many epochs have you set on 2stage?
\nAnd your CV score is calculated using only data of total vote >=10?

", "votes": 0, "replies": [ { "id": 2724030, "author_name": "Bartley", "author_url": "", "post_date": "2024-03-30T16:45:05.517000", "content": "

Yes, 5-fold GKF on total_votes >= 10.

", "votes": 1, "replies": [] } ] } ] }, { "id": 2689229, "author_name": "Aindriú", "author_url": "", "post_date": "2024-03-09T18:50:30.573000", "content": "

Update: Image 0.31, raw 0.29, combo 0.28

", "votes": 7, "replies": [] }, { "id": 2689794, "author_name": "Quan Vu", "author_url": "", "post_date": "2024-03-10T06:01:12.190000", "content": "

Single model, no two-stage, Group kfold by patient id, CV 0.48-LB 0.33.

", "votes": 6, "replies": [] }, { "id": 2689400, "author_name": "xavier fernando", "author_url": "", "post_date": "2024-03-09T20:19:25.247000", "content": "

Signal model (two-stage train)
\nEfficientNetB2 -> CV 0.3186 - LB 0.32

", "votes": 3, "replies": [ { "id": 2689420, "author_name": "Cody_Null", "author_url": "", "post_date": "2024-03-09T20:34:56.940000", "content": "

How are you doing CV? Not sure I have seen any 2 stage models with good CV yet! Great work!

", "votes": 1, "replies": [ { "id": 2689647, "author_name": "", "author_url": "", "post_date": "2024-03-10T03:22:01.683000", "content": "", "votes": 0, "replies": [] }, { "id": 2689699, "author_name": "xavier fernando", "author_url": "", "post_date": "2024-03-10T04:38:04.673000", "content": "

Cv only for stage 2 data.. Not for full training data..since the public lb is similar to stage 2 training data it seems. So LB is good.. Not sure about private LB

", "votes": 1, "replies": [] }, { "id": 2691019, "author_name": "gezi", "author_url": "", "post_date": "2024-03-11T00:30:34.143000", "content": "

I would guess PB is the same as LB😃

", "votes": 2, "replies": [] }, { "id": 2691101, "author_name": "xavier fernando", "author_url": "", "post_date": "2024-03-11T03:56:28.610000", "content": "

So chances of shake up is less, since most of the good score public LB is two stage training.

", "votes": 1, "replies": [] }, { "id": 2691905, "author_name": "Cody_Null", "author_url": "", "post_date": "2024-03-11T14:22:54.233000", "content": "

Unless data is closer to the training data than the public data. In which case the top would fall behind significantly

", "votes": 2, "replies": [] } ] } ] }, { "id": 2688837, "author_name": "Cody_Null", "author_url": "", "post_date": "2024-03-09T13:56:50.203000", "content": "

Thanks for sharing! I have actually been seeing different things in my testing which has me curious. In many of the examples I have worked with the smaller models preform the best. I think my best solo model is 0.33 and then 0.36 with the efficientnet models

", "votes": 3, "replies": [] }, { "id": 2688789, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2024-03-09T13:16:13.987000", "content": "

The best LB depends on the method of preprocessing of signals/eeg graghs, feature engineering, augments and finetuning hyperparameters and layers, etc, more than default sota itself. What you list are far from the pure/default pretrain sota itself. Different neural networks, GBDTs or others focus on different tasks, and all of them can reach good LB as I tried (I'm trying reaching better LB).

\n

For example, I can reach 0.34 using lightgbm. In the beginning of the competition, I tried RegNet and got 0.4, but it cost too much time.

\n

I can tell that best single model is below 0.3, but the limit I don't know.

", "votes": 3, "replies": [ { "id": 2688848, "author_name": "Yuri Sun", "author_url": "", "post_date": "2024-03-09T14:02:41.960000", "content": "

Signal model (two-stage train):
\nKaggle spec. -> LB 0.34 (EfficientNetB0)
\nKaggle raw eeg -> LB 0.36 (EfficientNetB0)

", "votes": 4, "replies": [ { "id": 2695071, "author_name": "Priyanshu Chaudhary", "author_url": "", "post_date": "2024-03-13T13:07:54.477000", "content": "

great results, so your best score is ensemble of various models @sunyuri ?

", "votes": 0, "replies": [] }, { "id": 2702198, "author_name": "Yuri Sun", "author_url": "", "post_date": "2024-03-17T13:21:25.080000", "content": "

Single model with multiple inputs (spec., raw eeg, and stft)

", "votes": 0, "replies": [] } ] }, { "id": 2688867, "author_name": "fireflysentinel", "author_url": "", "post_date": "2024-03-09T14:24:01.133000", "content": "

Thanks for the advice! I'm still learning and sorry for any confusion caused by my post🫡

", "votes": 0, "replies": [ { "id": 2689787, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2024-03-10T05:48:58.967000", "content": "

Not at all and It’s not an issue. I just think most sota can reach good score if using proper strategies.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2693610, "author_name": "Priyanshu Chaudhary", "author_url": "", "post_date": "2024-03-12T14:56:08.757000", "content": "

CV=0.30 LB = 0.30.
\nCurrently my best score is single model.

", "votes": 1, "replies": [ { "id": 2694048, "author_name": "Ricardo Colomer", "author_url": "", "post_date": "2024-03-12T20:14:03.057000", "content": "

The CV , LB correlation is awesome, congratulations

", "votes": 2, "replies": [] } ] }, { "id": 2691233, "author_name": "Zhaoge Qi", "author_url": "", "post_date": "2024-03-11T06:06:26.767000", "content": "

happy to see your sharing😀

", "votes": 1, "replies": [] }, { "id": 2689659, "author_name": "ynhuhu", "author_url": "", "post_date": "2024-03-10T03:39:16.517000", "content": "

spectrograms: LB 0.34 (EfficientNetB0)>0.34 (Convnext-atto).
\nSignal: LB 0.36(1DResnet_GRU).

", "votes": 1, "replies": [ { "id": 2697620, "author_name": "ynhuhu", "author_url": "", "post_date": "2024-03-15T03:00:01.630000", "content": "

mel + STFT spectrograms (Convnext-atto) ->0.32

", "votes": 2, "replies": [] } ] }, { "id": 2705526, "author_name": "Peilin Li", "author_url": "", "post_date": "2024-03-19T11:52:50.727000", "content": "

inspiring post😛

", "votes": 0, "replies": [] }, { "id": 2702112, "author_name": "Danial Zakaria", "author_url": "", "post_date": "2024-03-17T11:55:02.557000", "content": "

So far my single best model is EffecientNetB2 trained on kaggle's and EGG's spectrograms.
\nIt achieves 0.36 LB, it can be inferenced here

", "votes": 0, "replies": [] }, { "id": 2694312, "author_name": "AI Knight", "author_url": "", "post_date": "2024-03-13T01:52:27.247000", "content": "

mixnet eeg spectrogram LB:0.34

", "votes": 0, "replies": [] }, { "id": 2689751, "author_name": "Arun", "author_url": "", "post_date": "2024-03-10T05:37:16.693000", "content": "

Thanks for the info, so does EfficientNetB7 performs better then EfficientNetB0 or have you done any preprocessing and modifications in EfficientNetB7 which lead to its better score?

", "votes": 0, "replies": [ { "id": 2689892, "author_name": "fireflysentinel", "author_url": "", "post_date": "2024-03-10T07:26:11.567000", "content": "

I think mine is just a baseline B7 model created from timm. I used Adam optimizer, lr 1e-3, 5 epoch for the first stage of training, lr 1e-5 , 3 epoch for the second stage. For the training data, I used Kaggle's spectrograms + spectrograms created using Chris Deotte's code and only used xy masking for data augmentation. B7 actually has a much worse CV than B4, so I'm a bit surprised to see it runs 0.36 on LB.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2688648": "Hello everyone, I'm a Grade 12 student from Shanghai World Foreign Language Academy, and I'm participating in a Kaggle competition for the first time. After experimenting with various models based on spectrograms, I've documented the best LB I achieved with each model. Here's a summary of my findings:\n\n| Model | LB |\n|:-----------------------:|:--------:|\n| EfficientNetB7 | 0.36 |\n| EfficientNetB6 | 0.36 |\n| EfficientNetB5 | 0.37 |\n| EfficientNetB4 | **0.36** |\n| EfficientNetB3 | 0.38 |\n| EfficientNetB2 | 0.40 |\n| EfficientNetB1 | 0.41 |\n| EfficientNetB0 | 0.43 |\n| EfficientNetB2V2 | 0.38 |\n| EfficientNetB1V2 | **0.38** |\n| Resnet152d | 0.38 |\n| Resnet101d | 0.38 | \n| Resnet50d | **0.38** |\n| Resnet34d | 0.39 |\n\nI've also experimented with different variants of ResNet50, ViT-tiny, ViT-small, MobileNet, ConvNeXt, and DenseNet, but unfortunately, they didn't yield as promising results. In addition, I'm exploring models based on raw EEG signals, although my current best with WaveNet is at LB 0.47.\n\nI'm curious to know about the best LB that single models have achieved in your experiments. Thank you for your time and assistance🙏", "2696315": "Single model 4 folds with stage2 \n~~cv 0.265 and LB 0.27~~ \n~~update cv 0.265 LB 0.26~~ \n~~update cv 0.25 LB 0.25~~ \n~~update LB 0.24 with CV 0.244~~\nupdate LB 0.23 with CV 0.232\nBasicly stage2 cv and LB match well, my suggestion is to trust your vote >= 10 cv only. vote < 10 data is noise, label unconfident. \nFor myself I might choose 1 submission base on best LB and another on best local cv for votes >= 10.", "2689661": "Raw EEG -> 0.27\nEEG Spectrogram -> 0.26\nKaggle Spectrogram -> 0.35\n\nEdit: EEG spectrogram is ~~0.25~~ 0.24 now.\nEdit: Best single raw EEG model is 0.26 now. ", "2695805": "I'm excited to achieve single model `LB = 0.25` using all data, both Spectrogram and EEG.\n**UPDATE** Single model `LB = 0.24`, woohoo!", "2697515": "Current my best result of 3/5 fold & 3 seed ensemble(x9 models):\n\n* 1D (Raw EEG) -> ~~LB=0.28~~ LB=0.26\n* 2D (EEG Spectrogram w/o Kaggle spectrogram) -> LB=~~0.31~~ LB=0.27\n* Ensemble(1D + 2D) -> ~~LB=0.27~~ LB=0.24", "2690595": "Raw eeg custom network LB: 0.340X.\n\nUpdate: 0.32x (single stage)", "2720582": "A bit late, but for 5-fold GKF on total_votes >= 10.\n\nSingle Model: CV: 0.23, LB: 0.24\nEnsemble: CV: 0.21, LB: ~~0.23~~ 0.22", "2689229": "~~Image 0.33, raw 0.32, combo 0.32\n~~Update: Image 0.31, raw 0.29, combo 0.28", "2689794": "Single model, no two-stage, Group kfold by patient id, CV 0.48-LB 0.33.", "2689400": "Signal model (two-stage train)\nEfficientNetB2 -> CV 0.3186 - LB 0.32 ", "2688837": "Thanks for sharing! I have actually been seeing different things in my testing which has me curious. In many of the examples I have worked with the smaller models preform the best. I think my best solo model is 0.33 and then 0.36 with the efficientnet models ", "2688789": "The best LB depends on the method of preprocessing of signals/eeg graghs, feature engineering, augments and finetuning hyperparameters and layers, etc, more than default sota itself. What you list are far from the pure/default pretrain sota itself. Different neural networks, GBDTs or others focus on different tasks, and all of them can reach good LB as I tried (I'm trying reaching better LB).\n\nFor example, I can reach 0.34 using lightgbm. In the beginning of the competition, I tried RegNet and got 0.4, but it cost too much time.\n\nI can tell that best single model is below 0.3, but the limit I don't know.", "2693610": "CV=0.30 LB = 0.30.\nCurrently my best score is single model.", "2691233": "happy to see your sharing😀", "2689659": "spectrograms: LB 0.34 (EfficientNetB0)>0.34 (Convnext-atto).\nSignal: LB 0.36(1DResnet_GRU).", "2705526": "inspiring post😛", "2702112": "So far my single best model is EffecientNetB2 trained on kaggle's and EGG's spectrograms.\nIt achieves 0.36 LB, it can be inferenced [here](https://www.kaggle.com/code/nartaa/inference-features-head-starter-k-e-ke)", "2694312": "mixnet eeg spectrogram LB:0.34", "2689751": "Thanks for the info, so does EfficientNetB7 performs better then EfficientNetB0 or have you done any preprocessing and modifications in EfficientNetB7 which lead to its better score?" } }