{ "id": 492207, "title": "10th solution, good input image can generate a good single model (Opensource single model with LB233 PB287)", "url": "/competitions/hms-harmful-brain-activity-classification/discussion/492207", "author_name": "gezi", "post_date": "2024-04-09T00:50:14.184000", "votes": 56, "comment_count": 24, "views": 0, "content": "

Here is quick share, I might modify later.

\n
    \n
  1. Input with more info as image could lead a good single model, with LB 0.23 and PB 0.29.
    2. \n
  2. Some important parts what I found usefull.
    3. \n
\n\n


\nbone.

", "messages": [ { "id": 2742549, "postDate": "2024-04-09T00:50:14.183Z", "content": "

Here is quick share, I might modify later.

\n
    \n
  1. Input with more info as image could lead a good single model, with LB 0.23 and PB 0.29.
    2. \n
  2. Some important parts what I found usefull.
    3. \n
\n\n


\nbone.

", "rawMarkdown": "Here is quick share, I might modify later.\n1. Input with more info as image could lead a good single model, with LB 0.23 and PB 0.29.\n2. Some important parts what I found usefull.\n- Using 19 raw inputs(diff feature) of banana instead of 4(LL,LP,RL,RP), thanks to Chirs's great share.\n- Add center 10 sencods part as parts of the image.\n(Idea orginated from https://www.kaggle.com/code/rafaelzimmermann1/no-ensemble-new-spectrograms-label-refine)\n- Add raw eeg data as parts of the image\n- Use adan optimzier\n- Use efficientvit_b2 and mixnet_xl as timm backbone\n efficientvit much better then efficientnet , cv from 0.237 to 0.23.\n- Use torchaudio with stft and mel extractor, but my best single model still from using scipy.spectrogram, not sure why better then torchaudio.stft.\n- Train with exact eeg offset for each eeg id(random eeg sub id and high votes prefered)\n- Evaluate for all eeg sub ids and normalize for each eeg_id (each eeg id total weight as 1).\nMy strategy on dealing eeg sub ids is complex comparing to Chris's share, and now I confirm my strategy work worse, you could refer to https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/492631 \n- Train with votes >= 10 data with weight 60. \n 1 stage train is ok and stage 2 with votes >= 10 only seems could improve LB and PB a bit, because it lower down seizure prediction rate, but not enough time to verify (as cv might be hurt) and not used for final ensemble.\n As votes >= 10 and votes < 10 data distribtuion is different, to address this I used different linear head for those 2 datasets, which helped a bit both on cv and LB,PB.\nlike single efficientvit_b1 model improve on PB from 0.2967 to 0.29525.\n- Finally ensemble of 20 models, actually my models does not have much diversity, ensemble only improve my best single model from PB 2896 to 2866. I added 1 weak model which using kaggle spec only. One thing intesesting is comparing to using simple mean, using optuna for generating model weights could only improve cv a litte bit like from 0.206 to 0.2058 but online LB and PB show weighted results better.\n- Notebook for infererence single model(5 folds + 1 fulldata) with LB 0.2337 PB 0.287\nhttps://www.kaggle.com/code/goldenlock/single-model-lb0-233pb0-287-allinone?scriptVersionId=171674903\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F42245%2F6beb6f1a2ac27aa6f9a38954f4b656f2%2F5AD256BF437BA0D95950F0AEEF60BF5D.png?generation=1712623701878528&alt=media)\nbone.\n", "votes": 56 }, { "id": 2742680, "postDate": "2024-04-09T02:59:27.500Z", "content": "

Interesting. I tried stacking the raw channels as you, and found that using the pairwise difference was a bit better. Did you try it?

\n

Anyway, main point is that using 4 averages lose lots of info.

\n

It is when I saw you claimed LB 0.24 with only 2D that I decided to stop trying 1D models and focus on 2D.

", "rawMarkdown": "Interesting. I tried stacking the raw channels as you, and found that using the pairwise difference was a bit better. Did you try it?\n\nAnyway, main point is that using 4 averages lose lots of info.\n\nIt is when I saw you claimed LB 0.24 with only 2D that I decided to stop trying 1D models and focus on 2D.", "votes": 1, "replies": [ { "id": 2742740, "postDate": "2024-04-09T03:51:38.167Z", "content": "

Yes I used different channels for eeg to sepc, and also for raw eeg feature, but I also add original channel of eeg, it does not affect much, diffrence of channels more important.
\nBut I still invested a lot on 1d models… though all not work for me…

", "rawMarkdown": "Yes I used different channels for eeg to sepc, and also for raw eeg feature, but I also add original channel of eeg, it does not affect much, diffrence of channels more important.\nBut I still invested a lot on 1d models... though all not work for me...", "votes": 1 } ] }, { "id": 2742775, "postDate": "2024-04-09T04:32:28.563Z", "content": "
\n

Using 19 raw inputs of banana instead of 4(LL,LP,RL,RP), thanks to Chris's great share.

\n
\n

Great idea.

", "rawMarkdown": ">Using 19 raw inputs of banana instead of 4(LL,LP,RL,RP), thanks to Chris's great share.\n\nGreat idea. ", "votes": 2 }, { "id": 2742562, "postDate": "2024-04-09T00:59:23.267Z", "content": "

Congratz, man. No hard feelings though. What was the input size to the model? What is the hardware spec, if it isn't a secret?

", "rawMarkdown": "Congratz, man. No hard feelings though. What was the input size to the model? What is the hardware spec, if it isn't a secret?", "votes": 1, "replies": [ { "id": 2742566, "postDate": "2024-04-09T01:01:13.603Z", "content": "

Input size is 1286 * 626(h*w), basicly time len is 626 and for height it is kaggle spec 400 + 19 * 40 (eeg spec) + many raw eeg features, what do you mean hardware spec, it could be run using 1 4090 gpu.

", "rawMarkdown": "Input size is 1286 * 626(h*w), basicly time len is 626 and for height it is kaggle spec 400 + 19 * 40 (eeg spec) + many raw eeg features, what do you mean hardware spec, it could be run using 1 4090 gpu.", "votes": 2, "replies": [ { "id": 2742569, "postDate": "2024-04-09T01:03:00.780Z", "content": "

that's what I meant (what it was trained on), thank you. Did you have any luck with 1d models, if yes what was the best score?

", "rawMarkdown": "that's what I meant (what it was trained on), thank you. Did you have any luck with 1d models, if yes what was the best score? " }, { "id": 2742572, "postDate": "2024-04-09T01:04:27.883Z", "content": "

No I did not ensemble any 1d models. It could not get cv lower then 0.35 for me.

", "rawMarkdown": "No I did not ensemble any 1d models. It could not get cv lower then 0.35 for me.", "votes": 1 }, { "id": 2742947, "postDate": "2024-04-09T06:37:49.757Z", "content": "

How is the 626 composed?

", "rawMarkdown": "How is the 626 composed?" }, { "id": 2742950, "postDate": "2024-04-09T06:39:59.733Z", "content": "

Output ouf stft using win_length = 10_000 // 600

", "rawMarkdown": "Output ouf stft using win_length = 10_000 // 600" } ] } ] }, { "id": 2742797, "postDate": "2024-04-09T05:09:21.580Z", "content": "

Congrats~ May I ask how you did private LB probing?

", "rawMarkdown": "Congrats~ May I ask how you did private LB probing?", "replies": [ { "id": 2742801, "postDate": "2024-04-09T05:16:26.040Z", "content": "

https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/471287
\nThough might not be advised by official…

", "rawMarkdown": "https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/471287\nThough might not be advised by official...", "replies": [ { "id": 2742848, "postDate": "2024-04-09T05:42:24.240Z", "content": "

Sorry for not making myself clear. I know how to test how many data there are in the private test, but I don't know how you came to the conclusion to trust CV with votes >= 10. I remember you posted about your approach during the competition, but I can't find it now…

", "rawMarkdown": "Sorry for not making myself clear. I know how to test how many data there are in the private test, but I don't know how you came to the conclusion to trust CV with votes >= 10. I remember you posted about your approach during the competition, but I can't find it now..." }, { "id": 2742851, "postDate": "2024-04-09T05:45:01.683Z", "content": "

Oh you could train a model without considering n_votes, just train it on whole data with equal weights, then you could check your model prediction prob for seizure and you could probe online to see if the seizure rate is more likely to your local sezure rate of all or just nvotes<=10. Another way is to check you prediction entropy , they are similar.

", "rawMarkdown": "Oh you could train a model without considering n_votes, just train it on whole data with equal weights, then you could check your model prediction prob for seizure and you could probe online to see if the seizure rate is more likely to your local sezure rate of all or just nvotes<=10. Another way is to check you prediction entropy , they are similar.", "votes": 1 } ] } ] }, { "id": 2742748, "postDate": "2024-04-09T03:55:52.307Z", "content": "

How much better was efficientvit_b2 than efficientnet?

", "rawMarkdown": "How much better was efficientvit_b2 than efficientnet?", "replies": [ { "id": 2742759, "postDate": "2024-04-09T04:02:29.510Z", "content": "

Cv change like form 0.23(efficientvit b1) to 0.237(efficientnet b0)

", "rawMarkdown": "Cv change like form 0.23(efficientvit b1) to 0.237(efficientnet b0)", "votes": 1 } ] }, { "id": 2742636, "postDate": "2024-04-09T02:19:53.803Z", "content": "

Thank you very much for sharing your insights! 🌹 Could you provide some additional details on the criteria for determining ‘votes >= 10’ with a weight of 60? Our team has also categorized data with ‘votes >= 10’ as ‘high quality’ based on the original IIIC_SPaRCNet paper, but we’re unsure of the reasoning behind choosing 10 as the threshold instead of a different number.

", "rawMarkdown": "Thank you very much for sharing your insights! 🌹 Could you provide some additional details on the criteria for determining ‘votes >= 10’ with a weight of 60? Our team has also categorized data with ‘votes >= 10’ as ‘high quality’ based on the original IIIC_SPaRCNet paper, but we’re unsure of the reasoning behind choosing 10 as the threshold instead of a different number.", "replies": [ { "id": 2742745, "postDate": "2024-04-09T03:53:53.240Z", "content": "

At first we find votes >= 10 cv score is the most similar to online LB score. Second we could try many weights to seed your local cv score of votes>=10, we found 50~60 could be helpful.

", "rawMarkdown": "At first we find votes >= 10 cv score is the most similar to online LB score. Second we could try many weights to seed your local cv score of votes>=10, we found 50~60 could be helpful.", "votes": 1 } ] }, { "id": 2742592, "postDate": "2024-04-09T01:24:32.757Z", "content": "

Congrats for the solo gold! At the first glance, it's really straightforward, effective but nothing fancy.

", "rawMarkdown": "Congrats for the solo gold! At the first glance, it's really straightforward, effective but nothing fancy. ", "replies": [ { "id": 2742749, "postDate": "2024-04-09T03:56:40.083Z", "content": "

Thanks Wei , yes nothing fancy, it is mostly a solution of Chris😀 His idea of merging kaggle spec and eeg spec bring this competion to new level.

", "rawMarkdown": "Thanks Wei , yes nothing fancy, it is mostly a solution of Chris😀 His idea of merging kaggle spec and eeg spec bring this competion to new level.", "votes": 1 } ] }, { "id": 2742574, "postDate": "2024-04-09T01:08:37.523Z", "content": "

Congrats!!!, sorry but what is banana?.

\n

Thank in advanced!

", "rawMarkdown": "Congrats!!!, sorry but what is banana?.\n\nThank in advanced!", "replies": [ { "id": 2742578, "postDate": "2024-04-09T01:11:30.383Z", "content": "

The input kaggle spec is using eeg data as banana, you could refer to https://www.learningeeg.com/montages-and-technical-components

", "rawMarkdown": "The input kaggle spec is using eeg data as banana, you could refer to https://www.learningeeg.com/montages-and-technical-components", "votes": 4 } ] }, { "id": 2742944, "postDate": "2024-04-09T06:35:43.473Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2742617, "postDate": "2024-04-09T01:53:02.983Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2742747, "postDate": "2024-04-09T03:54:52.483Z", "content": "

Just use resize, most of the raw eeg features are diff features, but also add raw channels.

", "rawMarkdown": "Just use resize, most of the raw eeg features are diff features, but also add raw channels." } ] } ], "comments": [ { "id": 2742680, "author_name": "CPMP", "author_url": "", "post_date": "2024-04-09T02:59:27.500000", "content": "

Interesting. I tried stacking the raw channels as you, and found that using the pairwise difference was a bit better. Did you try it?

\n

Anyway, main point is that using 4 averages lose lots of info.

\n

It is when I saw you claimed LB 0.24 with only 2D that I decided to stop trying 1D models and focus on 2D.

", "votes": 1, "replies": [ { "id": 2742740, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T03:51:38.167000", "content": "

Yes I used different channels for eeg to sepc, and also for raw eeg feature, but I also add original channel of eeg, it does not affect much, diffrence of channels more important.
\nBut I still invested a lot on 1d models… though all not work for me…

", "votes": 1, "replies": [] } ] }, { "id": 2742775, "author_name": "Chris Deotte", "author_url": "", "post_date": "2024-04-09T04:32:28.563000", "content": "
\n

Using 19 raw inputs of banana instead of 4(LL,LP,RL,RP), thanks to Chris's great share.

\n
\n

Great idea.

", "votes": 2, "replies": [] }, { "id": 2742562, "author_name": "SSS", "author_url": "", "post_date": "2024-04-09T00:59:23.267000", "content": "

Congratz, man. No hard feelings though. What was the input size to the model? What is the hardware spec, if it isn't a secret?

", "votes": 1, "replies": [ { "id": 2742566, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T01:01:13.603000", "content": "

Input size is 1286 * 626(h*w), basicly time len is 626 and for height it is kaggle spec 400 + 19 * 40 (eeg spec) + many raw eeg features, what do you mean hardware spec, it could be run using 1 4090 gpu.

", "votes": 2, "replies": [ { "id": 2742569, "author_name": "SSS", "author_url": "", "post_date": "2024-04-09T01:03:00.780000", "content": "

that's what I meant (what it was trained on), thank you. Did you have any luck with 1d models, if yes what was the best score?

", "votes": 0, "replies": [] }, { "id": 2742572, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T01:04:27.883000", "content": "

No I did not ensemble any 1d models. It could not get cv lower then 0.35 for me.

", "votes": 1, "replies": [] }, { "id": 2742947, "author_name": "BladeRunner", "author_url": "", "post_date": "2024-04-09T06:37:49.757000", "content": "

How is the 626 composed?

", "votes": 0, "replies": [] }, { "id": 2742950, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T06:39:59.733000", "content": "

Output ouf stft using win_length = 10_000 // 600

", "votes": 0, "replies": [] } ] } ] }, { "id": 2742797, "author_name": "Wisp Vale", "author_url": "", "post_date": "2024-04-09T05:09:21.580000", "content": "

Congrats~ May I ask how you did private LB probing?

", "votes": 0, "replies": [ { "id": 2742801, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T05:16:26.040000", "content": "

https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/471287
\nThough might not be advised by official…

", "votes": 0, "replies": [ { "id": 2742848, "author_name": "Wisp Vale", "author_url": "", "post_date": "2024-04-09T05:42:24.240000", "content": "

Sorry for not making myself clear. I know how to test how many data there are in the private test, but I don't know how you came to the conclusion to trust CV with votes >= 10. I remember you posted about your approach during the competition, but I can't find it now…

", "votes": 0, "replies": [] }, { "id": 2742851, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T05:45:01.683000", "content": "

Oh you could train a model without considering n_votes, just train it on whole data with equal weights, then you could check your model prediction prob for seizure and you could probe online to see if the seizure rate is more likely to your local sezure rate of all or just nvotes<=10. Another way is to check you prediction entropy , they are similar.

", "votes": 1, "replies": [] } ] } ] }, { "id": 2742748, "author_name": "Bartley", "author_url": "", "post_date": "2024-04-09T03:55:52.307000", "content": "

How much better was efficientvit_b2 than efficientnet?

", "votes": 0, "replies": [ { "id": 2742759, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T04:02:29.510000", "content": "

Cv change like form 0.23(efficientvit b1) to 0.237(efficientnet b0)

", "votes": 1, "replies": [] } ] }, { "id": 2742636, "author_name": "fireflysentinel", "author_url": "", "post_date": "2024-04-09T02:19:53.803000", "content": "

Thank you very much for sharing your insights! 🌹 Could you provide some additional details on the criteria for determining ‘votes >= 10’ with a weight of 60? Our team has also categorized data with ‘votes >= 10’ as ‘high quality’ based on the original IIIC_SPaRCNet paper, but we’re unsure of the reasoning behind choosing 10 as the threshold instead of a different number.

", "votes": 0, "replies": [ { "id": 2742745, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T03:53:53.240000", "content": "

At first we find votes >= 10 cv score is the most similar to online LB score. Second we could try many weights to seed your local cv score of votes>=10, we found 50~60 could be helpful.

", "votes": 1, "replies": [] } ] }, { "id": 2742592, "author_name": "Roy Wei", "author_url": "", "post_date": "2024-04-09T01:24:32.757000", "content": "

Congrats for the solo gold! At the first glance, it's really straightforward, effective but nothing fancy.

", "votes": 0, "replies": [ { "id": 2742749, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T03:56:40.083000", "content": "

Thanks Wei , yes nothing fancy, it is mostly a solution of Chris😀 His idea of merging kaggle spec and eeg spec bring this competion to new level.

", "votes": 1, "replies": [] } ] }, { "id": 2742574, "author_name": "Pablo Larrosa", "author_url": "", "post_date": "2024-04-09T01:08:37.523000", "content": "

Congrats!!!, sorry but what is banana?.

\n

Thank in advanced!

", "votes": 0, "replies": [ { "id": 2742578, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T01:11:30.383000", "content": "

The input kaggle spec is using eeg data as banana, you could refer to https://www.learningeeg.com/montages-and-technical-components

", "votes": 4, "replies": [] } ] }, { "id": 2742944, "author_name": "", "author_url": "", "post_date": "2024-04-09T06:35:43.473000", "content": "", "votes": 0, "replies": [] }, { "id": 2742617, "author_name": "", "author_url": "", "post_date": "2024-04-09T01:53:02.983000", "content": "", "votes": 0, "replies": [ { "id": 2742747, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T03:54:52.483000", "content": "

Just use resize, most of the raw eeg features are diff features, but also add raw channels.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2742549": "Here is quick share, I might modify later.\n1. Input with more info as image could lead a good single model, with LB 0.23 and PB 0.29.\n2. Some important parts what I found usefull.\n- Using 19 raw inputs(diff feature) of banana instead of 4(LL,LP,RL,RP), thanks to Chirs's great share.\n- Add center 10 sencods part as parts of the image.\n(Idea orginated from https://www.kaggle.com/code/rafaelzimmermann1/no-ensemble-new-spectrograms-label-refine)\n- Add raw eeg data as parts of the image\n- Use adan optimzier\n- Use efficientvit_b2 and mixnet_xl as timm backbone\n efficientvit much better then efficientnet , cv from 0.237 to 0.23.\n- Use torchaudio with stft and mel extractor, but my best single model still from using scipy.spectrogram, not sure why better then torchaudio.stft.\n- Train with exact eeg offset for each eeg id(random eeg sub id and high votes prefered)\n- Evaluate for all eeg sub ids and normalize for each eeg_id (each eeg id total weight as 1).\nMy strategy on dealing eeg sub ids is complex comparing to Chris's share, and now I confirm my strategy work worse, you could refer to https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/492631 \n- Train with votes >= 10 data with weight 60. \n 1 stage train is ok and stage 2 with votes >= 10 only seems could improve LB and PB a bit, because it lower down seizure prediction rate, but not enough time to verify (as cv might be hurt) and not used for final ensemble.\n As votes >= 10 and votes < 10 data distribtuion is different, to address this I used different linear head for those 2 datasets, which helped a bit both on cv and LB,PB.\nlike single efficientvit_b1 model improve on PB from 0.2967 to 0.29525.\n- Finally ensemble of 20 models, actually my models does not have much diversity, ensemble only improve my best single model from PB 2896 to 2866. I added 1 weak model which using kaggle spec only. One thing intesesting is comparing to using simple mean, using optuna for generating model weights could only improve cv a litte bit like from 0.206 to 0.2058 but online LB and PB show weighted results better.\n- Notebook for infererence single model(5 folds + 1 fulldata) with LB 0.2337 PB 0.287\nhttps://www.kaggle.com/code/goldenlock/single-model-lb0-233pb0-287-allinone?scriptVersionId=171674903\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F42245%2F6beb6f1a2ac27aa6f9a38954f4b656f2%2F5AD256BF437BA0D95950F0AEEF60BF5D.png?generation=1712623701878528&alt=media)\nbone.\n", "2742680": "Interesting. I tried stacking the raw channels as you, and found that using the pairwise difference was a bit better. Did you try it?\n\nAnyway, main point is that using 4 averages lose lots of info.\n\nIt is when I saw you claimed LB 0.24 with only 2D that I decided to stop trying 1D models and focus on 2D.", "2742775": ">Using 19 raw inputs of banana instead of 4(LL,LP,RL,RP), thanks to Chris's great share.\n\nGreat idea. ", "2742562": "Congratz, man. No hard feelings though. What was the input size to the model? What is the hardware spec, if it isn't a secret?", "2742797": "Congrats~ May I ask how you did private LB probing?", "2742748": "How much better was efficientvit_b2 than efficientnet?", "2742636": "Thank you very much for sharing your insights! 🌹 Could you provide some additional details on the criteria for determining ‘votes >= 10’ with a weight of 60? Our team has also categorized data with ‘votes >= 10’ as ‘high quality’ based on the original IIIC_SPaRCNet paper, but we’re unsure of the reasoning behind choosing 10 as the threshold instead of a different number.", "2742592": "Congrats for the solo gold! At the first glance, it's really straightforward, effective but nothing fancy. ", "2742574": "Congrats!!!, sorry but what is banana?.\n\nThank in advanced!", "2742944": "", "2742617": "" } }