![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6343664%2Fe7c114b13fa506bbcc7ced308821b9b1%2FScreenshot%20from%202024-04-09%2019-21-12.png?generation=1712679944774259&alt=media\")
One of my models that provided the best LB score (~0.25) was efficientnet 3 trained on images that consisted of the following elements:
\nRaw eeg models:
\nOther 2D models were trained on images made via 1d resnet blocks and aggresive 1d convolutions with large strides.
\n1D model consisted of 1D resnet blocks with different kernel sizes + gru.
\nFor models trained on raw eegs, I have noticed that clipping was crucial for good convergence (-256 - 256) as well as bandpass filter 0.5-20 Hz. Also, it was beneficial to use the same weights for each signal in encoders.
Augmentations that worked for me:
\nCutMix on half signals(right in the middle cut)
\nAdd uniform noise to labels with higher than zero value(the range of uniform distribution depending on the number of votes, the more votes the more reliable labels we have, so less noise is allowed)
One of my models that provided the best LB score (~0.25) was efficientnet 3 trained on images that consisted of the following elements:
\nRaw eeg models:
\nOther 2D models were trained on images made via 1d resnet blocks and aggresive 1d convolutions with large strides.
\n1D model consisted of 1D resnet blocks with different kernel sizes + gru.
\nFor models trained on raw eegs, I have noticed that clipping was crucial for good convergence (-256 - 256) as well as bandpass filter 0.5-20 Hz. Also, it was beneficial to use the same weights for each signal in encoders.
Augmentations that worked for me:
\nCutMix on half signals(right in the middle cut)
\nAdd uniform noise to labels with higher than zero value(the range of uniform distribution depending on the number of votes, the more votes the more reliable labels we have, so less noise is allowed)
Interesting, you also used raw spectrograms plots, like @cdeotte . Both of you report strong gain from it, maybe this will become popular.
", "rawMarkdown": "Interesting, you also used raw spectrograms plots, like @cdeotte . Both of you report strong gain from it, maybe this will become popular.", "votes": null }, { "id": "2744102", "postDate": "04/09/2024 18:46:56", "content": "\n\nVisualisation of raw eegs(adding this gave about 0.02 improvment!!).
\n
Interesting. This also boosted my CV LB. I'm surprised that it works. Nice discovery and congratulations on your great finish!
", "rawMarkdown": ">Visualisation of raw eegs(adding this gave about 0.02 improvment!!).\n\nInteresting. This also boosted my CV LB. I'm surprised that it works. Nice discovery and congratulations on your great finish!", "votes": null }, { "id": "2744164", "postDate": "04/09/2024 19:28:45", "content": "I think spectrograms and visualizations of raw EEG work because they contain a highly compressed representation of data that 2D kernels can easily handle compared to processing 1d noisy signals where neseccary patterns can be stretched and hidden within long noise sequances.
\nSo top solutions encompass representations of the signals that are both compressed and diverse simultaneously.
Congratulations on securing 20th place in this competition. Thanks for your comments on what made the notebook improvements.
", "rawMarkdown": "Congratulations on securing 20th place in this competition. Thanks for your comments on what made the notebook improvements.", "votes": null }, { "id": "2747575", "postDate": "04/12/2024 01:15:08", "content": "thanks for sharing your solutions 🥳
\nI also have the similar insight about
\n\n\n16 spectrograms from banana montage with 2 sec(400 time stamps) window and clipping to 0-160 of dB range.
\n
but I never thought I can add \"Visualisation of raw eegs\" (I really wish I could come up with ways to do that too.)
", "rawMarkdown": "thanks for sharing your solutions 🥳\n\nI also have the similar insight about \n> 16 spectrograms from banana montage with 2 sec(400 time stamps) window and clipping to 0-160 of dB range.\n\nbut I never thought I can add \"Visualisation of raw eegs\" (I really wish I could come up with ways to do that too.)", "votes": null } ], "comments": [ { "id": 2744089, "author_name": "cpmpml", "author_url": "", "post_date": "04/09/2024 18:41:21", "content": "Interesting, you also used raw spectrograms plots, like @cdeotte . Both of you report strong gain from it, maybe this will become popular.
", "votes": null, "replies": [] }, { "id": 2744102, "author_name": "cdeotte", "author_url": "", "post_date": "04/09/2024 18:46:56", "content": "\n\nVisualisation of raw eegs(adding this gave about 0.02 improvment!!).
\n
Interesting. This also boosted my CV LB. I'm surprised that it works. Nice discovery and congratulations on your great finish!
", "votes": null, "replies": [ { "id": 2744164, "author_name": "nikitababich", "author_url": "", "post_date": "04/09/2024 19:28:45", "content": "I think spectrograms and visualizations of raw EEG work because they contain a highly compressed representation of data that 2D kernels can easily handle compared to processing 1d noisy signals where neseccary patterns can be stretched and hidden within long noise sequances.
\nSo top solutions encompass representations of the signals that are both compressed and diverse simultaneously.
Congratulations on securing 20th place in this competition. Thanks for your comments on what made the notebook improvements.
", "votes": null, "replies": [] }, { "id": 2747575, "author_name": "roger92", "author_url": "", "post_date": "04/12/2024 01:15:08", "content": "thanks for sharing your solutions 🥳
\nI also have the similar insight about
\n\n\n16 spectrograms from banana montage with 2 sec(400 time stamps) window and clipping to 0-160 of dB range.
\n
but I never thought I can add \"Visualisation of raw eegs\" (I really wish I could come up with ways to do that too.)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2743872": "One of my models that provided the best LB score (~0.25) was efficientnet 3 trained on images that consisted of the following elements:\n1. 16 spectrograms from banana montage with 2 sec(400 time stamps) window and clipping to 0-160 of dB range.\n2. Kaggle spectrograms.\n3. Visualisation of raw eegs(adding this gave about 0.02 improvment!!).\n\n**Raw eeg models**:\nOther 2D models were trained on images made via 1d resnet blocks and aggresive 1d convolutions with large strides. \n1D model consisted of 1D resnet blocks with different kernel sizes + gru. \nFor models trained on raw eegs, I have noticed that clipping was crucial for good convergence (-256 - 256) as well as bandpass filter 0.5-20 Hz. Also, it was beneficial to use the same weights for each signal in encoders.\n\n**Augmentations that worked for me:**\nCutMix on half signals(right in the middle cut)\nAdd uniform noise to labels with higher than zero value(the range of uniform distribution depending on the number of votes, the more votes the more reliable labels we have, so less noise is allowed)", "2744089": "Interesting, you also used raw spectrograms plots, like @cdeotte . Both of you report strong gain from it, maybe this will become popular.", "2744102": ">Visualisation of raw eegs(adding this gave about 0.02 improvment!!).\n\nInteresting. This also boosted my CV LB. I'm surprised that it works. Nice discovery and congratulations on your great finish!", "2744164": "I think spectrograms and visualizations of raw EEG work because they contain a highly compressed representation of data that 2D kernels can easily handle compared to processing 1d noisy signals where neseccary patterns can be stretched and hidden within long noise sequances. \nSo top solutions encompass representations of the signals that are both compressed and diverse simultaneously.", "2744691": "Congratulations on securing 20th place in this competition. Thanks for your comments on what made the notebook improvements.", "2747575": "thanks for sharing your solutions 🥳\n\nI also have the similar insight about \n> 16 spectrograms from banana montage with 2 sec(400 time stamps) window and clipping to 0-160 of dB range.\n\nbut I never thought I can add \"Visualisation of raw eegs\" (I really wish I could come up with ways to do that too.)" }, "source": "meta" }