\n
![\"solution](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1562037%2F9d2ab8354f85f39d092a75fd13922db7%2Fimg.png?generation=1715226166682613&alt=media\")
First of all, I would like to thanks to Kaggle and the organizers. I have learned a lot from this experience.
\nIn early time, feeding a bsx4xHxW input to the 2D-CNN produce worse result than the one-image method. I start to think why?
\nDue to the position-sensitive nature of the labels (LPD, GPD, etc.), We should take care of the channel dimension. 2D-CNN is not well-equipped to capture positional information within the channels. Because there is not pad in channel direction. And that's why we need to concat the spectrum into one image for a vision model other than a bsx16xHxW image( double banana montage).
\nSo I decided to use 3d-CNN for spectrum, and 2d-CNN model for raw-eeg signal.
diagram like:
\ntotal solution like this:
\n
After double banana montage, +-1024 clip and 0.5-20hz filter,
\nuse stft to extract the spectrum, then feed to a 3d-CNN(x3d-l).
![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1562037%2Fcbfe301faa8741dc262aa334e620e4c9%2Fimg_2.png?generation=1715226202040353&alt=media\")
The input data is a 16-channels spectrum image.
\nX3d-l cv 0.21+, public lb0.25 private lb0.29.
\nstft code like below, use it as a nn.Module :
\nIt's a 2d vision model, efficientnetb5, public lb 0.26 2490 private lb0.304877.
\nFor 2d model, I concat the 16-channels spectrum image like many people does.
\n = torch.reshape(image, shape=[n, , -, w])\n = image[:, :, ...]\n = image[:, :, ...]\n = torch.cat([x1, x2], dim=-)\n = torch.cat([image, image, image], dim=)\nI check the submission score, combine these two spectrum model, get 0.28 private lb.
\nView eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.
\nThe model define as below
rclass Net(nn.Module):\n ():\n (Net, self).__init__()\n self.model = timm.create_model(, pretrained=, in_chans=)\n self.pool = nn.AdaptiveAvgPool2d()\n self.fc = nn.Linear(, out_features=, bias=)\n self.dropout = nn.Dropout(p=)\n\n ():\n feature1 = self.model.forward_features(x)\n feature1\n\n ():\n bs = x.size()\n reshaped_tensor = x.view(bs, , , )\n reshaped_and_permuted_tensor = reshaped_tensor.permute(, , , )\n reshaped_and_permuted_tensor = reshaped_and_permuted_tensor.reshape(bs, * , )\n x = torch.unsqueeze(reshaped_and_permuted_tensor, dim=)\n\n x = torch.cat([x, x, x], dim=)\n bs = x.size()\n\n x = self.extract_features(x)\n x = self.pool(x)\n x = x.view(bs, -)\n x = self.dropout(x)\n x = self.fc(x)\n x\nThen, the eeg 'reshaped' to bsx3x160x1000. With efficinetnetb5, actives public lb 0.230 978, private lb 0.282 873.
\nThere are 3 models , but slightly different, mne-filter-efficinetnetb5, scipy.signal-filter-efficientnetb5 and mne-filter-hgnetb5.
\nAnd the scires are slightly different,also.
I use x3d-l to extract the spectrum feature ( with Transform50s only ), efficientnetb5 to extract the raw eeg feature,
\nlike the solution figure.
\nconcat the last feature. public lb 0.24, private lb 0.29 ? ? not sure
By combining these models, I think it could get my current score. However my score is 6 model emsemble, but not improvement that much (0.28->0.27 private lb).
\nFinal ensemble including:
With weights=[0.1,0.1,0.2,0.2,0.2,0.2].
\nps. with-mne.filter means use mne lib to do filter, butter filter means use scipy.signal, just to add some diversity.
\nI think raw eeg is more important in this task. Feeding raw EEG data into a 2D visual model is somewhat similar to how humans observe EEG signals. Observe in time and channel dimensions！
", "messages": [ { "id": 2742687, "postDate": "2024-04-09T03:01:28.943Z", "content": "First of all, I would like to thanks to Kaggle and the organizers. I have learned a lot from this experience.
\nIn early time, feeding a bsx4xHxW input to the 2D-CNN produce worse result than the one-image method. I start to think why?
\nDue to the position-sensitive nature of the labels (LPD, GPD, etc.), We should take care of the channel dimension. 2D-CNN is not well-equipped to capture positional information within the channels. Because there is not pad in channel direction. And that's why we need to concat the spectrum into one image for a vision model other than a bsx16xHxW image( double banana montage).
\nSo I decided to use 3d-CNN for spectrum, and 2d-CNN model for raw-eeg signal.
diagram like:
\ntotal solution like this:
\n
After double banana montage, +-1024 clip and 0.5-20hz filter,
\nuse stft to extract the spectrum, then feed to a 3d-CNN(x3d-l).
The input data is a 16-channels spectrum image.
\nX3d-l cv 0.21+, public lb0.25 private lb0.29.
\nstft code like below, use it as a nn.Module :
\nIt's a 2d vision model, efficientnetb5, public lb 0.26 2490 private lb0.304877.
\nFor 2d model, I concat the 16-channels spectrum image like many people does.
\n = torch.reshape(image, shape=[n, , -, w])\n = image[:, :, ...]\n = image[:, :, ...]\n = torch.cat([x1, x2], dim=-)\n = torch.cat([image, image, image], dim=)\nI check the submission score, combine these two spectrum model, get 0.28 private lb.
\nView eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.
\nThe model define as below
rclass Net(nn.Module):\n ():\n (Net, self).__init__()\n self.model = timm.create_model(, pretrained=, in_chans=)\n self.pool = nn.AdaptiveAvgPool2d()\n self.fc = nn.Linear(, out_features=, bias=)\n self.dropout = nn.Dropout(p=)\n\n ():\n feature1 = self.model.forward_features(x)\n feature1\n\n ():\n bs = x.size()\n reshaped_tensor = x.view(bs, , , )\n reshaped_and_permuted_tensor = reshaped_tensor.permute(, , , )\n reshaped_and_permuted_tensor = reshaped_and_permuted_tensor.reshape(bs, * , )\n x = torch.unsqueeze(reshaped_and_permuted_tensor, dim=)\n\n x = torch.cat([x, x, x], dim=)\n bs = x.size()\n\n x = self.extract_features(x)\n x = self.pool(x)\n x = x.view(bs, -)\n x = self.dropout(x)\n x = self.fc(x)\n x\nThen, the eeg 'reshaped' to bsx3x160x1000. With efficinetnetb5, actives public lb 0.230 978, private lb 0.282 873.
\nThere are 3 models , but slightly different, mne-filter-efficinetnetb5, scipy.signal-filter-efficientnetb5 and mne-filter-hgnetb5.
\nAnd the scires are slightly different,also.
I use x3d-l to extract the spectrum feature ( with Transform50s only ), efficientnetb5 to extract the raw eeg feature,
\nlike the solution figure.
\nconcat the last feature. public lb 0.24, private lb 0.29 ? ? not sure
By combining these models, I think it could get my current score. However my score is 6 model emsemble, but not improvement that much (0.28->0.27 private lb).
\nFinal ensemble including:
With weights=[0.1,0.1,0.2,0.2,0.2,0.2].
\nps. with-mne.filter means use mne lib to do filter, butter filter means use scipy.signal, just to add some diversity.
\nI think raw eeg is more important in this task. Feeding raw EEG data into a 2D visual model is somewhat similar to how humans observe EEG signals. Observe in time and channel dimensions！
", "rawMarkdown": "First of all, I would like to thanks to Kaggle and the organizers. I have learned a lot from this experience. \n\n\n\n## 1. model\n\nIn early time, feeding a bsx4xHxW input to the 2D-CNN produce worse result than the one-image method. I start to think why?\n\nDue to the position-sensitive nature of the labels (LPD, GPD, etc.), We should take care of the channel dimension. 2D-CNN is not well-equipped to capture positional information within the channels. Because there is not pad in channel direction. And that's why we need to concat the spectrum into one image for a vision model other than a bsx16xHxW image( double banana montage).\nSo I decided to use 3d-CNN for spectrum, and 2d-CNN model for raw-eeg signal.\n\n\ndiagram like:\n\n\ntotal solution like this:\n\n+ mne 0.5-20hz means use MNE-tool do filter.\n+ scipy.signal means use scipy.signal to do filter.\n+ For the reshape operator and the stft params please refer to the codes below.\n+ And the number in () means final weights of the ensemble.\n\n### 1.1 x3d-l (spectrum model)\n\n\nAfter double banana montage, +-1024 clip and 0.5-20hz filter,\nuse stft to extract the spectrum, then feed to a 3d-CNN(x3d-l).\n\n\n\nThe input data is a 16-channels spectrum image.\n\nX3d-l cv 0.21+, public lb0.25 private lb0.29.\n\n\n\nstft code like below, use it as a nn.Module :\n\n```\nclass Transform50s(nn.Module):\n def __init__(self, ):\n super().__init__()\n self.wave_transform = torchaudio.transforms.Spectrogram(n_fft=512, win_length=128, hop_length=50, power=1)\n\n def forward(self, x):\n image = self.wave_transform(x)\n image = torch.clip(image, min=0, max=10000) / 1000\n n, c, h, w = image.size()\n image = image[:, :, :int(20 / 100 * h + 10), :]\n return image\n\n\nclass Transform10s(nn.Module):\n def __init__(self, ):\n super().__init__()\n self.wave_transform = torchaudio.transforms.Spectrogram(n_fft=512, win_length=128, hop_length=10, power=1)\n\n def forward(self, x):\n image = self.wave_transform(x)\n image = torch.clip(image, min=0, max=10000) / 1000\n n, c, h, w = image.size()\n image = image[:, :, :int(20 / 100 * h + 10), :]\n return image\n\n\nclass Model(nn.Module):\n def __init__(self):\n super().__init__()\n\n model_name = \"x3d_l\"\n self.net = torch.hub.load('facebookresearch/pytorchvideo',\n model_name, pretrained=True)\n\n self.net.blocks[5].pool.pool = nn.AdaptiveAvgPool3d(1)\n # self.net.blocks[5]=nn.Identity()\n # self.net.avgpool = nn.Identity()\n self.net.blocks[5].dropout = nn.Identity()\n self.net.blocks[5].proj = nn.Identity()\n self.net.blocks[5].activation = nn.Identity()\n self.net.blocks[5].output_pool = nn.Identity()\n\n def forward(self, x):\n x = self.net(x)\n return x\n\n\nclass Net(nn.Module):\n def __init__(self, num_classes=1):\n super().__init__()\n\n self.preprocess50s = Transform50s()\n self.preprocess10s = Transform10s()\n\n self.model = Model()\n\n self.pool = nn.AdaptiveAvgPool3d(1)\n self.fc = nn.Linear(2048, 6, bias=True)\n self.drop = nn.Dropout(0.5)\n\n def forward(self, eeg):\n # do preprocess\n bs = eeg.size(0)\n eeg_50s = eeg\n eeg_10s = eeg[:, :, 4000:6000]\n x_50 = self.preprocess50s(eeg_50s)\n x_10 = self.preprocess10s(eeg_10s)\n x = torch.cat([x_10, x_50], dim=1)\n x = torch.unsqueeze(x, dim=1)\n x = torch.cat([x, x, x], dim=1)\n x = self.model(x)\n # x = self.pool(x)\n x = x.view(bs, -1)\n x = self.drop(x)\n x = self.fc(x)\n return x\n\n```\n\n### 1.2 oneimage\nIt's a 2d vision model, efficientnetb5, public lb 0.26 2490 private lb0.304877.\n\nFor 2d model, I concat the 16-channels spectrum image like many people does.\n\n```\n image = torch.reshape(image, shape=[n, 2, -1, w])\n x1 = image[:, 0:1, ...]\n x2 = image[:, 1:2, ...]\n image = torch.cat([x1, x2], dim=-1)\n image = torch.cat([image, image, image], dim=1)\n```\n\nI check the submission score, combine these two spectrum model, get 0.28 private lb.\n\n\n### 1.3 eeg model\n\nView eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.\nThe model define as below\n```python\nrclass Net(nn.Module):\n def __init__(self,):\n super(Net, self).__init__()\n self.model = timm.create_model('efficientnet_b5', pretrained=True, in_chans=3)\n self.pool = nn.AdaptiveAvgPool2d(1)\n self.fc = nn.Linear(2048, out_features=6, bias=True)\n self.dropout = nn.Dropout(p=0.5)\n\n def extract_features(self, x):\n feature1 = self.model.forward_features(x)\n return feature1\n\n def forward(self, x):\n bs = x.size(0)\n reshaped_tensor = x.view(bs, 16, 1000, 10)\n reshaped_and_permuted_tensor = reshaped_tensor.permute(0, 1, 3, 2)\n reshaped_and_permuted_tensor = reshaped_and_permuted_tensor.reshape(bs, 16 * 10, 1000)\n x = torch.unsqueeze(reshaped_and_permuted_tensor, dim=1)\n\n x = torch.cat([x, x, x], dim=1)\n bs = x.size(0)\n\n x = self.extract_features(x)\n x = self.pool(x)\n x = x.view(bs, -1)\n x = self.dropout(x)\n x = self.fc(x)\n return x\n```\nThen, the eeg 'reshaped' to bsx3x160x1000. With efficinetnetb5, actives public lb 0.230 978, private lb 0.282 873.\n\nThere are 3 models , but slightly different, mne-filter-efficinetnetb5, scipy.signal-filter-efficientnetb5 and mne-filter-hgnetb5. \nAnd the scires are slightly different,also.\n\n### 1.4 doublehead (eeg+spectrum)\n\nI use x3d-l to extract the spectrum feature ( with Transform50s only ), efficientnetb5 to extract the raw eeg feature, \nlike the solution figure.\nconcat the last feature. public lb 0.24, private lb 0.29 ? ? not sure\n\n# 2 Preprocess\n- 2.1.Double banana montage, eeg as 16x10000\n- 2.2.Filter with 0.5-20hz\n- 2.3.Clip by +-1024\n\n# 3. Train\n- 3.1 Stage1, 15 epochs, with loss weight voters_num/20, Adamw lr=0.001, cos scheduel\n- 3.2 Stage2, 5 epoch, loss weight=1, voters_num>=6. , Adamw lr=0.0001, cos scheduel\n- 3.3 use eeg_label_offset_seconds. I random choose an offset for each eegid, and the target is average according to eegid for each train iter.\n- 3.4 augmentation, mirror eeg, flip between left brain data and right brain data.\n- 3.5 10 folds, then move 1000 samples from val to train, left 709 samples in val set. And use vote_num>=6 to do validation. \n\n# 4. ensemble\n\nBy combining these models, I think it could get my current score. However my score is 6 model emsemble, but not improvement that much (0.28->0.27 private lb). \nFinal ensemble including: \n- 2 spectrum model (x3d, efficientnetb5), both use mne-filter\n- 3 raw eeg model( efficientnetb5 with-mne.filter, efficientnetb5 -butter filter, 1 hgnetb5 mne.filter),\n- 1 eeg-spectrum mix model, butter filter, \n\nWith weights=[0.1,0.1,0.2,0.2,0.2,0.2].\n\nps. with-mne.filter means use mne lib to do filter, butter filter means use scipy.signal, just to add some diversity.\n\n# Some thought\nI think raw eeg is more important in this task. Feeding raw EEG data into a 2D visual model is somewhat similar to how humans observe EEG signals. Observe in time and channel dimensions！\n\n", "votes": 133 }, { "id": 2743455, "postDate": "2024-04-09T13:08:35.670Z", "content": "\nCongratulations on your solo 2nd place. Very impressed with your very unique solution!
\n = x.view(bs, , , )\n = reshaped_tensor.permute(, , , )\n = reshaped_and_permuted_tensor.reshape(bs, * , )\n = torch.unsqueeze(reshaped_and_permuted_tensor, dim=)\n = torch.cat([x, x, x], dim=)\n\n\nThen, the eeg 'reshaped' to bsx3x80x1000.
\n
Is the size of the input to the model bsx3x80x1000 or bsx3x160x1000?
\nThe former seems to contradict the shape of x in the pseudo code.
It should be bsx3x160x1000, Thanks for pointing out the mistake.
", "rawMarkdown": "It should be bsx3x160x1000, Thanks for pointing out the mistake.", "votes": 5, "replies": [ { "id": 2743495, "postDate": "2024-04-09T13:36:35.067Z", "content": "Thank you for your prompt reply!
", "rawMarkdown": "Thank you for your prompt reply!", "votes": 1 }, { "id": 2743606, "postDate": "2024-04-09T14:43:52.207Z", "content": "thanks for the suggestion
", "rawMarkdown": "thanks for the suggestion\n", "votes": 1 } ] }, { "id": 2743504, "postDate": "2024-04-09T13:45:29.900Z", "content": "Very nice solution. Did you see performance differences with 3-channels vs 1-channel images?
\nx = torch.cat([x, x, x], dim=1)
I didn't try. I think there should be no difference after some tuning.
", "rawMarkdown": "I didn't try. I think there should be no difference after some tuning.", "votes": 3 } ] } ] }, { "id": 2742899, "postDate": "2024-04-09T06:15:00.850Z", "content": "Great solution, congratulations!
\n\n\n2D-CNN is not well-equipped to capture positional information within the channels
\n
What do you mean here, can you elaborate?
", "rawMarkdown": "Great solution, congratulations!\n> 2D-CNN is not well-equipped to capture positional information within the channels\n\nWhat do you mean here, can you elaborate?", "votes": 1, "replies": [ { "id": 2742917, "postDate": "2024-04-09T06:21:42.780Z", "content": "In a paper（I forget which one）said, the position in the CNN is obtained through padding, but in general, padding is not done in the channel dimension for vision models.
", "rawMarkdown": "In a paper（I forget which one）said, the position in the CNN is obtained through padding, but in general, padding is not done in the channel dimension for vision models. ", "votes": 2 }, { "id": 2742921, "postDate": "2024-04-09T06:23:17.757Z", "content": "《Position, Padding and Predictions:
\nA Deeper Look at Position Information in CNNs》 this paper
Ah I see, thanks for clarifying and for the paper link.
", "rawMarkdown": "Ah I see, thanks for clarifying and for the paper link. " } ] } ] }, { "id": 2742695, "postDate": "2024-04-09T03:08:30.883Z", "content": "In your opinion, would Vision transformer-based models for example CCT, CvT, and ViT do well on the spectrum stage?
", "rawMarkdown": "In your opinion, would Vision transformer-based models for example CCT, CvT, and ViT do well on the spectrum stage?", "votes": 1, "replies": [ { "id": 2742702, "postDate": "2024-04-09T03:12:12.973Z", "content": "Vit not work well in this task in my experiments. I think there is 2 reason. 1. the patch embeding has no overlap. 2. It's the same with plain vision model, not care too much on the channel direction.
", "rawMarkdown": "Vit not work well in this task in my experiments. I think there is 2 reason. 1. the patch embeding has no overlap. 2. It's the same with plain vision model, not care too much on the channel direction.", "votes": 1 }, { "id": 2742852, "postDate": "2024-04-09T05:45:45.023Z", "content": "However, because of the position encoding, vit should work better than the cnn arch. And it seems that was approved in other teams.
", "rawMarkdown": "However, because of the position encoding, vit should work better than the cnn arch. And it seems that was approved in other teams.", "votes": 1 }, { "id": 2742968, "postDate": "2024-04-09T06:50:16.687Z", "content": "I only using vit in all final submission. My single have CV 0.217, LB 0.22, Private 0.28.
", "rawMarkdown": "I only using vit in all final submission. My single have CV 0.217, LB 0.22, Private 0.28.", "votes": 1, "replies": [ { "id": 2743261, "postDate": "2024-04-09T10:20:03.643Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2743266, "postDate": "2024-04-09T10:25:04.437Z", "content": "Vit not work well, i mean there, with separate channel spectrum, not the one-image method.
\nAnd congratulation !!!
Using 3d cnn is a great idea. Like many I was puzzled by why using channels was not effective. But you moved past it and addressed it with 3d models. Well done, and well deserved 2nd prize!
", "rawMarkdown": "Using 3d cnn is a great idea. Like many I was puzzled by why using channels was not effective. But you moved past it and addressed it with 3d models. Well done, and well deserved 2nd prize!", "votes": 2, "replies": [ { "id": 2744852, "postDate": "2024-04-10T06:58:01.810Z", "content": "Thanks!
\nI actually spent a lot of time trying to figure out why using channels was not effective in the 2D vision model. Now i think it may because of the target. For example, lpd, gpd may have same type wave shape in some channels, but in different position( channels ),
\nthere position( channels ) means electrode placement.
Well done @cooolz on solo 2nd. Impressive 💪
\nI had not tried stft transform as preprocess; I am interested to see a snippet of the code for preprocessing the signal for input to the model; or understand how it was done. I guess the detail is important.
\nWe tried x3d_m and _l as 3d model of spectrograms. They performed well, but in a blend they added no extra information over our 2d CNN. I guess with spectrum as input data it would have performed better in the blend.
Thanks.
\nI update the stft codes in the post, torchaudio was used. I tried 50s-spectrum and 10s-spectrum concat to 32 channels, but they not ouperform that much with just 50s-spectrum.
@darraghdog mel is really similar to stsft. The main difference is that with mel the frequency scale is logarithmic (or close to it depending on the settings you use) while it is linear for stft. Mel zooms in the low frequency are compared to stft of same size.
", "rawMarkdown": "@darraghdog mel is really similar to stsft. The main difference is that with mel the frequency scale is logarithmic (or close to it depending on the settings you use) while it is linear for stft. Mel zooms in the low frequency are compared to stft of same size.", "votes": 4, "replies": [ { "id": 2744798, "postDate": "2024-04-10T06:15:48.013Z", "content": "Got it, thanks !
", "rawMarkdown": "Got it, thanks !" }, { "id": 2744838, "postDate": "2024-04-10T06:41:04.930Z", "content": "Yeah, from my experiments, they perform similar, actually stft is a bit better then mel, and interestingly is scipy.spectrogram still work best for me(better then torchaudio.stft).
", "rawMarkdown": "Yeah, from my experiments, they perform similar, actually stft is a bit better then mel, and interestingly is scipy.spectrogram still work best for me(better then torchaudio.stft).", "votes": 1 }, { "id": 2745399, "postDate": "2024-04-10T15:47:47.723Z", "content": "ah, interesting - will know for next time to try them out. I guess log vs linear frequency scaling may give some diversity on what is learned.
", "rawMarkdown": "ah, interesting - will know for next time to try them out. I guess log vs linear frequency scaling may give some diversity on what is learned.", "votes": 1 } ] } ] }, { "id": 2742774, "postDate": "2024-04-09T04:31:44.970Z", "content": "\n\nView eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.
\n
Wow, brilliant. Great idea!
", "rawMarkdown": ">View eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.\n\nWow, brilliant. Great idea!", "votes": 2 }, { "id": 2742773, "postDate": "2024-04-09T04:27:03.787Z", "content": "Thanks for sharing , 3d image model and your method of eeg reshape seems much better then juse simplly resize, thanks for sharing, learned a lot and congratulation on solo 2nd place!
", "rawMarkdown": "Thanks for sharing , 3d image model and your method of eeg reshape seems much better then juse simplly resize, thanks for sharing, learned a lot and congratulation on solo 2nd place!", "votes": 2 }, { "id": 2886257, "postDate": "2024-06-23T14:45:59.083Z", "content": "I will learn it in future
", "rawMarkdown": "I will learn it in future" }, { "id": 2825957, "postDate": "2024-05-20T16:41:02.520Z", "content": "nice solution for me💝
", "rawMarkdown": "nice solution for me💝" }, { "id": 2764034, "postDate": "2024-04-20T20:54:56.797Z", "content": "Congratulations! Thanks for sharing your work. Are you using transfer learning or fitting these models from scratch. As a beginner, can you please help me understand how you compare between model architectures (either with transfer learning or \"fresh\"? Do you make sure to use the same 2-stage regime or did 2-stage learning come after you have chosen the final model?
", "rawMarkdown": "\nCongratulations! Thanks for sharing your work. Are you using transfer learning or fitting these models from scratch. As a beginner, can you please help me understand how you compare between model architectures (either with transfer learning or \"fresh\"? Do you make sure to use the same 2-stage regime or did 2-stage learning come after you have chosen the final model?\n" }, { "id": 2747447, "postDate": "2024-04-11T22:13:58.230Z", "content": "Thank you for sharing your comprehensive approach and the detailed insights into your modeling techniques. Your innovative use of both spectral and raw EEG data with a blend of 3D and 2D CNNs provides valuable lessons on tackling position-sensitive label challenges in EEG analysis.
", "rawMarkdown": "Thank you for sharing your comprehensive approach and the detailed insights into your modeling techniques. Your innovative use of both spectral and raw EEG data with a blend of 3D and 2D CNNs provides valuable lessons on tackling position-sensitive label challenges in EEG analysis." }, { "id": 2746452, "postDate": "2024-04-11T08:14:37.863Z", "content": "Very helpful. 3D-CNN is a model structure that I have never thought of before, which has also inspired my personal research topic.
", "rawMarkdown": "Very helpful. 3D-CNN is a model structure that I have never thought of before, which has also inspired my personal research topic." }, { "id": 2744869, "postDate": "2024-04-10T07:10:03.123Z", "content": "Congratulations on second place and thanks for sharing this solution!
\nWe tried using 3D-CNN on the raw EEG data, but not on the spectrograms, so it is great to see that you tried it on the spectrograms and it worked.
Congratulations on winning the 2nd prize in this competition.
", "rawMarkdown": "Congratulations on winning the 2nd prize in this competition. " }, { "id": 2743571, "postDate": "2024-04-09T14:21:41.980Z", "content": "Hello, congratulations on winning second place~~
\nThis competition is not an easy task for anyone.
I would like to ask a question about the ensemble, which can shed some light on the technique and direction of thinking. If I have doubts about the results, where should I test or modify?
\nTHX!!
Interesting approach to eeg model. Thanks for sharing and congratulations!
", "rawMarkdown": "Interesting approach to eeg model. Thanks for sharing and congratulations!" }, { "id": 2742967, "postDate": "2024-04-09T06:50:13.413Z", "content": "An incredible solution! You have no idea how painful it is for me! At the very beginning of the competition, I tested the eeg as an image, but then it seemed to me that it was not working. I also tried to use 3d models, but training did not start on tensorflow (tensorflow clearly had a problem here). Congratulations again!
", "rawMarkdown": "An incredible solution! You have no idea how painful it is for me! At the very beginning of the competition, I tested the eeg as an image, but then it seemed to me that it was not working. I also tried to use 3d models, but training did not start on tensorflow (tensorflow clearly had a problem here). Congratulations again!" }, { "id": 2742896, "postDate": "2024-04-09T06:13:08.250Z", "content": "Our team never thought of 3d architecture, and I learn a lot from your method. Congratulation !
", "rawMarkdown": "Our team never thought of 3d architecture, and I learn a lot from your method. Congratulation !" }, { "id": 2742844, "postDate": "2024-04-09T05:41:32.697Z", "content": "Wow, great idea! Thanks for sharing your approach. It will be very helpful in the upcoming competitions.
", "rawMarkdown": "Wow, great idea! Thanks for sharing your approach. It will be very helpful in the upcoming competitions." }, { "id": 2742736, "postDate": "2024-04-09T03:49:37.430Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2742719, "postDate": "2024-04-09T03:40:53.780Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2743455, "author_name": "yu4u", "author_url": "", "post_date": "2024-04-09T13:08:35.670000", "content": "\nCongratulations on your solo 2nd place. Very impressed with your very unique solution!
\n = x.view(bs, , , )\n = reshaped_tensor.permute(, , , )\n = reshaped_and_permuted_tensor.reshape(bs, * , )\n = torch.unsqueeze(reshaped_and_permuted_tensor, dim=)\n = torch.cat([x, x, x], dim=)\n\n\nThen, the eeg 'reshaped' to bsx3x80x1000.
\n
Is the size of the input to the model bsx3x80x1000 or bsx3x160x1000?
\nThe former seems to contradict the shape of x in the pseudo code.
It should be bsx3x160x1000, Thanks for pointing out the mistake.
", "votes": 5, "replies": [ { "id": 2743495, "author_name": "yu4u", "author_url": "", "post_date": "2024-04-09T13:36:35.067000", "content": "Thank you for your prompt reply!
", "votes": 1, "replies": [] }, { "id": 2743606, "author_name": "ZACKY_ZAC", "author_url": "", "post_date": "2024-04-09T14:43:52.207000", "content": "thanks for the suggestion
", "votes": 1, "replies": [] } ] }, { "id": 2743504, "author_name": "Bartley", "author_url": "", "post_date": "2024-04-09T13:45:29.900000", "content": "Very nice solution. Did you see performance differences with 3-channels vs 1-channel images?
\nx = torch.cat([x, x, x], dim=1)
I didn't try. I think there should be no difference after some tuning.
", "votes": 3, "replies": [] } ] } ] }, { "id": 2742899, "author_name": "Viktor Cikojevic", "author_url": "", "post_date": "2024-04-09T06:15:00.850000", "content": "Great solution, congratulations!
\n\n\n2D-CNN is not well-equipped to capture positional information within the channels
\n
What do you mean here, can you elaborate?
", "votes": 1, "replies": [ { "id": 2742917, "author_name": "coolz", "author_url": "", "post_date": "2024-04-09T06:21:42.780000", "content": "In a paper（I forget which one）said, the position in the CNN is obtained through padding, but in general, padding is not done in the channel dimension for vision models.
", "votes": 2, "replies": [] }, { "id": 2742921, "author_name": "coolz", "author_url": "", "post_date": "2024-04-09T06:23:17.757000", "content": "《Position, Padding and Predictions:
\nA Deeper Look at Position Information in CNNs》 this paper
Ah I see, thanks for clarifying and for the paper link.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2742695, "author_name": "MoZayed", "author_url": "", "post_date": "2024-04-09T03:08:30.883000", "content": "In your opinion, would Vision transformer-based models for example CCT, CvT, and ViT do well on the spectrum stage?
", "votes": 1, "replies": [ { "id": 2742702, "author_name": "coolz", "author_url": "", "post_date": "2024-04-09T03:12:12.973000", "content": "Vit not work well in this task in my experiments. I think there is 2 reason. 1. the patch embeding has no overlap. 2. It's the same with plain vision model, not care too much on the channel direction.
", "votes": 1, "replies": [] }, { "id": 2742852, "author_name": "coolz", "author_url": "", "post_date": "2024-04-09T05:45:45.023000", "content": "However, because of the position encoding, vit should work better than the cnn arch. And it seems that was approved in other teams.
", "votes": 1, "replies": [] }, { "id": 2742968, "author_name": "Quan Vu", "author_url": "", "post_date": "2024-04-09T06:50:16.687000", "content": "I only using vit in all final submission. My single have CV 0.217, LB 0.22, Private 0.28.
", "votes": 1, "replies": [ { "id": 2743261, "author_name": "", "author_url": "", "post_date": "2024-04-09T10:20:03.643000", "content": "", "votes": 0, "replies": [] }, { "id": 2743266, "author_name": "coolz", "author_url": "", "post_date": "2024-04-09T10:25:04.437000", "content": "Vit not work well, i mean there, with separate channel spectrum, not the one-image method.
\nAnd congratulation !!!
Using 3d cnn is a great idea. Like many I was puzzled by why using channels was not effective. But you moved past it and addressed it with 3d models. Well done, and well deserved 2nd prize!
", "votes": 2, "replies": [ { "id": 2744852, "author_name": "coolz", "author_url": "", "post_date": "2024-04-10T06:58:01.810000", "content": "Thanks!
\nI actually spent a lot of time trying to figure out why using channels was not effective in the 2D vision model. Now i think it may because of the target. For example, lpd, gpd may have same type wave shape in some channels, but in different position( channels ),
\nthere position( channels ) means electrode placement.
Well done @cooolz on solo 2nd. Impressive 💪
\nI had not tried stft transform as preprocess; I am interested to see a snippet of the code for preprocessing the signal for input to the model; or understand how it was done. I guess the detail is important.
\nWe tried x3d_m and _l as 3d model of spectrograms. They performed well, but in a blend they added no extra information over our 2d CNN. I guess with spectrum as input data it would have performed better in the blend.
Thanks.
\nI update the stft codes in the post, torchaudio was used. I tried 50s-spectrum and 10s-spectrum concat to 32 channels, but they not ouperform that much with just 50s-spectrum.
@darraghdog mel is really similar to stsft. The main difference is that with mel the frequency scale is logarithmic (or close to it depending on the settings you use) while it is linear for stft. Mel zooms in the low frequency are compared to stft of same size.
", "votes": 4, "replies": [ { "id": 2744798, "author_name": "Darragh", "author_url": "", "post_date": "2024-04-10T06:15:48.013000", "content": "Got it, thanks !
", "votes": 0, "replies": [] }, { "id": 2744838, "author_name": "gezi", "author_url": "", "post_date": "2024-04-10T06:41:04.930000", "content": "Yeah, from my experiments, they perform similar, actually stft is a bit better then mel, and interestingly is scipy.spectrogram still work best for me(better then torchaudio.stft).
", "votes": 1, "replies": [] }, { "id": 2745399, "author_name": "Darragh", "author_url": "", "post_date": "2024-04-10T15:47:47.723000", "content": "ah, interesting - will know for next time to try them out. I guess log vs linear frequency scaling may give some diversity on what is learned.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2742774, "author_name": "Chris Deotte", "author_url": "", "post_date": "2024-04-09T04:31:44.970000", "content": "\n\nView eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.
\n
Wow, brilliant. Great idea!
", "votes": 2, "replies": [] }, { "id": 2742773, "author_name": "gezi", "author_url": "", "post_date": "2024-04-09T04:27:03.787000", "content": "Thanks for sharing , 3d image model and your method of eeg reshape seems much better then juse simplly resize, thanks for sharing, learned a lot and congratulation on solo 2nd place!
", "votes": 2, "replies": [] }, { "id": 2886257, "author_name": "DongLibin", "author_url": "", "post_date": "2024-06-23T14:45:59.083000", "content": "I will learn it in future
", "votes": 0, "replies": [] }, { "id": 2825957, "author_name": "Wachirawit Premthaisong-KMUTT", "author_url": "", "post_date": "2024-05-20T16:41:02.520000", "content": "nice solution for me💝
", "votes": 0, "replies": [] }, { "id": 2764034, "author_name": "Idith Haber", "author_url": "", "post_date": "2024-04-20T20:54:56.797000", "content": "Congratulations! Thanks for sharing your work. Are you using transfer learning or fitting these models from scratch. As a beginner, can you please help me understand how you compare between model architectures (either with transfer learning or \"fresh\"? Do you make sure to use the same 2-stage regime or did 2-stage learning come after you have chosen the final model?
", "votes": 0, "replies": [] }, { "id": 2747447, "author_name": "Suhasini Polampelly", "author_url": "", "post_date": "2024-04-11T22:13:58.230000", "content": "Thank you for sharing your comprehensive approach and the detailed insights into your modeling techniques. Your innovative use of both spectral and raw EEG data with a blend of 3D and 2D CNNs provides valuable lessons on tackling position-sensitive label challenges in EEG analysis.
", "votes": 0, "replies": [] }, { "id": 2746452, "author_name": "Yuri Sun", "author_url": "", "post_date": "2024-04-11T08:14:37.863000", "content": "Very helpful. 3D-CNN is a model structure that I have never thought of before, which has also inspired my personal research topic.
", "votes": 0, "replies": [] }, { "id": 2744869, "author_name": "movieminer", "author_url": "", "post_date": "2024-04-10T07:10:03.123000", "content": "Congratulations on second place and thanks for sharing this solution!
\nWe tried using 3D-CNN on the raw EEG data, but not on the spectrograms, so it is great to see that you tried it on the spectrograms and it worked.
Congratulations on winning the 2nd prize in this competition.
", "votes": 0, "replies": [] }, { "id": 2743571, "author_name": "warrenkuo", "author_url": "", "post_date": "2024-04-09T14:21:41.980000", "content": "Hello, congratulations on winning second place~~
\nThis competition is not an easy task for anyone.
I would like to ask a question about the ensemble, which can shed some light on the technique and direction of thinking. If I have doubts about the results, where should I test or modify?
\nTHX!!
Interesting approach to eeg model. Thanks for sharing and congratulations!
", "votes": 0, "replies": [] }, { "id": 2742967, "author_name": "Andrij", "author_url": "", "post_date": "2024-04-09T06:50:13.413000", "content": "An incredible solution! You have no idea how painful it is for me! At the very beginning of the competition, I tested the eeg as an image, but then it seemed to me that it was not working. I also tried to use 3d models, but training did not start on tensorflow (tensorflow clearly had a problem here). Congratulations again!
", "votes": 0, "replies": [] }, { "id": 2742896, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2024-04-09T06:13:08.250000", "content": "Our team never thought of 3d architecture, and I learn a lot from your method. Congratulation !
", "votes": 0, "replies": [] }, { "id": 2742844, "author_name": "Aaditya Porwal", "author_url": "", "post_date": "2024-04-09T05:41:32.697000", "content": "Wow, great idea! Thanks for sharing your approach. It will be very helpful in the upcoming competitions.
", "votes": 0, "replies": [] }, { "id": 2742736, "author_name": "", "author_url": "", "post_date": "2024-04-09T03:49:37.430000", "content": "", "votes": 0, "replies": [] }, { "id": 2742719, "author_name": "", "author_url": "", "post_date": "2024-04-09T03:40:53.780000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2742687": "First of all, I would like to thanks to Kaggle and the organizers. I have learned a lot from this experience. \n\n\n\n## 1. model\n\nIn early time, feeding a bsx4xHxW input to the 2D-CNN produce worse result than the one-image method. I start to think why?\n\nDue to the position-sensitive nature of the labels (LPD, GPD, etc.), We should take care of the channel dimension. 2D-CNN is not well-equipped to capture positional information within the channels. Because there is not pad in channel direction. And that's why we need to concat the spectrum into one image for a vision model other than a bsx16xHxW image( double banana montage).\nSo I decided to use 3d-CNN for spectrum, and 2d-CNN model for raw-eeg signal.\n\n\ndiagram like:\n\n\ntotal solution like this:\n\n+ mne 0.5-20hz means use MNE-tool do filter.\n+ scipy.signal means use scipy.signal to do filter.\n+ For the reshape operator and the stft params please refer to the codes below.\n+ And the number in () means final weights of the ensemble.\n\n### 1.1 x3d-l (spectrum model)\n\n\nAfter double banana montage, +-1024 clip and 0.5-20hz filter,\nuse stft to extract the spectrum, then feed to a 3d-CNN(x3d-l).\n\n\n\nThe input data is a 16-channels spectrum image.\n\nX3d-l cv 0.21+, public lb0.25 private lb0.29.\n\n\n\nstft code like below, use it as a nn.Module :\n\n```\nclass Transform50s(nn.Module):\n def __init__(self, ):\n super().__init__()\n self.wave_transform = torchaudio.transforms.Spectrogram(n_fft=512, win_length=128, hop_length=50, power=1)\n\n def forward(self, x):\n image = self.wave_transform(x)\n image = torch.clip(image, min=0, max=10000) / 1000\n n, c, h, w = image.size()\n image = image[:, :, :int(20 / 100 * h + 10), :]\n return image\n\n\nclass Transform10s(nn.Module):\n def __init__(self, ):\n super().__init__()\n self.wave_transform = torchaudio.transforms.Spectrogram(n_fft=512, win_length=128, hop_length=10, power=1)\n\n def forward(self, x):\n image = self.wave_transform(x)\n image = torch.clip(image, min=0, max=10000) / 1000\n n, c, h, w = image.size()\n image = image[:, :, :int(20 / 100 * h + 10), :]\n return image\n\n\nclass Model(nn.Module):\n def __init__(self):\n super().__init__()\n\n model_name = \"x3d_l\"\n self.net = torch.hub.load('facebookresearch/pytorchvideo',\n model_name, pretrained=True)\n\n self.net.blocks[5].pool.pool = nn.AdaptiveAvgPool3d(1)\n # self.net.blocks[5]=nn.Identity()\n # self.net.avgpool = nn.Identity()\n self.net.blocks[5].dropout = nn.Identity()\n self.net.blocks[5].proj = nn.Identity()\n self.net.blocks[5].activation = nn.Identity()\n self.net.blocks[5].output_pool = nn.Identity()\n\n def forward(self, x):\n x = self.net(x)\n return x\n\n\nclass Net(nn.Module):\n def __init__(self, num_classes=1):\n super().__init__()\n\n self.preprocess50s = Transform50s()\n self.preprocess10s = Transform10s()\n\n self.model = Model()\n\n self.pool = nn.AdaptiveAvgPool3d(1)\n self.fc = nn.Linear(2048, 6, bias=True)\n self.drop = nn.Dropout(0.5)\n\n def forward(self, eeg):\n # do preprocess\n bs = eeg.size(0)\n eeg_50s = eeg\n eeg_10s = eeg[:, :, 4000:6000]\n x_50 = self.preprocess50s(eeg_50s)\n x_10 = self.preprocess10s(eeg_10s)\n x = torch.cat([x_10, x_50], dim=1)\n x = torch.unsqueeze(x, dim=1)\n x = torch.cat([x, x, x], dim=1)\n x = self.model(x)\n # x = self.pool(x)\n x = x.view(bs, -1)\n x = self.drop(x)\n x = self.fc(x)\n return x\n\n```\n\n### 1.2 oneimage\nIt's a 2d vision model, efficientnetb5, public lb 0.26 2490 private lb0.304877.\n\nFor 2d model, I concat the 16-channels spectrum image like many people does.\n\n```\n image = torch.reshape(image, shape=[n, 2, -1, w])\n x1 = image[:, 0:1, ...]\n x2 = image[:, 1:2, ...]\n image = torch.cat([x1, x2], dim=-1)\n image = torch.cat([image, image, image], dim=1)\n```\n\nI check the submission score, combine these two spectrum model, get 0.28 private lb.\n\n\n### 1.3 eeg model\n\nView eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.\nThe model define as below\n```python\nrclass Net(nn.Module):\n def __init__(self,):\n super(Net, self).__init__()\n self.model = timm.create_model('efficientnet_b5', pretrained=True, in_chans=3)\n self.pool = nn.AdaptiveAvgPool2d(1)\n self.fc = nn.Linear(2048, out_features=6, bias=True)\n self.dropout = nn.Dropout(p=0.5)\n\n def extract_features(self, x):\n feature1 = self.model.forward_features(x)\n return feature1\n\n def forward(self, x):\n bs = x.size(0)\n reshaped_tensor = x.view(bs, 16, 1000, 10)\n reshaped_and_permuted_tensor = reshaped_tensor.permute(0, 1, 3, 2)\n reshaped_and_permuted_tensor = reshaped_and_permuted_tensor.reshape(bs, 16 * 10, 1000)\n x = torch.unsqueeze(reshaped_and_permuted_tensor, dim=1)\n\n x = torch.cat([x, x, x], dim=1)\n bs = x.size(0)\n\n x = self.extract_features(x)\n x = self.pool(x)\n x = x.view(bs, -1)\n x = self.dropout(x)\n x = self.fc(x)\n return x\n```\nThen, the eeg 'reshaped' to bsx3x160x1000. With efficinetnetb5, actives public lb 0.230 978, private lb 0.282 873.\n\nThere are 3 models , but slightly different, mne-filter-efficinetnetb5, scipy.signal-filter-efficientnetb5 and mne-filter-hgnetb5. \nAnd the scires are slightly different,also.\n\n### 1.4 doublehead (eeg+spectrum)\n\nI use x3d-l to extract the spectrum feature ( with Transform50s only ), efficientnetb5 to extract the raw eeg feature, \nlike the solution figure.\nconcat the last feature. public lb 0.24, private lb 0.29 ? ? not sure\n\n# 2 Preprocess\n- 2.1.Double banana montage, eeg as 16x10000\n- 2.2.Filter with 0.5-20hz\n- 2.3.Clip by +-1024\n\n# 3. Train\n- 3.1 Stage1, 15 epochs, with loss weight voters_num/20, Adamw lr=0.001, cos scheduel\n- 3.2 Stage2, 5 epoch, loss weight=1, voters_num>=6. , Adamw lr=0.0001, cos scheduel\n- 3.3 use eeg_label_offset_seconds. I random choose an offset for each eegid, and the target is average according to eegid for each train iter.\n- 3.4 augmentation, mirror eeg, flip between left brain data and right brain data.\n- 3.5 10 folds, then move 1000 samples from val to train, left 709 samples in val set. And use vote_num>=6 to do validation. \n\n# 4. ensemble\n\nBy combining these models, I think it could get my current score. However my score is 6 model emsemble, but not improvement that much (0.28->0.27 private lb). \nFinal ensemble including: \n- 2 spectrum model (x3d, efficientnetb5), both use mne-filter\n- 3 raw eeg model( efficientnetb5 with-mne.filter, efficientnetb5 -butter filter, 1 hgnetb5 mne.filter),\n- 1 eeg-spectrum mix model, butter filter, \n\nWith weights=[0.1,0.1,0.2,0.2,0.2,0.2].\n\nps. with-mne.filter means use mne lib to do filter, butter filter means use scipy.signal, just to add some diversity.\n\n# Some thought\nI think raw eeg is more important in this task. Feeding raw EEG data into a 2D visual model is somewhat similar to how humans observe EEG signals. Observe in time and channel dimensions！\n\n", "2743455": "@cooolz \n\nCongratulations on your solo 2nd place. Very impressed with your very unique solution!\n\n```\nreshaped_tensor = x.view(bs, 16, 1000, 10)\nreshaped_and_permuted_tensor = reshaped_tensor.permute(0, 1, 3, 2)\nreshaped_and_permuted_tensor = reshaped_and_permuted_tensor.reshape(bs, 16 * 10, 1000)\nx = torch.unsqueeze(reshaped_and_permuted_tensor, dim=1)\nx = torch.cat([x, x, x], dim=1)\n```\n\n> Then, the eeg 'reshaped' to bsx3x80x1000.\n\nIs the size of the input to the model `bsx3x80x1000` or `bsx3x160x1000`?\nThe former seems to contradict the shape of x in the pseudo code.", "2742899": "Great solution, congratulations!\n> 2D-CNN is not well-equipped to capture positional information within the channels\n\nWhat do you mean here, can you elaborate?", "2742695": "In your opinion, would Vision transformer-based models for example CCT, CvT, and ViT do well on the spectrum stage?", "2744746": "Using 3d cnn is a great idea. Like many I was puzzled by why using channels was not effective. But you moved past it and addressed it with 3d models. Well done, and well deserved 2nd prize!", "2743871": "Well done @cooolz on solo 2nd. Impressive 💪\nI had not tried stft transform as preprocess; I am interested to see a snippet of the code for preprocessing the signal for input to the model; or understand how it was done. I guess the detail is important. \nWe tried x3d_m and _l as 3d model of spectrograms. They performed well, but in a blend they added no extra information over our 2d CNN. I guess with spectrum as input data it would have performed better in the blend. ", "2742774": ">View eeg (bsx16x10000) as an image. So expand dim=1( bsx1x16x10000), but the time dimension is too large. Then I make a reshape.\n\nWow, brilliant. Great idea!", "2742773": "Thanks for sharing , 3d image model and your method of eeg reshape seems much better then juse simplly resize, thanks for sharing, learned a lot and congratulation on solo 2nd place!", "2886257": "I will learn it in future", "2825957": "nice solution for me💝", "2764034": "\nCongratulations! Thanks for sharing your work. Are you using transfer learning or fitting these models from scratch. As a beginner, can you please help me understand how you compare between model architectures (either with transfer learning or \"fresh\"? Do you make sure to use the same 2-stage regime or did 2-stage learning come after you have chosen the final model?\n", "2747447": "Thank you for sharing your comprehensive approach and the detailed insights into your modeling techniques. Your innovative use of both spectral and raw EEG data with a blend of 3D and 2D CNNs provides valuable lessons on tackling position-sensitive label challenges in EEG analysis.", "2746452": "Very helpful. 3D-CNN is a model structure that I have never thought of before, which has also inspired my personal research topic.", "2744869": "Congratulations on second place and thanks for sharing this solution!\nWe tried using 3D-CNN on the raw EEG data, but not on the spectrograms, so it is great to see that you tried it on the spectrograms and it worked.", "2744693": "Congratulations on winning the 2nd prize in this competition. ", "2743571": "Hello, congratulations on winning second place~~\nThis competition is not an easy task for anyone.\n\nI would like to ask a question about the ensemble, which can shed some light on the technique and direction of thinking. If I have doubts about the results, where should I test or modify?\nTHX!!", "2743009": "Interesting approach to eeg model. Thanks for sharing and congratulations!", "2742967": "An incredible solution! You have no idea how painful it is for me! At the very beginning of the competition, I tested the eeg as an image, but then it seemed to me that it was not working. I also tried to use 3d models, but training did not start on tensorflow (tensorflow clearly had a problem here). Congratulations again!", "2742896": "Our team never thought of 3d architecture, and I learn a lot from your method. Congratulation !", "2742844": "Wow, great idea! Thanks for sharing your approach. It will be very helpful in the upcoming competitions.", "2742736": "", "2742719": "" } }