{
  "id": 492429,
  "title": "9th Place Solution",
  "url": "/competitions/hms-harmful-brain-activity-classification/writeups/rist-waves-9th-place-solution",
  "author_name": "",
  "post_date": "2024-04-09T16:06:07.458378500Z",
  "votes": 43,
  "comment_count": 8,
  "views": 0,
  "content": "<p>First of all, we would like to express our gratitude to Harvard University and Kaggle for organizing such an interesting contest. It was a competition in which various approaches could be considered for preprocessing and modeling, allowing us to learn a great deal through trial and error.</p>\n<h1>Overview</h1>\n<ul>\n<li>Weighted ensemble of 2D models. CV score vote_num&gt;=10 was 0.2023.</li>\n<li>To efficiently blend a large number of oof predictions, L-BFGS-B was used.</li>\n</ul>\n<h1>ishikei part</h1>\n<h2>dataset</h2>\n<h3>raw EEG feature</h3>\n<ul>\n<li>The signal preprocessing is similar to sqrt4kaido's method.</li>\n<li>Waveforms with various channels were created, up to 50 channels (Longitudinal / Transverse Bipolar Montage, raw-waves). </li>\n<li>Some models also used EKG-ch as input. This contributed to the diversity in ensemble.</li>\n</ul>\n<h2>Model</h2>\n<ul>\n<li>Dual (raw-eeg, eeg-spec), Triple (raw-eeg, eeg-spec, kaggle-spec) Encoder Model.</li>\n</ul>\n<h3>FeatureExtractor</h3>\n<h4>raw EEG</h4>\n<ul>\n<li>2D wavegrams were created by cropping sequences with various combinations of time windows (50s and 10s, 40s and 10s, 30s and 10s, etc.) and extracting features with GRUs for each of these time windows.</li>\n</ul>\n<h4>EEG-spectrogram</h4>\n<ul>\n<li>EEG Mel-spectrogram was created according to chris's method (4ch).</li>\n</ul>\n<h3>Encoder, Classification head</h3>\n<ul>\n<li>Each wavegram, Mel-spectrogram, is not concatenated in the w,h axis, but is passed through the backbone individually and averaged after applying attention layers.</li>\n<li>Contrastive loss was applied between raw-eeg and eeg-spec features. This improved both CV and LB by about 0.01.</li>\n<li>As aux loss, the presence of NaN in the waveform was predicted. This slightly improves CV by about 0.002~5.</li>\n<li>efficientnetv2_s, efficientnetv2_m, and resnet34d are used as backbone.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2F0d58b82f47e89a2b07b42785585c3ea7%2Fishikei.png?generation=1712677137353370&amp;alt=media\" alt=\"ishikei pipeline\"></li>\n</ul>\n<h2>Training</h2>\n<ul>\n<li>5fold StratifiedGroupKFold</li>\n<li>Basically, one eeg_sub_id is selected from each eeg_id for train and test. The sub_id with the highest number of annotators and closest to the center of the sequence is selected from each eeg_id.</li>\n<li>First, trained on data with vote_num&gt;=4, then switched to vote_num&gt;=10 and fine-tuned.</li>\n<li>In some models, pseudo-labeling was performed on data with vote_num&lt;10. That is, after fine-tuning with vote_num&gt;=10, pseudo-label was given to data with vote_num&lt;10 using that checkpoint.</li>\n<li>Finally, several epochs of fine-tuning were performed using both the pseudo-labeled data with vote_num&lt;10 and the ground truth with vote_num&gt;=10.</li>\n</ul>\n<h2>Ensemble</h2>\n<ul>\n<li>L-BFGS-B was used to efficiently blend a large number of team members' oofs (about 150 models). This was superior to Nelder-Mead in terms of speed and accuracy.</li>\n<li>First, the entire 150model is used to find the ensemble weight. Since more than half of the models have almost zero weight, the ensemble weight is calculated again after removing them.</li>\n<li>In this way, the models with low weight are gradually removed, and the process is repeated until a well-balanced combination of CV and number of models is finally obtained. (Usually, it is about 20~30models).</li>\n</ul>\n<h1>sqrt4kaido part</h1>\n<h2>Total Pipeline</h2>\n<p>Used the 2nd stage pipeline:</p>\n<ul>\n<li>1st stage: vote_num &gt;= 4</li>\n<li>2nd stage: vote_num &gt;= 10</li>\n<li>CV: Stratified 5 (or 10) fold</li>\n</ul>\n<h2>Label</h2>\n<ul>\n<li>For training, randomly select one label from each eeg_id</li>\n<li>During training, if vote_num is 10 or more, increase or decrease the number of votes from classes with non-zero votes (label augmentation)</li>\n<li>For validation, infer all label_ids and average them for each eeg_id, then average these results for each eeg_id</li>\n</ul>\n<h2>Preprocessing</h2>\n<ul>\n<li>Apply 60Hz notch filter and 0.4Hz high-pass filter</li>\n<li>Use robust scaler</li>\n<li>Data augmentation techniques:<ul>\n<li>Vertical flip</li>\n<li>Multiply by -1</li>\n<li>Waveform-based mixup</li>\n<li>Channel swapping</li></ul></li>\n<li>Use 16 channels (Longitudinal Bipolar Montage)</li>\n</ul>\n<h2>Model</h2>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdcd9b4bd57e68c4e1c6480532384c86c%2FHMS_model%20(1).png?generation=1712678061350592&amp;alt=media\" alt=\"sqrt4kaido pipeline\"></p>\n<p>Created a model with 4 encoders:</p>\n<ul>\n<li>1D LSTM:<ul>\n<li>Pass the signal through an LSTM with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.</li></ul></li>\n<li>1D WaveNet:<ul>\n<li>Pass the signal through a WaveNet with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.</li></ul></li>\n<li>2D Spectrogram(by torchaudio):<ul>\n<li>Calculate the spectrogram for 50s data and input it into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 16 channels at once).</li>\n<li>Calculate the spectrogram for 10s data with finer resolution than 50s data and input it into a 2D model to obtain embeddings (process each channel separately, not all 16 channels at once).</li>\n<li>Concatenate the above results to create a 2D image.</li></ul></li>\n<li>Kaggle Spectrogram:<ul>\n<li>Input 4 channels (e.g., LL~) into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 4 channels at once).</li></ul></li>\n<li>Feed each of the above results into a subsequent 2D model (tf_efficientnet_b0_ns), then concatenate and pass through a linear layer.</li>\n<li>Loss: KL divergence loss from each output and contrastive loss</li>\n<li>CV: 0.2247 (vote_num &gt;= 10, 10-fold)</li>\n</ul>\n<h2>Ensemble</h2>\n<ul>\n<li>Combine models from each of the 4 encoders</li>\n<li>Use 5-fold and 10-fold cross-validation</li>\n<li>Utilize tf_efficientnet_b0_ns and tf_efficientnetv2_s.in21k_ft_in1k models</li>\n<li>16 channels or 30 channels (add Transverse Bipolar Montage)</li>\n</ul>\n<h1>2g part</h1>\n<p>(Only 1D model for EEG in 2g part)<br>\nWe combined various features, feature extractors, augmentations, and backbones for ensemble model's diversity.</p>\n<h2>Total pipeline</h2>\n<p>(almost same as ishikei and sqrt4kaido part)  </p>\n<p>1st Stage: vote_num&gt;=4 (with sample weight made by vote_num)<br>\n2nd Stage: fine-tuning with vote_num&gt;=10<br>\n3rd Stage: re-fine-tuning for 1st stage model with real labels (vote_num&gt;=10) and pseudo-labels (vote_num&lt;10) predicted by the 2nd stage model.</p>\n<h2>Preprocessing</h2>\n<ul>\n<li>The train data was grouped by EEG_id and label (seizure, lrpd, etc.).<br>\n(Sub_id of each record was selected randomly in training as a form of augmentation)</li>\n<li>Signal Preprocessing: Butter Low-Pass Filter (as shown <a href=\"https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52?scriptVersionId=160158478&amp;cellId=12\" target=\"_blank\">here</a>)</li>\n</ul>\n<h2>Features</h2>\n<ul>\n<li>Waveforms with various channels were created, up to 50 channels. Primarily used double banana chains and occasionally used other chains as well.</li>\n<li>Ratio of NAN and constant (noise? like 9999, -9999) ratio in 50s or 10s.<br>\n(Concatenated at the final layer of the model)</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdc808a9930ac432a05b2ab9dbaf29264%2F2g_1.png?generation=1712678309931037&amp;alt=media\" alt=\"2g_1\"></p>\n<h2>model</h2>\n<ul>\n<li>Feature extractor (thanks for sharing the great notebooks <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>, <a href=\"https://www.kaggle.com/abaojiang\" target=\"_blank\">@abaojiang</a>)<ul>\n<li><a href=\"https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52#Butter-Low-Pass-Filter\" target=\"_blank\">wavenet</a></li>\n<li><a href=\"https://www.kaggle.com/code/abaojiang/lb-0-46-dilatedinception-wavenet-training\" target=\"_blank\">DilatedInception WaveNet</a></li></ul></li>\n<li>EEG features for 50s and 10s are used.</li>\n<li>1D conv for downsampling before input feature extractor</li>\n<li>EEG features corresponding to the right and left brain were paired and processed through 1D conv for reducing to 1 channel and downsampling.</li>\n<li>Output of feature extractor was concatenated to form an image</li>\n<li>The image was processed with EfficientNet_b0_ns or EfficientNet_v2 for final prediction</li>\n<li>Binary prediction (other or not) also conducted with BCELoss<br>\n(Loss ratio: KLDLoss (6 class):BCELoss (binary) = 0.7 : 0.3)</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fe5228079a9569e55280335b4cb35da80%2F2g_2.png?generation=1712678515475132&amp;alt=media\" alt=\"2g pipeline\"></p>\n<h2>Training</h2>\n<ul>\n<li>5 fold StratifiedGroupKFold</li>\n<li>For training, randomly select one label (offset) from each EEG_id</li>\n<li>CV score calculation: infer all label_ids and average them for each EEG_id, then average these results for each EEG_id  (same as sqrt4kaido part)</li>\n</ul>\n<h3>augmentation</h3>\n<ul>\n<li>vertical flip</li>\n<li>cutout several EEG features (cut out the entirety of a specific channel).</li>\n<li>time shift：sometime effective, sometimes not.</li>\n</ul>",
  "messages": [
    {
      "id": "2743784",
      "postDate": "04/09/2024 16:06:07",
      "content": "<p>First of all, we would like to express our gratitude to Harvard University and Kaggle for organizing such an interesting contest. It was a competition in which various approaches could be considered for preprocessing and modeling, allowing us to learn a great deal through trial and error.</p>\n<h1>Overview</h1>\n<ul>\n<li>Weighted ensemble of 2D models. CV score vote_num&gt;=10 was 0.2023.</li>\n<li>To efficiently blend a large number of oof predictions, L-BFGS-B was used.</li>\n</ul>\n<h1>ishikei part</h1>\n<h2>dataset</h2>\n<h3>raw EEG feature</h3>\n<ul>\n<li>The signal preprocessing is similar to sqrt4kaido's method.</li>\n<li>Waveforms with various channels were created, up to 50 channels (Longitudinal / Transverse Bipolar Montage, raw-waves). </li>\n<li>Some models also used EKG-ch as input. This contributed to the diversity in ensemble.</li>\n</ul>\n<h2>Model</h2>\n<ul>\n<li>Dual (raw-eeg, eeg-spec), Triple (raw-eeg, eeg-spec, kaggle-spec) Encoder Model.</li>\n</ul>\n<h3>FeatureExtractor</h3>\n<h4>raw EEG</h4>\n<ul>\n<li>2D wavegrams were created by cropping sequences with various combinations of time windows (50s and 10s, 40s and 10s, 30s and 10s, etc.) and extracting features with GRUs for each of these time windows.</li>\n</ul>\n<h4>EEG-spectrogram</h4>\n<ul>\n<li>EEG Mel-spectrogram was created according to chris's method (4ch).</li>\n</ul>\n<h3>Encoder, Classification head</h3>\n<ul>\n<li>Each wavegram, Mel-spectrogram, is not concatenated in the w,h axis, but is passed through the backbone individually and averaged after applying attention layers.</li>\n<li>Contrastive loss was applied between raw-eeg and eeg-spec features. This improved both CV and LB by about 0.01.</li>\n<li>As aux loss, the presence of NaN in the waveform was predicted. This slightly improves CV by about 0.002~5.</li>\n<li>efficientnetv2_s, efficientnetv2_m, and resnet34d are used as backbone.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2F0d58b82f47e89a2b07b42785585c3ea7%2Fishikei.png?generation=1712677137353370&amp;alt=media\" alt=\"ishikei pipeline\"></li>\n</ul>\n<h2>Training</h2>\n<ul>\n<li>5fold StratifiedGroupKFold</li>\n<li>Basically, one eeg_sub_id is selected from each eeg_id for train and test. The sub_id with the highest number of annotators and closest to the center of the sequence is selected from each eeg_id.</li>\n<li>First, trained on data with vote_num&gt;=4, then switched to vote_num&gt;=10 and fine-tuned.</li>\n<li>In some models, pseudo-labeling was performed on data with vote_num&lt;10. That is, after fine-tuning with vote_num&gt;=10, pseudo-label was given to data with vote_num&lt;10 using that checkpoint.</li>\n<li>Finally, several epochs of fine-tuning were performed using both the pseudo-labeled data with vote_num&lt;10 and the ground truth with vote_num&gt;=10.</li>\n</ul>\n<h2>Ensemble</h2>\n<ul>\n<li>L-BFGS-B was used to efficiently blend a large number of team members' oofs (about 150 models). This was superior to Nelder-Mead in terms of speed and accuracy.</li>\n<li>First, the entire 150model is used to find the ensemble weight. Since more than half of the models have almost zero weight, the ensemble weight is calculated again after removing them.</li>\n<li>In this way, the models with low weight are gradually removed, and the process is repeated until a well-balanced combination of CV and number of models is finally obtained. (Usually, it is about 20~30models).</li>\n</ul>\n<h1>sqrt4kaido part</h1>\n<h2>Total Pipeline</h2>\n<p>Used the 2nd stage pipeline:</p>\n<ul>\n<li>1st stage: vote_num &gt;= 4</li>\n<li>2nd stage: vote_num &gt;= 10</li>\n<li>CV: Stratified 5 (or 10) fold</li>\n</ul>\n<h2>Label</h2>\n<ul>\n<li>For training, randomly select one label from each eeg_id</li>\n<li>During training, if vote_num is 10 or more, increase or decrease the number of votes from classes with non-zero votes (label augmentation)</li>\n<li>For validation, infer all label_ids and average them for each eeg_id, then average these results for each eeg_id</li>\n</ul>\n<h2>Preprocessing</h2>\n<ul>\n<li>Apply 60Hz notch filter and 0.4Hz high-pass filter</li>\n<li>Use robust scaler</li>\n<li>Data augmentation techniques:<ul>\n<li>Vertical flip</li>\n<li>Multiply by -1</li>\n<li>Waveform-based mixup</li>\n<li>Channel swapping</li></ul></li>\n<li>Use 16 channels (Longitudinal Bipolar Montage)</li>\n</ul>\n<h2>Model</h2>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdcd9b4bd57e68c4e1c6480532384c86c%2FHMS_model%20(1).png?generation=1712678061350592&amp;alt=media\" alt=\"sqrt4kaido pipeline\"></p>\n<p>Created a model with 4 encoders:</p>\n<ul>\n<li>1D LSTM:<ul>\n<li>Pass the signal through an LSTM with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.</li></ul></li>\n<li>1D WaveNet:<ul>\n<li>Pass the signal through a WaveNet with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.</li></ul></li>\n<li>2D Spectrogram(by torchaudio):<ul>\n<li>Calculate the spectrogram for 50s data and input it into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 16 channels at once).</li>\n<li>Calculate the spectrogram for 10s data with finer resolution than 50s data and input it into a 2D model to obtain embeddings (process each channel separately, not all 16 channels at once).</li>\n<li>Concatenate the above results to create a 2D image.</li></ul></li>\n<li>Kaggle Spectrogram:<ul>\n<li>Input 4 channels (e.g., LL~) into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 4 channels at once).</li></ul></li>\n<li>Feed each of the above results into a subsequent 2D model (tf_efficientnet_b0_ns), then concatenate and pass through a linear layer.</li>\n<li>Loss: KL divergence loss from each output and contrastive loss</li>\n<li>CV: 0.2247 (vote_num &gt;= 10, 10-fold)</li>\n</ul>\n<h2>Ensemble</h2>\n<ul>\n<li>Combine models from each of the 4 encoders</li>\n<li>Use 5-fold and 10-fold cross-validation</li>\n<li>Utilize tf_efficientnet_b0_ns and tf_efficientnetv2_s.in21k_ft_in1k models</li>\n<li>16 channels or 30 channels (add Transverse Bipolar Montage)</li>\n</ul>\n<h1>2g part</h1>\n<p>(Only 1D model for EEG in 2g part)<br>\nWe combined various features, feature extractors, augmentations, and backbones for ensemble model's diversity.</p>\n<h2>Total pipeline</h2>\n<p>(almost same as ishikei and sqrt4kaido part)  </p>\n<p>1st Stage: vote_num&gt;=4 (with sample weight made by vote_num)<br>\n2nd Stage: fine-tuning with vote_num&gt;=10<br>\n3rd Stage: re-fine-tuning for 1st stage model with real labels (vote_num&gt;=10) and pseudo-labels (vote_num&lt;10) predicted by the 2nd stage model.</p>\n<h2>Preprocessing</h2>\n<ul>\n<li>The train data was grouped by EEG_id and label (seizure, lrpd, etc.).<br>\n(Sub_id of each record was selected randomly in training as a form of augmentation)</li>\n<li>Signal Preprocessing: Butter Low-Pass Filter (as shown <a href=\"https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52?scriptVersionId=160158478&amp;cellId=12\" target=\"_blank\">here</a>)</li>\n</ul>\n<h2>Features</h2>\n<ul>\n<li>Waveforms with various channels were created, up to 50 channels. Primarily used double banana chains and occasionally used other chains as well.</li>\n<li>Ratio of NAN and constant (noise? like 9999, -9999) ratio in 50s or 10s.<br>\n(Concatenated at the final layer of the model)</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdc808a9930ac432a05b2ab9dbaf29264%2F2g_1.png?generation=1712678309931037&amp;alt=media\" alt=\"2g_1\"></p>\n<h2>model</h2>\n<ul>\n<li>Feature extractor (thanks for sharing the great notebooks <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>, <a href=\"https://www.kaggle.com/abaojiang\" target=\"_blank\">@abaojiang</a>)<ul>\n<li><a href=\"https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52#Butter-Low-Pass-Filter\" target=\"_blank\">wavenet</a></li>\n<li><a href=\"https://www.kaggle.com/code/abaojiang/lb-0-46-dilatedinception-wavenet-training\" target=\"_blank\">DilatedInception WaveNet</a></li></ul></li>\n<li>EEG features for 50s and 10s are used.</li>\n<li>1D conv for downsampling before input feature extractor</li>\n<li>EEG features corresponding to the right and left brain were paired and processed through 1D conv for reducing to 1 channel and downsampling.</li>\n<li>Output of feature extractor was concatenated to form an image</li>\n<li>The image was processed with EfficientNet_b0_ns or EfficientNet_v2 for final prediction</li>\n<li>Binary prediction (other or not) also conducted with BCELoss<br>\n(Loss ratio: KLDLoss (6 class):BCELoss (binary) = 0.7 : 0.3)</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fe5228079a9569e55280335b4cb35da80%2F2g_2.png?generation=1712678515475132&amp;alt=media\" alt=\"2g pipeline\"></p>\n<h2>Training</h2>\n<ul>\n<li>5 fold StratifiedGroupKFold</li>\n<li>For training, randomly select one label (offset) from each EEG_id</li>\n<li>CV score calculation: infer all label_ids and average them for each EEG_id, then average these results for each EEG_id  (same as sqrt4kaido part)</li>\n</ul>\n<h3>augmentation</h3>\n<ul>\n<li>vertical flip</li>\n<li>cutout several EEG features (cut out the entirety of a specific channel).</li>\n<li>time shift：sometime effective, sometimes not.</li>\n</ul>",
      "rawMarkdown": "First of all, we would like to express our gratitude to Harvard University and Kaggle for organizing such an interesting contest. It was a competition in which various approaches could be considered for preprocessing and modeling, allowing us to learn a great deal through trial and error.\n\n# Overview\n- Weighted ensemble of 2D models. CV score vote_num>=10 was 0.2023.\n- To efficiently blend a large number of oof predictions, L-BFGS-B was used.\n\n# ishikei part\n## dataset\n### raw EEG feature\n- The signal preprocessing is similar to sqrt4kaido's method.\n- Waveforms with various channels were created, up to 50 channels (Longitudinal / Transverse Bipolar Montage, raw-waves). \n- Some models also used EKG-ch as input. This contributed to the diversity in ensemble.\n\n## Model\n- Dual (raw-eeg, eeg-spec), Triple (raw-eeg, eeg-spec, kaggle-spec) Encoder Model.\n\n### FeatureExtractor\n#### raw EEG\n- 2D wavegrams were created by cropping sequences with various combinations of time windows (50s and 10s, 40s and 10s, 30s and 10s, etc.) and extracting features with GRUs for each of these time windows.\n#### EEG-spectrogram\n- EEG Mel-spectrogram was created according to chris's method (4ch).\n\n### Encoder, Classification head\n- Each wavegram, Mel-spectrogram, is not concatenated in the w,h axis, but is passed through the backbone individually and averaged after applying attention layers.\n- Contrastive loss was applied between raw-eeg and eeg-spec features. This improved both CV and LB by about 0.01.\n- As aux loss, the presence of NaN in the waveform was predicted. This slightly improves CV by about 0.002~5.\n- efficientnetv2_s, efficientnetv2_m, and resnet34d are used as backbone.\n![ishikei pipeline](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2F0d58b82f47e89a2b07b42785585c3ea7%2Fishikei.png?generation=1712677137353370&alt=media)\n\n## Training\n- 5fold StratifiedGroupKFold\n- Basically, one eeg_sub_id is selected from each eeg_id for train and test. The sub_id with the highest number of annotators and closest to the center of the sequence is selected from each eeg_id.\n- First, trained on data with vote_num>=4, then switched to vote_num>=10 and fine-tuned.\n- In some models, pseudo-labeling was performed on data with vote_num<10. That is, after fine-tuning with vote_num>=10, pseudo-label was given to data with vote_num<10 using that checkpoint.\n- Finally, several epochs of fine-tuning were performed using both the pseudo-labeled data with vote_num<10 and the ground truth with vote_num>=10.\n\n## Ensemble\n- L-BFGS-B was used to efficiently blend a large number of team members' oofs (about 150 models). This was superior to Nelder-Mead in terms of speed and accuracy.\n- First, the entire 150model is used to find the ensemble weight. Since more than half of the models have almost zero weight, the ensemble weight is calculated again after removing them.\n- In this way, the models with low weight are gradually removed, and the process is repeated until a well-balanced combination of CV and number of models is finally obtained. (Usually, it is about 20~30models).\n\n\n# sqrt4kaido part\n\n## Total Pipeline\nUsed the 2nd stage pipeline:\n- 1st stage: vote_num >= 4\n- 2nd stage: vote_num >= 10\n- CV: Stratified 5 (or 10) fold\n  \n## Label\n- For training, randomly select one label from each eeg_id\n- During training, if vote_num is 10 or more, increase or decrease the number of votes from classes with non-zero votes (label augmentation)\n- For validation, infer all label_ids and average them for each eeg_id, then average these results for each eeg_id\n  \n## Preprocessing\n- Apply 60Hz notch filter and 0.4Hz high-pass filter\n- Use robust scaler\n- Data augmentation techniques:\n  - Vertical flip\n  - Multiply by -1\n  - Waveform-based mixup\n  - Channel swapping\n- Use 16 channels (Longitudinal Bipolar Montage)\n  \n## Model\n![sqrt4kaido pipeline](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdcd9b4bd57e68c4e1c6480532384c86c%2FHMS_model%20(1).png?generation=1712678061350592&alt=media)\n\nCreated a model with 4 encoders:\n- 1D LSTM:\n  - Pass the signal through an LSTM with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.\n- 1D WaveNet:\n  - Pass the signal through a WaveNet with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.\n- 2D Spectrogram(by torchaudio):\n  - Calculate the spectrogram for 50s data and input it into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 16 channels at once).\n  - Calculate the spectrogram for 10s data with finer resolution than 50s data and input it into a 2D model to obtain embeddings (process each channel separately, not all 16 channels at once).\n  - Concatenate the above results to create a 2D image.\n- Kaggle Spectrogram:\n  - Input 4 channels (e.g., LL~) into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 4 channels at once).\n- Feed each of the above results into a subsequent 2D model (tf_efficientnet_b0_ns), then concatenate and pass through a linear layer.\n- Loss: KL divergence loss from each output and contrastive loss\n- CV: 0.2247 (vote_num >= 10, 10-fold)\n\n## Ensemble\n- Combine models from each of the 4 encoders\n- Use 5-fold and 10-fold cross-validation\n- Utilize tf_efficientnet_b0_ns and tf_efficientnetv2_s.in21k_ft_in1k models\n- 16 channels or 30 channels (add Transverse Bipolar Montage)\n\n\n# 2g part\n\n(Only 1D model for EEG in 2g part)\nWe combined various features, feature extractors, augmentations, and backbones for ensemble model's diversity.\n\n## Total pipeline\n(almost same as ishikei and sqrt4kaido part)  \n\n1st Stage: vote_num>=4 (with sample weight made by vote_num)\n2nd Stage: fine-tuning with vote_num>=10\n3rd Stage: re-fine-tuning for 1st stage model with real labels (vote_num>=10) and pseudo-labels (vote_num<10) predicted by the 2nd stage model.\n\n## Preprocessing\n\n- The train data was grouped by EEG_id and label (seizure, lrpd, etc.).\n(Sub_id of each record was selected randomly in training as a form of augmentation)\n- Signal Preprocessing: Butter Low-Pass Filter (as shown [here](https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52?scriptVersionId=160158478&cellId=12))\n\n## Features\n- Waveforms with various channels were created, up to 50 channels. Primarily used double banana chains and occasionally used other chains as well.\n- Ratio of NAN and constant (noise? like 9999, -9999) ratio in 50s or 10s.\n(Concatenated at the final layer of the model)\n\n![2g_1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdc808a9930ac432a05b2ab9dbaf29264%2F2g_1.png?generation=1712678309931037&alt=media)\n\n## model\n- Feature extractor (thanks for sharing the great notebooks @cdeotte, @abaojiang)\n    - [wavenet](https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52#Butter-Low-Pass-Filter)\n    - [DilatedInception WaveNet](https://www.kaggle.com/code/abaojiang/lb-0-46-dilatedinception-wavenet-training)\n- EEG features for 50s and 10s are used.\n- 1D conv for downsampling before input feature extractor\n- EEG features corresponding to the right and left brain were paired and processed through 1D conv for reducing to 1 channel and downsampling.\n- Output of feature extractor was concatenated to form an image\n- The image was processed with EfficientNet_b0_ns or EfficientNet_v2 for final prediction\n- Binary prediction (other or not) also conducted with BCELoss\n(Loss ratio: KLDLoss (6 class):BCELoss (binary) = 0.7 : 0.3)\n\n![2g pipeline](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fe5228079a9569e55280335b4cb35da80%2F2g_2.png?generation=1712678515475132&alt=media)\n\n## Training\n- 5 fold StratifiedGroupKFold\n- For training, randomly select one label (offset) from each EEG_id\n- CV score calculation: infer all label_ids and average them for each EEG_id, then average these results for each EEG_id  (same as sqrt4kaido part)\n\n### augmentation\n- vertical flip\n- cutout several EEG features (cut out the entirety of a specific channel).\n- time shift：sometime effective, sometimes not.",
      "votes": null
    },
    {
      "id": "2743986",
      "postDate": "04/09/2024 17:33:37",
      "content": "<p>Great work, I need to read up on contrastive loss. Looks like it helped a lot.<br>\nThis is clever -<code>As aux loss, the presence of NaN in the waveform was predicted.</code></p>",
      "rawMarkdown": "Great work, I need to read up on contrastive loss. Looks like it helped a lot.\nThis is clever -`As aux loss, the presence of NaN in the waveform was predicted.`",
      "votes": null
    },
    {
      "id": "2744514",
      "postDate": "04/10/2024 00:35:21",
      "content": "<p>Congratulation on a gold medal!<br>\nWhat was the shape after 1D feature extraction, and how did you convert them into a 2D image?</p>",
      "rawMarkdown": "Congratulation on a gold medal!\nWhat was the shape after 1D feature extraction, and how did you convert them into a 2D image?",
      "votes": null
    },
    {
      "id": "2744683",
      "postDate": "04/10/2024 04:23:23",
      "content": "<p>Congratulations on securing 9th place in this competition. Thanks for sharing  your implementation details.  </p>",
      "rawMarkdown": "Congratulations on securing 9th place in this competition. Thanks for sharing  your implementation details.",
      "votes": null
    },
    {
      "id": "2746012",
      "postDate": "04/11/2024 01:44:33",
      "content": "<p><a href=\"https://www.kaggle.com/tamotamo\" target=\"_blank\">@tamotamo</a> </p>\n<p>Thank you !!</p>\n<blockquote>\n  <p>What was the shape after 1D feature extraction, and how did you convert them into a 2D image?</p>\n</blockquote>\n<p>Whose part?</p>",
      "rawMarkdown": "tamotamo \n\nThank you !!\n\n>What was the shape after 1D feature extraction, and how did you convert them into a 2D image?\n\nWhose part?",
      "votes": null
    },
    {
      "id": "2746036",
      "postDate": "04/11/2024 02:27:12",
      "content": "<p><a href=\"https://www.kaggle.com/nomorevotch\" target=\"_blank\">@nomorevotch</a> <a href=\"https://www.kaggle.com/ystsuji\" target=\"_blank\">@ystsuji</a> <br>\nCongratulations. I liked the approach of 2-stage encoding. It seems versatile approach for low level feature extraction alternative to spectrograms. How to be motivated to use this idea?</p>",
      "rawMarkdown": "nomorevotch @ystsuji \nCongratulations. I liked the approach of 2-stage encoding. It seems versatile approach for low level feature extraction alternative to spectrograms. How to be motivated to use this idea?",
      "votes": null
    },
    {
      "id": "2746228",
      "postDate": "04/11/2024 05:36:25",
      "content": "<p>It seems that this approach is used in all parts, but how about <a href=\"https://www.kaggle.com/ystsuji\" target=\"_blank\">@ystsuji</a> 's part?</p>",
      "rawMarkdown": "It seems that this approach is used in all parts, but how about @ystsuji 's part?",
      "votes": null
    },
    {
      "id": "2746462",
      "postDate": "04/11/2024 08:23:50",
      "content": "<p><a href=\"https://www.kaggle.com/tamotamo\" target=\"_blank\">@tamotamo</a> <br>\nin my part…</p>\n<ul>\n<li>After passing through a feature extractor and pooling, a single EEG feature has [64ch x 480seq].  </li>\n<li>These are concatenated in the channel direction to create an image.  </li>\n<li>Due to the large size of the image, it is down to 2048×480 by pooling for after 2DCNN.</li>\n</ul>",
      "rawMarkdown": "tamotamo \nin my part...\n- After passing through a feature extractor and pooling, a single EEG feature has [64ch x 480seq].  \n- These are concatenated in the channel direction to create an image.  \n- Due to the large size of the image, it is down to 2048×480 by pooling for after 2DCNN.",
      "votes": null
    },
    {
      "id": "2746477",
      "postDate": "04/11/2024 08:41:41",
      "content": "<p>Congratulations on your 4th place finish as well.</p>\n<p>I was inspired by the code tubo used in the 「Child Mind Institute - Detect Sleep States」 competition. This pipeline was also very useful in this competition.</p>\n<p><a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940</a></p>",
      "rawMarkdown": "Congratulations on your 4th place finish as well.\n\nI was inspired by the code tubo used in the 「Child Mind Institute - Detect Sleep States」 competition. This pipeline was also very useful in this competition.\n\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2743986,
      "author_name": "darraghdog",
      "author_url": "",
      "post_date": "04/09/2024 17:33:37",
      "content": "<p>Great work, I need to read up on contrastive loss. Looks like it helped a lot.<br>\nThis is clever -<code>As aux loss, the presence of NaN in the waveform was predicted.</code></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2744514,
      "author_name": "tamotamo",
      "author_url": "",
      "post_date": "04/10/2024 00:35:21",
      "content": "<p>Congratulation on a gold medal!<br>\nWhat was the shape after 1D feature extraction, and how did you convert them into a 2D image?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2746012,
          "author_name": "ystsuji",
          "author_url": "",
          "post_date": "04/11/2024 01:44:33",
          "content": "<p><a href=\"https://www.kaggle.com/tamotamo\" target=\"_blank\">@tamotamo</a> </p>\n<p>Thank you !!</p>\n<blockquote>\n  <p>What was the shape after 1D feature extraction, and how did you convert them into a 2D image?</p>\n</blockquote>\n<p>Whose part?</p>",
          "votes": null,
          "replies": [
            {
              "id": 2746228,
              "author_name": "tamotamo",
              "author_url": "",
              "post_date": "04/11/2024 05:36:25",
              "content": "<p>It seems that this approach is used in all parts, but how about <a href=\"https://www.kaggle.com/ystsuji\" target=\"_blank\">@ystsuji</a> 's part?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2746462,
                  "author_name": "ystsuji",
                  "author_url": "",
                  "post_date": "04/11/2024 08:23:50",
                  "content": "<p><a href=\"https://www.kaggle.com/tamotamo\" target=\"_blank\">@tamotamo</a> <br>\nin my part…</p>\n<ul>\n<li>After passing through a feature extractor and pooling, a single EEG feature has [64ch x 480seq].  </li>\n<li>These are concatenated in the channel direction to create an image.  </li>\n<li>Due to the large size of the image, it is down to 2048×480 by pooling for after 2DCNN.</li>\n</ul>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2744683,
      "author_name": "crsuthikshnkumar",
      "author_url": "",
      "post_date": "04/10/2024 04:23:23",
      "content": "<p>Congratulations on securing 9th place in this competition. Thanks for sharing  your implementation details.  </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2746036,
      "author_name": "tatamikenn",
      "author_url": "",
      "post_date": "04/11/2024 02:27:12",
      "content": "<p><a href=\"https://www.kaggle.com/nomorevotch\" target=\"_blank\">@nomorevotch</a> <a href=\"https://www.kaggle.com/ystsuji\" target=\"_blank\">@ystsuji</a> <br>\nCongratulations. I liked the approach of 2-stage encoding. It seems versatile approach for low level feature extraction alternative to spectrograms. How to be motivated to use this idea?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2746477,
          "author_name": "nomorevotch",
          "author_url": "",
          "post_date": "04/11/2024 08:41:41",
          "content": "<p>Congratulations on your 4th place finish as well.</p>\n<p>I was inspired by the code tubo used in the 「Child Mind Institute - Detect Sleep States」 competition. This pipeline was also very useful in this competition.</p>\n<p><a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2743784": "First of all, we would like to express our gratitude to Harvard University and Kaggle for organizing such an interesting contest. It was a competition in which various approaches could be considered for preprocessing and modeling, allowing us to learn a great deal through trial and error.\n\n# Overview\n- Weighted ensemble of 2D models. CV score vote_num>=10 was 0.2023.\n- To efficiently blend a large number of oof predictions, L-BFGS-B was used.\n\n# ishikei part\n## dataset\n### raw EEG feature\n- The signal preprocessing is similar to sqrt4kaido's method.\n- Waveforms with various channels were created, up to 50 channels (Longitudinal / Transverse Bipolar Montage, raw-waves). \n- Some models also used EKG-ch as input. This contributed to the diversity in ensemble.\n\n## Model\n- Dual (raw-eeg, eeg-spec), Triple (raw-eeg, eeg-spec, kaggle-spec) Encoder Model.\n\n### FeatureExtractor\n#### raw EEG\n- 2D wavegrams were created by cropping sequences with various combinations of time windows (50s and 10s, 40s and 10s, 30s and 10s, etc.) and extracting features with GRUs for each of these time windows.\n#### EEG-spectrogram\n- EEG Mel-spectrogram was created according to chris's method (4ch).\n\n### Encoder, Classification head\n- Each wavegram, Mel-spectrogram, is not concatenated in the w,h axis, but is passed through the backbone individually and averaged after applying attention layers.\n- Contrastive loss was applied between raw-eeg and eeg-spec features. This improved both CV and LB by about 0.01.\n- As aux loss, the presence of NaN in the waveform was predicted. This slightly improves CV by about 0.002~5.\n- efficientnetv2_s, efficientnetv2_m, and resnet34d are used as backbone.\n![ishikei pipeline](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2F0d58b82f47e89a2b07b42785585c3ea7%2Fishikei.png?generation=1712677137353370&alt=media)\n\n## Training\n- 5fold StratifiedGroupKFold\n- Basically, one eeg_sub_id is selected from each eeg_id for train and test. The sub_id with the highest number of annotators and closest to the center of the sequence is selected from each eeg_id.\n- First, trained on data with vote_num>=4, then switched to vote_num>=10 and fine-tuned.\n- In some models, pseudo-labeling was performed on data with vote_num<10. That is, after fine-tuning with vote_num>=10, pseudo-label was given to data with vote_num<10 using that checkpoint.\n- Finally, several epochs of fine-tuning were performed using both the pseudo-labeled data with vote_num<10 and the ground truth with vote_num>=10.\n\n## Ensemble\n- L-BFGS-B was used to efficiently blend a large number of team members' oofs (about 150 models). This was superior to Nelder-Mead in terms of speed and accuracy.\n- First, the entire 150model is used to find the ensemble weight. Since more than half of the models have almost zero weight, the ensemble weight is calculated again after removing them.\n- In this way, the models with low weight are gradually removed, and the process is repeated until a well-balanced combination of CV and number of models is finally obtained. (Usually, it is about 20~30models).\n\n\n# sqrt4kaido part\n\n## Total Pipeline\nUsed the 2nd stage pipeline:\n- 1st stage: vote_num >= 4\n- 2nd stage: vote_num >= 10\n- CV: Stratified 5 (or 10) fold\n  \n## Label\n- For training, randomly select one label from each eeg_id\n- During training, if vote_num is 10 or more, increase or decrease the number of votes from classes with non-zero votes (label augmentation)\n- For validation, infer all label_ids and average them for each eeg_id, then average these results for each eeg_id\n  \n## Preprocessing\n- Apply 60Hz notch filter and 0.4Hz high-pass filter\n- Use robust scaler\n- Data augmentation techniques:\n  - Vertical flip\n  - Multiply by -1\n  - Waveform-based mixup\n  - Channel swapping\n- Use 16 channels (Longitudinal Bipolar Montage)\n  \n## Model\n![sqrt4kaido pipeline](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdcd9b4bd57e68c4e1c6480532384c86c%2FHMS_model%20(1).png?generation=1712678061350592&alt=media)\n\nCreated a model with 4 encoders:\n- 1D LSTM:\n  - Pass the signal through an LSTM with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.\n- 1D WaveNet:\n  - Pass the signal through a WaveNet with shared weights for each channel (process each channel separately, not all 16 channels at once). Do this for both 50s and 10s data, then concatenate the results to create a 2D image.\n- 2D Spectrogram(by torchaudio):\n  - Calculate the spectrogram for 50s data and input it into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 16 channels at once).\n  - Calculate the spectrogram for 10s data with finer resolution than 50s data and input it into a 2D model to obtain embeddings (process each channel separately, not all 16 channels at once).\n  - Concatenate the above results to create a 2D image.\n- Kaggle Spectrogram:\n  - Input 4 channels (e.g., LL~) into a 2D model (resnet18d) to obtain embeddings (process each channel separately, not all 4 channels at once).\n- Feed each of the above results into a subsequent 2D model (tf_efficientnet_b0_ns), then concatenate and pass through a linear layer.\n- Loss: KL divergence loss from each output and contrastive loss\n- CV: 0.2247 (vote_num >= 10, 10-fold)\n\n## Ensemble\n- Combine models from each of the 4 encoders\n- Use 5-fold and 10-fold cross-validation\n- Utilize tf_efficientnet_b0_ns and tf_efficientnetv2_s.in21k_ft_in1k models\n- 16 channels or 30 channels (add Transverse Bipolar Montage)\n\n\n# 2g part\n\n(Only 1D model for EEG in 2g part)\nWe combined various features, feature extractors, augmentations, and backbones for ensemble model's diversity.\n\n## Total pipeline\n(almost same as ishikei and sqrt4kaido part)  \n\n1st Stage: vote_num>=4 (with sample weight made by vote_num)\n2nd Stage: fine-tuning with vote_num>=10\n3rd Stage: re-fine-tuning for 1st stage model with real labels (vote_num>=10) and pseudo-labels (vote_num<10) predicted by the 2nd stage model.\n\n## Preprocessing\n\n- The train data was grouped by EEG_id and label (seizure, lrpd, etc.).\n(Sub_id of each record was selected randomly in training as a form of augmentation)\n- Signal Preprocessing: Butter Low-Pass Filter (as shown [here](https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52?scriptVersionId=160158478&cellId=12))\n\n## Features\n- Waveforms with various channels were created, up to 50 channels. Primarily used double banana chains and occasionally used other chains as well.\n- Ratio of NAN and constant (noise? like 9999, -9999) ratio in 50s or 10s.\n(Concatenated at the final layer of the model)\n\n![2g_1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fdc808a9930ac432a05b2ab9dbaf29264%2F2g_1.png?generation=1712678309931037&alt=media)\n\n## model\n- Feature extractor (thanks for sharing the great notebooks @cdeotte, @abaojiang)\n    - [wavenet](https://www.kaggle.com/code/cdeotte/wavenet-starter-lb-0-52#Butter-Low-Pass-Filter)\n    - [DilatedInception WaveNet](https://www.kaggle.com/code/abaojiang/lb-0-46-dilatedinception-wavenet-training)\n- EEG features for 50s and 10s are used.\n- 1D conv for downsampling before input feature extractor\n- EEG features corresponding to the right and left brain were paired and processed through 1D conv for reducing to 1 channel and downsampling.\n- Output of feature extractor was concatenated to form an image\n- The image was processed with EfficientNet_b0_ns or EfficientNet_v2 for final prediction\n- Binary prediction (other or not) also conducted with BCELoss\n(Loss ratio: KLDLoss (6 class):BCELoss (binary) = 0.7 : 0.3)\n\n![2g pipeline](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1739577%2Fe5228079a9569e55280335b4cb35da80%2F2g_2.png?generation=1712678515475132&alt=media)\n\n## Training\n- 5 fold StratifiedGroupKFold\n- For training, randomly select one label (offset) from each EEG_id\n- CV score calculation: infer all label_ids and average them for each EEG_id, then average these results for each EEG_id  (same as sqrt4kaido part)\n\n### augmentation\n- vertical flip\n- cutout several EEG features (cut out the entirety of a specific channel).\n- time shift：sometime effective, sometimes not.",
    "2743986": "Great work, I need to read up on contrastive loss. Looks like it helped a lot.\nThis is clever -`As aux loss, the presence of NaN in the waveform was predicted.`",
    "2744514": "Congratulation on a gold medal!\nWhat was the shape after 1D feature extraction, and how did you convert them into a 2D image?",
    "2744683": "Congratulations on securing 9th place in this competition. Thanks for sharing  your implementation details.",
    "2746012": "tamotamo \n\nThank you !!\n\n>What was the shape after 1D feature extraction, and how did you convert them into a 2D image?\n\nWhose part?",
    "2746036": "nomorevotch @ystsuji \nCongratulations. I liked the approach of 2-stage encoding. It seems versatile approach for low level feature extraction alternative to spectrograms. How to be motivated to use this idea?",
    "2746228": "It seems that this approach is used in all parts, but how about @ystsuji 's part?",
    "2746462": "tamotamo \nin my part...\n- After passing through a feature extractor and pooling, a single EEG feature has [64ch x 480seq].  \n- These are concatenated in the channel direction to create an image.  \n- Due to the large size of the image, it is down to 2048×480 by pooling for after 2DCNN.",
    "2746477": "Congratulations on your 4th place finish as well.\n\nI was inspired by the code tubo used in the 「Child Mind Institute - Detect Sleep States」 competition. This pipeline was also very useful in this competition.\n\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940"
  },
  "source": "meta"
}