{
  "id": 275360,
  "title": "Public 69th / Private 54th solution",
  "url": "/competitions/g2net-gravitational-wave-detection/writeups/yuki-schulta-chizuchizu-public-69th-private-54th-s",
  "author_name": "",
  "post_date": "2021-10-01T01:55:28.190Z",
  "votes": 17,
  "comment_count": 2,
  "views": 0,
  "content": "<h2>Acknowledgements</h2>\n<p>Thanks to Kaggle and the hosts for holding this exciting competition! I've learned many things in this competition. Also thanks to all participants especially my teammate schulta( <a href=\"https://www.kaggle.com/schulta\" target=\"_blank\">@schulta</a> ) and Chizuchizu( <a href=\"https://www.kaggle.com/chizuchizu\" target=\"_blank\">@chizuchizu</a> )</p>\n<h2>Models</h2>\n<p>Scores of each model were as follows;</p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>EfficientB7ns</td>\n<td>0.87706</td>\n<td>0.8798</td>\n</tr>\n<tr>\n<td>EfficientB3ns</td>\n<td>0.87267</td>\n<td>0.8747</td>\n</tr>\n<tr>\n<td>1dCNN</td>\n<td>0.87281</td>\n<td>0.8769</td>\n</tr>\n<tr>\n<td>swin transformer</td>\n<td>0.86977</td>\n<td>0.8723</td>\n</tr>\n</tbody>\n</table>\n<p>Detail of each model was as follows;</p>\n<p><strong>CQT</strong></p>\n<ul>\n<li>EfficientNetB7ns<ul>\n<li>cpt parameter: {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}</li>\n<li>5fold validation</li>\n<li>Image size: 512 x 512</li>\n<li>Tripret Attention (<a href=\"https://arxiv.org/abs/2010.03045\" target=\"_blank\">paper</a>)</li>\n<li>Augmentation: shift in x-axis direction</li></ul></li>\n<li>Swin Transformer<ul>\n<li>cqt parameter {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}</li>\n<li>5fold validation</li>\n<li>Image size: 384 x 384</li>\n<li>Augmentation: shift in x-axis direction</li></ul></li>\n</ul>\n<p><strong>CWT</strong></p>\n<ul>\n<li>EfficientNetB3ns</li>\n<li>5fold validation</li>\n<li>image size : 256 x 256</li>\n<li>Tripret Attention </li>\n<li>Augmentation: shift in x-axis direction</li>\n</ul>\n<p><strong>1dCNN</strong></p>\n<ul>\n<li>We used <a href=\"https://www.kaggle.com/scaomath/g2net-1d-cnn-gem-pool-pytorch-train-inference\" target=\"_blank\">Public Kernel</a> as a baseline.Thanks to Shuhao Cao( <a href=\"https://www.kaggle.com/scaomath\" target=\"_blank\">@scaomath</a> )</li>\n<li>5fold validation</li>\n<li>add one more conv layer</li>\n<li>bandpass_params = dict(lf=25, hf=500)</li>\n</ul>\n<h2>Other things</h2>\n<p><strong>Worked</strong></p>\n<ul>\n<li>SAM optimizer</li>\n<li>Normalization (separately for each gravitational wave observatory)</li>\n<li>align spectrograms from each gravitational wave interferometers on channel axis</li>\n</ul>\n<p><strong>Not Worked</strong></p>\n<ul>\n<li>Mix up</li>\n<li>VQT</li>\n<li>pretrain using SETI data</li>\n</ul>\n<p><strong>Idea</strong></p>\n<ul>\n<li>matched filtering</li>\n<li>2 stage learning</li>\n<li>denoising auto-encoder</li>\n</ul>\n<p>If you have a question about this solution, feel free to ask!<br>\nThank you for reading!</p>",
  "messages": [
    {
      "id": "1529001",
      "postDate": "09/30/2021 04:45:55",
      "content": "<h2>Acknowledgements</h2>\n<p>Thanks to Kaggle and the hosts for holding this exciting competition! I've learned many things in this competition. Also thanks to all participants especially my teammate schulta( <a href=\"https://www.kaggle.com/schulta\" target=\"_blank\">@schulta</a> ) and Chizuchizu( <a href=\"https://www.kaggle.com/chizuchizu\" target=\"_blank\">@chizuchizu</a> )</p>\n<h2>Models</h2>\n<p>Scores of each model were as follows;</p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>EfficientB7ns</td>\n<td>0.87706</td>\n<td>0.8798</td>\n</tr>\n<tr>\n<td>EfficientB3ns</td>\n<td>0.87267</td>\n<td>0.8747</td>\n</tr>\n<tr>\n<td>1dCNN</td>\n<td>0.87281</td>\n<td>0.8769</td>\n</tr>\n<tr>\n<td>swin transformer</td>\n<td>0.86977</td>\n<td>0.8723</td>\n</tr>\n</tbody>\n</table>\n<p>Detail of each model was as follows;</p>\n<p><strong>CQT</strong></p>\n<ul>\n<li>EfficientNetB7ns<ul>\n<li>cpt parameter: {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}</li>\n<li>5fold validation</li>\n<li>Image size: 512 x 512</li>\n<li>Tripret Attention (<a href=\"https://arxiv.org/abs/2010.03045\" target=\"_blank\">paper</a>)</li>\n<li>Augmentation: shift in x-axis direction</li></ul></li>\n<li>Swin Transformer<ul>\n<li>cqt parameter {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}</li>\n<li>5fold validation</li>\n<li>Image size: 384 x 384</li>\n<li>Augmentation: shift in x-axis direction</li></ul></li>\n</ul>\n<p><strong>CWT</strong></p>\n<ul>\n<li>EfficientNetB3ns</li>\n<li>5fold validation</li>\n<li>image size : 256 x 256</li>\n<li>Tripret Attention </li>\n<li>Augmentation: shift in x-axis direction</li>\n</ul>\n<p><strong>1dCNN</strong></p>\n<ul>\n<li>We used <a href=\"https://www.kaggle.com/scaomath/g2net-1d-cnn-gem-pool-pytorch-train-inference\" target=\"_blank\">Public Kernel</a> as a baseline.Thanks to Shuhao Cao( <a href=\"https://www.kaggle.com/scaomath\" target=\"_blank\">@scaomath</a> )</li>\n<li>5fold validation</li>\n<li>add one more conv layer</li>\n<li>bandpass_params = dict(lf=25, hf=500)</li>\n</ul>\n<h2>Other things</h2>\n<p><strong>Worked</strong></p>\n<ul>\n<li>SAM optimizer</li>\n<li>Normalization (separately for each gravitational wave observatory)</li>\n<li>align spectrograms from each gravitational wave interferometers on channel axis</li>\n</ul>\n<p><strong>Not Worked</strong></p>\n<ul>\n<li>Mix up</li>\n<li>VQT</li>\n<li>pretrain using SETI data</li>\n</ul>\n<p><strong>Idea</strong></p>\n<ul>\n<li>matched filtering</li>\n<li>2 stage learning</li>\n<li>denoising auto-encoder</li>\n</ul>\n<p>If you have a question about this solution, feel free to ask!<br>\nThank you for reading!</p>",
      "rawMarkdown": "## Acknowledgements\n\nThanks to Kaggle and the hosts for holding this exciting competition! I've learned many things in this competition. Also thanks to all participants especially my teammate schulta( @schulta ) and Chizuchizu( @chizuchizu )\n\n## Models\n\nScores of each model were as follows;\n\n| Model | CV | LB |\n| :---: | :---: | :---: |\n| EfficientB7ns | 0.87706 | 0.8798 |\n| EfficientB3ns | 0.87267 | 0.8747 |\n| 1dCNN | 0.87281 | 0.8769 |\n| swin transformer | 0.86977 | 0.8723 |\n\nDetail of each model was as follows;\n\n**CQT**\n- EfficientNetB7ns\n   - cpt parameter: {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}\n   - 5fold validation\n   - Image size: 512 x 512\n   - Tripret Attention ([paper](https://arxiv.org/abs/2010.03045))\n   - Augmentation: shift in x-axis direction\n- Swin Transformer\n   - cqt parameter {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}\n   - 5fold validation\n   - Image size: 384 x 384\n   - Augmentation: shift in x-axis direction\n\n**CWT**\n- EfficientNetB3ns\n- 5fold validation\n- image size : 256 x 256\n- Tripret Attention \n- Augmentation: shift in x-axis direction\n\n**1dCNN**\n- We used [Public Kernel](https://www.kaggle.com/scaomath/g2net-1d-cnn-gem-pool-pytorch-train-inference) as a baseline.Thanks to Shuhao Cao( @scaomath )\n- 5fold validation\n- add one more conv layer\n- bandpass_params = dict(lf=25, hf=500)\n\n## Other things\n**Worked**\n- SAM optimizer\n- Normalization (separately for each gravitational wave observatory)\n- align spectrograms from each gravitational wave interferometers on channel axis\n\n**Not Worked**\n- Mix up\n- VQT\n- pretrain using SETI data\n\n**Idea**\n- matched filtering\n- 2 stage learning\n- denoising auto-encoder\n  \nIf you have a question about this solution, feel free to ask!\nThank you for reading!",
      "votes": null
    },
    {
      "id": "1559767",
      "postDate": "10/27/2021 07:22:54",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    },
    {
      "id": "1559876",
      "postDate": "10/27/2021 08:04:24",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1559767,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/27/2021 07:22:54",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1559876,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/27/2021 08:04:24",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1529001": "## Acknowledgements\n\nThanks to Kaggle and the hosts for holding this exciting competition! I've learned many things in this competition. Also thanks to all participants especially my teammate schulta( @schulta ) and Chizuchizu( @chizuchizu )\n\n## Models\n\nScores of each model were as follows;\n\n| Model | CV | LB |\n| :---: | :---: | :---: |\n| EfficientB7ns | 0.87706 | 0.8798 |\n| EfficientB3ns | 0.87267 | 0.8747 |\n| 1dCNN | 0.87281 | 0.8769 |\n| swin transformer | 0.86977 | 0.8723 |\n\nDetail of each model was as follows;\n\n**CQT**\n- EfficientNetB7ns\n   - cpt parameter: {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}\n   - 5fold validation\n   - Image size: 512 x 512\n   - Tripret Attention ([paper](https://arxiv.org/abs/2010.03045))\n   - Augmentation: shift in x-axis direction\n- Swin Transformer\n   - cqt parameter {“sr”: 2048, “fmin”: 20, “fmax”: 500, “hop_length”: 8, “bins_per_octave”: 12, “filter_scale”: 0.7}\n   - 5fold validation\n   - Image size: 384 x 384\n   - Augmentation: shift in x-axis direction\n\n**CWT**\n- EfficientNetB3ns\n- 5fold validation\n- image size : 256 x 256\n- Tripret Attention \n- Augmentation: shift in x-axis direction\n\n**1dCNN**\n- We used [Public Kernel](https://www.kaggle.com/scaomath/g2net-1d-cnn-gem-pool-pytorch-train-inference) as a baseline.Thanks to Shuhao Cao( @scaomath )\n- 5fold validation\n- add one more conv layer\n- bandpass_params = dict(lf=25, hf=500)\n\n## Other things\n**Worked**\n- SAM optimizer\n- Normalization (separately for each gravitational wave observatory)\n- align spectrograms from each gravitational wave interferometers on channel axis\n\n**Not Worked**\n- Mix up\n- VQT\n- pretrain using SETI data\n\n**Idea**\n- matched filtering\n- 2 stage learning\n- denoising auto-encoder\n  \nIf you have a question about this solution, feel free to ask!\nThank you for reading!",
    "1559767": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
    "1559876": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
  },
  "source": "meta"
}