{
  "id": 275353,
  "title": "10th Place Solution: CWT->1D Conv + CNN",
  "url": "/competitions/g2net-gravitational-wave-detection/writeups/alex-myles-coolz-jonny-10th-place-solution-cwt-1d-",
  "author_name": "",
  "post_date": "2021-11-01T05:11:27.617Z",
  "votes": 27,
  "comment_count": 4,
  "views": 0,
  "content": "<p>First of all I want to thank the host and Kaggle for organizing this amazing competition. We learned a lot in this two months. </p>\n<h1>Brief summary of my part</h1>\n<h3>Pipeline</h3>\n<p>CWT/CQT is a kind of 1D Conv in nature. But it is only one layer and the kernel is too large. So making it trainable directly doesn't work. I used a little trick as below.</p>\n<ul>\n<li>CWT + Resnet34/EfNet, 488x512, 1/5 fold, train 5 epochs -&gt; valid_auc: 0.87x</li>\n<li>Replace CWT with a multiple layers 1D Conv, the same size as CWT, train 20 epochs -&gt; valid_auc: 0.880x</li>\n<li>Fine-tune the model with full data, 5 epochs<br>\nSingle model can get 0.8790/0.8808 (private/public)</li>\n</ul>\n<p>Using different parameter sets(bandpass, time shift, random channel off etc.) to train 10 models above. Ensemble them all can get 0.8803/0.8818 (private/public)</p>\n<p>Then ensemble with my other models, the result is 0.8816/0.8828 (private/public) Unfortunately we didn't choose it as our final score.</p>\n<h3>Augmentation</h3>\n<ul>\n<li>Trim the original wave to (3, 3904), Random shift +- 65</li>\n<li>Random shift each channel +- 5</li>\n<li>Random turn off one channel</li>\n<li>Random switch channel of two waves (target=0 only)</li>\n</ul>\n<h3>Others</h3>\n<ul>\n<li>Using Tensorflow and TPU</li>\n<li>CWT/CQT conversion is on-the-fly</li>\n</ul>",
  "messages": [
    {
      "id": "1528928",
      "postDate": "09/30/2021 03:29:09",
      "content": "<p>First of all I want to thank the host and Kaggle for organizing this amazing competition. We learned a lot in this two months. </p>\n<h1>Brief summary of my part</h1>\n<h3>Pipeline</h3>\n<p>CWT/CQT is a kind of 1D Conv in nature. But it is only one layer and the kernel is too large. So making it trainable directly doesn't work. I used a little trick as below.</p>\n<ul>\n<li>CWT + Resnet34/EfNet, 488x512, 1/5 fold, train 5 epochs -&gt; valid_auc: 0.87x</li>\n<li>Replace CWT with a multiple layers 1D Conv, the same size as CWT, train 20 epochs -&gt; valid_auc: 0.880x</li>\n<li>Fine-tune the model with full data, 5 epochs<br>\nSingle model can get 0.8790/0.8808 (private/public)</li>\n</ul>\n<p>Using different parameter sets(bandpass, time shift, random channel off etc.) to train 10 models above. Ensemble them all can get 0.8803/0.8818 (private/public)</p>\n<p>Then ensemble with my other models, the result is 0.8816/0.8828 (private/public) Unfortunately we didn't choose it as our final score.</p>\n<h3>Augmentation</h3>\n<ul>\n<li>Trim the original wave to (3, 3904), Random shift +- 65</li>\n<li>Random shift each channel +- 5</li>\n<li>Random turn off one channel</li>\n<li>Random switch channel of two waves (target=0 only)</li>\n</ul>\n<h3>Others</h3>\n<ul>\n<li>Using Tensorflow and TPU</li>\n<li>CWT/CQT conversion is on-the-fly</li>\n</ul>",
      "rawMarkdown": "First of all I want to thank the host and Kaggle for organizing this amazing competition. We learned a lot in this two months. \n\n<h1>Brief summary of my part</h1>\n\n<h3>Pipeline</h3>\nCWT/CQT is a kind of 1D Conv in nature. But it is only one layer and the kernel is too large. So making it trainable directly doesn't work. I used a little trick as below.\n\n* CWT + Resnet34/EfNet, 488x512, 1/5 fold, train 5 epochs -> valid_auc: 0.87x\n* Replace CWT with a multiple layers 1D Conv, the same size as CWT, train 20 epochs -> valid_auc: 0.880x\n* Fine-tune the model with full data, 5 epochs\nSingle model can get 0.8790/0.8808 (private/public)\n\nUsing different parameter sets(bandpass, time shift, random channel off etc.) to train 10 models above. Ensemble them all can get 0.8803/0.8818 (private/public)\n\nThen ensemble with my other models, the result is 0.8816/0.8828 (private/public) Unfortunately we didn't choose it as our final score.\n\n<h3>Augmentation</h3>\n* Trim the original wave to (3, 3904), Random shift +- 65\n* Random shift each channel +- 5\n* Random turn off one channel\n* Random switch channel of two waves (target=0 only)\n\n<h3>Others</h3>\n* Using Tensorflow and TPU\n* CWT/CQT conversion is on-the-fly",
      "votes": null
    },
    {
      "id": "1532094",
      "postDate": "10/02/2021 16:05:35",
      "content": "<p>Thank you for sharing your solution. The discussion about Frontend(Replace CWT) is very clear and I regret that I did not try it.<br>\nI have a question, regarding \"Random turn off one channel\" in Augmentation, does it mean that you randomly set all the wave values of a channel to 0 ? Also, how much did that help CV or LB?<br>\nI built a model for each detector, but it was not very effective, so I thought it was an absolute requirement to use 3 waves. So I am curious.</p>",
      "rawMarkdown": "Thank you for sharing your solution. The discussion about Frontend(Replace CWT) is very clear and I regret that I did not try it.\nI have a question, regarding \"Random turn off one channel\" in Augmentation, does it mean that you randomly set all the wave values of a channel to 0 ? Also, how much did that help CV or LB?\nI built a model for each detector, but it was not very effective, so I thought it was an absolute requirement to use 3 waves. So I am curious.",
      "votes": null
    },
    {
      "id": "1532405",
      "postDate": "10/03/2021 00:25:55",
      "content": "<p>Yes, I set each channel to 0 with the probability of 0.1. I tested it at very early stage, but with only 1/8 of data. It can prevent my model from overfitting a little.<br>\nBTW, it doesn't work on 1D CNN model. </p>",
      "rawMarkdown": "Yes, I set each channel to 0 with the probability of 0.1. I tested it at very early stage, but with only 1/8 of data. It can prevent my model from overfitting a little.\nBTW, it doesn't work on 1D CNN model.",
      "votes": null
    },
    {
      "id": "1532410",
      "postDate": "10/03/2021 00:40:00",
      "content": "<p>It's nice. Thank you for your quick answer!</p>",
      "rawMarkdown": "It's nice. Thank you for your quick answer!",
      "votes": null
    },
    {
      "id": "1559903",
      "postDate": "10/27/2021 08:07:32",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1532094,
      "author_name": "naoism",
      "author_url": "",
      "post_date": "10/02/2021 16:05:35",
      "content": "<p>Thank you for sharing your solution. The discussion about Frontend(Replace CWT) is very clear and I regret that I did not try it.<br>\nI have a question, regarding \"Random turn off one channel\" in Augmentation, does it mean that you randomly set all the wave values of a channel to 0 ? Also, how much did that help CV or LB?<br>\nI built a model for each detector, but it was not very effective, so I thought it was an absolute requirement to use 3 waves. So I am curious.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1532405,
          "author_name": "wuliaokaola",
          "author_url": "",
          "post_date": "10/03/2021 00:25:55",
          "content": "<p>Yes, I set each channel to 0 with the probability of 0.1. I tested it at very early stage, but with only 1/8 of data. It can prevent my model from overfitting a little.<br>\nBTW, it doesn't work on 1D CNN model. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1532410,
          "author_name": "naoism",
          "author_url": "",
          "post_date": "10/03/2021 00:40:00",
          "content": "<p>It's nice. Thank you for your quick answer!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1559903,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/27/2021 08:07:32",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1528928": "First of all I want to thank the host and Kaggle for organizing this amazing competition. We learned a lot in this two months. \n\n<h1>Brief summary of my part</h1>\n\n<h3>Pipeline</h3>\nCWT/CQT is a kind of 1D Conv in nature. But it is only one layer and the kernel is too large. So making it trainable directly doesn't work. I used a little trick as below.\n\n* CWT + Resnet34/EfNet, 488x512, 1/5 fold, train 5 epochs -> valid_auc: 0.87x\n* Replace CWT with a multiple layers 1D Conv, the same size as CWT, train 20 epochs -> valid_auc: 0.880x\n* Fine-tune the model with full data, 5 epochs\nSingle model can get 0.8790/0.8808 (private/public)\n\nUsing different parameter sets(bandpass, time shift, random channel off etc.) to train 10 models above. Ensemble them all can get 0.8803/0.8818 (private/public)\n\nThen ensemble with my other models, the result is 0.8816/0.8828 (private/public) Unfortunately we didn't choose it as our final score.\n\n<h3>Augmentation</h3>\n* Trim the original wave to (3, 3904), Random shift +- 65\n* Random shift each channel +- 5\n* Random turn off one channel\n* Random switch channel of two waves (target=0 only)\n\n<h3>Others</h3>\n* Using Tensorflow and TPU\n* CWT/CQT conversion is on-the-fly",
    "1532094": "Thank you for sharing your solution. The discussion about Frontend(Replace CWT) is very clear and I regret that I did not try it.\nI have a question, regarding \"Random turn off one channel\" in Augmentation, does it mean that you randomly set all the wave values of a channel to 0 ? Also, how much did that help CV or LB?\nI built a model for each detector, but it was not very effective, so I thought it was an absolute requirement to use 3 waves. So I am curious.",
    "1532405": "Yes, I set each channel to 0 with the probability of 0.1. I tested it at very early stage, but with only 1/8 of data. It can prevent my model from overfitting a little.\nBTW, it doesn't work on 1D CNN model.",
    "1532410": "It's nice. Thank you for your quick answer!",
    "1559903": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
  },
  "source": "meta"
}