{ "id": 275431, "title": "15th place solution ", "url": "/competitions/g2net-gravitational-wave-detection/writeups/fumihiro-kaneko-15th-place-solution", "author_name": "", "post_date": "2021-10-01T12:07:29.183Z", "votes": 14, "comment_count": 10, "views": 0, "content": "

Thanks to kaggle and organizers hosting such a nice competition.
\nMy approach is using a simple Conv2D net like below.
\nSadly other complex approaches did not work well on my experiments.
\n\"model\"

\n", "messages": [ { "id": "1529450", "postDate": "09/30/2021 12:15:32", "content": "

Thanks to kaggle and organizers hosting such a nice competition.
\nMy approach is using a simple Conv2D net like below.
\nSadly other complex approaches did not work well on my experiments.
\n\"model\"

\n", "rawMarkdown": "Thanks to kaggle and organizers hosting such a nice competition.\nMy approach is using a simple Conv2D net like below.\nSadly other complex approaches did not work well on my experiments.\n\n![model](https://user-images.githubusercontent.com/61892693/135449819-bcc73b16-eb7c-41e1-ba1a-6b870a535dfa.png)\n\n* Whitening: Using average PSD. Averaging over all noise samples for each site.\n* CQT Scaling with `filter_scale = 8/bins_per_octave` and (fmin, fmax)=(20, 1024). Both abs and angle part were used.\n* Augmentation\n * Horizontal/time shift\n * Pad both side and then horizontal random crop to get time shift image. -> ROC +0.002.\n * Mixup, prevent from overfitting\n* GeM Fixed power 3 was better than the trainable case. -> ROC +0.001\n\n* Scores\n\n\n| net | spec | height | width | PB score |\n| --- | --- | --- | --- | --- |\n| effnet b0 | Log STFT | 256 | 513 | 0.8760 |\n| effnet b0 | CQT | 181 | 513 | 0.8768 |\n| effnet b3 | CQT | 181 | 1024 | 0.8797 |\n| effnet b3 | CQT | 273 | 1024 | 0.8802 |\n\nMy final score is ensemble of Log STFT/CQT models.\n\n* My code is available [here](https://github.com/Fkaneko/kaggle_g2net_gravitational_wave_detection)", "votes": null }, { "id": "1529451", "postDate": "09/30/2021 12:18:54", "content": "

Hey. Congratulations, I tried Log STFT in the competition but it didn't perform well (0.85x). Could you elaborate on it and share the parameters for the same if used any?

", "rawMarkdown": "Hey. Congratulations, I tried Log STFT in the competition but it didn't perform well (0.85x). Could you elaborate on it and share the parameters for the same if used any?", "votes": null }, { "id": "1529456", "postDate": "09/30/2021 12:22:03", "content": "
\n

Whitening: Using average PSD. Averaging over all noise samples for each site

\n
\n

You mean you do whitening by global average of only-noise samples and not each sample average?

", "rawMarkdown": "> Whitening: Using average PSD. Averaging over all noise samples for each site\n\nYou mean you do whitening by global average of only-noise samples and not each sample average?", "votes": null }, { "id": "1529460", "postDate": "09/30/2021 12:30:13", "content": "

This is my settings for Log STFT with nn.Audio.

\n
  stft_params:\n    n_fft: 512\n    win_length: 256\n    hop_length: 8\n    window: hann\n    freq_scale: log   #log/linear\n    iSTFT: False\n    fmin: 10\n    fmax: 1024\n    sr: 2048\n    output_format: Complex\n
\n

After this nn.Audio processings, I converted it to dB scale like librosa.amplitude_to_db with the following parameters.

\n
    amin: 1.0e-8\n    top_db: 200\n    ref: 1.0\n
\n

After that offset +135dB and finally get a [0, 255] image.

", "rawMarkdown": "This is my settings for Log STFT with nn.Audio.\n```yaml\n stft_params:\n n_fft: 512\n win_length: 256\n hop_length: 8\n window: hann\n freq_scale: log #log/linear\n iSTFT: False\n fmin: 10\n fmax: 1024\n sr: 2048\n output_format: Complex\n```\nAfter this nn.Audio processings, I converted it to dB scale like [librosa.amplitude_to_db](https://librosa.org/doc/latest/generated/librosa.amplitude_to_db.html#librosa.amplitude_to_db) with the following parameters. \n```yaml\n amin: 1.0e-8\n top_db: 200\n ref: 1.0\n```\nAfter that offset +135dB and finally get a [0, 255] image.", "votes": null }, { "id": "1529489", "postDate": "09/30/2021 12:52:49", "content": "

Yes I used global average of only-noise sample. Here is plot for three global average cases.
\nUsing all samples, only positive and only noise.
\n\"avg_psd\"

", "rawMarkdown": "Yes I used global average of only-noise sample. Here is plot for three global average cases.\nUsing all samples, only positive and only noise.\n![avg_psd](https://user-images.githubusercontent.com/61892693/135459283-def629bb-20da-42ab-a215-d016ede30d0b.png)", "votes": null }, { "id": "1529519", "postDate": "09/30/2021 13:11:47", "content": "

Congrats/. Interesting that non trainable GEM works better. We used trainable GEM, LOL.

\n

We also tried mixup but it didn't work for us.

\n

And we had the same idea for whitening!

", "rawMarkdown": "Congrats/. Interesting that non trainable GEM works better. We used trainable GEM, LOL.\n\nWe also tried mixup but it didn't work for us. \n\nAnd we had the same idea for whitening!", "votes": null }, { "id": "1529558", "postDate": "09/30/2021 13:29:26", "content": "

Thanks, is there any justification for why you used only-noise samples instead of all? It's kinda hard to see for me but the three curves seems very similar to me.

", "rawMarkdown": "Thanks, is there any justification for why you used only-noise samples instead of all? It's kinda hard to see for me but the three curves seems very similar to me.", "votes": null }, { "id": "1529618", "postDate": "09/30/2021 14:21:54", "content": "

At my early experiment, heavy overfitting is fixed with Mixup.
\n\"roc\"
\nThis is validation ROC for each epoch.

\n

About whitening I choose this method but I'm still not confident on how to do it properly…

", "rawMarkdown": "At my early experiment, heavy overfitting is fixed with Mixup. \n![roc](https://user-images.githubusercontent.com/61892693/135468694-7dc429a7-9efd-4379-907e-0e12c5d270f8.png)\nThis is validation ROC for each epoch. \n\nAbout whitening I choose this method but I'm still not confident on how to do it properly...", "votes": null }, { "id": "1529640", "postDate": "09/30/2021 14:47:18", "content": "

Yeah, These three curves are quite similar. and hard to decide which one is the best.
\nThe following figure is psd distribution on target range 35-350hz. There are some clear separation between target and noise sample. I tried to keep this distribution as possible as I can.
\nThat's why I choose noise curve, but not sure for this effect. And my original motivation for this psd calculation is determining one global psd curve, so I didn't require 100% justification on this point…

\n

\"Screenshot

", "rawMarkdown": "Yeah, These three curves are quite similar. and hard to decide which one is the best.\nThe following figure is psd distribution on target range 35-350hz. There are some clear separation between target and noise sample. I tried to keep this distribution as possible as I can. \nThat's why I choose noise curve, but not sure for this effect. And my original motivation for this psd calculation is determining one global psd curve, so I didn't require 100% justification on this point...\n\n![Screenshot from 2021-09-30 23-24-08](https://user-images.githubusercontent.com/61892693/135473960-71e60c21-1a24-4095-b184-cf0a3e21a502.png)", "votes": null }, { "id": "1529645", "postDate": "09/30/2021 14:54:06", "content": "

Mixup doesn't work in my solution, too.
\nRandom switching channel of two waves (target=0 only) works well.

", "rawMarkdown": "Mixup doesn't work in my solution, too. \nRandom switching channel of two waves (target=0 only) works well.", "votes": null }, { "id": "1559888", "postDate": "10/27/2021 08:06:14", "content": "

Hey All,

\n

Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.

\n

Thank you all,

\n

Regards,
\nChris

", "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris", "votes": null } ], "comments": [ { "id": 1529451, "author_name": "nischaydnk", "author_url": "", "post_date": "09/30/2021 12:18:54", "content": "

Hey. Congratulations, I tried Log STFT in the competition but it didn't perform well (0.85x). Could you elaborate on it and share the parameters for the same if used any?

", "votes": null, "replies": [ { "id": 1529460, "author_name": "sai11fkaneko", "author_url": "", "post_date": "09/30/2021 12:30:13", "content": "

This is my settings for Log STFT with nn.Audio.

\n
  stft_params:\n    n_fft: 512\n    win_length: 256\n    hop_length: 8\n    window: hann\n    freq_scale: log   #log/linear\n    iSTFT: False\n    fmin: 10\n    fmax: 1024\n    sr: 2048\n    output_format: Complex\n
\n

After this nn.Audio processings, I converted it to dB scale like librosa.amplitude_to_db with the following parameters.

\n
    amin: 1.0e-8\n    top_db: 200\n    ref: 1.0\n
\n

After that offset +135dB and finally get a [0, 255] image.

", "votes": null, "replies": [] } ] }, { "id": 1529456, "author_name": "brachester", "author_url": "", "post_date": "09/30/2021 12:22:03", "content": "
\n

Whitening: Using average PSD. Averaging over all noise samples for each site

\n
\n

You mean you do whitening by global average of only-noise samples and not each sample average?

", "votes": null, "replies": [ { "id": 1529489, "author_name": "sai11fkaneko", "author_url": "", "post_date": "09/30/2021 12:52:49", "content": "

Yes I used global average of only-noise sample. Here is plot for three global average cases.
\nUsing all samples, only positive and only noise.
\n\"avg_psd\"

", "votes": null, "replies": [] }, { "id": 1529558, "author_name": "brachester", "author_url": "", "post_date": "09/30/2021 13:29:26", "content": "

Thanks, is there any justification for why you used only-noise samples instead of all? It's kinda hard to see for me but the three curves seems very similar to me.

", "votes": null, "replies": [] }, { "id": 1529640, "author_name": "sai11fkaneko", "author_url": "", "post_date": "09/30/2021 14:47:18", "content": "

Yeah, These three curves are quite similar. and hard to decide which one is the best.
\nThe following figure is psd distribution on target range 35-350hz. There are some clear separation between target and noise sample. I tried to keep this distribution as possible as I can.
\nThat's why I choose noise curve, but not sure for this effect. And my original motivation for this psd calculation is determining one global psd curve, so I didn't require 100% justification on this point…

\n

\"Screenshot

", "votes": null, "replies": [] } ] }, { "id": 1529519, "author_name": "cpmpml", "author_url": "", "post_date": "09/30/2021 13:11:47", "content": "

Congrats/. Interesting that non trainable GEM works better. We used trainable GEM, LOL.

\n

We also tried mixup but it didn't work for us.

\n

And we had the same idea for whitening!

", "votes": null, "replies": [ { "id": 1529618, "author_name": "sai11fkaneko", "author_url": "", "post_date": "09/30/2021 14:21:54", "content": "

At my early experiment, heavy overfitting is fixed with Mixup.
\n\"roc\"
\nThis is validation ROC for each epoch.

\n

About whitening I choose this method but I'm still not confident on how to do it properly…

", "votes": null, "replies": [] }, { "id": 1529645, "author_name": "wuliaokaola", "author_url": "", "post_date": "09/30/2021 14:54:06", "content": "

Mixup doesn't work in my solution, too.
\nRandom switching channel of two waves (target=0 only) works well.

", "votes": null, "replies": [] } ] }, { "id": 1559888, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 08:06:14", "content": "

Hey All,

\n

Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.

\n

Thank you all,

\n

Regards,
\nChris

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1529450": "Thanks to kaggle and organizers hosting such a nice competition.\nMy approach is using a simple Conv2D net like below.\nSadly other complex approaches did not work well on my experiments.\n\n![model](https://user-images.githubusercontent.com/61892693/135449819-bcc73b16-eb7c-41e1-ba1a-6b870a535dfa.png)\n\n* Whitening: Using average PSD. Averaging over all noise samples for each site.\n* CQT Scaling with `filter_scale = 8/bins_per_octave` and (fmin, fmax)=(20, 1024). Both abs and angle part were used.\n* Augmentation\n * Horizontal/time shift\n * Pad both side and then horizontal random crop to get time shift image. -> ROC +0.002.\n * Mixup, prevent from overfitting\n* GeM Fixed power 3 was better than the trainable case. -> ROC +0.001\n\n* Scores\n\n\n| net | spec | height | width | PB score |\n| --- | --- | --- | --- | --- |\n| effnet b0 | Log STFT | 256 | 513 | 0.8760 |\n| effnet b0 | CQT | 181 | 513 | 0.8768 |\n| effnet b3 | CQT | 181 | 1024 | 0.8797 |\n| effnet b3 | CQT | 273 | 1024 | 0.8802 |\n\nMy final score is ensemble of Log STFT/CQT models.\n\n* My code is available [here](https://github.com/Fkaneko/kaggle_g2net_gravitational_wave_detection)", "1529451": "Hey. Congratulations, I tried Log STFT in the competition but it didn't perform well (0.85x). Could you elaborate on it and share the parameters for the same if used any?", "1529456": "> Whitening: Using average PSD. Averaging over all noise samples for each site\n\nYou mean you do whitening by global average of only-noise samples and not each sample average?", "1529460": "This is my settings for Log STFT with nn.Audio.\n```yaml\n stft_params:\n n_fft: 512\n win_length: 256\n hop_length: 8\n window: hann\n freq_scale: log #log/linear\n iSTFT: False\n fmin: 10\n fmax: 1024\n sr: 2048\n output_format: Complex\n```\nAfter this nn.Audio processings, I converted it to dB scale like [librosa.amplitude_to_db](https://librosa.org/doc/latest/generated/librosa.amplitude_to_db.html#librosa.amplitude_to_db) with the following parameters. \n```yaml\n amin: 1.0e-8\n top_db: 200\n ref: 1.0\n```\nAfter that offset +135dB and finally get a [0, 255] image.", "1529489": "Yes I used global average of only-noise sample. Here is plot for three global average cases.\nUsing all samples, only positive and only noise.\n![avg_psd](https://user-images.githubusercontent.com/61892693/135459283-def629bb-20da-42ab-a215-d016ede30d0b.png)", "1529519": "Congrats/. Interesting that non trainable GEM works better. We used trainable GEM, LOL.\n\nWe also tried mixup but it didn't work for us. \n\nAnd we had the same idea for whitening!", "1529558": "Thanks, is there any justification for why you used only-noise samples instead of all? It's kinda hard to see for me but the three curves seems very similar to me.", "1529618": "At my early experiment, heavy overfitting is fixed with Mixup. \n![roc](https://user-images.githubusercontent.com/61892693/135468694-7dc429a7-9efd-4379-907e-0e12c5d270f8.png)\nThis is validation ROC for each epoch. \n\nAbout whitening I choose this method but I'm still not confident on how to do it properly...", "1529640": "Yeah, These three curves are quite similar. and hard to decide which one is the best.\nThe following figure is psd distribution on target range 35-350hz. There are some clear separation between target and noise sample. I tried to keep this distribution as possible as I can. \nThat's why I choose noise curve, but not sure for this effect. And my original motivation for this psd calculation is determining one global psd curve, so I didn't require 100% justification on this point...\n\n![Screenshot from 2021-09-30 23-24-08](https://user-images.githubusercontent.com/61892693/135473960-71e60c21-1a24-4095-b184-cf0a3e21a502.png)", "1529645": "Mixup doesn't work in my solution, too. \nRandom switching channel of two waves (target=0 only) works well.", "1559888": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris" }, "source": "meta" }