{ "id": 264969, "title": "nnAudio Counterpart Implemented with TensorFlow", "url": "/competitions/g2net-gravitational-wave-detection/discussion/264969", "author_name": "", "post_date": "2021-08-14T02:50:47.959522500Z", "votes": 30, "comment_count": 10, "views": 0, "content": "

Hi all,

\n

I've made my notebooks public which have implemented nnAudio counterpart with Tensorflow.

\n\n

You can use on-the-fly CQT computation with TPU training/inference. My EfficientNetB0 model trained on on-the-fly CQT shows better result compared to Welf's EfficientNetB7 model or Xuxu's EfficientNetB7 model.

\n

Just scaling up the model or image size will give you better score I think.

\n

Happy Kaggling!

\n
\n

UPDATE

\n

I've run EfficientNetB7 on version2 of the training notebook, version3 of the inference notebook. It achieves CV 0.8669 LB 0.869.

", "messages": [ { "id": "1471184", "postDate": "08/14/2021 02:50:47", "content": "

Hi all,

\n

I've made my notebooks public which have implemented nnAudio counterpart with Tensorflow.

\n\n

You can use on-the-fly CQT computation with TPU training/inference. My EfficientNetB0 model trained on on-the-fly CQT shows better result compared to Welf's EfficientNetB7 model or Xuxu's EfficientNetB7 model.

\n

Just scaling up the model or image size will give you better score I think.

\n

Happy Kaggling!

\n
\n

UPDATE

\n

I've run EfficientNetB7 on version2 of the training notebook, version3 of the inference notebook. It achieves CV 0.8669 LB 0.869.

", "rawMarkdown": "Hi all,\n\nI've made my notebooks public which have implemented nnAudio counterpart with Tensorflow.\n\n* [Training notebook](https://www.kaggle.com/hidehisaarai1213/g2net-tf-on-the-fly-cqt-tpu-training)\n* [Inference notebook](https://www.kaggle.com/hidehisaarai1213/g2net-tf-on-the-fly-cqt-tpu-inference)\n\nYou can use on-the-fly CQT computation with TPU training/inference. My EfficientNetB0 model trained on on-the-fly CQT shows better result compared to [Welf's EfficientNetB7 model](https://www.kaggle.com/miklgr500/cqt-g2net-efficientnetb7-tpu-inference) or [Xuxu's EfficientNetB7 model](https://www.kaggle.com/xuxu1234/lb-0-866-g2net-efficientnetb7-tpu-inference).\n\nJust scaling up the model or image size will give you better score I think.\n\nHappy Kaggling!\n\n---\nUPDATE\n\nI've run EfficientNetB7 on version2 of the training notebook, version3 of the inference notebook. It achieves CV 0.8669 LB 0.869.", "votes": null }, { "id": "1471186", "postDate": "08/14/2021 02:54:25", "content": "

And longer training also.
\nIn my notebook, models are trained 15 epochs, but it seems they can be better if we add more training epochs.

", "rawMarkdown": "And longer training also.\nIn my notebook, models are trained 15 epochs, but it seems they can be better if we add more training epochs.", "votes": null }, { "id": "1471501", "postDate": "08/14/2021 08:52:27", "content": "

wow！thanks for share. for TPU training I have not been able to break 0.866 for model generation😭

", "rawMarkdown": "wow！thanks for share. for TPU training I have not been able to break 0.866 for model generation😭", "votes": null }, { "id": "1471911", "postDate": "08/14/2021 14:40:06", "content": "

Thank you for the notebook. Since pre-processing seems to be important in this competition, could you tell us about your pre-processing steps(if any) when creating tfrecord files?

", "rawMarkdown": "Thank you for the notebook. Since pre-processing seems to be important in this competition, could you tell us about your pre-processing steps(if any) when creating tfrecord files?", "votes": null }, { "id": "1471946", "postDate": "08/14/2021 15:07:46", "content": "

Yes, of course
\nbut actually there's nothing interesting in my TFRecord creation steps.
\nJust load npy file and put it into TFRecord files.

\n

Here's the notebooks (sorry, there are 6 notebooks, but the only difference is which part of train/test npy files do they pack into tfrec)

\n", "rawMarkdown": "Yes, of course\nbut actually there's nothing interesting in my TFRecord creation steps.\nJust load npy file and put it into TFRecord files.\n\nHere's the notebooks (sorry, there are 6 notebooks, but the only difference is which part of train/test npy files do they pack into tfrec)\n\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-5-9\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-10-14\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-15-19\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-test-0-4\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-test-5-9", "votes": null }, { "id": "1474831", "postDate": "08/16/2021 09:38:36", "content": "

I get 0874 with a single model using

\n", "rawMarkdown": "I get 0874 with a single model using\n* TFrecords that include 20-500Hz band pass: I picked this as the SNR is very poor below 20Hz, and most of the GW events are below 500Hz.\n* CWT written in TF2 as a Keras Layer configured to create a 256x256x3 scalogram\n* B7\nand run for 15 epoch in under 2hrs.\n\nSee this repo for the CWT https://github.com/Kevin-McIsaac/cmorlet-tensorflow and how I optimised it.\n\nI've tried larger scalograms (512) but this did not improve the results. I'm now looking at how to \"denoise\" the input signal.", "votes": null }, { "id": "1476125", "postDate": "08/17/2021 02:19:53", "content": "

do you just resize the image after CWT?

", "rawMarkdown": "do you just resize the image after CWT?", "votes": null }, { "id": "1476657", "postDate": "08/17/2021 07:42:11", "content": "

No, I change n_scales and stride parameters to create the output size I want. For example I want a spectrogram that is 256x256 then set:

\n", "rawMarkdown": "No, I change n_scales and stride parameters to create the output size I want. For example I want a spectrogram that is 256x256 then set:\n* n_scale = 256 \n* stride = 16. This is calculated 4096/256, where 4096 is the length of the sample.", "votes": null }, { "id": "1476671", "postDate": "08/17/2021 07:50:35", "content": "

OK, I see. Thanks.

", "rawMarkdown": "OK, I see. Thanks.", "votes": null }, { "id": "1476915", "postDate": "08/17/2021 09:40:54", "content": "

n_scale = 256 is a lot! Maybe I try that :-)

", "rawMarkdown": "n_scale = 256 is a lot! Maybe I try that :-)", "votes": null }, { "id": "1561214", "postDate": "10/27/2021 12:20:35", "content": "

Hey All,

\n

Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.

\n

Thank you all,

\n

Regards,
\nChris

", "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris", "votes": null } ], "comments": [ { "id": 1471186, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "08/14/2021 02:54:25", "content": "

And longer training also.
\nIn my notebook, models are trained 15 epochs, but it seems they can be better if we add more training epochs.

", "votes": null, "replies": [] }, { "id": 1471501, "author_name": "xuxu1234", "author_url": "", "post_date": "08/14/2021 08:52:27", "content": "

wow！thanks for share. for TPU training I have not been able to break 0.866 for model generation😭

", "votes": null, "replies": [] }, { "id": 1471911, "author_name": "bibek777", "author_url": "", "post_date": "08/14/2021 14:40:06", "content": "

Thank you for the notebook. Since pre-processing seems to be important in this competition, could you tell us about your pre-processing steps(if any) when creating tfrecord files?

", "votes": null, "replies": [ { "id": 1471946, "author_name": "hidehisaarai1213", "author_url": "", "post_date": "08/14/2021 15:07:46", "content": "

Yes, of course
\nbut actually there's nothing interesting in my TFRecord creation steps.
\nJust load npy file and put it into TFRecord files.

\n

Here's the notebooks (sorry, there are 6 notebooks, but the only difference is which part of train/test npy files do they pack into tfrec)

\n", "votes": null, "replies": [] } ] }, { "id": 1474831, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/16/2021 09:38:36", "content": "

I get 0874 with a single model using

\n", "votes": null, "replies": [ { "id": 1476125, "author_name": "mozhiwenmzw", "author_url": "", "post_date": "08/17/2021 02:19:53", "content": "

do you just resize the image after CWT?

", "votes": null, "replies": [] }, { "id": 1476657, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/17/2021 07:42:11", "content": "

No, I change n_scales and stride parameters to create the output size I want. For example I want a spectrogram that is 256x256 then set:

\n", "votes": null, "replies": [] }, { "id": 1476671, "author_name": "mozhiwenmzw", "author_url": "", "post_date": "08/17/2021 07:50:35", "content": "

OK, I see. Thanks.

", "votes": null, "replies": [] }, { "id": 1476915, "author_name": "hannes82", "author_url": "", "post_date": "08/17/2021 09:40:54", "content": "

n_scale = 256 is a lot! Maybe I try that :-)

", "votes": null, "replies": [] } ] }, { "id": 1561214, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 12:20:35", "content": "

Hey All,

\n

Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.

\n

Thank you all,

\n

Regards,
\nChris

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1471184": "Hi all,\n\nI've made my notebooks public which have implemented nnAudio counterpart with Tensorflow.\n\n* [Training notebook](https://www.kaggle.com/hidehisaarai1213/g2net-tf-on-the-fly-cqt-tpu-training)\n* [Inference notebook](https://www.kaggle.com/hidehisaarai1213/g2net-tf-on-the-fly-cqt-tpu-inference)\n\nYou can use on-the-fly CQT computation with TPU training/inference. My EfficientNetB0 model trained on on-the-fly CQT shows better result compared to [Welf's EfficientNetB7 model](https://www.kaggle.com/miklgr500/cqt-g2net-efficientnetb7-tpu-inference) or [Xuxu's EfficientNetB7 model](https://www.kaggle.com/xuxu1234/lb-0-866-g2net-efficientnetb7-tpu-inference).\n\nJust scaling up the model or image size will give you better score I think.\n\nHappy Kaggling!\n\n---\nUPDATE\n\nI've run EfficientNetB7 on version2 of the training notebook, version3 of the inference notebook. It achieves CV 0.8669 LB 0.869.", "1471186": "And longer training also.\nIn my notebook, models are trained 15 epochs, but it seems they can be better if we add more training epochs.", "1471501": "wow！thanks for share. for TPU training I have not been able to break 0.866 for model generation😭", "1471911": "Thank you for the notebook. Since pre-processing seems to be important in this competition, could you tell us about your pre-processing steps(if any) when creating tfrecord files?", "1471946": "Yes, of course\nbut actually there's nothing interesting in my TFRecord creation steps.\nJust load npy file and put it into TFRecord files.\n\nHere's the notebooks (sorry, there are 6 notebooks, but the only difference is which part of train/test npy files do they pack into tfrec)\n\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-5-9\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-10-14\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-15-19\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-test-0-4\n* https://www.kaggle.com/hidehisaarai1213/g2net-waveform-tfrecords-creation-test-5-9", "1474831": "I get 0874 with a single model using\n* TFrecords that include 20-500Hz band pass: I picked this as the SNR is very poor below 20Hz, and most of the GW events are below 500Hz.\n* CWT written in TF2 as a Keras Layer configured to create a 256x256x3 scalogram\n* B7\nand run for 15 epoch in under 2hrs.\n\nSee this repo for the CWT https://github.com/Kevin-McIsaac/cmorlet-tensorflow and how I optimised it.\n\nI've tried larger scalograms (512) but this did not improve the results. I'm now looking at how to \"denoise\" the input signal.", "1476125": "do you just resize the image after CWT?", "1476657": "No, I change n_scales and stride parameters to create the output size I want. For example I want a spectrogram that is 256x256 then set:\n* n_scale = 256 \n* stride = 16. This is calculated 4096/256, where 4096 is the length of the sample.", "1476671": "OK, I see. Thanks.", "1476915": "n_scale = 256 is a lot! Maybe I try that :-)", "1561214": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris" }, "source": "meta" }