\n
I am using this great notebook as a starting point.
\nAs you can see in the notebook, it uses CQT features and is able to predict with a good ROC score (around 0.864).
\nHowever, if we check the Grad-CAM of some of the predictions (check the following screenshots from the notebook), we see that the model seems to focus on minor details, minor in the sense it is hard for me to distinguish between target 1 and 0 given the CQT alone.
\n
\n
Indeed, some (most?) CQT for target 1 or 0 look very similar.
\nIs it the color scale choice not very good for the CQT plots (we need maybe more colors to see through the purple?) or am I missing something else?
\nAny help/comments/suggestions is (are) appreciated. Thanks. :)
", "messages": [ { "id": "1473642", "postDate": "08/15/2021 16:39:11", "content": "I am using this great notebook as a starting point.
\nAs you can see in the notebook, it uses CQT features and is able to predict with a good ROC score (around 0.864).
\nHowever, if we check the Grad-CAM of some of the predictions (check the following screenshots from the notebook), we see that the model seems to focus on minor details, minor in the sense it is hard for me to distinguish between target 1 and 0 given the CQT alone.
\n
\n
Indeed, some (most?) CQT for target 1 or 0 look very similar.
\nIs it the color scale choice not very good for the CQT plots (we need maybe more colors to see through the purple?) or am I missing something else?
\nAny help/comments/suggestions is (are) appreciated. Thanks. :)
", "rawMarkdown": "I am using this great [notebook](https://www.kaggle.com/yasufuminakama/g2net-efficientnet-b7-baseline-training.) as a starting point. \n\nAs you can see in the notebook, it uses [**CQT**](https://en.wikipedia.org/wiki/Constant-Q_transform) features and is able to predict **with a good ROC score** (around 0.864). \n\nHowever, if we check the [**Grad-CAM**](https://arxiv.org/abs/1610.02391) of some of the predictions (check the following screenshots from the notebook), we see that the model seems to focus on minor details, minor in the sense it is hard for me to distinguish between target 1 and 0 given the CQT alone. \n\n\n\n\n\n\nIndeed, some (most?) CQT for target 1 or 0 look very similar.\n\n\n\nIs it the color scale choice not very good for the CQT plots (we need maybe more colors to see through the purple?) or am I missing something else?\n\nAny help/comments/suggestions is (are) appreciated. Thanks. :)", "votes": null }, { "id": "1474518", "postDate": "08/16/2021 06:41:51", "content": "I can recommend you make whitening, LP & HP filter and Tukey window for obtaining CQT features similar with CQT from gwpy or pycbc. I may recommend you read G2Net Spectral Whitening
\nMy implementation tukey window on torch:
\nfrom math import cos\n\ndef torch_tukey(size, alpha, dtype):\n window = torch.zeros((size,), dtype=dtype)\n for n in range(size // 2 + 1):\n if n < alpha * size / 2:\n window[n] = window[-n] = 1 / 2 * (1 - cos(2 * 3.14 * n / (size * alpha)))\n else:\n window[n] = window[-n] = 1\n return window\nCQT with whitening & applying tukey window
\n![\"\"](\"https://i.postimg.cc/25Mx2bTc/00017d3cf3.png\")
I also tried grad-cam on my research with model on TensorFlow and CQT calculated with PyCBC package and obtain expected result for me.
", "rawMarkdown": "I can recommend you make whitening, LP & HP filter and Tukey window for obtaining CQT features similar with CQT from gwpy or pycbc. I may recommend you read [G2Net Spectral Whitening](https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening)\n\nMy implementation tukey window on torch:\n```\nfrom math import cos\n\ndef torch_tukey(size, alpha, dtype):\n window = torch.zeros((size,), dtype=dtype)\n for n in range(size // 2 + 1):\n if n < alpha * size / 2:\n window[n] = window[-n] = 1 / 2 * (1 - cos(2 * 3.14 * n / (size * alpha)))\n else:\n window[n] = window[-n] = 1\n return window\n```\n\nCQT with whitening & applying tukey window\n\n\nI also tried grad-cam [on my research with model on TensorFlow and CQT](https://www.kaggle.com/miklgr500/eda-g2net-efficientnetb1-embeding-grad-cam) calculated with PyCBC package and obtain expected result for me.", "votes": null }, { "id": "1474812", "postDate": "08/16/2021 09:26:48", "content": "I'm able to get 0.874 using CWT (similar to CQT) that creates a 256x256 image and efficientnet B7.
", "rawMarkdown": "I'm able to get 0.874 using CWT (similar to CQT) that creates a 256x256 image and efficientnet B7.", "votes": null }, { "id": "1474834", "postDate": "08/16/2021 09:40:01", "content": "@kevinmcisaac what is difference between two. ANy notebook for CWT..
", "rawMarkdown": "kevinmcisaac what is difference between two. ANy notebook for CWT..", "votes": null }, { "id": "1474917", "postDate": "08/16/2021 10:46:32", "content": "That's awesome, thanks for those details!
", "rawMarkdown": "That's awesome, thanks for those details!", "votes": null }, { "id": "1475607", "postDate": "08/16/2021 18:47:46", "content": "@jaideepvalani CWT stands for continuous wavelet transform so it is quite different to the CQT one.
\nAs far as I know, you can use the scipy implementation here.
\nNotice that this won't be as optimized as the CQT nnAudio one (since it runs on GPU) but if you search more, you should be able to find one.
\nI hope this helps.
", "rawMarkdown": "jaideepvalani [CWT](https://en.wikipedia.org/wiki/Continuous_wavelet_transform) stands for **continuous wavelet transform** so it is quite different to the CQT one. \n\nAs far as I know, you can use the **scipy** implementation [here](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.cwt.html). \n\nNotice that this won't be as optimized as the CQT nnAudio one (since it runs on GPU) but if you search more, you should be able to find one. \n\nI hope this helps.", "votes": null }, { "id": "1475784", "postDate": "08/16/2021 20:34:33", "content": "Here is the whitening code from the https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening notebook (thanks again @kevinmcisaac) for those that want it available quickly. Notice that the
\ncode uses PyTorch so it should be fast. Also, the whitening function here is the Hann window.
import torch\nfrom torch.fft import fft, rfft, ifft\nimport numpy as np\n\n\ndef whiten(signal):\n # From here: https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening\n hann = torch.hann_window(len(signal), periodic=True, dtype=float)\n spec = fft(torch.from_numpy(signal).float()* hann)\n mag = torch.sqrt(torch.real(spec*torch.conj(spec))) \n\n return torch.real(ifft(spec/mag)).numpy() * np.sqrt(len(signal)/2)\nGreat discussion guys! Are you aware of a PyTorch equivalent for a bandpass filter?
\nI am aware of torchaudio.functional.lfilter but this introduces a phase shift. I started trying to port scipy.signal.filtfilt to PyTorch but it's quite tricky due to the initial conditions (zi)… 😵️
I'm use
\nfrom torchaudio.functional import bandpass_biquad\n\ndef butter_bandpass_filter(data, lowcut, highcut, fs):\n return bandpass_biquad(data, fs, (highcut + lowcut) / 2, (highcut - lowcut) / (highcut + lowcut))\nAwesome thank you! I knew the answer was somewhere in those biquad filters :)
", "rawMarkdown": "Awesome thank you! I knew the answer was somewhere in those biquad filters :)", "votes": null }, { "id": "1476584", "postDate": "08/17/2021 07:10:45", "content": "Yet another great function, thanks @miklgr500. 👍
", "rawMarkdown": "Yet another great function, thanks @miklgr500. 👍", "votes": null }, { "id": "1476609", "postDate": "08/17/2021 07:21:34", "content": "CQT and CWT preform a similar function, ie.., convert a 1D waveform (i.e., time vs amplitude) to a 2D spectrogram (i.e., time vs amplitude). When you show a CWT of a GW event you get a very similar image to the CQT.
\nMostly I chose CWT because I found a TF implementation of CWT that I could convert to TF2.0 and make a Keras Layer, then use this in my existing Keras model (Efficient Net) that when feed with TFrecords and run on a TPU executes an epoch in under 3 min. This allowed me to experiment very quick.
\nThere is also the option to use different types of wavelets (which I've not yet done), that might help. However I think largely CWT and CQT will give very similar results.
", "rawMarkdown": "CQT and CWT preform a similar function, ie.., convert a 1D waveform (i.e., time vs amplitude) to a 2D spectrogram (i.e., time vs amplitude). When you show a CWT of a GW event you get a very similar image to the CQT. \n\nMostly I chose CWT because I found a TF implementation of CWT that I could convert to TF2.0 and make a Keras Layer, then use this in my existing Keras model (Efficient Net) that when feed with TFrecords and run on a TPU executes an epoch in under 3 min. This allowed me to experiment very quick.\n\nThere is also the option to use different types of wavelets (which I've not yet done), that might help. However I think largely CWT and CQT will give very similar results.", "votes": null }, { "id": "1476621", "postDate": "08/17/2021 07:26:43", "content": "BTW, my implementation in TF2/Keras is in github. WHen used with a GPU and large batches the performance is excellent, i.e., 0.04ms / sample
", "rawMarkdown": "BTW, my implementation in TF2/Keras is in [github](https://github.com/Kevin-McIsaac/cmorlet-tensorflow). WHen used with a GPU and large batches the performance is excellent, i.e., 0.04ms / sample", "votes": null }, { "id": "1477169", "postDate": "08/17/2021 11:33:03", "content": "One other option is to have many models each one using a type of features: let's say one model trained on CWT and one on CQT. Then average both (or a more clever ensemble). Or even train one model on a mix of both features. Lots of creative ways to combine these so I guess the best way to know is to experiment. :)
", "rawMarkdown": "One other option is to have many models each one using a type of features: let's say one model trained on CWT and one on CQT. Then average both (or a more clever ensemble). Or even train one model on a mix of both features. Lots of creative ways to combine these so I guess the best way to know is to experiment. :)", "votes": null }, { "id": "1477386", "postDate": "08/17/2021 13:08:24", "content": "any sample code for CWT … the one give in uses synthetic signal.
\nI tried putting our actual strain but get very big size data
\ntorch.Size([32, 1, 30, 12288])
\nthis is unmanageable for GPU
@jaideepvalani you should have the same size as in CQT so this is indeed weird. If I find a good implementation for PyTorch of CWT, I will let you know. 👌
", "rawMarkdown": "jaideepvalani you should have the same size as in CQT so this is indeed weird. If I find a good implementation for PyTorch of CWT, I will let you know. 👌", "votes": null }, { "id": "1477779", "postDate": "08/17/2021 16:20:18", "content": "i use this
\npycwt = CWT(widths, \"ricker\", 1)\nprint(data.shape)\n#data = torch.tensor(sig , dtype=torch.float32)\ndata = torch.tensor(sig1 , dtype=torch.float32).view(-1,1).squeeze(-1)\n\nprint('data',sig.shape,data.shape)\ndata = torch.stack([data] *1) # 3 channels\ndata = torch.stack([data] * 32) # Batch of 32\nout = pycwt(data)\n@jaideepvalani At the bottom of my notebook there is an example with GW data. The output shape is [1, 3, 80, 4096].
Unfortunately, strided convolutions with padding='same' are not yet supported (GitHub issue). This means you'll have to resize the images to something more manageable using torch.nn.functional.interpolate.
Another alternative is to thin the data along the time axis like this: out[:, :, :, ::8] which has a shape of [1, 3, 80, 512]
You should definitely try different preprocessing and compare which works better :)
", "rawMarkdown": "You should definitely try different preprocessing and compare which works better :)", "votes": null }, { "id": "1478987", "postDate": "08/18/2021 08:28:28", "content": "@yassinealouini a little bit of hint from my experiments it appears that preprocessing data before feeding it to CQT kernels can improve model's performance ! Think about it . Eg: in this kernel , using better preprocessing techniques you can boost oof score from 0.8669 to 0.86714 !
", "rawMarkdown": "yassinealouini a little bit of hint from my experiments it appears that preprocessing data before feeding it to CQT kernels can improve model's performance ! Think about it . Eg: in this [kernel](https://www.kaggle.com/hidehisaarai1213/g2net-tf-on-the-fly-cqt-tpu-training) , using better preprocessing techniques you can boost oof score from 0.8669 to 0.86714 !", "votes": null }, { "id": "1479457", "postDate": "08/18/2021 13:31:22", "content": "Thanks for sharing! You can upgrade torch to 1.9, then padding='same' works.
", "rawMarkdown": "Thanks for sharing! You can upgrade torch to 1.9, then padding='same' works.", "votes": null }, { "id": "1479480", "postDate": "08/18/2021 13:38:48", "content": "Unfortunately not when stride > 1
", "rawMarkdown": "Unfortunately not when stride > 1", "votes": null }, { "id": "1479605", "postDate": "08/18/2021 14:39:57", "content": "Oh ok I see. So far I have always resized the image afterwards.
", "rawMarkdown": "Oh ok I see. So far I have always resized the image afterwards.", "votes": null }, { "id": "1484790", "postDate": "08/21/2021 15:18:03", "content": "Hi @miklgr500 this might be a very dumb question but can you tell us that are you using hann window for whitening similar to kernel here : https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening
\nAlso you follow this order right : Signal --> Whiten --> BandPass Filtering --> CQT using Tuckey window
", "rawMarkdown": "Hi @miklgr500 this might be a very dumb question but can you tell us that are you using hann window for whitening similar to kernel here : https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening\n\nAlso you follow this order right : Signal --> Whiten --> BandPass Filtering --> CQT using Tuckey window", "votes": null }, { "id": "1484846", "postDate": "08/21/2021 15:49:18", "content": "I'm use use Tukey widow for whiten and Hann for CQT
", "rawMarkdown": "I'm use use Tukey widow for whiten and Hann for CQT", "votes": null }, { "id": "1484860", "postDate": "08/21/2021 16:08:27", "content": "@tanulsingh077 : As far as I know, the CQT transform already contains bandpass filtering.
", "rawMarkdown": "tanulsingh077 : As far as I know, the CQT transform already contains bandpass filtering.", "votes": null }, { "id": "1484967", "postDate": "08/21/2021 17:31:41", "content": "Sorry for the dumb question, just jumping in. For those who are doing processing on gpu (using torch), are you passing float64s -> gpu?
", "rawMarkdown": "Sorry for the dumb question, just jumping in. For those who are doing processing on gpu (using torch), are you passing float64s -> gpu?", "votes": null }, { "id": "1485294", "postDate": "08/22/2021 01:08:40", "content": "Our current score (0.880) is obtained with Q Transform. Our best single 5 fold model achieves 0.880 LB score. We will be back now that SETI is over.
", "rawMarkdown": "Our current score (0.880) is obtained with Q Transform. Our best single 5 fold model achieves 0.880 LB score. We will be back now that SETI is over.", "votes": null }, { "id": "1485327", "postDate": "08/22/2021 02:23:24", "content": "Is there a digit missing in there somewhere?
", "rawMarkdown": "Is there a digit missing in there somewhere?", "votes": null }, { "id": "1485331", "postDate": "08/22/2021 02:36:17", "content": "yes! tx, I fixed it.
", "rawMarkdown": "yes! tx, I fixed it.", "votes": null }, { "id": "1485377", "postDate": "08/22/2021 04:07:11", "content": "@cpmpml does dim of cqt output has got any significant impact my current single model 4f gets 87.4 ,still far from desired levels
", "rawMarkdown": "cpmpml does dim of cqt output has got any significant impact my current single model 4f gets 87.4 ,still far from desired levels", "votes": null }, { "id": "1485381", "postDate": "08/22/2021 04:11:51", "content": "@miklgr500 which image is whitened and which is post tukey.
\nIs tukey window and whitening used by cqt also or they are additional transformation before applying cqt
@tanulsingh077 if using BPF and whitening, bpf would have to occur first imho
", "rawMarkdown": "tanulsingh077 if using BPF and whitening, bpf would have to occur first imho", "votes": null }, { "id": "1485718", "postDate": "08/22/2021 11:04:47", "content": "That's impressive, thanks for sharing some of the deatils. 👌
", "rawMarkdown": "That's impressive, thanks for sharing some of the deatils. 👌", "votes": null }, { "id": "1486497", "postDate": "08/23/2021 02:59:25", "content": "@miklgr500 : Did you find that spectral whitening worked? I tried spectral whitening using Tukey window (window=(\"tukey\",0.15)) but according to the experiments, the valid AUC was only around 0.81 either or not using the bandpass filtering (20Hz-500Hz with Hann window); whilst my best valid AUC was about 0.87 without spectral whitening.
", "rawMarkdown": "miklgr500 : Did you find that spectral whitening worked? I tried spectral whitening using Tukey window (window=(\"tukey\",0.15)) but according to the experiments, the valid AUC was only around 0.81 either or not using the bandpass filtering (20Hz-500Hz with Hann window); whilst my best valid AUC was about 0.87 without spectral whitening.", "votes": null }, { "id": "1486545", "postDate": "08/23/2021 04:09:00", "content": "I too found it made the AUC worse, see Spectral Whitening
", "rawMarkdown": "I too found it made the AUC worse, see [Spectral Whitening](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/252396)", "votes": null }, { "id": "1486579", "postDate": "08/23/2021 04:56:20", "content": "Hi @sayedathar11 , did you mean spectral whitening when you meant better preprocessing ?
", "rawMarkdown": "Hi @sayedathar11 , did you mean spectral whitening when you meant better preprocessing ?", "votes": null }, { "id": "1486688", "postDate": "08/23/2021 06:49:54", "content": "Hi @cpmpml are you using the time series from all the three detectors stacked on top of each other for input?
", "rawMarkdown": "Hi @cpmpml are you using the time series from all the three detectors stacked on top of each other for input?", "votes": null }, { "id": "1487911", "postDate": "08/24/2021 00:18:43", "content": "I've ran about 10 experiments so far with CV from 0.860 to 0.867 and the usual LB bump. What I've noticed so far is that simple ensembling (forward, N-M weight opt, rank, mean, etc) all fails and actually removes the LB bump from the expected OOF. I wonder if this is what @cpmpml also experienced with his teams best model being the one driving LB score. I have yet to start stacking (would like a few more models in the arsenal first), but I have my CV setup ready for that.
", "rawMarkdown": "I've ran about 10 experiments so far with CV from 0.860 to 0.867 and the usual LB bump. What I've noticed so far is that simple ensembling (forward, N-M weight opt, rank, mean, etc) all fails and actually removes the LB bump from the expected OOF. I wonder if this is what @cpmpml also experienced with his teams best model being the one driving LB score. I have yet to start stacking (would like a few more models in the arsenal first), but I have my CV setup ready for that.", "votes": null }, { "id": "1488519", "postDate": "08/24/2021 10:46:29", "content": "Ensembling brings little indeed. In my experience roc-auc metric is almost never improved a lot by ensembling.
", "rawMarkdown": "Ensembling brings little indeed. In my experience roc-auc metric is almost never improved a lot by ensembling.", "votes": null }, { "id": "1561188", "postDate": "10/27/2021 12:15:36", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
I can recommend you make whitening, LP & HP filter and Tukey window for obtaining CQT features similar with CQT from gwpy or pycbc. I may recommend you read G2Net Spectral Whitening
\nMy implementation tukey window on torch:
\nfrom math import cos\n\ndef torch_tukey(size, alpha, dtype):\n window = torch.zeros((size,), dtype=dtype)\n for n in range(size // 2 + 1):\n if n < alpha * size / 2:\n window[n] = window[-n] = 1 / 2 * (1 - cos(2 * 3.14 * n / (size * alpha)))\n else:\n window[n] = window[-n] = 1\n return window\nCQT with whitening & applying tukey window
\n
I also tried grad-cam on my research with model on TensorFlow and CQT calculated with PyCBC package and obtain expected result for me.
", "votes": null, "replies": [ { "id": 1474917, "author_name": "yassinealouini", "author_url": "", "post_date": "08/16/2021 10:46:32", "content": "That's awesome, thanks for those details!
", "votes": null, "replies": [] }, { "id": 1475784, "author_name": "yassinealouini", "author_url": "", "post_date": "08/16/2021 20:34:33", "content": "Here is the whitening code from the https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening notebook (thanks again @kevinmcisaac) for those that want it available quickly. Notice that the
\ncode uses PyTorch so it should be fast. Also, the whitening function here is the Hann window.
import torch\nfrom torch.fft import fft, rfft, ifft\nimport numpy as np\n\n\ndef whiten(signal):\n # From here: https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening\n hann = torch.hann_window(len(signal), periodic=True, dtype=float)\n spec = fft(torch.from_numpy(signal).float()* hann)\n mag = torch.sqrt(torch.real(spec*torch.conj(spec))) \n\n return torch.real(ifft(spec/mag)).numpy() * np.sqrt(len(signal)/2)\nGreat discussion guys! Are you aware of a PyTorch equivalent for a bandpass filter?
\nI am aware of torchaudio.functional.lfilter but this introduces a phase shift. I started trying to port scipy.signal.filtfilt to PyTorch but it's quite tricky due to the initial conditions (zi)… 😵️
I'm use
\nfrom torchaudio.functional import bandpass_biquad\n\ndef butter_bandpass_filter(data, lowcut, highcut, fs):\n return bandpass_biquad(data, fs, (highcut + lowcut) / 2, (highcut - lowcut) / (highcut + lowcut))\nAwesome thank you! I knew the answer was somewhere in those biquad filters :)
", "votes": null, "replies": [] }, { "id": 1476584, "author_name": "yassinealouini", "author_url": "", "post_date": "08/17/2021 07:10:45", "content": "Yet another great function, thanks @miklgr500. 👍
", "votes": null, "replies": [] }, { "id": 1484790, "author_name": "tanulsingh077", "author_url": "", "post_date": "08/21/2021 15:18:03", "content": "Hi @miklgr500 this might be a very dumb question but can you tell us that are you using hann window for whitening similar to kernel here : https://www.kaggle.com/kevinmcisaac/g2net-spectral-whitening
\nAlso you follow this order right : Signal --> Whiten --> BandPass Filtering --> CQT using Tuckey window
", "votes": null, "replies": [] }, { "id": 1484846, "author_name": "miklgr500", "author_url": "", "post_date": "08/21/2021 15:49:18", "content": "I'm use use Tukey widow for whiten and Hann for CQT
", "votes": null, "replies": [] }, { "id": 1484860, "author_name": "solosquad1999", "author_url": "", "post_date": "08/21/2021 16:08:27", "content": "@tanulsingh077 : As far as I know, the CQT transform already contains bandpass filtering.
", "votes": null, "replies": [] }, { "id": 1484967, "author_name": "authman", "author_url": "", "post_date": "08/21/2021 17:31:41", "content": "Sorry for the dumb question, just jumping in. For those who are doing processing on gpu (using torch), are you passing float64s -> gpu?
", "votes": null, "replies": [] }, { "id": 1485381, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/22/2021 04:11:51", "content": "@miklgr500 which image is whitened and which is post tukey.
\nIs tukey window and whitening used by cqt also or they are additional transformation before applying cqt
@tanulsingh077 if using BPF and whitening, bpf would have to occur first imho
", "votes": null, "replies": [] }, { "id": 1486497, "author_name": "solosquad1999", "author_url": "", "post_date": "08/23/2021 02:59:25", "content": "@miklgr500 : Did you find that spectral whitening worked? I tried spectral whitening using Tukey window (window=(\"tukey\",0.15)) but according to the experiments, the valid AUC was only around 0.81 either or not using the bandpass filtering (20Hz-500Hz with Hann window); whilst my best valid AUC was about 0.87 without spectral whitening.
", "votes": null, "replies": [] }, { "id": 1486545, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/23/2021 04:09:00", "content": "I too found it made the AUC worse, see Spectral Whitening
", "votes": null, "replies": [] } ] }, { "id": 1474812, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/16/2021 09:26:48", "content": "I'm able to get 0.874 using CWT (similar to CQT) that creates a 256x256 image and efficientnet B7.
", "votes": null, "replies": [ { "id": 1474834, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/16/2021 09:40:01", "content": "@kevinmcisaac what is difference between two. ANy notebook for CWT..
", "votes": null, "replies": [] }, { "id": 1475607, "author_name": "yassinealouini", "author_url": "", "post_date": "08/16/2021 18:47:46", "content": "@jaideepvalani CWT stands for continuous wavelet transform so it is quite different to the CQT one.
\nAs far as I know, you can use the scipy implementation here.
\nNotice that this won't be as optimized as the CQT nnAudio one (since it runs on GPU) but if you search more, you should be able to find one.
\nI hope this helps.
", "votes": null, "replies": [] }, { "id": 1476609, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/17/2021 07:21:34", "content": "CQT and CWT preform a similar function, ie.., convert a 1D waveform (i.e., time vs amplitude) to a 2D spectrogram (i.e., time vs amplitude). When you show a CWT of a GW event you get a very similar image to the CQT.
\nMostly I chose CWT because I found a TF implementation of CWT that I could convert to TF2.0 and make a Keras Layer, then use this in my existing Keras model (Efficient Net) that when feed with TFrecords and run on a TPU executes an epoch in under 3 min. This allowed me to experiment very quick.
\nThere is also the option to use different types of wavelets (which I've not yet done), that might help. However I think largely CWT and CQT will give very similar results.
", "votes": null, "replies": [] }, { "id": 1476621, "author_name": "kevinmcisaac", "author_url": "", "post_date": "08/17/2021 07:26:43", "content": "BTW, my implementation in TF2/Keras is in github. WHen used with a GPU and large batches the performance is excellent, i.e., 0.04ms / sample
", "votes": null, "replies": [] }, { "id": 1477169, "author_name": "yassinealouini", "author_url": "", "post_date": "08/17/2021 11:33:03", "content": "One other option is to have many models each one using a type of features: let's say one model trained on CWT and one on CQT. Then average both (or a more clever ensemble). Or even train one model on a mix of both features. Lots of creative ways to combine these so I guess the best way to know is to experiment. :)
", "votes": null, "replies": [] }, { "id": 1477386, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/17/2021 13:08:24", "content": "any sample code for CWT … the one give in uses synthetic signal.
\nI tried putting our actual strain but get very big size data
\ntorch.Size([32, 1, 30, 12288])
\nthis is unmanageable for GPU
@jaideepvalani you should have the same size as in CQT so this is indeed weird. If I find a good implementation for PyTorch of CWT, I will let you know. 👌
", "votes": null, "replies": [] }, { "id": 1477779, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/17/2021 16:20:18", "content": "i use this
\npycwt = CWT(widths, \"ricker\", 1)\nprint(data.shape)\n#data = torch.tensor(sig , dtype=torch.float32)\ndata = torch.tensor(sig1 , dtype=torch.float32).view(-1,1).squeeze(-1)\n\nprint('data',sig.shape,data.shape)\ndata = torch.stack([data] *1) # 3 channels\ndata = torch.stack([data] * 32) # Batch of 32\nout = pycwt(data)\n@jaideepvalani At the bottom of my notebook there is an example with GW data. The output shape is [1, 3, 80, 4096].
Unfortunately, strided convolutions with padding='same' are not yet supported (GitHub issue). This means you'll have to resize the images to something more manageable using torch.nn.functional.interpolate.
Another alternative is to thin the data along the time axis like this: out[:, :, :, ::8] which has a shape of [1, 3, 80, 512]
Thanks for sharing! You can upgrade torch to 1.9, then padding='same' works.
", "votes": null, "replies": [] }, { "id": 1479480, "author_name": "anjum48", "author_url": "", "post_date": "08/18/2021 13:38:48", "content": "Unfortunately not when stride > 1
", "votes": null, "replies": [] }, { "id": 1479605, "author_name": "hannes82", "author_url": "", "post_date": "08/18/2021 14:39:57", "content": "Oh ok I see. So far I have always resized the image afterwards.
", "votes": null, "replies": [] } ] }, { "id": 1478964, "author_name": "zarif98sjs", "author_url": "", "post_date": "08/18/2021 08:14:11", "content": "You should definitely try different preprocessing and compare which works better :)
", "votes": null, "replies": [] }, { "id": 1478987, "author_name": "sayedathar11", "author_url": "", "post_date": "08/18/2021 08:28:28", "content": "@yassinealouini a little bit of hint from my experiments it appears that preprocessing data before feeding it to CQT kernels can improve model's performance ! Think about it . Eg: in this kernel , using better preprocessing techniques you can boost oof score from 0.8669 to 0.86714 !
", "votes": null, "replies": [ { "id": 1486579, "author_name": "zarif98sjs", "author_url": "", "post_date": "08/23/2021 04:56:20", "content": "Hi @sayedathar11 , did you mean spectral whitening when you meant better preprocessing ?
", "votes": null, "replies": [] } ] }, { "id": 1485294, "author_name": "cpmpml", "author_url": "", "post_date": "08/22/2021 01:08:40", "content": "Our current score (0.880) is obtained with Q Transform. Our best single 5 fold model achieves 0.880 LB score. We will be back now that SETI is over.
", "votes": null, "replies": [ { "id": 1485327, "author_name": "authman", "author_url": "", "post_date": "08/22/2021 02:23:24", "content": "Is there a digit missing in there somewhere?
", "votes": null, "replies": [] }, { "id": 1485331, "author_name": "cpmpml", "author_url": "", "post_date": "08/22/2021 02:36:17", "content": "yes! tx, I fixed it.
", "votes": null, "replies": [] }, { "id": 1485377, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/22/2021 04:07:11", "content": "@cpmpml does dim of cqt output has got any significant impact my current single model 4f gets 87.4 ,still far from desired levels
", "votes": null, "replies": [] }, { "id": 1485718, "author_name": "yassinealouini", "author_url": "", "post_date": "08/22/2021 11:04:47", "content": "That's impressive, thanks for sharing some of the deatils. 👌
", "votes": null, "replies": [] }, { "id": 1486688, "author_name": "tanulsingh077", "author_url": "", "post_date": "08/23/2021 06:49:54", "content": "Hi @cpmpml are you using the time series from all the three detectors stacked on top of each other for input?
", "votes": null, "replies": [] }, { "id": 1487911, "author_name": "authman", "author_url": "", "post_date": "08/24/2021 00:18:43", "content": "I've ran about 10 experiments so far with CV from 0.860 to 0.867 and the usual LB bump. What I've noticed so far is that simple ensembling (forward, N-M weight opt, rank, mean, etc) all fails and actually removes the LB bump from the expected OOF. I wonder if this is what @cpmpml also experienced with his teams best model being the one driving LB score. I have yet to start stacking (would like a few more models in the arsenal first), but I have my CV setup ready for that.
", "votes": null, "replies": [] }, { "id": 1488519, "author_name": "cpmpml", "author_url": "", "post_date": "08/24/2021 10:46:29", "content": "Ensembling brings little indeed. In my experience roc-auc metric is almost never improved a lot by ensembling.
", "votes": null, "replies": [] } ] }, { "id": 1561188, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 12:15:36", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris