![\"https://i.ibb.co/vP5P1mR/Selection-760.png\"](\"https://i.ibb.co/vP5P1mR/Selection-760.png\")
some of the GW signal are not visible on CQT. If you can whiten the signal and confirm its position in time, you can
\n
some of the GW signal are not visible on CQT. If you can whiten the signal and confirm its position in time, you can
\nNice plots! For those interested, I made a discussion post in SETI comp explaining what \"class activation maps\" (i.e. grad cam) are here and a starter notebook here
", "rawMarkdown": "Nice plots! For those interested, I made a discussion post in SETI comp explaining what \"class activation maps\" (i.e. grad cam) are [here][1] and a starter notebook [here][2]\n\n[1]: https://www.kaggle.com/c/seti-breakthrough-listen/discussion/268314\n[2]: https://www.kaggle.com/cdeotte/silver-medal-with-grad-cam-lb-0-780", "votes": 11, "replies": [ { "id": 1494908, "postDate": "2021-08-29T06:59:02.787Z", "content": "Thanks for the notebook
", "rawMarkdown": "Thanks for the notebook", "votes": 4 } ] }, { "id": 1497791, "postDate": "2021-08-31T12:44:25.417Z", "content": "an interesting paper:
\nhttps://arxiv.org/pdf/1904.12069.pdf
Improving Deep Speech Denoising by Noisy2Noisy Signal Mapping
\nif we treat:
\nLivingston wave = signal + noise1
\nHanford wave = signal + noise2
we can learn the denoising in an unsupervised manner as described in the paper
", "rawMarkdown": "an interesting paper:\nhttps://arxiv.org/pdf/1904.12069.pdf\n\nImproving Deep Speech Denoising by Noisy2Noisy Signal Mapping\n\nif we treat:\n\nLivingston wave = signal + noise1\nHanford wave = signal + noise2\n\nwe can learn the denoising in an unsupervised manner as described in the paper\n", "votes": 3, "replies": [ { "id": 1499049, "postDate": "2021-09-01T12:03:35.353Z", "content": "we can use a network trained on 3 detector input to enforce the consistency of a network trained with one detector.
\nsmiliarly, the consistency between network of using different transform, difference image size, ….
\nmaybe such self-supervised learning on labelled and unlabelled data can help
", "rawMarkdown": "we can use a network trained on 3 detector input to enforce the consistency of a network trained with one detector.\n\nsmiliarly, the consistency between network of using different transform, difference image size, ....\n\nmaybe such self-supervised learning on labelled and unlabelled data can help" }, { "id": 1499260, "postDate": "2021-09-01T14:27:15.867Z", "content": "I did try autoencoders on the signals but apparently, The network ignores minute details on the signal which hurts the outcome. The network tends to learn dominant frequencies.
\nIn my case the network learnt 306 Hz on detector 1 & 2. Thats when I posted this
", "rawMarkdown": "I did try autoencoders on the signals but apparently, The network ignores minute details on the signal which hurts the outcome. The network tends to learn dominant frequencies. \n\nIn my case the network learnt 306 Hz on detector 1 & 2. Thats when I posted [this](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/263995) ", "replies": [ { "id": 2078822, "postDate": "2022-12-28T17:17:59.950Z", "content": "Hi, I know this competition is over but I am trying to improve myself by trying out new ideas. I tried Noise2Noise but id dosent seem to work. My understanding is that it's just a UNET with Hanford CQT as Input and Livingston CQT as Output? Did you ever get this to work for denoising? I guess one issue is not having a zero mean distribution and as also mentioned above learning the dominant noise frequencies as signal
", "rawMarkdown": "Hi, I know this competition is over but I am trying to improve myself by trying out new ideas. I tried Noise2Noise but id dosent seem to work. My understanding is that it's just a UNET with Hanford CQT as Input and Livingston CQT as Output? Did you ever get this to work for denoising? I guess one issue is not having a zero mean distribution and as also mentioned above learning the dominant noise frequencies as signal" } ] } ] }, { "id": 1493591, "postDate": "2021-08-28T03:50:39.763Z", "content": "![\"https://i.ibb.co/n72pHhB/Selection-762.png\"](\"https://i.ibb.co/n72pHhB/Selection-762.png\")
a complicated way to modify the SNR of pos samples.
\nthe sample way is just to add up the two waveform: good pos sample + neg sample
\nor construct the PSD. instead of using PSD to denoise, use it to increase noise (or decrease signal)
another weird idea?
\ndivide the samples according to label and score. use unpaired-cyclic GAN to translate from one domain to another
i realise gan can go both forward and backward
\n![\"https://i.ibb.co/G9wvgnb/Selection-865.png\"](\"https://i.ibb.co/G9wvgnb/Selection-865.png\")
maybe we need to use the raw wave as input:
\n![\"https://i.ibb.co/bWvwZP3/Selection-761.png\"](\"https://i.ibb.co/bWvwZP3/Selection-761.png\")
https://arxiv.org/pdf/1701.00008.pdf
\nhttps://arxiv.org/pdf/2012.13101.pdf
That image you just posted is even a better illustration of the point I was trying to make above. Also, from what I recall about reading that paper 2-3 weeks back, they used 1) whitened signal not raw, and 2) a regular old spectrogram to make that image as opposed to, e.g. CQT, or even a melspectrogram. See y-axis.
", "rawMarkdown": "That image you just posted is even a better illustration of the point I was trying to make above. Also, from what I recall about reading that paper 2-3 weeks back, they used 1) whitened signal not raw, and 2) a regular old spectrogram to make that image as opposed to, e.g. CQT, or even a melspectrogram. See y-axis.", "votes": 3 }, { "id": 1495991, "postDate": "2021-08-30T01:42:35.193Z", "content": "I tested raw wave input and a simple 1d conv model, but AUC was never over 0.5. Did anyone manage to make it work?
", "rawMarkdown": "I tested raw wave input and a simple 1d conv model, but AUC was never over 0.5. Did anyone manage to make it work?", "votes": 5 }, { "id": 1496021, "postDate": "2021-08-30T03:03:27.607Z", "content": "I tried raw wave input with LSTM and cwt with LSTM. But it didn't work.
", "rawMarkdown": "I tried raw wave input with LSTM and cwt with LSTM. But it didn't work.", "votes": 1 }, { "id": 1496023, "postDate": "2021-08-30T03:06:13.383Z", "content": "By \"raw\", I assume you mean whitened or at least BP'd? In either case, I tried processed wave with GRU and with transformer (1D-ViT) and neither converged.
", "rawMarkdown": "By \"raw\", I assume you mean whitened or at least BP'd? In either case, I tried processed wave with GRU and with transformer (1D-ViT) and neither converged." }, { "id": 1496029, "postDate": "2021-08-30T03:17:40.217Z", "content": "BTW-one of the crazier papers I had read said they fed a FFN the real and the imaginary components (separately) of the signal transformed and were able to get that to converge, which is pretty wild. I tried that as well and it very much hugged 0.5 AUC. I do plan on tinkering with it again though in a week or so once I run out of ideas.
", "rawMarkdown": "BTW-one of the crazier papers I had read said they fed a FFN the real and the imaginary components (separately) of the signal transformed and were able to get that to converge, which is pretty wild. I tried that as well and it ***very*** much hugged 0.5 AUC. I do plan on tinkering with it again though in a week or so once I run out of ideas.", "votes": 2 }, { "id": 1496549, "postDate": "2021-08-30T13:33:32.783Z", "content": "I had the same experience with transformers. Strangely it works with mel spectograms but not with CWT/CQT..
", "rawMarkdown": "I had the same experience with transformers. Strangely it works with mel spectograms but not with CWT/CQT..", "votes": 1 }, { "id": 1496650, "postDate": "2021-08-30T14:34:10.730Z", "content": "i can train vision transformer (after some difficulty).
\nthe reason is this: https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/269154
33% of the +ve label is similar to noise
\na shortcut is to used channel based attention transformer:
\nhttps://github.com/facebookresearch/xcit/blob/master/xcit.py
\n(also available in TIMM)
Thanks for sharing this insight! This clears up things.
", "rawMarkdown": "Thanks for sharing this insight! This clears up things." }, { "id": 1500660, "postDate": "2021-09-02T14:51:46.740Z", "content": "we made 1d model work really well
", "rawMarkdown": "we made 1d model work really well", "votes": 2 }, { "id": 1501128, "postDate": "2021-09-03T00:54:40.567Z", "content": "@richx86 That sounds very interesting. Is your implementation based on papers presented in this discussion?
", "rawMarkdown": "@richx86 That sounds very interesting. Is your implementation based on papers presented in this discussion?", "votes": 1 }, { "id": 1501192, "postDate": "2021-09-03T02:50:42.200Z", "content": ":) Yes, in some sense. @analokamus
", "rawMarkdown": ":) Yes, in some sense. @analokamus ", "votes": 2 }, { "id": 1501495, "postDate": "2021-09-03T09:18:59.997Z", "content": "I work on a 1D model too. CV max is 0,83 for this moments. I think we have enough data to build a good model with learnt filter which are more efficient than CQT.
", "rawMarkdown": "I work on a 1D model too. CV max is 0,83 for this moments. I think we have enough data to build a good model with learnt filter which are more efficient than CQT. ", "votes": 1 }, { "id": 1509655, "postDate": "2021-09-11T14:16:19.520Z", "content": "Very interesting, I used Galerkin Transformer+Fourier Neural Operator approach acting on the 1D raw signal. The AUC on train converges to 1 real quick like 5 epochs, but the AUC on valid never leaves 0.5…
", "rawMarkdown": "Very interesting, I used Galerkin Transformer+Fourier Neural Operator approach acting on the 1D raw signal. The AUC on train converges to 1 real quick like 5 epochs, but the AUC on valid never leaves 0.5..." } ] }, { "id": 1498310, "postDate": "2021-08-31T21:10:00.770Z", "content": "Hi :),
\nI think that using CQT or spectrogram won’t be enough if the signal has a very low SNR. Only good pre processing or temporal convolution (with learnt filter) can. At this moment I try CNN1D. We dont need the three channel. With one channel only i have (AUC onCV)0,74, 2 channel 0,80 and all 0,83.
\nI use only custom archecture. Some resnet block and attention layer is enough (8 millions parameters). I think this is the way to reach a good score but I wont have enough time to try all my ideas alone
", "rawMarkdown": "Hi :),\n\nI think that using CQT or spectrogram won’t be enough if the signal has a very low SNR. Only good pre processing or temporal convolution (with learnt filter) can. At this moment I try CNN1D. We dont need the three channel. With one channel only i have (AUC onCV)0,74, 2 channel 0,80 and all 0,83.\n\nI use only custom archecture. Some resnet block and attention layer is enough (8 millions parameters). I think this is the way to reach a good score but I wont have enough time to try all my ideas alone", "votes": 4 }, { "id": 1493571, "postDate": "2021-08-28T03:26:46.323Z", "content": "Noise2Noise: Learning Image Restoration without Clean Data
\nhttps://arxiv.org/pdf/1803.04189.pdf
application to audio
\n![\"https://i.ibb.co/7zSn9cK/3-Figure1-1.png\"](\"https://i.ibb.co/7zSn9cK/3-Figure1-1.png\")
Underwater Signal Denoising Using Deep Learning Approach
\n…. In this context, we propose WaveN2N that is able to learn noise removal and clean signal reconstruction from multi-channels array data in a self-supervised learning setting. …
", "rawMarkdown": "Noise2Noise: Learning Image Restoration without Clean Data\nhttps://arxiv.org/pdf/1803.04189.pdf\n\napplication to audio\n\n\n\n\nUnderwater Signal Denoising Using Deep Learning Approach\n \n .... In this context, we propose WaveN2N that is able to learn noise removal and clean signal reconstruction from multi-channels array data in a self-supervised learning setting. ...", "votes": 4 }, { "id": 1493346, "postDate": "2021-08-27T19:33:15.383Z", "content": "I haven't visually verified this myself-but on the forums, it's been reported that GW events occur in the 0.5 - 1.0 portion of the generated signals in train set. One of the experiments I ran last week was simply lopping off the first 1-10% of the signal (after preprocessing!) to see if getting rid of trash helps. None of these experiments helped. I went a bit further and tried training by chopping off random 1-10% and then rescaling the remaining back to full width and testing with original images, and I tried the inverse (training will full signal, but then doing inference by chopping off 1,2,3,…10% of the signal). All of these experiments resulted in lower RoC-AUC.
\n![\"\"](\"https://i.imgur.com/Q1DauBa.jpg\")
Looking at images like the above, I think it makes sense that having the tail of the GW can be beneficial to the model. If anything, I think it might be beneficial to experiment with SED, find the most likey GW candidate like max across all 3 wave images closest in time, then chop off whatever happens after that point. While we aren't aware of where in the signal a GW occurs, we are aware that there aren't more than 2 simulated GW's per sample. And even if there are, if we find in a sample the most likely GW candidate position, then even if there is another GW in the signal, by definition, it has a less likelihood of being a TP anyway.
\nDisclaimer-I haven't ran that proposed experiment yet.
", "rawMarkdown": "I haven't visually verified this myself-but on the forums, [it's been reported](https://www.kaggle.com/mistag/reverse-engineering-create-clean-gw-signals#1397273) that GW events occur in the 0.5 - 1.0 portion of the generated signals in train set. One of the experiments I ran last week was simply lopping off the first 1-10% of the signal (after preprocessing!) to see if getting rid of trash helps. None of these experiments helped. I went a bit further and tried training by chopping off random 1-10% and then rescaling the remaining back to full width and testing with original images, and I tried the inverse (training will full signal, but then doing inference by chopping off 1,2,3,...10% of the signal). All of these experiments resulted in lower RoC-AUC.\n\n\n\nLooking at images like the above, I think it makes sense that having the tail of the GW can be beneficial to the model. If anything, I think it might be beneficial to experiment with SED, find the most likey GW candidate like max across all 3 wave images closest in time, then chop off whatever happens _after_ that point. While we aren't aware of where in the signal a GW occurs, we are aware that there aren't more than 2 simulated GW's per sample. And even if there are, if we find in a sample the most likely GW candidate position, then even if there is another GW in the signal, by definition, it has a less likelihood of being a TP anyway.\n\nDisclaimer-I haven't ran that proposed experiment yet.", "votes": 4, "replies": [ { "id": 1493356, "postDate": "2021-08-27T19:48:27.080Z", "content": "i am looking for open source parameter estimation of GW. Then I would like to use this as the additional label to train a DNN or CNN
", "rawMarkdown": "i am looking for open source parameter estimation of GW. Then I would like to use this as the additional label to train a DNN or CNN", "votes": 1 }, { "id": 1493361, "postDate": "2021-08-27T19:55:52.577Z", "content": "It might be easier to go the other way round. That is, train a network on TN images with synthetically injected signals. The dataset is well balanced with 50% pos and 50% neg labels, and that first link in my post is to a kernel that can simulate merger events with 3-4 of the 15 generating parameters.
", "rawMarkdown": "It might be easier to go the other way round. That is, train a network on TN images with synthetically injected signals. The dataset is well balanced with 50% pos and 50% neg labels, and that first link in my post is to a kernel that can simulate merger events with 3-4 of the 15 generating parameters.", "votes": 1 } ] }, { "id": 1508181, "postDate": "2021-09-10T01:16:32.130Z", "content": "1d CNN is here !!!!!!!!!!!!
\nhttps://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py
thank you very much to @kit716
\nhis notebook at: https://www.kaggle.com/kit716/grav-wave-detection
", "rawMarkdown": "1d CNN is here !!!!!!!!!!!!\nhttps://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py\n\nthank you very much to @kit716 \n\nhis notebook at: https://www.kaggle.com/kit716/grav-wave-detection", "votes": 1, "replies": [ { "id": 1508197, "postDate": "2021-09-10T01:59:11.703Z", "content": "It looks like the cat is out of the bag. It was bound to happen anyway. 1D CNNs are indeed effective (0.87450726, 0.8774) with the right architectures and have the added benefit that you can fit the entire dataset in ram (~25 gb float32s). But I'm still of the mind 2D is king here.
", "rawMarkdown": "It looks like the cat is out of the bag. It was bound to happen anyway. 1D CNNs are indeed effective (0.87450726, 0.8774) with the right architectures and have the added benefit that you can fit the entire dataset in ram (~25 gb float32s). But I'm still of the mind 2D is king here.", "votes": 1 }, { "id": 1508205, "postDate": "2021-09-10T02:23:17.337Z", "content": "1d cnn can localise the GW in time (e.g. via activation map or learned attention)
\none can use this to set localisation consistency between 2d cnn.
on a side note:
\nGravity Kills Schrödinger's Cat
\nTheorists argue that warped spacetime prevents quantum superpositions of large-scale objects
now i am waiting for the transformer in public notebook
", "rawMarkdown": "now i am waiting for the transformer in public notebook", "votes": 1 }, { "id": 1508220, "postDate": "2021-09-10T02:44:20.567Z", "content": "I think that 1D is the king here. I have 87,2xxx AUC on LB with 1D and you 0,8774 which is better. The first are maybe using 2D but for 0,003-4 better AUC ol but which the prize? Bigger model with efficientnet ? Mine is 8million parameters, your I dont know. One epoch is into 27 seconds. Researcher can continue the training with more data simulated to overcome the small between 1D and 2D which is only on 16% on the test for moments.
\nWe would need SNR value to make a curriculum learning (good SNR and step by step add lower and lower snr)
", "rawMarkdown": "I think that 1D is the king here. I have 87,2xxx AUC on LB with 1D and you 0,8774 which is better. The first are maybe using 2D but for 0,003-4 better AUC ol but which the prize? Bigger model with efficientnet ? Mine is 8million parameters, your I dont know. One epoch is into 27 seconds. Researcher can continue the training with more data simulated to overcome the small between 1D and 2D which is only on 16% on the test for moments. \n\nWe would need SNR value to make a curriculum learning (good SNR and step by step add lower and lower snr) " }, { "id": 1508233, "postDate": "2021-09-10T03:09:27.140Z", "content": "Well, there's two trains of thought. >LB or >help scientific community. For a business problem, the latter makes sense, but for kaggle… 0.00001 makes the difference. For the GW researchers, anything that anyone does here will be more optimal than matched filtering by orders of magnitude. Even if its just used as first stage filtering.
", "rawMarkdown": "Well, there's two trains of thought. >LB or >help scientific community. For a business problem, the latter makes sense, but for kaggle... 0.00001 makes the difference. For the GW researchers, anything that anyone does here will be more optimal than matched filtering by orders of magnitude. Even if its just used as first stage filtering." }, { "id": 1508555, "postDate": "2021-09-10T10:40:53.860Z", "content": "So I haven't tried the code yet but the architecture seems quite straight forward and similar to what I've tried. I'm honestly curious as to why this managed to converge and my model did not. Perhaps the batchnorm is the key since I didn't include it.
", "rawMarkdown": "So I haven't tried the code yet but the architecture seems quite straight forward and similar to what I've tried. I'm honestly curious as to why this managed to converge and my model did not. Perhaps the batchnorm is the key since I didn't include it." } ] }, { "id": 1503140, "postDate": "2021-09-05T03:55:47.853Z", "content": "another model to try: 1d auto regressive flow network:
\nhttps://arxiv.org/pdf/2002.07656.pdf
@hengck23 can you guide me which library you are using for CQT transform as I am not getting the input plots which you are getting
", "rawMarkdown": "@hengck23 can you guide me which library you are using for CQT transform as I am not getting the input plots which you are getting", "votes": 1, "replies": [ { "id": 1503926, "postDate": "2021-09-05T21:56:19.523Z", "content": "I think that the key to get the diagam there is getting/not getting the normalization right
", "rawMarkdown": "I think that the key to get the diagam there is getting/not getting the normalization right" } ] }, { "id": 1495987, "postDate": "2021-08-30T01:25:35.197Z", "content": "I wonder if the grad-cam results can be used a segmentation mask and could be used for pre-training
", "rawMarkdown": "I wonder if the grad-cam results can be used a segmentation mask and could be used for pre-training", "votes": 1, "replies": [ { "id": 1496656, "postDate": "2021-08-30T14:40:08.083Z", "content": "you can try unet like encoder and decoder and just use maxpool or GME pool at the output.
\nwe assume there is only one wave in the positive sample.
Thanks for sharing! What is the advantage of using GME pool, if there is any?
", "rawMarkdown": "Thanks for sharing! What is the advantage of using GME pool, if there is any?" }, { "id": 1497872, "postDate": "2021-08-31T13:22:58.770Z", "content": "Before GEM, people used either average pooling or max pooling. GEM is an ensemble of both average pooling and max pooling. Also during training, GEM learns how much average to use and how much max to use
", "rawMarkdown": "Before GEM, people used either average pooling or max pooling. GEM is an ensemble of both average pooling and max pooling. Also during training, GEM learns how much average to use and how much max to use", "votes": 8 }, { "id": 1498149, "postDate": "2021-08-31T17:45:13.143Z", "content": "Thanks a lot for the clear explanation!
", "rawMarkdown": "Thanks a lot for the clear explanation!", "votes": 1 } ] }, { "id": 1493386, "postDate": "2021-08-27T20:40:49.587Z", "content": "![\"https://i.ibb.co/k1YF9HX/11-Figure3-1.png\"](\"https://i.ibb.co/k1YF9HX/11-Figure3-1.png\")
\nhttps://arxiv.org/abs/1904.08693
yet another GW wavenet paper: https://www.sciencedirect.com/science/article/pii/S0370269320308327?via%3Dihub
", "rawMarkdown": "yet another GW wavenet paper: https://www.sciencedirect.com/science/article/pii/S0370269320308327?via%3Dihub" }, { "id": 1512091, "postDate": "2021-09-14T00:28:00.770Z", "content": "![\"https://github.com/mravanelli/SincNet/blob/master/SincNet.png\"](\"https://github.com/mravanelli/SincNet/blob/master/SincNet.png\")
https://towardsdatascience.com/whats-up-with-waveform-based-vggs-15ff7c3afc28
\nhttps://github.com/mravanelli/SincNet
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
fourier layer
\nhttps://zongyi-li.github.io/blog/2020/fourier-pde/
![\"http://zongyi-li.github.io/assets/img/fourier_layer.png\"](\"http://zongyi-li.github.io/assets/img/fourier_layer.png\")
I have re-implemented with a few added tweaks using the Fourier Neural Operator combined with the Galerkin Transformer feature extractor (attention on a specific frequency band), but unfortunately the val auc stopped at 0.8 on a small subset of the data.
", "rawMarkdown": "I have re-implemented with a few added tweaks using the Fourier Neural Operator combined with the Galerkin Transformer feature extractor (attention on a specific frequency band), but unfortunately the val auc stopped at 0.8 on a small subset of the data." }, { "id": 1511807, "postDate": "2021-09-13T17:49:23.317Z", "content": "i suggest a simple stacked 1d conv model as the first step. you should get cv in the range 0.87+.
\nthen you can replace the 1d conv with Fourier Neural Operator.
and finally add Transformer.
", "rawMarkdown": "i suggest a simple stacked 1d conv model as the first step. you should get cv in the range 0.87+.\nthen you can replace the 1d conv with Fourier Neural Operator.\n\nand finally add Transformer. " }, { "id": 1511812, "postDate": "2021-09-13T17:52:54.097Z", "content": "Like the conv+deconv in U-net but in 1D?
", "rawMarkdown": "Like the conv+deconv in U-net but in 1D?" }, { "id": 1511829, "postDate": "2021-09-13T18:01:33.430Z", "content": "https://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py
\nthe above notebook is good enough for initial experiments
\nclass Model1DCNN(nn.Module):\n 1D convolutional neural network. Classifier of the gravitational waves.\n Architecture from there https://journals.aps.org/prl/pdf/10.1103/PhysRevLett.120.141103\nyou can modify the head (if you want to analyse the CAM activation map.)
\nyou can can increase the num of parameters for better results.
check also my post at https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/271576
", "rawMarkdown": "https://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py\n\nthe above notebook is good enough for initial experiments\n\n```\nclass Model1DCNN(nn.Module):\n 1D convolutional neural network. Classifier of the gravitational waves.\n Architecture from there https://journals.aps.org/prl/pdf/10.1103/PhysRevLett.120.141103\n \n```\n\nyou can modify the head (if you want to analyse the CAM activation map.)\nyou can can increase the num of parameters for better results.\n\ncheck also my post at https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/271576", "votes": 2 }, { "id": 1522951, "postDate": "2021-09-24T18:48:12.390Z", "content": "\nwhitening is dividing the wave by noise PSD. this is some form of weighing in the frequency domain.
\nisn't this the same as attention in frequency? maybe you can write a paper on Galerkin Transformer for whitening
", "rawMarkdown": "@scaomath \n\nwhitening is dividing the wave by noise PSD. this is some form of weighing in the frequency domain.\n\nisn't this the same as attention in frequency? maybe you can write a paper on Galerkin Transformer for whitening", "votes": 1 } ] }, { "id": 1495206, "postDate": "2021-08-29T11:41:43.937Z", "content": "![\"https://i.ibb.co/34WYZTb/Selection-768.png\"](\"https://i.ibb.co/34WYZTb/Selection-768.png\")
if you know the generator of the synthetic data … the regression can predict the 15 generating parameters
\nanother idea is to treat the template as query vectors (you need to learn the encoding net). these query vectors are input to and transform net (together with the input stain). the output will be if each of the query is present or not
", "rawMarkdown": "\n\nif you know the generator of the synthetic data ... the regression can predict the 15 generating parameters\n\nanother idea is to treat the template as query vectors (you need to learn the encoding net). these query vectors are input to and transform net (together with the input stain). the output will be if each of the query is present or not" }, { "id": 1493593, "postDate": "2021-08-28T04:04:46.067Z", "content": "![\"https://i.ibb.co/wQ5rx9w/Selection-764.png\"](\"https://i.ibb.co/wQ5rx9w/Selection-764.png\")
@hengck23 : Could you elaborate the second idea about the role of positional encoding and CNN to prevent FP? What I understand from yours is that the temporal feature of 1D signals might play an important role in filtering FP. I wonder that whether attention mechanisms on 2D spectrogram can help too, maybe should I give it a try.
", "rawMarkdown": "@hengck23 : Could you elaborate the second idea about the role of positional encoding and CNN to prevent FP? What I understand from yours is that the temporal feature of 1D signals might play an important role in filtering FP. I wonder that whether **attention mechanisms** on 2D spectrogram can help too, maybe should I give it a try.", "replies": [ { "id": 1493513, "postDate": "2021-08-28T01:43:04.800Z", "content": "CNNs are known to be translation invariant; but the chirp occurring in certain areas of the 2d spectrogram are almost guaranteed fp (in our case, basically anywhere on the left hand side, or anywhere in the middle-top of the image. it's possible to add PE to CNN, either as input channels into the network, or alternatively, each CNN layer can also encode that information. Maybe simpler though to handcraft the attention mask since we already know this information.
", "rawMarkdown": "CNNs are known to be translation invariant; but the chirp occurring in certain areas of the 2d spectrogram are almost guaranteed fp (in our case, basically anywhere on the left hand side, or anywhere in the middle-top of the image. it's possible to add PE to CNN, either as input channels into the network, or alternatively, each CNN layer can also encode that information. Maybe simpler though to handcraft the attention mask since we already know this information.", "votes": 1 }, { "id": 1493538, "postDate": "2021-08-28T02:46:19.520Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1493565, "postDate": "2021-08-28T03:16:35.380Z", "content": "I'm new to signal processing, and I wonder that whether it is possible to find the mapping (a function or the like) of a specific interval in 1D signal to its corresponding region in 2D spectrogram (as shown in this figure) and how to do it. Thanks in advance.
", "rawMarkdown": "\n\nI'm new to signal processing, and I wonder that whether it is possible to find the mapping (a function or the like) of a specific interval in 1D signal to its corresponding region in 2D spectrogram (as shown in this figure) and how to do it. Thanks in advance." } ] }, { "id": 1498989, "postDate": "2021-09-01T11:27:48.127Z", "content": "Nice plots! Thank u for sharing =))
", "rawMarkdown": "Nice plots! Thank u for sharing =))", "votes": 1 } ], "comments": [ { "id": 1493520, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-08-28T02:07:34.147000", "content": "Nice plots! For those interested, I made a discussion post in SETI comp explaining what \"class activation maps\" (i.e. grad cam) are here and a starter notebook here
", "votes": 11, "replies": [ { "id": 1494908, "author_name": "killua", "author_url": "", "post_date": "2021-08-29T06:59:02.787000", "content": "Thanks for the notebook
", "votes": 4, "replies": [] } ] }, { "id": 1497791, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-31T12:44:25.417000", "content": "an interesting paper:
\nhttps://arxiv.org/pdf/1904.12069.pdf
Improving Deep Speech Denoising by Noisy2Noisy Signal Mapping
\nif we treat:
\nLivingston wave = signal + noise1
\nHanford wave = signal + noise2
we can learn the denoising in an unsupervised manner as described in the paper
", "votes": 3, "replies": [ { "id": 1499049, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-01T12:03:35.353000", "content": "we can use a network trained on 3 detector input to enforce the consistency of a network trained with one detector.
\nsmiliarly, the consistency between network of using different transform, difference image size, ….
\nmaybe such self-supervised learning on labelled and unlabelled data can help
", "votes": 0, "replies": [] }, { "id": 1499260, "author_name": "Harsh Patel", "author_url": "", "post_date": "2021-09-01T14:27:15.867000", "content": "I did try autoencoders on the signals but apparently, The network ignores minute details on the signal which hurts the outcome. The network tends to learn dominant frequencies.
\nIn my case the network learnt 306 Hz on detector 1 & 2. Thats when I posted this
", "votes": 0, "replies": [ { "id": 2078822, "author_name": "Akshay Ghalsasi", "author_url": "", "post_date": "2022-12-28T17:17:59.950000", "content": "Hi, I know this competition is over but I am trying to improve myself by trying out new ideas. I tried Noise2Noise but id dosent seem to work. My understanding is that it's just a UNET with Hanford CQT as Input and Livingston CQT as Output? Did you ever get this to work for denoising? I guess one issue is not having a zero mean distribution and as also mentioned above learning the dominant noise frequencies as signal
", "votes": 0, "replies": [] } ] } ] }, { "id": 1493591, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-28T03:50:39.763000", "content": "
a complicated way to modify the SNR of pos samples.
\nthe sample way is just to add up the two waveform: good pos sample + neg sample
\nor construct the PSD. instead of using PSD to denoise, use it to increase noise (or decrease signal)
another weird idea?
\ndivide the samples according to label and score. use unpaired-cyclic GAN to translate from one domain to another
i realise gan can go both forward and backward
\n
maybe we need to use the raw wave as input:
\n
https://arxiv.org/pdf/1701.00008.pdf
\nhttps://arxiv.org/pdf/2012.13101.pdf
That image you just posted is even a better illustration of the point I was trying to make above. Also, from what I recall about reading that paper 2-3 weeks back, they used 1) whitened signal not raw, and 2) a regular old spectrogram to make that image as opposed to, e.g. CQT, or even a melspectrogram. See y-axis.
", "votes": 3, "replies": [] }, { "id": 1495991, "author_name": "RabotniKuma", "author_url": "", "post_date": "2021-08-30T01:42:35.193000", "content": "I tested raw wave input and a simple 1d conv model, but AUC was never over 0.5. Did anyone manage to make it work?
", "votes": 5, "replies": [] }, { "id": 1496021, "author_name": "Johnny Lee", "author_url": "", "post_date": "2021-08-30T03:03:27.607000", "content": "I tried raw wave input with LSTM and cwt with LSTM. But it didn't work.
", "votes": 1, "replies": [] }, { "id": 1496023, "author_name": "عثمان", "author_url": "", "post_date": "2021-08-30T03:06:13.383000", "content": "By \"raw\", I assume you mean whitened or at least BP'd? In either case, I tried processed wave with GRU and with transformer (1D-ViT) and neither converged.
", "votes": 0, "replies": [] }, { "id": 1496029, "author_name": "عثمان", "author_url": "", "post_date": "2021-08-30T03:17:40.217000", "content": "BTW-one of the crazier papers I had read said they fed a FFN the real and the imaginary components (separately) of the signal transformed and were able to get that to converge, which is pretty wild. I tried that as well and it very much hugged 0.5 AUC. I do plan on tinkering with it again though in a week or so once I run out of ideas.
", "votes": 2, "replies": [] }, { "id": 1496549, "author_name": "Hannes Öhler", "author_url": "", "post_date": "2021-08-30T13:33:32.783000", "content": "I had the same experience with transformers. Strangely it works with mel spectograms but not with CWT/CQT..
", "votes": 1, "replies": [] }, { "id": 1496650, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-30T14:34:10.730000", "content": "i can train vision transformer (after some difficulty).
\nthe reason is this: https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/269154
33% of the +ve label is similar to noise
\na shortcut is to used channel based attention transformer:
\nhttps://github.com/facebookresearch/xcit/blob/master/xcit.py
\n(also available in TIMM)
Thanks for sharing this insight! This clears up things.
", "votes": 0, "replies": [] }, { "id": 1500660, "author_name": "Richard Xing", "author_url": "", "post_date": "2021-09-02T14:51:46.740000", "content": "we made 1d model work really well
", "votes": 2, "replies": [] }, { "id": 1501128, "author_name": "RabotniKuma", "author_url": "", "post_date": "2021-09-03T00:54:40.567000", "content": "@richx86 That sounds very interesting. Is your implementation based on papers presented in this discussion?
", "votes": 1, "replies": [] }, { "id": 1501192, "author_name": "Richard Xing", "author_url": "", "post_date": "2021-09-03T02:50:42.200000", "content": ":) Yes, in some sense. @analokamus
", "votes": 2, "replies": [] }, { "id": 1501495, "author_name": "Benjamin35", "author_url": "", "post_date": "2021-09-03T09:18:59.997000", "content": "I work on a 1D model too. CV max is 0,83 for this moments. I think we have enough data to build a good model with learnt filter which are more efficient than CQT.
", "votes": 1, "replies": [] }, { "id": 1509655, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-11T14:16:19.520000", "content": "Very interesting, I used Galerkin Transformer+Fourier Neural Operator approach acting on the 1D raw signal. The AUC on train converges to 1 real quick like 5 epochs, but the AUC on valid never leaves 0.5…
", "votes": 0, "replies": [] } ] }, { "id": 1498310, "author_name": "Benjamin35", "author_url": "", "post_date": "2021-08-31T21:10:00.770000", "content": "Hi :),
\nI think that using CQT or spectrogram won’t be enough if the signal has a very low SNR. Only good pre processing or temporal convolution (with learnt filter) can. At this moment I try CNN1D. We dont need the three channel. With one channel only i have (AUC onCV)0,74, 2 channel 0,80 and all 0,83.
\nI use only custom archecture. Some resnet block and attention layer is enough (8 millions parameters). I think this is the way to reach a good score but I wont have enough time to try all my ideas alone
", "votes": 4, "replies": [] }, { "id": 1493571, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-28T03:26:46.323000", "content": "Noise2Noise: Learning Image Restoration without Clean Data
\nhttps://arxiv.org/pdf/1803.04189.pdf
application to audio
\n
Underwater Signal Denoising Using Deep Learning Approach
\n…. In this context, we propose WaveN2N that is able to learn noise removal and clean signal reconstruction from multi-channels array data in a self-supervised learning setting. …
", "votes": 4, "replies": [] }, { "id": 1493346, "author_name": "عثمان", "author_url": "", "post_date": "2021-08-27T19:33:15.383000", "content": "I haven't visually verified this myself-but on the forums, it's been reported that GW events occur in the 0.5 - 1.0 portion of the generated signals in train set. One of the experiments I ran last week was simply lopping off the first 1-10% of the signal (after preprocessing!) to see if getting rid of trash helps. None of these experiments helped. I went a bit further and tried training by chopping off random 1-10% and then rescaling the remaining back to full width and testing with original images, and I tried the inverse (training will full signal, but then doing inference by chopping off 1,2,3,…10% of the signal). All of these experiments resulted in lower RoC-AUC.
\n
Looking at images like the above, I think it makes sense that having the tail of the GW can be beneficial to the model. If anything, I think it might be beneficial to experiment with SED, find the most likey GW candidate like max across all 3 wave images closest in time, then chop off whatever happens after that point. While we aren't aware of where in the signal a GW occurs, we are aware that there aren't more than 2 simulated GW's per sample. And even if there are, if we find in a sample the most likely GW candidate position, then even if there is another GW in the signal, by definition, it has a less likelihood of being a TP anyway.
\nDisclaimer-I haven't ran that proposed experiment yet.
", "votes": 4, "replies": [ { "id": 1493356, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-27T19:48:27.080000", "content": "i am looking for open source parameter estimation of GW. Then I would like to use this as the additional label to train a DNN or CNN
", "votes": 1, "replies": [] }, { "id": 1493361, "author_name": "عثمان", "author_url": "", "post_date": "2021-08-27T19:55:52.577000", "content": "It might be easier to go the other way round. That is, train a network on TN images with synthetically injected signals. The dataset is well balanced with 50% pos and 50% neg labels, and that first link in my post is to a kernel that can simulate merger events with 3-4 of the 15 generating parameters.
", "votes": 1, "replies": [] } ] }, { "id": 1508181, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-10T01:16:32.130000", "content": "1d CNN is here !!!!!!!!!!!!
\nhttps://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py
thank you very much to @kit716
\nhis notebook at: https://www.kaggle.com/kit716/grav-wave-detection
", "votes": 1, "replies": [ { "id": 1508197, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-10T01:59:11.703000", "content": "It looks like the cat is out of the bag. It was bound to happen anyway. 1D CNNs are indeed effective (0.87450726, 0.8774) with the right architectures and have the added benefit that you can fit the entire dataset in ram (~25 gb float32s). But I'm still of the mind 2D is king here.
", "votes": 1, "replies": [] }, { "id": 1508205, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-10T02:23:17.337000", "content": "1d cnn can localise the GW in time (e.g. via activation map or learned attention)
\none can use this to set localisation consistency between 2d cnn.
on a side note:
\nGravity Kills Schrödinger's Cat
\nTheorists argue that warped spacetime prevents quantum superpositions of large-scale objects
now i am waiting for the transformer in public notebook
", "votes": 1, "replies": [] }, { "id": 1508220, "author_name": "Benjamin35", "author_url": "", "post_date": "2021-09-10T02:44:20.567000", "content": "I think that 1D is the king here. I have 87,2xxx AUC on LB with 1D and you 0,8774 which is better. The first are maybe using 2D but for 0,003-4 better AUC ol but which the prize? Bigger model with efficientnet ? Mine is 8million parameters, your I dont know. One epoch is into 27 seconds. Researcher can continue the training with more data simulated to overcome the small between 1D and 2D which is only on 16% on the test for moments.
\nWe would need SNR value to make a curriculum learning (good SNR and step by step add lower and lower snr)
", "votes": 0, "replies": [] }, { "id": 1508233, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-10T03:09:27.140000", "content": "Well, there's two trains of thought. >LB or >help scientific community. For a business problem, the latter makes sense, but for kaggle… 0.00001 makes the difference. For the GW researchers, anything that anyone does here will be more optimal than matched filtering by orders of magnitude. Even if its just used as first stage filtering.
", "votes": 0, "replies": [] }, { "id": 1508555, "author_name": "brachester", "author_url": "", "post_date": "2021-09-10T10:40:53.860000", "content": "So I haven't tried the code yet but the architecture seems quite straight forward and similar to what I've tried. I'm honestly curious as to why this managed to converge and my model did not. Perhaps the batchnorm is the key since I didn't include it.
", "votes": 0, "replies": [] } ] }, { "id": 1503140, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-05T03:55:47.853000", "content": "another model to try: 1d auto regressive flow network:
\nhttps://arxiv.org/pdf/2002.07656.pdf
@hengck23 can you guide me which library you are using for CQT transform as I am not getting the input plots which you are getting
", "votes": 1, "replies": [ { "id": 1503926, "author_name": "Felipe Bivort Haiek", "author_url": "", "post_date": "2021-09-05T21:56:19.523000", "content": "I think that the key to get the diagam there is getting/not getting the normalization right
", "votes": 0, "replies": [] } ] }, { "id": 1495987, "author_name": "Bibek", "author_url": "", "post_date": "2021-08-30T01:25:35.197000", "content": "I wonder if the grad-cam results can be used a segmentation mask and could be used for pre-training
", "votes": 1, "replies": [ { "id": 1496656, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-30T14:40:08.083000", "content": "you can try unet like encoder and decoder and just use maxpool or GME pool at the output.
\nwe assume there is only one wave in the positive sample.
Thanks for sharing! What is the advantage of using GME pool, if there is any?
", "votes": 0, "replies": [] }, { "id": 1497872, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-08-31T13:22:58.770000", "content": "Before GEM, people used either average pooling or max pooling. GEM is an ensemble of both average pooling and max pooling. Also during training, GEM learns how much average to use and how much max to use
", "votes": 8, "replies": [] }, { "id": 1498149, "author_name": "Richard Xing", "author_url": "", "post_date": "2021-08-31T17:45:13.143000", "content": "Thanks a lot for the clear explanation!
", "votes": 1, "replies": [] } ] }, { "id": 1493386, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-27T20:40:49.587000", "content": "
\nhttps://arxiv.org/abs/1904.08693
yet another GW wavenet paper: https://www.sciencedirect.com/science/article/pii/S0370269320308327?via%3Dihub
", "votes": 0, "replies": [] }, { "id": 1512091, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-14T00:28:00.770000", "content": "
https://towardsdatascience.com/whats-up-with-waveform-based-vggs-15ff7c3afc28
\nhttps://github.com/mravanelli/SincNet
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
fourier layer
\nhttps://zongyi-li.github.io/blog/2020/fourier-pde/
I have re-implemented with a few added tweaks using the Fourier Neural Operator combined with the Galerkin Transformer feature extractor (attention on a specific frequency band), but unfortunately the val auc stopped at 0.8 on a small subset of the data.
", "votes": 0, "replies": [] }, { "id": 1511807, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-13T17:49:23.317000", "content": "i suggest a simple stacked 1d conv model as the first step. you should get cv in the range 0.87+.
\nthen you can replace the 1d conv with Fourier Neural Operator.
and finally add Transformer.
", "votes": 0, "replies": [] }, { "id": 1511812, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-13T17:52:54.097000", "content": "Like the conv+deconv in U-net but in 1D?
", "votes": 0, "replies": [] }, { "id": 1511829, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-13T18:01:33.430000", "content": "https://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py
\nthe above notebook is good enough for initial experiments
\n![\"image\"](\"https://imgur.com/a/xz1pFHY\")
class Model1DCNN(nn.Module):\n 1D convolutional neural network. Classifier of the gravitational waves.\n Architecture from there https://journals.aps.org/prl/pdf/10.1103/PhysRevLett.120.141103\nyou can modify the head (if you want to analyse the CAM activation map.)
\nyou can can increase the num of parameters for better results.
check also my post at https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/271576
", "votes": 2, "replies": [] }, { "id": 1522951, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-24T18:48:12.390000", "content": "\nwhitening is dividing the wave by noise PSD. this is some form of weighing in the frequency domain.
\nisn't this the same as attention in frequency? maybe you can write a paper on Galerkin Transformer for whitening
", "votes": 1, "replies": [] } ] }, { "id": 1495206, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-29T11:41:43.937000", "content": "
if you know the generator of the synthetic data … the regression can predict the 15 generating parameters
\nanother idea is to treat the template as query vectors (you need to learn the encoding net). these query vectors are input to and transform net (together with the input stain). the output will be if each of the query is present or not
", "votes": 0, "replies": [] }, { "id": 1493593, "author_name": "hengck23", "author_url": "", "post_date": "2021-08-28T04:04:46.067000", "content": "
@hengck23 : Could you elaborate the second idea about the role of positional encoding and CNN to prevent FP? What I understand from yours is that the temporal feature of 1D signals might play an important role in filtering FP. I wonder that whether attention mechanisms on 2D spectrogram can help too, maybe should I give it a try.
", "votes": 0, "replies": [ { "id": 1493513, "author_name": "عثمان", "author_url": "", "post_date": "2021-08-28T01:43:04.800000", "content": "CNNs are known to be translation invariant; but the chirp occurring in certain areas of the 2d spectrogram are almost guaranteed fp (in our case, basically anywhere on the left hand side, or anywhere in the middle-top of the image. it's possible to add PE to CNN, either as input channels into the network, or alternatively, each CNN layer can also encode that information. Maybe simpler though to handcraft the attention mask since we already know this information.
", "votes": 1, "replies": [] }, { "id": 1493538, "author_name": "", "author_url": "", "post_date": "2021-08-28T02:46:19.520000", "content": "", "votes": 0, "replies": [] }, { "id": 1493565, "author_name": "The fearless", "author_url": "", "post_date": "2021-08-28T03:16:35.380000", "content": "I'm new to signal processing, and I wonder that whether it is possible to find the mapping (a function or the like) of a specific interval in 1D signal to its corresponding region in 2D spectrogram (as shown in this figure) and how to do it. Thanks in advance.
", "votes": 0, "replies": [] } ] }, { "id": 1498989, "author_name": "fireflies", "author_url": "", "post_date": "2021-09-01T11:27:48.127000", "content": "Nice plots! Thank u for sharing =))
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "1493333": "\n\nsome of the GW signal are not visible on CQT. If you can whiten the signal and confirm its position in time, you can\n1. see of the GW is \"always\" occurring at some fix time\n2. if it is, then positional encoding + CNN should get rid of the false positives", "1493520": "Nice plots! For those interested, I made a discussion post in SETI comp explaining what \"class activation maps\" (i.e. grad cam) are [here][1] and a starter notebook [here][2]\n\n[1]: https://www.kaggle.com/c/seti-breakthrough-listen/discussion/268314\n[2]: https://www.kaggle.com/cdeotte/silver-medal-with-grad-cam-lb-0-780", "1497791": "an interesting paper:\nhttps://arxiv.org/pdf/1904.12069.pdf\n\nImproving Deep Speech Denoising by Noisy2Noisy Signal Mapping\n\nif we treat:\n\nLivingston wave = signal + noise1\nHanford wave = signal + noise2\n\nwe can learn the denoising in an unsupervised manner as described in the paper\n", "1493591": "\n\na complicated way to modify the SNR of pos samples.\nthe sample way is just to add up the two waveform: good pos sample + neg sample\nor construct the PSD. instead of using PSD to denoise, use it to increase noise (or decrease signal)", "1493345": "maybe we need to use the raw wave as input:\n\n\n\nhttps://arxiv.org/pdf/1701.00008.pdf\nhttps://arxiv.org/pdf/2012.13101.pdf", "1498310": "Hi :),\n\nI think that using CQT or spectrogram won’t be enough if the signal has a very low SNR. Only good pre processing or temporal convolution (with learnt filter) can. At this moment I try CNN1D. We dont need the three channel. With one channel only i have (AUC onCV)0,74, 2 channel 0,80 and all 0,83.\n\nI use only custom archecture. Some resnet block and attention layer is enough (8 millions parameters). I think this is the way to reach a good score but I wont have enough time to try all my ideas alone", "1493571": "Noise2Noise: Learning Image Restoration without Clean Data\nhttps://arxiv.org/pdf/1803.04189.pdf\n\napplication to audio\n\n\n\n\nUnderwater Signal Denoising Using Deep Learning Approach\n \n .... In this context, we propose WaveN2N that is able to learn noise removal and clean signal reconstruction from multi-channels array data in a self-supervised learning setting. ...", "1493346": "I haven't visually verified this myself-but on the forums, [it's been reported](https://www.kaggle.com/mistag/reverse-engineering-create-clean-gw-signals#1397273) that GW events occur in the 0.5 - 1.0 portion of the generated signals in train set. One of the experiments I ran last week was simply lopping off the first 1-10% of the signal (after preprocessing!) to see if getting rid of trash helps. None of these experiments helped. I went a bit further and tried training by chopping off random 1-10% and then rescaling the remaining back to full width and testing with original images, and I tried the inverse (training will full signal, but then doing inference by chopping off 1,2,3,...10% of the signal). All of these experiments resulted in lower RoC-AUC.\n\n\n\nLooking at images like the above, I think it makes sense that having the tail of the GW can be beneficial to the model. If anything, I think it might be beneficial to experiment with SED, find the most likey GW candidate like max across all 3 wave images closest in time, then chop off whatever happens _after_ that point. While we aren't aware of where in the signal a GW occurs, we are aware that there aren't more than 2 simulated GW's per sample. And even if there are, if we find in a sample the most likely GW candidate position, then even if there is another GW in the signal, by definition, it has a less likelihood of being a TP anyway.\n\nDisclaimer-I haven't ran that proposed experiment yet.", "1508181": "1d CNN is here !!!!!!!!!!!!\nhttps://www.kaggle.com/kit716/grav-wave-detection?select=g2net_models.py\n\nthank you very much to @kit716 \n\nhis notebook at: https://www.kaggle.com/kit716/grav-wave-detection", "1503140": "another model to try: 1d auto regressive flow network:\nhttps://arxiv.org/pdf/2002.07656.pdf\n", "1499849": "@hengck23 can you guide me which library you are using for CQT transform as I am not getting the input plots which you are getting", "1495987": "I wonder if the grad-cam results can be used a segmentation mask and could be used for pre-training", "1493386": "\nhttps://arxiv.org/abs/1904.08693", "1559734": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris", "1508307": "fourier layer\nhttps://zongyi-li.github.io/blog/2020/fourier-pde/\n\n \n", "1495206": "\n\nif you know the generator of the synthetic data ... the regression can predict the 15 generating parameters\n\nanother idea is to treat the template as query vectors (you need to learn the encoding net). these query vectors are input to and transform net (together with the input stain). the output will be if each of the query is present or not", "1493593": "", "1493490": "@hengck23 : Could you elaborate the second idea about the role of positional encoding and CNN to prevent FP? What I understand from yours is that the temporal feature of 1D signals might play an important role in filtering FP. I wonder that whether **attention mechanisms** on 2D spectrogram can help too, maybe should I give it a try.", "1498989": "Nice plots! Thank u for sharing =))" } }