\n
![\"https://i.ibb.co/GJkYKmY/Selection-876.png\"](\"https://i.ibb.co/GJkYKmY/Selection-876.png\")
Just focus on single model best cv and lb.
\nEfficientNet-B0 - 5 folds
\nApply Q-Transform more details from this link
\nBasic Data Augmentation：None
\nEpochs: 3
\nLoss: BCEWithLogitsLoss
\nNo TTA
\nCV: 0.862 , LB: 0.867
Just focus on single model best cv and lb.
\nEfficientNet-B0 - 5 folds
\nApply Q-Transform more details from this link
\nBasic Data Augmentation：None
\nEpochs: 3
\nLoss: BCEWithLogitsLoss
\nNo TTA
\nCV: 0.862 , LB: 0.867
Ok, let's do it.
\nResNet34
\nCV: 0.8840
\nLB: 0.8858
Amazing :)
", "rawMarkdown": "Amazing :)", "votes": 2 }, { "id": 1519425, "postDate": "2021-09-21T15:46:04.560Z", "content": "when i write my 1d cnn, i realised that it is probably easier if you spend time understanding the signal processing and write your own network.
", "rawMarkdown": "when i write my 1d cnn, i realised that it is probably easier if you spend time understanding the signal processing and write your own network.", "votes": 4 }, { "id": 1519528, "postDate": "2021-09-21T17:43:24.730Z", "content": "I wonder how many teams just started a resnet34 training.
\nI am more interested in the signal processing part, but I doubt we will learn much about it before competition end.
", "rawMarkdown": "I wonder how many teams just started a resnet34 training.\n\nI am more interested in the signal processing part, but I doubt we will learn much about it before competition end.", "votes": 9 }, { "id": 1519537, "postDate": "2021-09-21T17:57:02.030Z", "content": "True story. It is more about preprocessing raw signal rather than NN architectures. Unless you look at the difference between 1D CNN and 2D one 😄
", "rawMarkdown": "True story. It is more about preprocessing raw signal rather than NN architectures. Unless you look at the difference between 1D CNN and 2D one 😄", "votes": 3 }, { "id": 1519621, "postDate": "2021-09-21T19:18:13.260Z", "content": "In case the magic resides in the 34, I just started a training ;)
\nI doubt it though … and looks forward to the post-mortem !
There is no magic in resnet34, it's just faster and performs slightly better than effnets here. It could be different for your setup though.
", "rawMarkdown": "There is no magic in resnet34, it's just faster and performs slightly better than effnets here. It could be different for your setup though. ", "votes": 6 }, { "id": 1519791, "postDate": "2021-09-21T22:52:55.750Z", "content": "8 days to go :-). Time to redo all signal processing and analysis code
", "rawMarkdown": "8 days to go :-). Time to redo all signal processing and analysis code", "votes": 2 }, { "id": 1519820, "postDate": "2021-09-22T00:25:06.763Z", "content": "for those who want to compare signal processing and automatic feature learning (deep models), one can google for per-channel energy normalization (PCEN). This is a good technique in audio processing.
\ne.g. https://ai.googleblog.com/2018/10/acoustic-detection-of-humpback-whales.html
unfortunately, PCEN is not usable here. But this is a good case to show the power of signal processing.
\naccuracy improves a lot with good signal processing (a 24% reduction in error rate of whale call detection).
And I was proud of my 0.8813 single model…
\nCan't wait to learn from top teams.
", "rawMarkdown": "And I was proud of my 0.8813 single model...\n\nCan't wait to learn from top teams.", "votes": 5 }, { "id": 1520374, "postDate": "2021-09-22T09:49:27.533Z", "content": "Can't wait to read your solution. Lots of domain specific stuff I imagine. All the best. :)
", "rawMarkdown": "Can't wait to read your solution. Lots of domain specific stuff I imagine. All the best. :) " }, { "id": 1520393, "postDate": "2021-09-22T10:00:06.060Z", "content": "Just trained a resnet34, it is great. :)
", "rawMarkdown": "Just trained a resnet34, it is great. :)", "votes": 6 }, { "id": 1520438, "postDate": "2021-09-22T10:27:40.147Z", "content": "@cpmpml , 0.8813 single model is achievement to be proud of. We know.
", "rawMarkdown": "@cpmpml , 0.8813 single model is achievement to be proud of. We know. ", "votes": 2 }, { "id": 1520468, "postDate": "2021-09-22T10:43:33.247Z", "content": "@denisbsu Thanks! Still, we are not in same league.
", "rawMarkdown": "@denisbsu Thanks! Still, we are not in same league." }, { "id": 1520946, "postDate": "2021-09-22T18:03:47.310Z", "content": "@cpmpml that is cool enough. how much is your ensemble score..
", "rawMarkdown": "@cpmpml that is cool enough. how much is your ensemble score..\n", "votes": -7 }, { "id": 1521144, "postDate": "2021-09-23T00:10:17.070Z", "content": "\n\nhow much is your ensemble score..
\n
You can see it on LB.
", "rawMarkdown": "> how much is your ensemble score..\n\nYou can see it on LB. ", "votes": 3 }, { "id": 1526304, "postDate": "2021-09-28T02:50:56.283Z", "content": "ResNet34d is awesome. Blazing fast training, not sure why the hype about EfficientNet in academia…
", "rawMarkdown": "ResNet34d is awesome. Blazing fast training, not sure why the hype about EfficientNet in academia..." } ] }, { "id": 1483879, "postDate": "2021-08-21T00:09:41.467Z", "content": "EfficientNet-b2, 5 folds
\nCV: 0.878
\nLB: 0.880
CV/LB correlation looks good to me.
", "rawMarkdown": "EfficientNet-b2, 5 folds\nCV: 0.878\nLB: 0.880\n\nCV/LB correlation looks good to me.", "votes": 23, "replies": [ { "id": 1483886, "postDate": "2021-08-21T00:22:19.330Z", "content": "Very solid! How long does it take to train your models on average?
", "rawMarkdown": "Very solid! How long does it take to train your models on average?" }, { "id": 1484302, "postDate": "2021-08-21T07:45:01.137Z", "content": "Hi @analokamus thats a great result can you tell us if you use CWT or CQT ?
", "rawMarkdown": "Hi @analokamus thats a great result can you tell us if you use CWT or CQT ?", "votes": -1 }, { "id": 1484509, "postDate": "2021-08-21T11:21:37.780Z", "content": "@snnclsr
\nIt takes around 40 mins per epoch using a single RTX 3090.
@tanulsingh077
\nI used CWT for the best model, but CQT with similar settings performed just as good as CWT.
@analokamus did u modify any CQT or CWT params also to work better ?
\nDid u use all your data or subset of data ?
\nAny pre processing that could be working well for lower models ?
Custom architecture (based on EfficientNet) / 5 fold
\nCV: 0.8798
\nLB: 0.8823
It's amazing how you did it ！
", "rawMarkdown": "It's amazing how you did it ！" }, { "id": 1519440, "postDate": "2021-09-21T15:56:06.620Z", "content": "@analokamus great..
\nsome XYZ augs may be ?
@jaideepvalani Yes, augmentation helped a bit.
", "rawMarkdown": "@jaideepvalani Yes, augmentation helped a bit.", "votes": 4 } ] }, { "id": 1514316, "postDate": "2021-09-16T00:44:28.467Z", "content": "LB 0.8805
\nCV 0.8787
\nres 128x128
\nNot EfficientNet model
By \"res 128x128\" do you mean you're reshaping to 128x128?
", "rawMarkdown": "By \"res 128x128\" do you mean you're reshaping to 128x128?" }, { "id": 1514323, "postDate": "2021-09-16T01:02:38.690Z", "content": "yep, it is the res of the input to the model
", "rawMarkdown": "yep, it is the res of the input to the model" }, { "id": 1514637, "postDate": "2021-09-16T09:02:42.487Z", "content": "128x128, that's amazing.
\nBtw I use 512x4096, but the CV is only 0.878x.
you can modify the stride in your model.
\nalso for the transformer, there is no stride.
", "rawMarkdown": "you can modify the stride in your model.\n\nalso for the transformer, there is no stride.", "votes": 4 }, { "id": 1514657, "postDate": "2021-09-16T09:54:59.473Z", "content": "I just tried a crazy idea. :)
", "rawMarkdown": "I just tried a crazy idea. :)", "votes": 2 }, { "id": 1514702, "postDate": "2021-09-16T11:07:26.430Z", "content": "Oh my gosh. I use 5 models in blend to get 0.8798 LB. And all of them have 0.976+- as solo, but pair correlations are less then 0.97-0.96. I think it is one of the rare competitions where top people will have really different solutions.
", "rawMarkdown": "Oh my gosh. I use 5 models in blend to get 0.8798 LB. And all of them have 0.976+- as solo, but pair correlations are less then 0.97-0.96. I think it is one of the rare competitions where top people will have really different solutions.", "votes": 3 }, { "id": 1516261, "postDate": "2021-09-18T06:03:35.803Z", "content": "Great work! My best b0 cv=0.8745. Your work will motivate me to continue to optimize my single model. Thank you so much for sharing。
", "rawMarkdown": "Great work! My best b0 cv=0.8745. Your work will motivate me to continue to optimize my single model. Thank you so much for sharing。" }, { "id": 1517062, "postDate": "2021-09-19T06:59:15.187Z", "content": "Your comment just inspired me to design new non correlation loss as aux loss
", "rawMarkdown": "Your comment just inspired me to design new non correlation loss as aux loss ", "votes": 3 }, { "id": 1517838, "postDate": "2021-09-20T05:47:46.150Z", "content": "@hengck23 , Thank you, your comment also has encouraged me to look into 1D models more)
", "rawMarkdown": "@hengck23 , Thank you, your comment also has encouraged me to look into 1D models more)", "votes": 1 }, { "id": 1518580, "postDate": "2021-09-20T19:09:55.893Z", "content": "@iafoss So, that means your current ensemble doesn't include any 1d models? 👀
", "rawMarkdown": "@iafoss So, that means your current ensemble doesn't include any 1d models? 👀" }, { "id": 1518678, "postDate": "2021-09-20T22:52:48.247Z", "content": "We did some basic stuff…
", "rawMarkdown": "We did some basic stuff... ", "votes": 1 } ] }, { "id": 1516108, "postDate": "2021-09-17T22:48:53.703Z", "content": "public lb: 0.8782
\ncv(3 fold) : 0.87715, 0.87796 , 0.87648
1dCNN (input is raw waveform)
", "rawMarkdown": "public lb: 0.8782\ncv(3 fold) : 0.87715, 0.87796 , 0.87648 \n\n1dCNN (input is raw waveform)\n \n", "votes": 12, "replies": [ { "id": 1516121, "postDate": "2021-09-17T23:39:10.850Z", "content": "Impressive. It seems you were able to implement the Dr.'s whitening layer from mxnet. Congrats!
", "rawMarkdown": "Impressive. It seems you were able to implement the Dr.'s whitening layer from mxnet. Congrats!", "votes": 2 }, { "id": 1516122, "postDate": "2021-09-17T23:48:06.490Z", "content": "no whitening. just simple 1d cnn with parameter opmization
", "rawMarkdown": "no whitening. just simple 1d cnn with parameter opmization", "votes": 5 }, { "id": 1516260, "postDate": "2021-09-18T06:01:11.857Z", "content": "Great work! Thank you for your sharing！
", "rawMarkdown": "Great work! Thank you for your sharing！" }, { "id": 1516644, "postDate": "2021-09-18T14:56:46.610Z", "content": "No bandpass either?
", "rawMarkdown": "No bandpass either?", "votes": 1 }, { "id": 1516909, "postDate": "2021-09-18T22:24:22.430Z", "content": "bandpass is applied
", "rawMarkdown": "bandpass is applied", "votes": 2 }, { "id": 1528938, "postDate": "2021-09-30T03:37:01.010Z", "content": "this is the model for the 1dCNN that achieved the results reported
\nhttps://gist.github.com/hengck23/8739ac7537ad5a47d382b8c23b61253f
", "rawMarkdown": "this is the model for the 1dCNN that achieved the results reported\n\n\nhttps://gist.github.com/hengck23/8739ac7537ad5a47d382b8c23b61253f", "votes": 1 } ] }, { "id": 1520987, "postDate": "2021-09-22T18:53:40.470Z", "content": "1d CNN + Galerkin Transformer in the frequency domain
\nsimple bandpass + some simple tricks
\n1 fold
\nLB: 0.8780
\nCV: 0.8771
Update on Sept 29:
\nWell done. I tried swin transformer last night. Converged for about 20 batches then started to explode, even with low lrs and a large bs. Was going to give up on it but perhaps I should push further. Also, this is my first time hearing about galerkin transformer.
", "rawMarkdown": "Well done. I tried swin transformer last night. Converged for about 20 batches then started to explode, even with low lrs and a large bs. Was going to give up on it but perhaps I should push further. Also, this is my first time hearing about galerkin transformer." }, { "id": 1522138, "postDate": "2021-09-23T22:22:38.707Z", "content": "Galerkin transformer is the idea by Shuhao Cao
", "rawMarkdown": "Galerkin transformer is the idea by Shuhao Cao", "votes": 1 }, { "id": 1522241, "postDate": "2021-09-24T02:55:57.143Z", "content": "there is nothing more satisfying than applied research :) Congratulations @scaomath
", "rawMarkdown": "there is nothing more satisfying than applied research :) Congratulations @scaomath " }, { "id": 1522244, "postDate": "2021-09-24T03:08:26.747Z", "content": "Haha, yeah. Thanks. I started this competition kinda late (last week). At first experiencing some EDA problem (train AUC 1 but valid AUC 0.5), then reading and being inspired by Heng @hengck23 's posts, I realized that signal processing is more important. So the idea is basically to let the black box to do the signal processing.
\nYet I do believe all top teams have found the weird \"artifacts\" in LIGOs but not the Virgo (if any or I could be wrong as I am pretty amateur in terms of EDA and signals). I can't wait to see the top team's methods of handling the data.
", "rawMarkdown": "Haha, yeah. Thanks. I started this competition kinda late (last week). At first experiencing some EDA problem (train AUC 1 but valid AUC 0.5), then reading and being inspired by Heng @hengck23 's posts, I realized that signal processing is more important. So the idea is basically to let the black box to do the signal processing.\n\nYet I do believe all top teams have found the weird \"artifacts\" in LIGOs but not the Virgo (if any or I could be wrong as I am pretty amateur in terms of EDA and signals). I can't wait to see the top team's methods of handling the data.", "votes": 1 }, { "id": 1522259, "postDate": "2021-09-24T03:32:37.503Z", "content": "\n\nI do believe all top teams have found the weird \"artifacts\" in LIGOs but not the Virgo
\n
Perhaps the model has but the user hasn't 💦. Time to resume EDA while this thing trains…
", "rawMarkdown": "> I do believe all top teams have found the weird \"artifacts\" in LIGOs but not the Virgo\n\nPerhaps the model has but the user hasn't 💦. Time to resume EDA while this thing trains...", "votes": 2 }, { "id": 1522314, "postDate": "2021-09-24T04:50:54.483Z", "content": "…. weird \"artifacts\" in LIGOs …
\nlast time when I work in signature detection in gene (DNA sequence), 1dcnn + max pool works well for this. you can verify if there is any signature/artifacts using this kind of network
", "rawMarkdown": ".... weird \"artifacts\" in LIGOs ...\n\nlast time when I work in signature detection in gene (DNA sequence), 1dcnn + max pool works well for this. you can verify if there is any signature/artifacts using this kind of network", "votes": 1 }, { "id": 1526215, "postDate": "2021-09-28T00:26:01.323Z", "content": "… top teams have found the weird \"artifacts\"
\nmaybe not artifacts (since this is not real data), but injection parameters (or injection software or injection assumption). e.g. injection at Q = 4,10, …
summary of the results from various kagglers in graph:
\n
@hengck23 but still we need to resize the freq axis ? as i get rectangular img output from CQT. with lesser value of freq domain.
", "rawMarkdown": "@hengck23 but still we need to resize the freq axis ? as i get rectangular img output from CQT. with lesser value of freq domain." } ] }, { "id": 1481952, "postDate": "2021-08-19T18:48:01.057Z", "content": "efficientnet_b0, 5 folds
\nCV: 0.874
\nLB: 0.878
WOW, that's impressive especially for B0!
", "rawMarkdown": "WOW, that's impressive especially for B0!", "votes": 1 }, { "id": 1482228, "postDate": "2021-08-20T00:52:33.040Z", "content": "Great work!
", "rawMarkdown": "Great work!" }, { "id": 1482556, "postDate": "2021-08-20T06:35:42.503Z", "content": "Great! Always believed large models like b7 is not necessary
", "rawMarkdown": "Great! Always believed large models like b7 is not necessary" }, { "id": 1482801, "postDate": "2021-08-20T09:38:17.007Z", "content": "It's not certain. He may be better than us in preprocessing. Maybe he only tested B0 instead of trying a model with higher complexity, such as B7… These days, I'm going to try a large model, my 5fold
\nB0 is only LB 0.872. I think preprocessing and data enhancement are the key points of this project
", "rawMarkdown": "It's not certain. He may be better than us in preprocessing. Maybe he only tested B0 instead of trying a model with higher complexity, such as B7... These days, I'm going to try a large model, my 5fold\n\nB0 is only LB 0.872. I think preprocessing and data enhancement are the key points of this project", "votes": 2 } ] }, { "id": 1499021, "postDate": "2021-09-01T11:46:19.290Z", "content": "baseline performance should be LB 0.87+
\nmany of the public kernels are in the range of LB 0.86+.
\nI haven't checked but their sample normalization could be wrong.
you should investigate the following
\nwave = read npy\nimage = cqt_transform(wave)\n\nwhich of the below is more correct and why?\n\nwave = normalised(wave) ... or no norm , or once per detector or once for 3 detectors?\nimage = normalised(image) ... or no norm , or once per detector or once for 3 detectors?\n\nshould we just use constant value, instance/global std or instance/global min/max ?\nWhen I got into the competition, I spent an evening reading over all the forums to that point. It was noted by two people who had tested that there is some discriminating information in the relative amplitudes of the various detectors. Therefore I believe that norm-once-for-all-three-detectors should be used. Separately, @kevinmcisaac's post already mentions the importance of norm, and from a DL perspective, I don't believe it makes sense not to do it.
", "rawMarkdown": "When I got into the competition, I spent an evening reading over all the forums to that point. It was noted by two people who had tested that there is some discriminating information in the relative amplitudes of the various detectors. Therefore I believe that norm-once-for-all-three-detectors should be used. Separately, @kevinmcisaac's post already mentions the importance of norm, and from a DL perspective, I don't believe it makes sense *not* to do it.", "votes": 6 }, { "id": 1499199, "postDate": "2021-09-01T13:50:43.720Z", "content": "I have the same doubt @hengck23, my normalization on wave is /(max of 3 detectors), and on image it's min/max normalization on 3 detecotor(I guess it's global min/max you mean).
\nAnd on both wave and image I found normalization on 3 detectors is slightly better, compared to normalize them separately. I'm not quite clear why. Also I tried normalize image with global std, result is worse than global min/max.
\nAnd I also read from pytoch document here, I'm wonder if normalize imaget to this instance std is better.
\n\n", "rawMarkdown": "I have the same doubt @hengck23, my normalization on wave is /(max of 3 detectors), and on image it's min/max normalization on 3 detecotor(I guess it's global min/max you mean). \n\nAnd on both wave and image I found normalization on 3 detectors is slightly better, compared to normalize them separately. I'm not quite clear why. Also I tried normalize image with global std, result is worse than global min/max.\n\nAnd I also read from pytoch document [here](https://pytorch.org/vision/stable/models.html), I'm wonder if normalize imaget to this instance std is better.\n> All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]\n", "votes": 2 }, { "id": 1499360, "postDate": "2021-09-01T15:30:24.857Z", "content": "All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]
\n
the data are synthetically generate based on equations. Does your normalization destroy the original relationship?
", "rawMarkdown": "the data are synthetically generate based on equations. Does your normalization destroy the original relationship?", "votes": 3 }, { "id": 1499380, "postDate": "2021-09-01T15:46:13.647Z", "content": "When normalizing a disparate datasource to imagenet, what needs to be considered isn't the mean and std used to perform the normalization so much as mu/sigma of the resulting final distribution / network input.
", "rawMarkdown": "When normalizing a disparate datasource to imagenet, what needs to be considered isn't the mean and std used to perform the normalization so much as mu/sigma of the resulting final distribution / network input.", "votes": 3 }, { "id": 1499413, "postDate": "2021-09-01T16:11:22.210Z", "content": "@authman Norm once for all detector would mean this ?
\ndef apply_preprocess(self, waves):\n waves = np.hstack(waves)\n waves = waves / np.max(waves)\n return waves\nI just looked at my preprocessing code more closely and it looks like what I'm using now is global norm across entire dataset. I recall using something similar to this to get the values.
\nSo basically, 1) preprocess, 2) standardize with global stats (a single mean and std for the entire dataset, not channel specific), and then 3) feed into model.
\nLooking at my earlier experiments, I had a nn.InstanceNorm layer right on the input, which was later replaced LayerNorm layer, which now is replaced by standardization across the entire dataset in my dataloader.
", "rawMarkdown": "I just looked at my preprocessing code more closely and it looks like what I'm using now is global norm across entire dataset. I recall using something [similar to this](https://www.kaggle.com/kozodoi/seti-mean-and-std-of-new-data) to get the values.\n\nSo basically, 1) preprocess, 2) standardize with global stats (a single mean and std for the entire dataset, not channel specific), and then 3) feed into model.\n\nLooking at my earlier experiments, I had a nn.InstanceNorm layer right on the input, which was later replaced LayerNorm layer, which now is replaced by standardization across the entire dataset in my dataloader.", "votes": 1 }, { "id": 1499469, "postDate": "2021-09-01T16:51:23.020Z", "content": "Thanku.. what is diff between 1 and 2.. 1 is using max ?
", "rawMarkdown": "Thanku.. what is diff between 1 and 2.. 1 is using max ?" }, { "id": 1499488, "postDate": "2021-09-01T17:04:20.013Z", "content": "I never used .max() manually, I had always used .std().
", "rawMarkdown": "I never used .max() manually, I had always used .std()." }, { "id": 1499563, "postDate": "2021-09-01T18:01:44.263Z", "content": "you can do quick experiments to verify results.
\nthese pre-processing steps affect most networks, even the smallest ones, and smallest image size.
\nso it is fast to verify which normalization is effective
how much difference u see with and without. I have been using .max only so far. also without max no big difference
", "rawMarkdown": "how much difference u see with and without. I have been using .max only so far. also without max no big difference" } ] }, { "id": 1471090, "postDate": "2021-08-13T23:46:19.807Z", "content": "Bandpass(20-500Hz) => CWT( 256x256 resolution) => EfficientnetB7
\nTPU using TFRecords, batch size 512
\nLB:0.874 CV: 0.8614
\n12 epochs
\n7.4m per epoch
See here for details
", "rawMarkdown": "Bandpass(20-500Hz) => CWT( 256x256 resolution) => EfficientnetB7 \nTPU using TFRecords, batch size 512\nLB:0.874 CV: 0.8614\n12 epochs\n7.4m per epoch\n\nSee [here](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/261721) for details", "votes": 8, "replies": [ { "id": 1472082, "postDate": "2021-08-14T16:30:48.430Z", "content": "@kevinmcisaac did CWT work better for you comparing to CQT? to be honest im not even really sure about the difference between the two, looks like they are very similar, to my understanding CWT is a multiresolution transformation given a choice of wavelet, and for a set of scale and translation you can get the scalogram. And CQT seems like a variant to STFT where the frequency is somehow in log scale. Do you have recommendations on readings about CQT and how it differs from CWT and STFT?
\ndoes the choice of wavelet matter for you?
", "rawMarkdown": "@kevinmcisaac did CWT work better for you comparing to CQT? to be honest im not even really sure about the difference between the two, looks like they are very similar, to my understanding CWT is a multiresolution transformation given a choice of wavelet, and for a set of scale and translation you can get the scalogram. And CQT seems like a variant to STFT where the frequency is somehow in log scale. Do you have recommendations on readings about CQT and how it differs from CWT and STFT?\n\ndoes the choice of wavelet matter for you?" }, { "id": 1472499, "postDate": "2021-08-14T22:52:10.447Z", "content": "CWT did not seem to make much difference, however I found a Keras implementation and was able to optimise. This enables me to compute the transforms on the fly in the NN which gives me flexibility in choosing CWT parameter. This has been beneficial.
\nI've only tried the complex morlet wave so far, perhaps I''l try some other.
", "rawMarkdown": "CWT did not seem to make much difference, however I found a Keras implementation and was able to optimise. This enables me to compute the transforms on the fly in the NN which gives me flexibility in choosing CWT parameter. This has been beneficial.\n\nI've only tried the complex morlet wave so far, perhaps I''l try some other.", "votes": 6 }, { "id": 1475033, "postDate": "2021-08-16T12:09:46.337Z", "content": "Your LB seems to be so high compared to CV. What is the reason?
", "rawMarkdown": "Your LB seems to be so high compared to CV. What is the reason?", "votes": 5 } ] }, { "id": 1528512, "postDate": "2021-09-29T17:43:44.030Z", "content": "I hope it is the best single 1D model:
\nCV 0.8819, LB ?? ~0.883X based on the previous similar subs
Did it score 0.8840 ?? Still not letting us sleep LOL
", "rawMarkdown": "Did it score 0.8840 ?? Still not letting us sleep LOL", "votes": 3 }, { "id": 1528536, "postDate": "2021-09-29T18:07:03.653Z", "content": "It is already combined. So you may relax before the the reset.
", "rawMarkdown": "It is already combined. So you may relax before the the reset.", "votes": 3 }, { "id": 1528563, "postDate": "2021-09-29T18:30:27.453Z", "content": "I hope you will share how the hell did you trained it! well done!
", "rawMarkdown": "I hope you will share how the hell did you trained it! well done!", "votes": 2 }, { "id": 1528587, "postDate": "2021-09-29T19:05:20.667Z", "content": "Thanks. The main boost is coming from the model architecture: it should address they way how GW are detected. But there is also a number of other things our team will share when the competition is finished
", "rawMarkdown": "Thanks. The main boost is coming from the model architecture: it should address they way how GW are detected. But there is also a number of other things our team will share when the competition is finished", "votes": 5 }, { "id": 1528630, "postDate": "2021-09-29T20:31:19.070Z", "content": "Such a great score with 1D models. We stopped tuning their architecture from like 2 weeks ago once they got 0.879x LB and focused on the pipeline to combine them with 2D ones, incredibly regret now :'( Your team performed really well indeed, good luck!
", "rawMarkdown": "Such a great score with 1D models. We stopped tuning their architecture from like 2 weeks ago once they got 0.879x LB and focused on the pipeline to combine them with 2D ones, incredibly regret now :'( Your team performed really well indeed, good luck!", "votes": 2 }, { "id": 1528642, "postDate": "2021-09-29T20:48:12.893Z", "content": "Thanks @solosquad1999 , best luck to you too.
\nI got motivated by @hengck23 and my teammate @richx86 to look into 1D in more details, and it was really a good chose. It's unfortunate that 1D method was not that popular in this competition: it is much faster to train and also it has quite a good performance (though, I'm not sure what top1 team did to get so high score for 2D resnet model).
\" I'm not sure what top1 team did to get so high score for 2D resnet model\"
\nthe way to learn 2d resnet model and 1dCNN is the \"same\".
\nlet's wait for a few more hours …
well, we have zero 1D model in our blend. Now we know what we missed LOL
", "rawMarkdown": "well, we have zero 1D model in our blend. Now we know what we missed LOL", "votes": 3 }, { "id": 1528678, "postDate": "2021-09-29T21:48:03.473Z", "content": "\n\nwell, we have zero 1D model in our blend. Now we know what we missed LOL
\n
we can say it too ahahah! what a pity
", "rawMarkdown": "> well, we have zero 1D model in our blend. Now we know what we missed LOL\n\nwe can say it too ahahah! what a pity", "votes": 1 }, { "id": 1528682, "postDate": "2021-09-29T21:53:03.023Z", "content": "train one now! 1dcnn are fast to train
", "rawMarkdown": "train one now! 1dcnn are fast to train", "votes": 1 }, { "id": 1528684, "postDate": "2021-09-29T21:53:39.003Z", "content": "One of our guesses was that top 1 team makes synthetic data to work very nicely.
", "rawMarkdown": "One of our guesses was that top 1 team makes synthetic data to work very nicely." }, { "id": 1528687, "postDate": "2021-09-29T21:58:54.043Z", "content": "hyperparameters are not that easy to find xD
", "rawMarkdown": "hyperparameters are not that easy to find xD" }, { "id": 1528689, "postDate": "2021-09-29T21:59:53.803Z", "content": "maybe synthetic noise (i.e. not the wave)
", "rawMarkdown": "maybe synthetic noise (i.e. not the wave)", "votes": 1 }, { "id": 1528690, "postDate": "2021-09-29T22:06:43.950Z", "content": "Yes, it is quite possible. Synthetic GW data itself gives just a minor boost, nothing to compare with the score of top1( or probably we missed something about it.
", "rawMarkdown": "Yes, it is quite possible. Synthetic GW data itself gives just a minor boost, nothing to compare with the score of top1( or probably we missed something about it." }, { "id": 1528718, "postDate": "2021-09-29T22:51:13.893Z", "content": "Couldn't resist comparing our 1d models after the competition.
\n 1D CNN 5 fold mean cv: 0.8817, lb: 0.8838
Do all of you add residual in CNN 1d? Hard to imagine Inception-like structure can be trained well with so much noisy data.
", "rawMarkdown": "Do all of you add residual in CNN 1d? Hard to imagine Inception-like structure can be trained well with so much noisy data.", "votes": 1 }, { "id": 1528763, "postDate": "2021-09-30T00:24:20.473Z", "content": "@nischaydnk , what is your private LB score for the model. My one got the following: Private LB: 0.8820, Public LB: 0.8827. I guess you model might be better)
", "rawMarkdown": "@nischaydnk , what is your private LB score for the model. My one got the following: Private LB: 0.8820, Public LB: 0.8827. I guess you model might be better)", "votes": 1 }, { "id": 1528769, "postDate": "2021-09-30T00:28:52.033Z", "content": "Our model scored 0.8823 on Private LB
", "rawMarkdown": "Our model scored 0.8823 on Private LB", "votes": 2 }, { "id": 1528770, "postDate": "2021-09-30T00:29:05.247Z", "content": "@iafoss cograts ;)
\nIt scored 0.8823 in private leaderboard
Congratulations with the best 1D model and gold medal. I'm looking forwards to your writeup.
", "rawMarkdown": "Congratulations with the best 1D model and gold medal. I'm looking forwards to your writeup.", "votes": 1 } ] }, { "id": 1504051, "postDate": "2021-09-06T03:20:41.803Z", "content": "EfficientNetB7 - single fold
\nCV: 0.8776
\nLB: 0.8791
\nWe still do not know what the scores of 5 folds would be.
EfficientB3:
\n1/5 folds
\nCV: 0.8785
\nLB: 0.8776
update 9/2
\nEfficientB3:
\n1/5 folds
\nCV: 0.8791
\nLB: 0.8784
Bravo, your got such decent score from single fold, not from ensemble of 5 folds, right? @wuliaokaola
", "rawMarkdown": "Bravo, your got such decent score from single fold, not from ensemble of 5 folds, right? @wuliaokaola " }, { "id": 1499232, "postDate": "2021-09-01T14:09:46.260Z", "content": "过奖了。
\nYes, 0.8776 is single fold. And my current LB is an ensemble of 4 models.
Great, wish you can win a gold this time.
\nAnd my current best single model is CV 0.8720, LB 0.8748, with EfficietNet_B0 single fold, will continue digging to catch up 😃
", "rawMarkdown": "Great, wish you can win a gold this time.\n\nAnd my current best single model is CV 0.8720, LB 0.8748, with EfficietNet_B0 single fold, will continue digging to catch up 😃 ", "votes": 4 }, { "id": 1499460, "postDate": "2021-09-01T16:42:58.707Z", "content": "@wuliaokaola this is impressive.. any kind of preprocessing you used.
", "rawMarkdown": "@wuliaokaola this is impressive.. any kind of preprocessing you used. ", "votes": -8 } ] }, { "id": 1524389, "postDate": "2021-09-26T13:52:24.507Z", "content": "EfficientNet
\nCV: 8791, LB: 8808
My best single model is an Efficientnet-b3, with CWT
\nLB: 0.8800 (Full Training Data)
\nLB: 0.8793 (One Fold)
Great work! Thanks for your sharing!
", "rawMarkdown": "Great work! Thanks for your sharing!" } ] }, { "id": 1511256, "postDate": "2021-09-13T09:07:55.593Z", "content": "Resnet18d, CQT, 1 of 4 folds
\nCV: 0.8734
\nLB: 0.8763
roughly the same for me:
\nResnet18d, CQT, average of 5 folds
\nCV: 0.8740
\nLB: 0.8759
Nice to hear that small models are good enough. Especially nice to see 4 min per epoch on old 1080Ti ;)
", "rawMarkdown": "Nice to hear that small models are good enough. Especially nice to see 4 min per epoch on old 1080Ti ;)", "votes": 5 } ] }, { "id": 1503782, "postDate": "2021-09-05T17:46:23.297Z", "content": "Effnet B2
\nCV 0.8750, LB 0.8780
\n5 folds, no augmentations
My feeling here is there's a lot of room to improve before ensembling anything
", "rawMarkdown": "Effnet B2\nCV 0.8750, LB 0.8780\n5 folds, no augmentations\n\nMy feeling here is there's a lot of room to improve before ensembling anything", "votes": 3, "replies": [ { "id": 1512568, "postDate": "2021-09-14T11:17:56.263Z", "content": "update: it's not my best but with a B0 based model, 5 folds: CV 0.8751, LB 0.8776
", "rawMarkdown": "update: it's not my best but with a B0 based model, 5 folds: CV 0.8751, LB 0.8776", "votes": 1 } ] }, { "id": 1503053, "postDate": "2021-09-04T22:51:11.077Z", "content": "CV 0.87392066
\nLB 0.8780
\nFolds 5
\nEpochs 7
\nBCEWithLogitsLoss
\nNo TTA
CV 0.8750
\nLB 0.8788
@authman What is the model?
", "rawMarkdown": "@authman What is the model?" }, { "id": 1517443, "postDate": "2021-09-19T15:51:53.160Z", "content": "custom 1d cnn
", "rawMarkdown": "custom 1d cnn", "votes": 3 }, { "id": 1517906, "postDate": "2021-09-20T07:36:35.723Z", "content": "Your 1DCNN's gap is quite gaping, ours got CV 0.8766 but LB only 0.8762.
", "rawMarkdown": "Your 1DCNN's gap is quite gaping, ours got CV 0.8766 but LB only 0.8762.", "votes": 1 } ] }, { "id": 1528144, "postDate": "2021-09-29T12:06:15.303Z", "content": "Effnetv2s, CV 0.8803 LB 0.8826
\n3 subs left and half a day to make ensembling work… LOL
", "rawMarkdown": "Effnetv2s, CV 0.8803 LB 0.8826\n\n3 subs left and half a day to make ensembling work... LOL", "votes": 4, "replies": [ { "id": 1528154, "postDate": "2021-09-29T12:13:48.300Z", "content": "You can do it
", "rawMarkdown": "You can do it", "votes": 2 }, { "id": 1528162, "postDate": "2021-09-29T12:24:33.573Z", "content": "Thanks! I hope you are right. But ensembling aith roc-auc is not very effective anyway.
", "rawMarkdown": "Thanks! I hope you are right. But ensembling aith roc-auc is not very effective anyway.\n" }, { "id": 1528168, "postDate": "2021-09-29T12:30:44.283Z", "content": "I also hope you can make it to gold. This is a tough competition.
", "rawMarkdown": "I also hope you can make it to gold. This is a tough competition.\n", "votes": 1 }, { "id": 1528178, "postDate": "2021-09-29T12:41:52.540Z", "content": "Hi neighbors. same here.
\nGood luck!
Hope to see all in gold zone in a few hours 😂 fingers crossed ahaha
", "rawMarkdown": "Hope to see all in gold zone in a few hours 😂 fingers crossed ahaha ", "votes": 3 }, { "id": 1528199, "postDate": "2021-09-29T13:05:49.073Z", "content": "PS: I bet a flight to India if we win a gold medal. Looks like they really want to meet me 😂
", "rawMarkdown": "PS: I bet a flight to India if we win a gold medal. Looks like they really want to meet me 😂", "votes": 4 }, { "id": 1528216, "postDate": "2021-09-29T13:21:38.777Z", "content": "1D + 2D ensemble --> BoOm
", "rawMarkdown": "1D + 2D ensemble --> BoOm ", "votes": 1 }, { "id": 1528220, "postDate": "2021-09-29T13:27:37.213Z", "content": "@nischaydnk shhh…… 🤐
", "rawMarkdown": "@nischaydnk shhh...... 🤐", "votes": 1 }, { "id": 1528228, "postDate": "2021-09-29T13:37:44.687Z", "content": "I bet many teams will now try there last attempt with 1d models😂
", "rawMarkdown": "I bet many teams will now try there last attempt with 1d models😂" }, { "id": 1528234, "postDate": "2021-09-29T13:43:37.800Z", "content": "Most of the teams who've shared the best single models here had (0.0005 - 0.001) better scores on the leaderboard at the time of posting. Everyone must have got some boost from the ensemble.
", "rawMarkdown": "Most of the teams who've shared the best single models here had (0.0005 - 0.001) better scores on the leaderboard at the time of posting. Everyone must have got some boost from the ensemble." }, { "id": 1528237, "postDate": "2021-09-29T13:45:57.213Z", "content": "there is something else than 1dCNN. it seems that the top teams know about it. we will know tmr
", "rawMarkdown": "there is something else than 1dCNN. it seems that the top teams know about it. we will know tmr", "votes": 5 }, { "id": 1528242, "postDate": "2021-09-29T13:50:04.017Z", "content": "WaveNet-like bidirectional LSTM? Or unsupervised pretrain of a denoising autoencoder in the frequency domain.
", "rawMarkdown": "WaveNet-like bidirectional LSTM? Or unsupervised pretrain of a denoising autoencoder in the frequency domain." }, { "id": 1528247, "postDate": "2021-09-29T13:52:54.557Z", "content": "Resnet34 --> boom 💥
", "rawMarkdown": "Resnet34 --> boom 💥 ", "votes": 2 }, { "id": 1528251, "postDate": "2021-09-29T13:56:35.117Z", "content": "torch.randn()
@pheadrus My brain 🧠 ---> boom 💥
", "rawMarkdown": "@pheadrus My brain 🧠 ---> boom 💥" }, { "id": 1528255, "postDate": "2021-09-29T13:59:08.207Z", "content": "\n\n\nunsupervised pretrain of a denoising autoencoder in the frequency domain.
\n
Another very interesting Idea , just wanted to add , I loved galerkin transformer although I am still a kid to be able to completely understand the math behind it but I got the intuition . I also made a failed attempt to use it , but I am glad I was introduced to it , maybe I will make one more attempt after all this is over
", "rawMarkdown": "@scaomath \n> unsupervised pretrain of a denoising autoencoder in the frequency domain.\n\nAnother very interesting Idea , just wanted to add , I loved galerkin transformer although I am still a kid to be able to completely understand the math behind it but I got the intuition . I also made a failed attempt to use it , but I am glad I was introduced to it , maybe I will make one more attempt after all this is over", "votes": 1 }, { "id": 1528299, "postDate": "2021-09-29T14:42:03.410Z", "content": "I tried to pretrain encoder with unsupervised AE (in every competition i entered). For me, didnt work
", "rawMarkdown": "I tried to pretrain encoder with unsupervised AE (in every competition i entered). For me, didnt work", "votes": 3 }, { "id": 1528493, "postDate": "2021-09-29T17:31:54.857Z", "content": "3 downvotes seriously :)
\nIs it even a secret sauce that no one was aware of?
Yes, seriously. Sharing few hours before deadline is not a good practice. In the past it has led hundreds of people to lose their medal once when someone shared a good silver medal info. No wonder some are very sensitive to late sharing.
", "rawMarkdown": "Yes, seriously. Sharing few hours before deadline is not a good practice. In the past it has led hundreds of people to lose their medal once when someone shared a good silver medal info. No wonder some are very sensitive to late sharing.", "votes": 4 }, { "id": 1528943, "postDate": "2021-09-30T03:42:16.597Z", "content": "I too tried AE with 10 15 percent of background noise but it dint help , it was overfitting
", "rawMarkdown": "I too tried AE with 10 15 percent of background noise but it dint help , it was overfitting " } ] }, { "id": 1516477, "postDate": "2021-09-18T11:16:18.093Z", "content": "public lb: 0.8790
\ncv(4 fold): 0.87599, 0.87609, 0.874753, 0.876685
effnet_b2: CQT 256x786
\npublic lb: 0.8788
\ncv(5 fold): 0.87456, 0.87623, 0.87815, 0.87642, 0.87757
effnet_b7_ns: CQT 256x512
", "rawMarkdown": "public lb: 0.8790\ncv(4 fold): 0.87599, 0.87609, 0.874753, 0.876685\n\neffnet_b2: CQT 256x786\n\n---\npublic lb: 0.8788\ncv(5 fold): 0.87456, 0.87623, 0.87815, 0.87642, 0.87757 \n\neffnet_b7_ns: CQT 256x512", "votes": 4, "replies": [ { "id": 1522910, "postDate": "2021-09-24T17:45:21.353Z", "content": "can't get the same results as the top, but the performance of resnet34 is a surprise for me.
\nsingle fold resner34 (modified) + 126x129 cqt (modified)
\ncv :0.875115
\nlb : 0.8760
single fold resner34 (modified) + 126x129 cqt (another modified)
\ncv :0.877104
\nlb : 0.8772
single fold resner34 (modified) + 256x257 cqt (modified)
\ncv : 0.878053
\nlb : 0.8799
\nlb: 0.8812 (5fold)
cv : 0.87915 (tta-single-fold)
\nlb: 0.8803 (tta-single-fold)
single fold resner34 (modified) + 256x257 cqt (another modified)
\ncv :0.878869
\nlb: 0.8805
me too :)
\nsingle fold resner34
\ncv : 0.8833
\nlb : 0.8816
Glad that it helped :)
", "rawMarkdown": "Glad that it helped :)", "votes": 3 }, { "id": 1528694, "postDate": "2021-09-29T22:15:13.330Z", "content": "Hot from the oven!
\nsingle fold resner34 (modified) + 126x256 cqt (yet another modified)
\ncv :0.87904
\ncv-tta : 0.88013
\npublic-lb-tta : 0.8812
EfficientNetB2, 5-fold, CQT
\nCV: 0.8760
\nLB: 0.8794
Efficientnet-b3, CQT
\nCV(5-folds): 0.87624, 0.87563, 0.87724, 0.87594, 0.87692
\nLB: 0.8794
EfficientnetV2M, CQT 512x512
\nNo CV(training data 100%. See here)
\nLB: 0.8789
hmm … \"training data 100%\" could be a trade secret. i will try it
", "rawMarkdown": "hmm ... \"training data 100%\" could be a trade secret. i will try it", "votes": 1 } ] }, { "id": 1502439, "postDate": "2021-09-04T08:49:31.083Z", "content": "Best Single Model Update : CV 87.4 LB 87.71
", "rawMarkdown": "Best Single Model Update : CV 87.4 LB 87.71", "votes": 1, "replies": [ { "id": 1502689, "postDate": "2021-09-04T14:39:00.223Z", "content": "You are using keras cwt?
", "rawMarkdown": "You are using keras cwt?" }, { "id": 1502729, "postDate": "2021-09-04T15:36:12.137Z", "content": "i use CQT only.
", "rawMarkdown": "i use CQT only.", "votes": 1 }, { "id": 1502749, "postDate": "2021-09-04T15:54:43.297Z", "content": "If you don’t mind sharing, what's your base model architecture ?
", "rawMarkdown": "If you don’t mind sharing, what's your base model architecture ?" }, { "id": 1502844, "postDate": "2021-09-04T17:25:59.010Z", "content": "sorry lb is highly volatile.. every piece of info is adding to people score :) tomorrow i will be down to 50 . you can go through public nbks . every one is around same only..
", "rawMarkdown": "sorry lb is highly volatile.. every piece of info is adding to people score :) tomorrow i will be down to 50 . you can go through public nbks . every one is around same only..\n", "votes": 1 }, { "id": 1508018, "postDate": "2021-09-09T18:14:59.793Z", "content": "Lower model,base version- CV 87.24-Further improvable to 87.44 ,LB-8771
", "rawMarkdown": "Lower model,base version- CV 87.24-Further improvable to 87.44 ,LB-8771", "votes": -1 } ] }, { "id": 1475808, "postDate": "2021-08-16T20:48:58.720Z", "content": "EfficientB0:
\n5folds
\nCV: 0.866
\nLB: 0.868
Thanks, how long does one epoch take?
", "rawMarkdown": "Thanks, how long does one epoch take?", "votes": 1, "replies": [ { "id": 1467895, "postDate": "2021-08-12T07:09:15.320Z", "content": "Almost 3 minutes in 4-v100 gpus.
", "rawMarkdown": "Almost 3 minutes in 4-v100 gpus.", "votes": 1 } ] }, { "id": 1507896, "postDate": "2021-09-09T16:23:32.713Z", "content": "EfficientnetB7 4fold CV:0.8750 LB:0.8785
", "rawMarkdown": "EfficientnetB7 4fold CV:0.8750 LB:0.8785", "votes": 2 }, { "id": 1502790, "postDate": "2021-09-04T16:41:12.367Z", "content": "Effnet B0 CV 87.2 LB 87.65
", "rawMarkdown": "Effnet B0 CV 87.2 LB 87.65", "votes": 2 }, { "id": 1518689, "postDate": "2021-09-20T23:46:32.333Z", "content": "surprising results?
\nlb : 0.8762 (one fold)
\ncv : 0.874475
wide mobilenetv2 (width multiplier x3) + Coordinate attention :
\nqct = 256x256
this is an experiment for trying wide instead of deep. it is trained from scratch
", "rawMarkdown": "surprising results?\nlb : 0.8762 (one fold)\ncv : 0.874475\n\nwide mobilenetv2 (width multiplier x3) + Coordinate attention : \nqct = 256x256\n\nthis is an experiment for trying wide instead of deep. it is trained from scratch" }, { "id": 1486870, "postDate": "2021-08-23T09:15:09.350Z", "content": "EfficientNet-b0, 1 fold
\nCV: 0.873
\nLB: 0.875
@hanson0910 i tried lower eff net models but all dsnt reach to 87 also .
\n. Did u use same cqt as this https://www.kaggle.com/yasufuminakama/g2net-efficientnet-b7-baseline-training?scriptVersionId=69947710
\nor different..
\neven this kernel b7 is not reaching 87
CQT is best for me,but i change some config.do not always focus on the big net.
", "rawMarkdown": "CQT is best for me,but i change some config.do not always focus on the big net.", "votes": 1 } ] }, { "id": 1470011, "postDate": "2021-08-13T08:25:48.433Z", "content": "EfficientNet-B7 - total: 5-folds run 1 fold
\nMove CQT layer insider model
\nmore details from this link
\nEpochs: 3
\nLoss: BCEWithLogitsLoss
\nCV:0.861 LB:0.865
I tried several models , all around 0.86-0.87 with CQT. some people mentioned that continuous wavelet transform has better result.
", "rawMarkdown": "I tried several models , all around 0.86-0.87 with CQT. some people mentioned that continuous wavelet transform has better result. ", "votes": 1 }, { "id": 1482165, "postDate": "2021-08-19T22:51:03.563Z", "content": "I use CWT and have a single model with 0.873, however that requires B7 and the CWT configured to create 256 frequency steps and 256 time steps
", "rawMarkdown": "I use CWT and have a single model with 0.873, however that requires B7 and the CWT configured to create 256 frequency steps and 256 time steps\n", "votes": 1 }, { "id": 1482177, "postDate": "2021-08-19T23:13:19.453Z", "content": "\nHow long does it take to train your models on average?
", "rawMarkdown": "Hi @kevinmcisaac \n\nHow long does it take to train your models on average?" }, { "id": 1482205, "postDate": "2021-08-20T00:17:12.303Z", "content": "An hour I think but that is based on using a TF implementation of CWT, tfrecords and a TPU.
", "rawMarkdown": "An hour I think but that is based on using a TF implementation of CWT, tfrecords and a TPU. " }, { "id": 1483814, "postDate": "2021-08-20T22:11:11.450Z", "content": "Wow, an hour is pretty fast to me. Is it one fold or complete training with all folds?
", "rawMarkdown": "Wow, an hour is pretty fast to me. Is it one fold or complete training with all folds?" }, { "id": 1483823, "postDate": "2021-08-20T22:24:26.097Z", "content": "Yes, just single fold with about 20 epochs. To achieve this I used a TPU, read data using TFRecords and create an optimised TF2.0 implementation of CWT then created a simple Keras model.
", "rawMarkdown": "Yes, just single fold with about 20 epochs. To achieve this I used a TPU, read data using TFRecords and create an optimised TF2.0 implementation of [CWT](https://github.com/Kevin-McIsaac/cmorlet-tensorflow) then created a simple Keras model.", "votes": 2 }, { "id": 1483825, "postDate": "2021-08-20T22:30:19.157Z", "content": "Sounds good. Achieving over .873 with a single fold is great! Thanks.
", "rawMarkdown": "Sounds good. Achieving over .873 with a single fold is great! Thanks." } ] }, { "id": 1486801, "postDate": "2021-08-23T08:19:37.353Z", "content": "efficientnet_b7, 4 folds
\nCV: 0.873
\nLB: 0.875
Another thread with same theme : https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/251549
", "rawMarkdown": "Another thread with same theme : https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/251549" }, { "id": 1559765, "postDate": "2021-10-27T07:22:34.903Z", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
EffNetB7, CQT :
\nBest single model - CV : 87.58
\n5-fold - CV : 87.73
CV: 86.7
\nLB: 86.7
Ok, let's do it.
\nResNet34
\nCV: 0.8840
\nLB: 0.8858
Amazing :)
", "votes": 2, "replies": [] }, { "id": 1519425, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-21T15:46:04.560000", "content": "when i write my 1d cnn, i realised that it is probably easier if you spend time understanding the signal processing and write your own network.
", "votes": 4, "replies": [] }, { "id": 1519528, "author_name": "CPMP", "author_url": "", "post_date": "2021-09-21T17:43:24.730000", "content": "I wonder how many teams just started a resnet34 training.
\nI am more interested in the signal processing part, but I doubt we will learn much about it before competition end.
", "votes": 9, "replies": [] }, { "id": 1519537, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-09-21T17:57:02.030000", "content": "True story. It is more about preprocessing raw signal rather than NN architectures. Unless you look at the difference between 1D CNN and 2D one 😄
", "votes": 3, "replies": [] }, { "id": 1519621, "author_name": "FabienDaniel", "author_url": "", "post_date": "2021-09-21T19:18:13.260000", "content": "In case the magic resides in the 34, I just started a training ;)
\nI doubt it though … and looks forward to the post-mortem !
There is no magic in resnet34, it's just faster and performs slightly better than effnets here. It could be different for your setup though.
", "votes": 6, "replies": [] }, { "id": 1519791, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-21T22:52:55.750000", "content": "8 days to go :-). Time to redo all signal processing and analysis code
", "votes": 2, "replies": [] }, { "id": 1519820, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-22T00:25:06.763000", "content": "for those who want to compare signal processing and automatic feature learning (deep models), one can google for per-channel energy normalization (PCEN). This is a good technique in audio processing.
\ne.g. https://ai.googleblog.com/2018/10/acoustic-detection-of-humpback-whales.html
unfortunately, PCEN is not usable here. But this is a good case to show the power of signal processing.
\naccuracy improves a lot with good signal processing (a 24% reduction in error rate of whale call detection).
And I was proud of my 0.8813 single model…
\nCan't wait to learn from top teams.
", "votes": 5, "replies": [] }, { "id": 1520374, "author_name": "Phaedrus", "author_url": "", "post_date": "2021-09-22T09:49:27.533000", "content": "Can't wait to read your solution. Lots of domain specific stuff I imagine. All the best. :)
", "votes": 0, "replies": [] }, { "id": 1520393, "author_name": "outrunner", "author_url": "", "post_date": "2021-09-22T10:00:06.060000", "content": "Just trained a resnet34, it is great. :)
", "votes": 6, "replies": [] }, { "id": 1520438, "author_name": "Denis Kanonik", "author_url": "", "post_date": "2021-09-22T10:27:40.147000", "content": "@cpmpml , 0.8813 single model is achievement to be proud of. We know.
", "votes": 2, "replies": [] }, { "id": 1520468, "author_name": "CPMP", "author_url": "", "post_date": "2021-09-22T10:43:33.247000", "content": "@denisbsu Thanks! Still, we are not in same league.
", "votes": 0, "replies": [] }, { "id": 1520946, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-22T18:03:47.310000", "content": "@cpmpml that is cool enough. how much is your ensemble score..
", "votes": -7, "replies": [] }, { "id": 1521144, "author_name": "CPMP", "author_url": "", "post_date": "2021-09-23T00:10:17.070000", "content": "\n\nhow much is your ensemble score..
\n
You can see it on LB.
", "votes": 3, "replies": [] }, { "id": 1526304, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-28T02:50:56.283000", "content": "ResNet34d is awesome. Blazing fast training, not sure why the hype about EfficientNet in academia…
", "votes": 0, "replies": [] } ] }, { "id": 1483879, "author_name": "RabotniKuma", "author_url": "", "post_date": "2021-08-21T00:09:41.467000", "content": "EfficientNet-b2, 5 folds
\nCV: 0.878
\nLB: 0.880
CV/LB correlation looks good to me.
", "votes": 23, "replies": [ { "id": 1483886, "author_name": "Sinan Calisir", "author_url": "", "post_date": "2021-08-21T00:22:19.330000", "content": "Very solid! How long does it take to train your models on average?
", "votes": 0, "replies": [] }, { "id": 1484302, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2021-08-21T07:45:01.137000", "content": "Hi @analokamus thats a great result can you tell us if you use CWT or CQT ?
", "votes": -1, "replies": [] }, { "id": 1484509, "author_name": "RabotniKuma", "author_url": "", "post_date": "2021-08-21T11:21:37.780000", "content": "@snnclsr
\nIt takes around 40 mins per epoch using a single RTX 3090.
@tanulsingh077
\nI used CWT for the best model, but CQT with similar settings performed just as good as CWT.
@analokamus did u modify any CQT or CWT params also to work better ?
\nDid u use all your data or subset of data ?
\nAny pre processing that could be working well for lower models ?
Custom architecture (based on EfficientNet) / 5 fold
\nCV: 0.8798
\nLB: 0.8823
It's amazing how you did it ！
", "votes": 0, "replies": [] }, { "id": 1519440, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-21T15:56:06.620000", "content": "@analokamus great..
\nsome XYZ augs may be ?
@jaideepvalani Yes, augmentation helped a bit.
", "votes": 4, "replies": [] } ] }, { "id": 1514316, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-16T00:44:28.467000", "content": "LB 0.8805
\nCV 0.8787
\nres 128x128
\nNot EfficientNet model
By \"res 128x128\" do you mean you're reshaping to 128x128?
", "votes": 0, "replies": [] }, { "id": 1514323, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-16T01:02:38.690000", "content": "yep, it is the res of the input to the model
", "votes": 0, "replies": [] }, { "id": 1514637, "author_name": "Johnny Lee", "author_url": "", "post_date": "2021-09-16T09:02:42.487000", "content": "128x128, that's amazing.
\nBtw I use 512x4096, but the CV is only 0.878x.
you can modify the stride in your model.
\nalso for the transformer, there is no stride.
", "votes": 4, "replies": [] }, { "id": 1514657, "author_name": "Johnny Lee", "author_url": "", "post_date": "2021-09-16T09:54:59.473000", "content": "I just tried a crazy idea. :)
", "votes": 2, "replies": [] }, { "id": 1514702, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-09-16T11:07:26.430000", "content": "Oh my gosh. I use 5 models in blend to get 0.8798 LB. And all of them have 0.976+- as solo, but pair correlations are less then 0.97-0.96. I think it is one of the rare competitions where top people will have really different solutions.
", "votes": 3, "replies": [] }, { "id": 1516261, "author_name": "Roc", "author_url": "", "post_date": "2021-09-18T06:03:35.803000", "content": "Great work! My best b0 cv=0.8745. Your work will motivate me to continue to optimize my single model. Thank you so much for sharing。
", "votes": 0, "replies": [] }, { "id": 1517062, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-19T06:59:15.187000", "content": "Your comment just inspired me to design new non correlation loss as aux loss
", "votes": 3, "replies": [] }, { "id": 1517838, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-20T05:47:46.150000", "content": "@hengck23 , Thank you, your comment also has encouraged me to look into 1D models more)
", "votes": 1, "replies": [] }, { "id": 1518580, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2021-09-20T19:09:55.893000", "content": "@iafoss So, that means your current ensemble doesn't include any 1d models? 👀
", "votes": 0, "replies": [] }, { "id": 1518678, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-20T22:52:48.247000", "content": "We did some basic stuff…
", "votes": 1, "replies": [] } ] }, { "id": 1516108, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-17T22:48:53.703000", "content": "public lb: 0.8782
\ncv(3 fold) : 0.87715, 0.87796 , 0.87648
1dCNN (input is raw waveform)
", "votes": 12, "replies": [ { "id": 1516121, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-17T23:39:10.850000", "content": "Impressive. It seems you were able to implement the Dr.'s whitening layer from mxnet. Congrats!
", "votes": 2, "replies": [] }, { "id": 1516122, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-17T23:48:06.490000", "content": "no whitening. just simple 1d cnn with parameter opmization
", "votes": 5, "replies": [] }, { "id": 1516260, "author_name": "Roc", "author_url": "", "post_date": "2021-09-18T06:01:11.857000", "content": "Great work! Thank you for your sharing！
", "votes": 0, "replies": [] }, { "id": 1516644, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-18T14:56:46.610000", "content": "No bandpass either?
", "votes": 1, "replies": [] }, { "id": 1516909, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-18T22:24:22.430000", "content": "bandpass is applied
", "votes": 2, "replies": [] }, { "id": 1528938, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-30T03:37:01.010000", "content": "this is the model for the 1dCNN that achieved the results reported
\nhttps://gist.github.com/hengck23/8739ac7537ad5a47d382b8c23b61253f
", "votes": 1, "replies": [] } ] }, { "id": 1520987, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-22T18:53:40.470000", "content": "1d CNN + Galerkin Transformer in the frequency domain
\nsimple bandpass + some simple tricks
\n1 fold
\nLB: 0.8780
\nCV: 0.8771
Update on Sept 29:
\nWell done. I tried swin transformer last night. Converged for about 20 batches then started to explode, even with low lrs and a large bs. Was going to give up on it but perhaps I should push further. Also, this is my first time hearing about galerkin transformer.
", "votes": 0, "replies": [] }, { "id": 1522138, "author_name": "Egor Trushin", "author_url": "", "post_date": "2021-09-23T22:22:38.707000", "content": "Galerkin transformer is the idea by Shuhao Cao
", "votes": 1, "replies": [] }, { "id": 1522241, "author_name": "yukiya", "author_url": "", "post_date": "2021-09-24T02:55:57.143000", "content": "there is nothing more satisfying than applied research :) Congratulations @scaomath
", "votes": 0, "replies": [] }, { "id": 1522244, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-24T03:08:26.747000", "content": "Haha, yeah. Thanks. I started this competition kinda late (last week). At first experiencing some EDA problem (train AUC 1 but valid AUC 0.5), then reading and being inspired by Heng @hengck23 's posts, I realized that signal processing is more important. So the idea is basically to let the black box to do the signal processing.
\nYet I do believe all top teams have found the weird \"artifacts\" in LIGOs but not the Virgo (if any or I could be wrong as I am pretty amateur in terms of EDA and signals). I can't wait to see the top team's methods of handling the data.
", "votes": 1, "replies": [] }, { "id": 1522259, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-24T03:32:37.503000", "content": "\n\nI do believe all top teams have found the weird \"artifacts\" in LIGOs but not the Virgo
\n
Perhaps the model has but the user hasn't 💦. Time to resume EDA while this thing trains…
", "votes": 2, "replies": [] }, { "id": 1522314, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-24T04:50:54.483000", "content": "…. weird \"artifacts\" in LIGOs …
\nlast time when I work in signature detection in gene (DNA sequence), 1dcnn + max pool works well for this. you can verify if there is any signature/artifacts using this kind of network
", "votes": 1, "replies": [] }, { "id": 1526215, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-28T00:26:01.323000", "content": "… top teams have found the weird \"artifacts\"
\nmaybe not artifacts (since this is not real data), but injection parameters (or injection software or injection assumption). e.g. injection at Q = 4,10, …
summary of the results from various kagglers in graph:
\n
@hengck23 but still we need to resize the freq axis ? as i get rectangular img output from CQT. with lesser value of freq domain.
", "votes": 0, "replies": [] } ] }, { "id": 1481952, "author_name": "Mark Tenenholtz", "author_url": "", "post_date": "2021-08-19T18:48:01.057000", "content": "efficientnet_b0, 5 folds
\nCV: 0.874
\nLB: 0.878
WOW, that's impressive especially for B0!
", "votes": 1, "replies": [] }, { "id": 1482228, "author_name": "Roc", "author_url": "", "post_date": "2021-08-20T00:52:33.040000", "content": "Great work!
", "votes": 0, "replies": [] }, { "id": 1482556, "author_name": "dan", "author_url": "", "post_date": "2021-08-20T06:35:42.503000", "content": "Great! Always believed large models like b7 is not necessary
", "votes": 0, "replies": [] }, { "id": 1482801, "author_name": "zhangeng", "author_url": "", "post_date": "2021-08-20T09:38:17.007000", "content": "It's not certain. He may be better than us in preprocessing. Maybe he only tested B0 instead of trying a model with higher complexity, such as B7… These days, I'm going to try a large model, my 5fold
\nB0 is only LB 0.872. I think preprocessing and data enhancement are the key points of this project
", "votes": 2, "replies": [] } ] }, { "id": 1499021, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-01T11:46:19.290000", "content": "baseline performance should be LB 0.87+
\nmany of the public kernels are in the range of LB 0.86+.
\nI haven't checked but their sample normalization could be wrong.
you should investigate the following
\nwave = read npy\nimage = cqt_transform(wave)\n\nwhich of the below is more correct and why?\n\nwave = normalised(wave) ... or no norm , or once per detector or once for 3 detectors?\nimage = normalised(image) ... or no norm , or once per detector or once for 3 detectors?\n\nshould we just use constant value, instance/global std or instance/global min/max ?\nWhen I got into the competition, I spent an evening reading over all the forums to that point. It was noted by two people who had tested that there is some discriminating information in the relative amplitudes of the various detectors. Therefore I believe that norm-once-for-all-three-detectors should be used. Separately, @kevinmcisaac's post already mentions the importance of norm, and from a DL perspective, I don't believe it makes sense not to do it.
", "votes": 6, "replies": [] }, { "id": 1499199, "author_name": "Hao", "author_url": "", "post_date": "2021-09-01T13:50:43.720000", "content": "I have the same doubt @hengck23, my normalization on wave is /(max of 3 detectors), and on image it's min/max normalization on 3 detecotor(I guess it's global min/max you mean).
\nAnd on both wave and image I found normalization on 3 detectors is slightly better, compared to normalize them separately. I'm not quite clear why. Also I tried normalize image with global std, result is worse than global min/max.
\nAnd I also read from pytoch document here, I'm wonder if normalize imaget to this instance std is better.
\n\n", "votes": 2, "replies": [] }, { "id": 1499360, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-01T15:30:24.857000", "content": "All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]
\n
the data are synthetically generate based on equations. Does your normalization destroy the original relationship?
", "votes": 3, "replies": [] }, { "id": 1499380, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-01T15:46:13.647000", "content": "When normalizing a disparate datasource to imagenet, what needs to be considered isn't the mean and std used to perform the normalization so much as mu/sigma of the resulting final distribution / network input.
", "votes": 3, "replies": [] }, { "id": 1499413, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-01T16:11:22.210000", "content": "@authman Norm once for all detector would mean this ?
\ndef apply_preprocess(self, waves):\n waves = np.hstack(waves)\n waves = waves / np.max(waves)\n return waves\nI just looked at my preprocessing code more closely and it looks like what I'm using now is global norm across entire dataset. I recall using something similar to this to get the values.
\nSo basically, 1) preprocess, 2) standardize with global stats (a single mean and std for the entire dataset, not channel specific), and then 3) feed into model.
\nLooking at my earlier experiments, I had a nn.InstanceNorm layer right on the input, which was later replaced LayerNorm layer, which now is replaced by standardization across the entire dataset in my dataloader.
", "votes": 1, "replies": [] }, { "id": 1499469, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-01T16:51:23.020000", "content": "Thanku.. what is diff between 1 and 2.. 1 is using max ?
", "votes": 0, "replies": [] }, { "id": 1499488, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-01T17:04:20.013000", "content": "I never used .max() manually, I had always used .std().
", "votes": 0, "replies": [] }, { "id": 1499563, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-01T18:01:44.263000", "content": "you can do quick experiments to verify results.
\nthese pre-processing steps affect most networks, even the smallest ones, and smallest image size.
\nso it is fast to verify which normalization is effective
how much difference u see with and without. I have been using .max only so far. also without max no big difference
", "votes": 0, "replies": [] } ] }, { "id": 1471090, "author_name": "Fractal Feelings", "author_url": "", "post_date": "2021-08-13T23:46:19.807000", "content": "Bandpass(20-500Hz) => CWT( 256x256 resolution) => EfficientnetB7
\nTPU using TFRecords, batch size 512
\nLB:0.874 CV: 0.8614
\n12 epochs
\n7.4m per epoch
See here for details
", "votes": 8, "replies": [ { "id": 1472082, "author_name": "samshipengs", "author_url": "", "post_date": "2021-08-14T16:30:48.430000", "content": "@kevinmcisaac did CWT work better for you comparing to CQT? to be honest im not even really sure about the difference between the two, looks like they are very similar, to my understanding CWT is a multiresolution transformation given a choice of wavelet, and for a set of scale and translation you can get the scalogram. And CQT seems like a variant to STFT where the frequency is somehow in log scale. Do you have recommendations on readings about CQT and how it differs from CWT and STFT?
\ndoes the choice of wavelet matter for you?
", "votes": 0, "replies": [] }, { "id": 1472499, "author_name": "Fractal Feelings", "author_url": "", "post_date": "2021-08-14T22:52:10.447000", "content": "CWT did not seem to make much difference, however I found a Keras implementation and was able to optimise. This enables me to compute the transforms on the fly in the NN which gives me flexibility in choosing CWT parameter. This has been beneficial.
\nI've only tried the complex morlet wave so far, perhaps I''l try some other.
", "votes": 6, "replies": [] }, { "id": 1475033, "author_name": "S. Tomizawa", "author_url": "", "post_date": "2021-08-16T12:09:46.337000", "content": "Your LB seems to be so high compared to CV. What is the reason?
", "votes": 5, "replies": [] } ] }, { "id": 1528512, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-29T17:43:44.030000", "content": "I hope it is the best single 1D model:
\nCV 0.8819, LB ?? ~0.883X based on the previous similar subs
Did it score 0.8840 ?? Still not letting us sleep LOL
", "votes": 3, "replies": [] }, { "id": 1528536, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-29T18:07:03.653000", "content": "It is already combined. So you may relax before the the reset.
", "votes": 3, "replies": [] }, { "id": 1528563, "author_name": "Riccardo", "author_url": "", "post_date": "2021-09-29T18:30:27.453000", "content": "I hope you will share how the hell did you trained it! well done!
", "votes": 2, "replies": [] }, { "id": 1528587, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-29T19:05:20.667000", "content": "Thanks. The main boost is coming from the model architecture: it should address they way how GW are detected. But there is also a number of other things our team will share when the competition is finished
", "votes": 5, "replies": [] }, { "id": 1528630, "author_name": "The fearless", "author_url": "", "post_date": "2021-09-29T20:31:19.070000", "content": "Such a great score with 1D models. We stopped tuning their architecture from like 2 weeks ago once they got 0.879x LB and focused on the pipeline to combine them with 2D ones, incredibly regret now :'( Your team performed really well indeed, good luck!
", "votes": 2, "replies": [] }, { "id": 1528642, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-29T20:48:12.893000", "content": "Thanks @solosquad1999 , best luck to you too.
\nI got motivated by @hengck23 and my teammate @richx86 to look into 1D in more details, and it was really a good chose. It's unfortunate that 1D method was not that popular in this competition: it is much faster to train and also it has quite a good performance (though, I'm not sure what top1 team did to get so high score for 2D resnet model).
\" I'm not sure what top1 team did to get so high score for 2D resnet model\"
\nthe way to learn 2d resnet model and 1dCNN is the \"same\".
\nlet's wait for a few more hours …
well, we have zero 1D model in our blend. Now we know what we missed LOL
", "votes": 3, "replies": [] }, { "id": 1528678, "author_name": "Riccardo", "author_url": "", "post_date": "2021-09-29T21:48:03.473000", "content": "\n\nwell, we have zero 1D model in our blend. Now we know what we missed LOL
\n
we can say it too ahahah! what a pity
", "votes": 1, "replies": [] }, { "id": 1528682, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-29T21:53:03.023000", "content": "train one now! 1dcnn are fast to train
", "votes": 1, "replies": [] }, { "id": 1528684, "author_name": "Richard Xing", "author_url": "", "post_date": "2021-09-29T21:53:39.003000", "content": "One of our guesses was that top 1 team makes synthetic data to work very nicely.
", "votes": 0, "replies": [] }, { "id": 1528687, "author_name": "Riccardo", "author_url": "", "post_date": "2021-09-29T21:58:54.043000", "content": "hyperparameters are not that easy to find xD
", "votes": 0, "replies": [] }, { "id": 1528689, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-29T21:59:53.803000", "content": "maybe synthetic noise (i.e. not the wave)
", "votes": 1, "replies": [] }, { "id": 1528690, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-29T22:06:43.950000", "content": "Yes, it is quite possible. Synthetic GW data itself gives just a minor boost, nothing to compare with the score of top1( or probably we missed something about it.
", "votes": 0, "replies": [] }, { "id": 1528718, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2021-09-29T22:51:13.893000", "content": "Couldn't resist comparing our 1d models after the competition.
\n 1D CNN 5 fold mean cv: 0.8817, lb: 0.8838
Do all of you add residual in CNN 1d? Hard to imagine Inception-like structure can be trained well with so much noisy data.
", "votes": 1, "replies": [] }, { "id": 1528763, "author_name": "Iafoss", "author_url": "", "post_date": "2021-09-30T00:24:20.473000", "content": "@nischaydnk , what is your private LB score for the model. My one got the following: Private LB: 0.8820, Public LB: 0.8827. I guess you model might be better)
", "votes": 1, "replies": [] }, { "id": 1528769, "author_name": "Ayushman Buragohain", "author_url": "", "post_date": "2021-09-30T00:28:52.033000", "content": "Our model scored 0.8823 on Private LB
", "votes": 2, "replies": [] }, { "id": 1528770, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2021-09-30T00:29:05.247000", "content": "@iafoss cograts ;)
\nIt scored 0.8823 in private leaderboard
Congratulations with the best 1D model and gold medal. I'm looking forwards to your writeup.
", "votes": 1, "replies": [] } ] }, { "id": 1504051, "author_name": "The fearless", "author_url": "", "post_date": "2021-09-06T03:20:41.803000", "content": "EfficientNetB7 - single fold
\nCV: 0.8776
\nLB: 0.8791
\nWe still do not know what the scores of 5 folds would be.
EfficientB3:
\n1/5 folds
\nCV: 0.8785
\nLB: 0.8776
update 9/2
\nEfficientB3:
\n1/5 folds
\nCV: 0.8791
\nLB: 0.8784
Bravo, your got such decent score from single fold, not from ensemble of 5 folds, right? @wuliaokaola
", "votes": 0, "replies": [] }, { "id": 1499232, "author_name": "Johnny Lee", "author_url": "", "post_date": "2021-09-01T14:09:46.260000", "content": "过奖了。
\nYes, 0.8776 is single fold. And my current LB is an ensemble of 4 models.
Great, wish you can win a gold this time.
\nAnd my current best single model is CV 0.8720, LB 0.8748, with EfficietNet_B0 single fold, will continue digging to catch up 😃
", "votes": 4, "replies": [] }, { "id": 1499460, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-01T16:42:58.707000", "content": "@wuliaokaola this is impressive.. any kind of preprocessing you used.
", "votes": -8, "replies": [] } ] }, { "id": 1524389, "author_name": "Riccardo", "author_url": "", "post_date": "2021-09-26T13:52:24.507000", "content": "EfficientNet
\nCV: 8791, LB: 8808
My best single model is an Efficientnet-b3, with CWT
\nLB: 0.8800 (Full Training Data)
\nLB: 0.8793 (One Fold)
Great work! Thanks for your sharing!
", "votes": 0, "replies": [] } ] }, { "id": 1511256, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-09-13T09:07:55.593000", "content": "Resnet18d, CQT, 1 of 4 folds
\nCV: 0.8734
\nLB: 0.8763
roughly the same for me:
\nResnet18d, CQT, average of 5 folds
\nCV: 0.8740
\nLB: 0.8759
Nice to hear that small models are good enough. Especially nice to see 4 min per epoch on old 1080Ti ;)
", "votes": 5, "replies": [] } ] }, { "id": 1503782, "author_name": "Hamish", "author_url": "", "post_date": "2021-09-05T17:46:23.297000", "content": "Effnet B2
\nCV 0.8750, LB 0.8780
\n5 folds, no augmentations
My feeling here is there's a lot of room to improve before ensembling anything
", "votes": 3, "replies": [ { "id": 1512568, "author_name": "Hamish", "author_url": "", "post_date": "2021-09-14T11:17:56.263000", "content": "update: it's not my best but with a B0 based model, 5 folds: CV 0.8751, LB 0.8776
", "votes": 1, "replies": [] } ] }, { "id": 1503053, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-04T22:51:11.077000", "content": "CV 0.87392066
\nLB 0.8780
\nFolds 5
\nEpochs 7
\nBCEWithLogitsLoss
\nNo TTA
CV 0.8750
\nLB 0.8788
@authman What is the model?
", "votes": 0, "replies": [] }, { "id": 1517443, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-19T15:51:53.160000", "content": "custom 1d cnn
", "votes": 3, "replies": [] }, { "id": 1517906, "author_name": "The fearless", "author_url": "", "post_date": "2021-09-20T07:36:35.723000", "content": "Your 1DCNN's gap is quite gaping, ours got CV 0.8766 but LB only 0.8762.
", "votes": 1, "replies": [] } ] }, { "id": 1528144, "author_name": "CPMP", "author_url": "", "post_date": "2021-09-29T12:06:15.303000", "content": "Effnetv2s, CV 0.8803 LB 0.8826
\n3 subs left and half a day to make ensembling work… LOL
", "votes": 4, "replies": [ { "id": 1528154, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-29T12:13:48.300000", "content": "You can do it
", "votes": 2, "replies": [] }, { "id": 1528162, "author_name": "CPMP", "author_url": "", "post_date": "2021-09-29T12:24:33.573000", "content": "Thanks! I hope you are right. But ensembling aith roc-auc is not very effective anyway.
", "votes": 0, "replies": [] }, { "id": 1528168, "author_name": "CPMP", "author_url": "", "post_date": "2021-09-29T12:30:44.283000", "content": "I also hope you can make it to gold. This is a tough competition.
", "votes": 1, "replies": [] }, { "id": 1528178, "author_name": "Johnny Lee", "author_url": "", "post_date": "2021-09-29T12:41:52.540000", "content": "Hi neighbors. same here.
\nGood luck!
Hope to see all in gold zone in a few hours 😂 fingers crossed ahaha
", "votes": 3, "replies": [] }, { "id": 1528199, "author_name": "Riccardo", "author_url": "", "post_date": "2021-09-29T13:05:49.073000", "content": "PS: I bet a flight to India if we win a gold medal. Looks like they really want to meet me 😂
", "votes": 4, "replies": [] }, { "id": 1528216, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2021-09-29T13:21:38.777000", "content": "1D + 2D ensemble --> BoOm
", "votes": 1, "replies": [] }, { "id": 1528220, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2021-09-29T13:27:37.213000", "content": "@nischaydnk shhh…… 🤐
", "votes": 1, "replies": [] }, { "id": 1528228, "author_name": "shiroe", "author_url": "", "post_date": "2021-09-29T13:37:44.687000", "content": "I bet many teams will now try there last attempt with 1d models😂
", "votes": 0, "replies": [] }, { "id": 1528234, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2021-09-29T13:43:37.800000", "content": "Most of the teams who've shared the best single models here had (0.0005 - 0.001) better scores on the leaderboard at the time of posting. Everyone must have got some boost from the ensemble.
", "votes": 0, "replies": [] }, { "id": 1528237, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-29T13:45:57.213000", "content": "there is something else than 1dCNN. it seems that the top teams know about it. we will know tmr
", "votes": 5, "replies": [] }, { "id": 1528242, "author_name": "Shuhao Cao", "author_url": "", "post_date": "2021-09-29T13:50:04.017000", "content": "WaveNet-like bidirectional LSTM? Or unsupervised pretrain of a denoising autoencoder in the frequency domain.
", "votes": 0, "replies": [] }, { "id": 1528247, "author_name": "Phaedrus", "author_url": "", "post_date": "2021-09-29T13:52:54.557000", "content": "Resnet34 --> boom 💥
", "votes": 2, "replies": [] }, { "id": 1528251, "author_name": "عثمان", "author_url": "", "post_date": "2021-09-29T13:56:35.117000", "content": "torch.randn()
@pheadrus My brain 🧠 ---> boom 💥
", "votes": 0, "replies": [] }, { "id": 1528255, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2021-09-29T13:59:08.207000", "content": "\n\n\nunsupervised pretrain of a denoising autoencoder in the frequency domain.
\n
Another very interesting Idea , just wanted to add , I loved galerkin transformer although I am still a kid to be able to completely understand the math behind it but I got the intuition . I also made a failed attempt to use it , but I am glad I was introduced to it , maybe I will make one more attempt after all this is over
", "votes": 1, "replies": [] }, { "id": 1528299, "author_name": "Gleb", "author_url": "", "post_date": "2021-09-29T14:42:03.410000", "content": "I tried to pretrain encoder with unsupervised AE (in every competition i entered). For me, didnt work
", "votes": 3, "replies": [] }, { "id": 1528493, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2021-09-29T17:31:54.857000", "content": "3 downvotes seriously :)
\nIs it even a secret sauce that no one was aware of?
Yes, seriously. Sharing few hours before deadline is not a good practice. In the past it has led hundreds of people to lose their medal once when someone shared a good silver medal info. No wonder some are very sensitive to late sharing.
", "votes": 4, "replies": [] }, { "id": 1528943, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-30T03:42:16.597000", "content": "I too tried AE with 10 15 percent of background noise but it dint help , it was overfitting
", "votes": 0, "replies": [] } ] }, { "id": 1516477, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-18T11:16:18.093000", "content": "public lb: 0.8790
\ncv(4 fold): 0.87599, 0.87609, 0.874753, 0.876685
effnet_b2: CQT 256x786
\npublic lb: 0.8788
\ncv(5 fold): 0.87456, 0.87623, 0.87815, 0.87642, 0.87757
effnet_b7_ns: CQT 256x512
", "votes": 4, "replies": [ { "id": 1522910, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-24T17:45:21.353000", "content": "can't get the same results as the top, but the performance of resnet34 is a surprise for me.
\nsingle fold resner34 (modified) + 126x129 cqt (modified)
\ncv :0.875115
\nlb : 0.8760
single fold resner34 (modified) + 126x129 cqt (another modified)
\ncv :0.877104
\nlb : 0.8772
single fold resner34 (modified) + 256x257 cqt (modified)
\ncv : 0.878053
\nlb : 0.8799
\nlb: 0.8812 (5fold)
cv : 0.87915 (tta-single-fold)
\nlb: 0.8803 (tta-single-fold)
single fold resner34 (modified) + 256x257 cqt (another modified)
\ncv :0.878869
\nlb: 0.8805
me too :)
\nsingle fold resner34
\ncv : 0.8833
\nlb : 0.8816
Glad that it helped :)
", "votes": 3, "replies": [] }, { "id": 1528694, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-29T22:15:13.330000", "content": "Hot from the oven!
\nsingle fold resner34 (modified) + 126x256 cqt (yet another modified)
\ncv :0.87904
\ncv-tta : 0.88013
\npublic-lb-tta : 0.8812
EfficientNetB2, 5-fold, CQT
\nCV: 0.8760
\nLB: 0.8794
Efficientnet-b3, CQT
\nCV(5-folds): 0.87624, 0.87563, 0.87724, 0.87594, 0.87692
\nLB: 0.8794
EfficientnetV2M, CQT 512x512
\nNo CV(training data 100%. See here)
\nLB: 0.8789
hmm … \"training data 100%\" could be a trade secret. i will try it
", "votes": 1, "replies": [] } ] }, { "id": 1502439, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-04T08:49:31.083000", "content": "Best Single Model Update : CV 87.4 LB 87.71
", "votes": 1, "replies": [ { "id": 1502689, "author_name": "Phaedrus", "author_url": "", "post_date": "2021-09-04T14:39:00.223000", "content": "You are using keras cwt?
", "votes": 0, "replies": [] }, { "id": 1502729, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-04T15:36:12.137000", "content": "i use CQT only.
", "votes": 1, "replies": [] }, { "id": 1502749, "author_name": "Ayushman Buragohain", "author_url": "", "post_date": "2021-09-04T15:54:43.297000", "content": "If you don’t mind sharing, what's your base model architecture ?
", "votes": 0, "replies": [] }, { "id": 1502844, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-04T17:25:59.010000", "content": "sorry lb is highly volatile.. every piece of info is adding to people score :) tomorrow i will be down to 50 . you can go through public nbks . every one is around same only..
", "votes": 1, "replies": [] }, { "id": 1508018, "author_name": "Jaideep", "author_url": "", "post_date": "2021-09-09T18:14:59.793000", "content": "Lower model,base version- CV 87.24-Further improvable to 87.44 ,LB-8771
", "votes": -1, "replies": [] } ] }, { "id": 1475808, "author_name": "Bibek", "author_url": "", "post_date": "2021-08-16T20:48:58.720000", "content": "EfficientB0:
\n5folds
\nCV: 0.866
\nLB: 0.868
Thanks, how long does one epoch take?
", "votes": 1, "replies": [ { "id": 1467895, "author_name": "Roc", "author_url": "", "post_date": "2021-08-12T07:09:15.320000", "content": "Almost 3 minutes in 4-v100 gpus.
", "votes": 1, "replies": [] } ] }, { "id": 1507896, "author_name": "imori", "author_url": "", "post_date": "2021-09-09T16:23:32.713000", "content": "EfficientnetB7 4fold CV:0.8750 LB:0.8785
", "votes": 2, "replies": [] }, { "id": 1502790, "author_name": "Phaedrus", "author_url": "", "post_date": "2021-09-04T16:41:12.367000", "content": "Effnet B0 CV 87.2 LB 87.65
", "votes": 2, "replies": [] }, { "id": 1518689, "author_name": "hengck23", "author_url": "", "post_date": "2021-09-20T23:46:32.333000", "content": "surprising results?
\nlb : 0.8762 (one fold)
\ncv : 0.874475
wide mobilenetv2 (width multiplier x3) + Coordinate attention :
\nqct = 256x256
this is an experiment for trying wide instead of deep. it is trained from scratch
", "votes": 0, "replies": [] }, { "id": 1486870, "author_name": "", "author_url": "", "post_date": "2021-08-23T09:15:09.350000", "content": "", "votes": 2, "replies": [ { "id": 1489081, "author_name": "", "author_url": "", "post_date": "2021-08-24T17:38:09.950000", "content": "", "votes": -7, "replies": [] }, { "id": 1489720, "author_name": "", "author_url": "", "post_date": "2021-08-25T08:01:12.793000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 1470011, "author_name": "", "author_url": "", "post_date": "2021-08-13T08:25:48.433000", "content": "", "votes": 2, "replies": [ { "id": 1481917, "author_name": "", "author_url": "", "post_date": "2021-08-19T18:15:27.460000", "content": "", "votes": 1, "replies": [] }, { "id": 1482165, "author_name": "", "author_url": "", "post_date": "2021-08-19T22:51:03.563000", "content": "", "votes": 1, "replies": [] }, { "id": 1482177, "author_name": "", "author_url": "", "post_date": "2021-08-19T23:13:19.453000", "content": "", "votes": 0, "replies": [] }, { "id": 1482205, "author_name": "", "author_url": "", "post_date": "2021-08-20T00:17:12.303000", "content": "", "votes": 0, "replies": [] }, { "id": 1483814, "author_name": "", "author_url": "", "post_date": "2021-08-20T22:11:11.450000", "content": "", "votes": 0, "replies": [] }, { "id": 1483823, "author_name": "", "author_url": "", "post_date": "2021-08-20T22:24:26.097000", "content": "", "votes": 2, "replies": [] }, { "id": 1483825, "author_name": "", "author_url": "", "post_date": "2021-08-20T22:30:19.157000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1486801, "author_name": "", "author_url": "", "post_date": "2021-08-23T08:19:37.353000", "content": "", "votes": 0, "replies": [] }, { "id": 1486148, "author_name": "", "author_url": "", "post_date": "2021-08-22T17:36:22.330000", "content": "", "votes": 0, "replies": [] }, { "id": 1559765, "author_name": "", "author_url": "", "post_date": "2021-10-27T07:22:34.903000", "content": "", "votes": 0, "replies": [] }, { "id": 1559764, "author_name": "", "author_url": "", "post_date": "2021-10-27T07:22:14.680000", "content": "", "votes": 0, "replies": [] }, { "id": 1509537, "author_name": "", "author_url": "", "post_date": "2021-09-11T11:46:46.260000", "content": "", "votes": 0, "replies": [] }, { "id": 1475908, "author_name": "", "author_url": "", "post_date": "2021-08-16T22:13:09.717000", "content": "", "votes": 0, "replies": [] }, { "id": 1518869, "author_name": "", "author_url": "", "post_date": "2021-09-21T06:45:59.800000", "content": "", "votes": 0, "replies": [] }, { "id": 1514883, "author_name": "", "author_url": "", "post_date": "2021-09-16T14:17:21.383000", "content": "", "votes": 0, "replies": [] }, { "id": 1485702, "author_name": "", "author_url": "", "post_date": "2021-08-22T10:54:33.113000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1467409": "Just focus on single model best cv and lb.\n\nEfficientNet-B0 - 5 folds \nApply Q-Transform more details from this [link](https://www.kaggle.com/yasufuminakama/g2net-efficientnet-b7-baseline-training?scriptVersionId=69947710)\nBasic Data Augmentation：None\nEpochs: 3\nLoss: BCEWithLogitsLoss\nNo TTA\nCV: 0.862 , LB: 0.867", "1519333": "Ok, let's do it.\n\nResNet34\nCV: 0.8840\nLB: 0.8858", "1483879": "EfficientNet-b2, 5 folds\nCV: 0.878\nLB: 0.880\n\nCV/LB correlation looks good to me.", "1519227": "Custom architecture (based on EfficientNet) / 5 fold\nCV: 0.8798\nLB: 0.8823", "1514316": "LB 0.8805\nCV 0.8787\nres 128x128\nNot EfficientNet model", "1516108": "public lb: 0.8782\ncv(3 fold) : 0.87715, 0.87796 , 0.87648 \n\n1dCNN (input is raw waveform)\n \n", "1520987": "1d CNN + Galerkin Transformer in the frequency domain\nsimple bandpass + some simple tricks \n1 fold\nLB: 0.8780\nCV: 0.8771\n\nUpdate on Sept 29: \n- somehow I finished training 5 folds today 1 hour before the deadline, and the OOF AUC is worse than the 1 fold...public is 0.8793 after blending with the 5 folds EfficientNet B7 and ResNet34d.\n- the simple tricks are just layer normalization and skip-connection in the spacial domain.", "1513772": "summary of the results from various kagglers in graph:\n", "1481952": "efficientnet_b0, 5 folds\nCV: 0.874\nLB: 0.878", "1499021": "baseline performance should be LB 0.87+\n\nmany of the public kernels are in the range of LB 0.86+.\nI haven't checked but their sample normalization could be wrong.\n\nyou should investigate the following\n\n```\nwave = read npy\nimage = cqt_transform(wave)\n\nwhich of the below is more correct and why?\n\nwave = normalised(wave) ... or no norm , or once per detector or once for 3 detectors?\nimage = normalised(image) ... or no norm , or once per detector or once for 3 detectors?\n\nshould we just use constant value, instance/global std or instance/global min/max ?\n\n```", "1471090": "Bandpass(20-500Hz) => CWT( 256x256 resolution) => EfficientnetB7 \nTPU using TFRecords, batch size 512\nLB:0.874 CV: 0.8614\n12 epochs\n7.4m per epoch\n\nSee [here](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/261721) for details", "1528512": "I hope it is the best single 1D model:\nCV 0.8819, LB ?? ~0.883X based on the previous similar subs", "1504051": "EfficientNetB7 - single fold\nCV: 0.8776\nLB: 0.8791\nWe still do not know what the scores of 5 folds would be.", "1498962": "EfficientB3:\n1/5 folds\nCV: 0.8785\nLB: 0.8776\n\nupdate 9/2\nEfficientB3:\n1/5 folds\nCV: 0.8791\nLB: 0.8784", "1524389": "EfficientNet\nCV: 8791, LB: 8808", "1518932": "My best single model is an Efficientnet-b3, with CWT \n\nLB: 0.8800 (Full Training Data)\nLB: 0.8793 (One Fold)\n\n\n\n\n\n", "1511256": "Resnet18d, CQT, 1 of 4 folds\n\nCV: 0.8734\nLB: 0.8763\n", "1503782": "Effnet B2\nCV 0.8750, LB 0.8780\n5 folds, no augmentations\n\nMy feeling here is there's a lot of room to improve before ensembling anything", "1503053": "CV 0.87392066\nLB 0.8780\nFolds 5\nEpochs 7\nBCEWithLogitsLoss\nNo TTA", "1528144": "Effnetv2s, CV 0.8803 LB 0.8826\n\n3 subs left and half a day to make ensembling work... LOL", "1516477": "public lb: 0.8790\ncv(4 fold): 0.87599, 0.87609, 0.874753, 0.876685\n\neffnet_b2: CQT 256x786\n\n---\npublic lb: 0.8788\ncv(5 fold): 0.87456, 0.87623, 0.87815, 0.87642, 0.87757 \n\neffnet_b7_ns: CQT 256x512", "1507849": "EfficientNetB2, 5-fold, CQT\n\nCV: 0.8760\nLB: 0.8794", "1517582": "Efficientnet-b3, CQT\nCV(5-folds): 0.87624, 0.87563, 0.87724, 0.87594, 0.87692\nLB: 0.8794", "1517305": "EfficientnetV2M, CQT 512x512\n\nNo CV(training data 100%. See [here](https://www.kaggle.com/ragnar123/g2net-effb7-100-seed-21))\nLB: 0.8789", "1502439": "Best Single Model Update : CV 87.4 LB 87.71", "1475808": "EfficientB0:\n5folds \nCV: 0.866\nLB: 0.868", "1467806": "Thanks, how long does one epoch take?", "1507896": "EfficientnetB7 4fold CV:0.8750 LB:0.8785", "1502790": "Effnet B0 CV 87.2 LB 87.65", "1518689": "surprising results?\nlb : 0.8762 (one fold)\ncv : 0.874475\n\nwide mobilenetv2 (width multiplier x3) + Coordinate attention : \nqct = 256x256\n\nthis is an experiment for trying wide instead of deep. it is trained from scratch", "1486870": "EfficientNet-b0, 1 fold\nCV: 0.873\nLB: 0.875", "1470011": "EfficientNet-B7 - total: 5-folds run 1 fold\nMove CQT layer insider model\nmore details from this [link](https://www.kaggle.com/mozhiwenmzw/g2net-efficientnet-b7-nontrainable-cqt-layer?scriptVersionId=71153284)\nEpochs: 3\nLoss: BCEWithLogitsLoss \nCV:0.861 LB:0.865", "1486801": "efficientnet_b7, 4 folds\nCV: 0.873\nLB: 0.875", "1486148": "Another thread with same theme : https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/251549", "1559765": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris", "1559764": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris", "1509537": "EffNetB7, CQT : \nBest single model - CV : 87.58\n5-fold - CV : 87.73", "1475908": "* effnet_b1\n* no augmentation\n* CQT\n* 4-folds\n\nCV: 86.7\nLB: 86.7", "1518869": "", "1514883": "", "1485702": "" } }