\n
![\"\"](\"https://i.ibb.co/4PgdmLY/1Dmodel.png\")
2D Models [best single model: 0.87875/0.8805/0.8787 at CV, public, and private LB]
\n1D Models [best single model: 0.8819/0.8827/0.8820 at CV, public, and private LB]
\nTraining:
\nPreprocessing:
\nGW Simulation
\nEnsemble
\nOur team would like to thank organizers and kaggle for making this competition possible. Also, I want to express my gratitude to my outstanding teammates @vincentwang25, @richx86, @anjum48, and @yamsam for their incredible contribution towards our final result. In this competition I got the last gold medal required for getting kaggle GM, and also it is the first gold medal for @vincentwang25 and @richx86 . The provided write-up is the result of the cumulative efforts of all our team members, and I just got the privilege to post it.
\nLike many of the participants, our team initially focused on 2D models. Among the various approaches we tried, we found that whitened signals, which we will discuss later, was the most effective preprocessing for signals. We created the following 2D models using the CQT and CWT images generated from the whitened signal as input.
\nWe also tried various augmentations for 2D images (mixup, cutout, …), but most of them did not work here, however mixup on the input waveform prior to CQT/CWT did help counteract overfitting. The most effective one was the augmentation of swapping LIGO signals. This worked both for training and inference (TTA, Test Time Augmentation) and we found soft pseudo labeling also improved the score.
\nThe performance of our best single 2D model is 0.87875/0.8805/0.8787 at CV, public, and private LB.
\n1D models appeared to be the key component of our solution, even if we didn't make great improvement until the last 1.5 weeks of the competition. The reason why these models were not widely explored by most of the participants may be the need of using signal whitening to reach a performance comparable with 2D models (at least easily), and whitening is not straightforward for 2s long signals (see discussion below). However, 1D models are much faster to train, and they also outperform our 2D setup. The performance of our best single 1D model is 0.8819/0.8827/0.8820 at CV, public, and private LB. So it can reach top-7 LB after ~8 hours of training.
\nOne of the main contributions towards this result is the proper choice of the model architecture for GW detection task. Specifically, GW is not just a signal of the specific shape, but rather a correlation in signal between multiple detectors. Meanwhile, signals may be shifted by up to ~10-20 ms because of the time needed for the signal to cross the distance between detectors. So direct concatenation of signals into (3,4096) stack and then applying a joined convolution is not a good idea (our baseline V0 architecture with CV of 0.8768). Random shift between components prohibits the generation of joined features. Thus, we asked the question, why not split the network into branches for different detectors, like proposed in this paper? So the first layers, extractor and following Res blocks, learn how to extract features from a signal, a kind of learnable FFT or wavelet transform. So before merging the signals the network already has a basic understanding of what is going on. We also share weights between LIGO1 and LIGO2 branches because the signals are very similar.
\n
Merge of the extracted features instead of the original signal mitigates the effect of the relative shift (like in Short Time Fourier Transform correlation turns into a product of two aligned cells). So simple concatenation at this stage (instead of early concatenation) followed by several Res blocks (V1 architecture) gives an improvement from 0.8768 to 0.8789. However, the model after combining the signal and getting a better idea about GW, may still want to look into individual branches as a reference. So we extend our individual branches and perform the second concatenation at the next convolutional block (V2 architecture). It results in a further improvement of CV to 0.8804.
\nAfter the basic structure of the V2 model was defined, we performed a number of additional experiments for further model optimization. The model architecture tricks giving further improvement include the use of SiLU instead of ReLU, use of GeM instead of strided convolution, use of concatenation pooling at the end of the convolutional part. In one of our final runs, we used ResNeSt blocks (Split Attention convolution) having a comparable performance in the preliminary experiments, but it also performed slightly worse at the end of full training. Using CBAM modules and Stochastic Depth modules gave a slight boost and made the 1D models more diversified. The network width, n, is equal to 16 and 32 for our final models. One of our experiments is also performed for a combined 1D+2D models, pretrained separately and then finetuned jointly, which gave 0.8815/0.8831/0.8817 score.
\nThings that didn’t work: use of larger network depth (extra Res blocks), use of smaller/larger conv size in the extractor/first blocks, use of multi-head self-attention blocks before pooling or in the last stages of ResBlocks (i.e. Bottleneck Transformers), use of densely connected or ResNeXt blocks instead of ReBlocks, learnable CWT like extractors (FFT->multiplication by learnable weights->iFFT), WaveNet like network structures, pretrained ViT following the extractors (instead of customized resnet).
\nTTA based on multiplying the signal by -1 and first/second channel swap gave ~2bps boost. 64 fold MC dropout gave an additional ~1bps boost. Also during training in some experiments we used 0.05 spectral dropout: drop the given percentage of FFT spectrum during whitening.
\nRegarding whitening, direct use of packages, such as pycbc, doesn’t work mainly because of the short duration of the provided signals: only 2 seconds chunks in contrast to the almost unlimited length of data from LIGO/VIRGO detectors. To make the estimated PSD smooth, pycbc package uses an algorithm that corrupts the boundary of data, which is too costly for our dataset, whose duration of signals is only 2 seconds. We reduce the variance of estimated PSD by taking the average PSD for all negative training samples. This is the key to make whitening work (interestingly, this averaging idea came up independently to two of the team members). To further reduce the boundary effect from discontinuity and allow ourselves to use a window function that has a larger decay area (for example, Tukey window with a larger alpha), we first extend the wave while keeping the first derivative continuous at boundaries. Finally, we apply the window function to each data and use the average PSD to normalize the wave for different frequencies.
\nFor 1D model:
\nFor 2D model:
\n![\"\"](\"https://i.ibb.co/CtL2nk6/whitening.png\")
This idea is coming from curriculum learning, and in this paper, it mentioned that “We find that the deep learning algorithms can generalize low signal-to-noise ratio (SNR) signals to high SNR ones but not vice versa”, so we follow it and try to generate a signal and inject into the noise with low SNR. Even though there are around 15 parameters, we found that the most important one is the total mass and mass ratio (maybe we are wrong) because it affects the shape of GW the most through eyeballing. So we adjust the total mass and mass ratio using different distributions and inject the signal into the noise with a given SNR following max(random.gauss(3.6,1),1) distribution. This SNR distribution is determined by checking the training loss trend: we want it to follow the trend of original data (not too hard, not too simple). This way gives us a 2~8bps increase depending on the model we use.
\nWe also tried to follow this idea by giving Hard Positive samples from the train data more weight but due to the time constraints, we didn’t make it work. It could be a potential win here.
First, to confirm that train and test data are similar and do not have any hidden peculiarities we used adversarial validation that gave 0.5 AUC = train and test data are indistinguishable.
\nWe tried many different methods to ensemble the models and saw the following relative performance trend: CME-ES with Logit > CME-ES with rank> Scipy Optimization > Neural Network > other methods. We also tried to use sklearn.preprocessing.PolynomialFeatures with probability prediction to do the CME-ES optimization. It brings the highest CV and LB but with a little chance of overfitting. We are glad that it turns out to be our best 0.8829 submission.
Our second submission is based on a simulation of the private LB by excluding 16% of our OOF data out of CV optimization. So the produced model weights are more robust to the data noise and potentially can lead to a better performance at the private LB. We do this because we found the CV and LB correlation is very high and we also used adversarial validation to check that indeed they are similar. So We bootstrapped 16% OOF data which has a similar CV to public LB score and the optimized CV for the remaining data matched the same as the private LB (0.8828 for one of our submissions).
\nSome of the team members used JarvisLabs.ai for their GPU workstations. The cloud platform is easy to use, stable, and very affordable. The pause functionality is great. Recommend this platform to Kagglers.
", "messages": [ { "id": "1530199", "postDate": "10/01/2021 02:48:17", "content": "2D Models [best single model: 0.87875/0.8805/0.8787 at CV, public, and private LB]
\n1D Models [best single model: 0.8819/0.8827/0.8820 at CV, public, and private LB]
\nTraining:
\nPreprocessing:
\nGW Simulation
\nEnsemble
\nOur team would like to thank organizers and kaggle for making this competition possible. Also, I want to express my gratitude to my outstanding teammates @vincentwang25, @richx86, @anjum48, and @yamsam for their incredible contribution towards our final result. In this competition I got the last gold medal required for getting kaggle GM, and also it is the first gold medal for @vincentwang25 and @richx86 . The provided write-up is the result of the cumulative efforts of all our team members, and I just got the privilege to post it.
\nLike many of the participants, our team initially focused on 2D models. Among the various approaches we tried, we found that whitened signals, which we will discuss later, was the most effective preprocessing for signals. We created the following 2D models using the CQT and CWT images generated from the whitened signal as input.
\nWe also tried various augmentations for 2D images (mixup, cutout, …), but most of them did not work here, however mixup on the input waveform prior to CQT/CWT did help counteract overfitting. The most effective one was the augmentation of swapping LIGO signals. This worked both for training and inference (TTA, Test Time Augmentation) and we found soft pseudo labeling also improved the score.
\nThe performance of our best single 2D model is 0.87875/0.8805/0.8787 at CV, public, and private LB.
\n1D models appeared to be the key component of our solution, even if we didn't make great improvement until the last 1.5 weeks of the competition. The reason why these models were not widely explored by most of the participants may be the need of using signal whitening to reach a performance comparable with 2D models (at least easily), and whitening is not straightforward for 2s long signals (see discussion below). However, 1D models are much faster to train, and they also outperform our 2D setup. The performance of our best single 1D model is 0.8819/0.8827/0.8820 at CV, public, and private LB. So it can reach top-7 LB after ~8 hours of training.
\nOne of the main contributions towards this result is the proper choice of the model architecture for GW detection task. Specifically, GW is not just a signal of the specific shape, but rather a correlation in signal between multiple detectors. Meanwhile, signals may be shifted by up to ~10-20 ms because of the time needed for the signal to cross the distance between detectors. So direct concatenation of signals into (3,4096) stack and then applying a joined convolution is not a good idea (our baseline V0 architecture with CV of 0.8768). Random shift between components prohibits the generation of joined features. Thus, we asked the question, why not split the network into branches for different detectors, like proposed in this paper? So the first layers, extractor and following Res blocks, learn how to extract features from a signal, a kind of learnable FFT or wavelet transform. So before merging the signals the network already has a basic understanding of what is going on. We also share weights between LIGO1 and LIGO2 branches because the signals are very similar.
\n
Merge of the extracted features instead of the original signal mitigates the effect of the relative shift (like in Short Time Fourier Transform correlation turns into a product of two aligned cells). So simple concatenation at this stage (instead of early concatenation) followed by several Res blocks (V1 architecture) gives an improvement from 0.8768 to 0.8789. However, the model after combining the signal and getting a better idea about GW, may still want to look into individual branches as a reference. So we extend our individual branches and perform the second concatenation at the next convolutional block (V2 architecture). It results in a further improvement of CV to 0.8804.
\nAfter the basic structure of the V2 model was defined, we performed a number of additional experiments for further model optimization. The model architecture tricks giving further improvement include the use of SiLU instead of ReLU, use of GeM instead of strided convolution, use of concatenation pooling at the end of the convolutional part. In one of our final runs, we used ResNeSt blocks (Split Attention convolution) having a comparable performance in the preliminary experiments, but it also performed slightly worse at the end of full training. Using CBAM modules and Stochastic Depth modules gave a slight boost and made the 1D models more diversified. The network width, n, is equal to 16 and 32 for our final models. One of our experiments is also performed for a combined 1D+2D models, pretrained separately and then finetuned jointly, which gave 0.8815/0.8831/0.8817 score.
\nThings that didn’t work: use of larger network depth (extra Res blocks), use of smaller/larger conv size in the extractor/first blocks, use of multi-head self-attention blocks before pooling or in the last stages of ResBlocks (i.e. Bottleneck Transformers), use of densely connected or ResNeXt blocks instead of ReBlocks, learnable CWT like extractors (FFT->multiplication by learnable weights->iFFT), WaveNet like network structures, pretrained ViT following the extractors (instead of customized resnet).
\nTTA based on multiplying the signal by -1 and first/second channel swap gave ~2bps boost. 64 fold MC dropout gave an additional ~1bps boost. Also during training in some experiments we used 0.05 spectral dropout: drop the given percentage of FFT spectrum during whitening.
\nRegarding whitening, direct use of packages, such as pycbc, doesn’t work mainly because of the short duration of the provided signals: only 2 seconds chunks in contrast to the almost unlimited length of data from LIGO/VIRGO detectors. To make the estimated PSD smooth, pycbc package uses an algorithm that corrupts the boundary of data, which is too costly for our dataset, whose duration of signals is only 2 seconds. We reduce the variance of estimated PSD by taking the average PSD for all negative training samples. This is the key to make whitening work (interestingly, this averaging idea came up independently to two of the team members). To further reduce the boundary effect from discontinuity and allow ourselves to use a window function that has a larger decay area (for example, Tukey window with a larger alpha), we first extend the wave while keeping the first derivative continuous at boundaries. Finally, we apply the window function to each data and use the average PSD to normalize the wave for different frequencies.
\nFor 1D model:
\nFor 2D model:
\n
This idea is coming from curriculum learning, and in this paper, it mentioned that “We find that the deep learning algorithms can generalize low signal-to-noise ratio (SNR) signals to high SNR ones but not vice versa”, so we follow it and try to generate a signal and inject into the noise with low SNR. Even though there are around 15 parameters, we found that the most important one is the total mass and mass ratio (maybe we are wrong) because it affects the shape of GW the most through eyeballing. So we adjust the total mass and mass ratio using different distributions and inject the signal into the noise with a given SNR following max(random.gauss(3.6,1),1) distribution. This SNR distribution is determined by checking the training loss trend: we want it to follow the trend of original data (not too hard, not too simple). This way gives us a 2~8bps increase depending on the model we use.
\nWe also tried to follow this idea by giving Hard Positive samples from the train data more weight but due to the time constraints, we didn’t make it work. It could be a potential win here.
First, to confirm that train and test data are similar and do not have any hidden peculiarities we used adversarial validation that gave 0.5 AUC = train and test data are indistinguishable.
\nWe tried many different methods to ensemble the models and saw the following relative performance trend: CME-ES with Logit > CME-ES with rank> Scipy Optimization > Neural Network > other methods. We also tried to use sklearn.preprocessing.PolynomialFeatures with probability prediction to do the CME-ES optimization. It brings the highest CV and LB but with a little chance of overfitting. We are glad that it turns out to be our best 0.8829 submission.
Our second submission is based on a simulation of the private LB by excluding 16% of our OOF data out of CV optimization. So the produced model weights are more robust to the data noise and potentially can lead to a better performance at the private LB. We do this because we found the CV and LB correlation is very high and we also used adversarial validation to check that indeed they are similar. So We bootstrapped 16% OOF data which has a similar CV to public LB score and the optimized CV for the remaining data matched the same as the private LB (0.8828 for one of our submissions).
\nSome of the team members used JarvisLabs.ai for their GPU workstations. The cloud platform is easy to use, stable, and very affordable. The pause functionality is great. Recommend this platform to Kagglers.
", "rawMarkdown": "### TL;DR\n##### Models we used in our best submission: 15 1D Model + 8 2D Model\n**2D Models** [best single model: **0.87875/0.8805/0.8787** at CV, public, and private LB]\n- CQT, CWT Transformation\n- Efficientnet, Resnext, InceptionV3...\n- TTA: shuffle LIGO channel\n\n**1D Models** [best single model: **0.8819/0.8827/0.8820** at CV, public, and private LB]\n- Customized architecture targeted at GW detection\n- TTA: vflip, shuffle LIGO channels, Gaussian noise, time shift, time mask, MC dropout\n\n**Training:**\n- Pretraining with GW\n- Training with Pseudo Label or Soft Label\n- BCE, Rank Loss\n- MixUp\n- AdamW, RangerLars+Lookahead optimizer \n\n**Preprocessing:**\n- Avg PSD of target 0 (Design Curves)\n- Extending waves\n- Whitening with Tukey window\n\n**GW Simulation**\n- Distribution of Total Mass\n- SNR injection ratio max(N(3.6,1),1)\n\n**Ensemble**\n- CMA-ES Optimization\n- Hacking Private LB\n\n### Introduction\nOur team would like to thank organizers and kaggle for making this competition possible. Also, I want to express my gratitude to my outstanding teammates @vincentwang25, @richx86, @anjum48, and @yamsam for their incredible contribution towards our final result. In this competition I got the last gold medal required for getting kaggle GM, and also it is the first gold medal for @vincentwang25 and @richx86 . The provided write-up is the result of the cumulative efforts of all our team members, and I just got the privilege to post it.\n\n### Details \n#### 2D Models\nLike many of the participants, our team initially focused on 2D models. Among the various approaches we tried, we found that whitened signals, which we will discuss later, was the most effective preprocessing for signals. We created the following 2D models using the CQT and CWT images generated from the whitened signal as input.\n\nWe also tried various augmentations for 2D images (mixup, cutout, ...), but most of them did not work here, however mixup on the input waveform prior to CQT/CWT did help counteract overfitting. The most effective one was the augmentation of swapping LIGO signals. This worked both for training and inference (TTA, Test Time Augmentation) and we found soft pseudo labeling also improved the score. \n\nThe performance of our best single 2D model is **0.87875/0.8805/0.8787** at CV, public, and private LB. \n- EfficientNet(B3, B4, B5, B7), EfficientNetV2(M), ResNet200D, Inception-V3 (also we performed a number of initial experiments with ResNeXt models)\n- CQT and CWT images generated based on the whitened signal\n- Image size 128 x 128 〜 512 x 512\n- Soft leak-free pseudo labeling from ensemble results\n- LIGO channel swap argumentation (randomly swapping LIGO channels) for both training and TTA\n- 1D mixup prior to CQT/CWT\n- Adding a 4th channel to the spectrogram/scalogram input which is just a linear gradient (-1, 1) along the frequency dimension used for frequency encoding (similar to positional encoding in transformers)\n#### 1D Models\n1D models appeared to be the key component of our solution, even if we didn't make great improvement until the last 1.5 weeks of the competition. The reason why these models were not widely explored by most of the participants may be the need of using signal whitening to reach a performance comparable with 2D models (at least easily), and whitening is not straightforward for 2s long signals (see discussion below). However, 1D models are much faster to train, and they also outperform our 2D setup. The performance of our best single 1D model is **0.8819/0.8827/0.8820** at CV, public, and private LB. So **it can reach top-7 LB after ~8 hours of training**.\n\nOne of the main contributions towards this result is the proper choice of the model architecture for GW detection task. Specifically, **GW is not just a signal of the specific shape, but rather a correlation in signal between multiple detectors**. Meanwhile, signals may be shifted by up to ~10-20 ms because of the time needed for the signal to cross the distance between detectors. So direct concatenation of signals into (3,4096) stack and then applying a joined convolution is not a good idea (our baseline V0 architecture with CV of 0.8768). Random shift between components prohibits the generation of joined features. Thus, we asked the question, why not split the network into branches for different detectors, like proposed in [this paper](https://www.sciencedirect.com/science/article/pii/S0370269320308327)? So the first layers, extractor and following Res blocks, learn how to extract features from a signal, a kind of learnable FFT or wavelet transform. So before merging the signals the network already has a basic understanding of what is going on. We also share weights between LIGO1 and LIGO2 branches because the signals are very similar.\n\n\nMerge of the extracted features instead of the original signal mitigates the effect of the relative shift (like in Short Time Fourier Transform correlation turns into a product of two aligned cells). So simple concatenation at this stage (instead of early concatenation) followed by several Res blocks (V1 architecture) gives an improvement from **0.8768 to 0.8789**. However, the model after combining the signal and getting a better idea about GW, may still want to look into individual branches as a reference. So we extend our individual branches and perform the second concatenation at the next convolutional block (V2 architecture). It results in a further improvement of CV to **0.8804**. \n\nAfter the basic structure of the V2 model was defined, we performed a number of additional experiments for further model optimization. The model architecture tricks giving further improvement include the use of SiLU instead of ReLU, use of GeM instead of strided convolution, use of concatenation pooling at the end of the convolutional part. In one of our final runs, we used ResNeSt blocks (Split Attention convolution) having a comparable performance in the preliminary experiments, but it also performed slightly worse at the end of full training. Using CBAM modules and Stochastic Depth modules gave a slight boost and made the 1D models more diversified. The network width, n, is equal to 16 and 32 for our final models. One of our experiments is also performed for a combined 1D+2D models, pretrained separately and then finetuned jointly, which gave 0.8815/0.8831/0.8817 score.\n\n**Things that didn’t work:** use of larger network depth (extra Res blocks), use of smaller/larger conv size in the extractor/first blocks, use of multi-head self-attention blocks before pooling or in the last stages of ResBlocks (i.e. Bottleneck Transformers), use of densely connected or ResNeXt blocks instead of ReBlocks, learnable CWT like extractors (FFT->multiplication by learnable weights->iFFT), WaveNet like network structures, pretrained ViT following the extractors (instead of customized resnet).\n\n##### Training (1D models)\n- Pretraining with Simulated GW for 2~4 epochs (**2-8bps** boost)\n- Training with Pseudo-labeling data for 4~6 epochs depending on the model\n- Training with rank loss and low lr for 2 epoch (**~1bps** performance boost)\n\nTTA based on multiplying the signal by -1 and first/second channel swap gave **~2bps** boost. 64 fold MC dropout gave an additional **~1bps** boost. Also during training in some experiments we used 0.05 spectral dropout: drop the given percentage of FFT spectrum during whitening.\n\n### Preprocessing\nRegarding whitening, direct use of packages, such as pycbc, doesn’t work mainly because of the short duration of the provided signals: only 2 seconds chunks in contrast to the almost unlimited length of data from LIGO/VIRGO detectors. To make the estimated PSD smooth, pycbc package uses an algorithm that corrupts the boundary of data, which is too costly for our dataset, whose duration of signals is only 2 seconds. We reduce the variance of estimated PSD by taking the average PSD for all negative training samples. This is the key to make whitening work (interestingly, this averaging idea came up independently to two of the team members). To further reduce the boundary effect from discontinuity and allow ourselves to use a window function that has a larger decay area (for example, Tukey window with a larger alpha), we first **extend the wave while keeping the first derivative continuous at boundaries**. Finally, we apply the window function to each data and **use the average PSD to normalize the wave for different frequencies**.\n\n**For 1D model:**\n- Extend the signal to (3,8192)\n- Tukey window with alpha 0.5\n- Whitening with PSD based on the average FFT of all negative training examples\n\n**For 2D model:**\n- Using the whiten data and apply CQT transform with the following parameters:\n- CQT1992v2(sr=2048, fmin=20, fmax=1000, window='flattop', bins_per_octave=48, filter_scale=0.25, hop_length=8)\n- Resize (128x128, 256x256 …. ) \n\n\n### GW Simulation\nThis idea is coming from curriculum learning, and in [this paper](https://arxiv.org/abs/2106.03741), it mentioned that “We find that the deep learning algorithms can generalize **low signal-to-noise ratio (SNR) signals to high SNR ones but not vice versa**”, so we follow it and try to generate a signal and inject into the noise with low SNR. Even though there are around 15 parameters, we found that the most important one is the total mass and mass ratio (maybe we are wrong) because it affects the shape of GW the most through eyeballing. So we adjust the total mass and mass ratio using different distributions and inject the signal into the noise with a given SNR following max(random.gauss(3.6,1),1) distribution. This SNR distribution is determined by checking the training loss trend: we want it to follow the trend of original data (not too hard, not too simple). This way gives us a **2~8bps increase** depending on the model we use.\nWe also tried to follow this idea by giving Hard Positive samples from the train data more weight but due to the time constraints, we didn’t make it work. It could be a potential win here.\n### Ensemble\nFirst, to confirm that train and test data are similar and do not have any hidden peculiarities we used **adversarial validation** that gave 0.5 AUC = train and test data are indistinguishable.\n\nWe tried many different methods to ensemble the models and saw the following relative performance trend: [CME-ES](https://github.com/CMA-ES/pycma) with Logit > CME-ES with rank> Scipy Optimization > Neural Network > other methods. We also tried to use `sklearn.preprocessing.PolynomialFeatures` with probability prediction to do the CME-ES optimization. It brings the highest CV and LB but with a little chance of overfitting. We are glad that it turns out to be our best 0.8829 submission.\n\nOur second submission is based on a simulation of the private LB by excluding 16% of our OOF data out of CV optimization. So the produced model weights are more robust to the data noise and potentially can lead to a better performance at the private LB. We do this because we found the CV and LB correlation is very high and we also used adversarial validation to check that indeed they are similar. So We bootstrapped 16% OOF data which has a similar CV to public LB score and **the optimized CV for the remaining data matched the same as the private LB (0.8828 for one of our submissions)**. \n### Acknowledgment\nSome of the team members used JarvisLabs.ai for their GPU workstations. The cloud platform is easy to use, stable, and very affordable. The pause functionality is great. Recommend this platform to Kagglers.", "votes": null }, { "id": "1530200", "postDate": "10/01/2021 02:55:56", "content": "Congratulations @iafoss on becoming competition GM!
", "rawMarkdown": "Congratulations @iafoss on becoming competition GM!", "votes": null }, { "id": "1530205", "postDate": "10/01/2021 02:59:21", "content": "Thank you everyone! We will post our github repository here once it get cleaned and organized :D
", "rawMarkdown": "Thank you everyone! We will post our github repository here once it get cleaned and organized :D", "votes": null }, { "id": "1530217", "postDate": "10/01/2021 03:17:13", "content": "Thanks for the writeup. Congrats on becoming GM ! Looks like we missed out on the 1D models, should have tried that at the end
", "rawMarkdown": "Thanks for the writeup. Congrats on becoming GM ! Looks like we missed out on the 1D models, should have tried that at the end", "votes": null }, { "id": "1530218", "postDate": "10/01/2021 03:17:25", "content": "Thanks so much @richx86 , it was a long journey for me but finally I reached this point. The time to post some datasets/random posts… just joking.
\nI was happy to work with you on this competition. It was really great teamwork, which at the end got rewarded.
I was happy and honored to work with you! Great teamwork indeed! 1+1+1+1+1>10 (not binary😊)
", "rawMarkdown": "I was happy and honored to work with you! Great teamwork indeed! 1+1+1+1+1>10 (not binary😊)", "votes": null }, { "id": "1530228", "postDate": "10/01/2021 03:31:36", "content": "Thanks so much. 1D models were really super effective here. Fortunately, ~1.5 weeks before the end of the competition we realized that proper architecture may easily give a quite high performance.
\nIt was quite a strange feeling when after ~12 hours of 1D architecture turning I got better performance than for 2D models after a month of experimenting with them… Architecture turning is really a cool thing. It was a way how I also got my first gold medal.
Thank you for the tips 😁 I actually started looking into 1D architectures in roughly the same time, in last week, but my final exams were going on and couldn't make time for the training :( Need to try this on some other competition.
", "rawMarkdown": "Thank you for the tips 😁 I actually started looking into 1D architectures in roughly the same time, in last week, but my final exams were going on and couldn't make time for the training :( Need to try this on some other competition.", "votes": null }, { "id": "1530304", "postDate": "10/01/2021 05:01:58", "content": "Congrats on becoming GM @iafoss.
", "rawMarkdown": "Congrats on becoming GM @iafoss.", "votes": null }, { "id": "1530333", "postDate": "10/01/2021 05:28:17", "content": "Thank you so much @urvishp80. Unfortunately, we missed gold in the previous competition we worked togather, but eventually I received the last gold required for GM.
", "rawMarkdown": "Thank you so much @urvishp80. Unfortunately, we missed gold in the previous competition we worked togather, but eventually I received the last gold required for GM.", "votes": null }, { "id": "1530335", "postDate": "10/01/2021 05:28:57", "content": "Definitely!
", "rawMarkdown": "Definitely!", "votes": null }, { "id": "1530338", "postDate": "10/01/2021 05:32:54", "content": "yeah @iafoss but I am very happy that we worked together and now you are GM. There was a time when I used to fork your kernels to start the competition like Bengali.Ai one. So, seeing you becoming GM is really inspiring for me.
", "rawMarkdown": "yeah @iafoss but I am very happy that we worked together and now you are GM. There was a time when I used to fork your kernels to start the competition like Bengali.Ai one. So, seeing you becoming GM is really inspiring for me.", "votes": null }, { "id": "1530499", "postDate": "10/01/2021 07:42:43", "content": "Thank you @iafoss, @yamsam, @vincentwang25 & @richx86! This was one of the most fun competitions I've taken part in and I had a great time shooting for the top with you guys!
", "rawMarkdown": "Thank you @iafoss, @yamsam, @vincentwang25 & @richx86! This was one of the most fun competitions I've taken part in and I had a great time shooting for the top with you guys!", "votes": null }, { "id": "1530575", "postDate": "10/01/2021 08:48:48", "content": "Congratz @iafoss for finally reaching competitions GM, and to your entire team for the impressive finish !
", "rawMarkdown": "Congratz @iafoss for finally reaching competitions GM, and to your entire team for the impressive finish !", "votes": null }, { "id": "1530916", "postDate": "10/01/2021 13:35:03", "content": "Thanks you @anjum48, likewise, it was really great working with you.
", "rawMarkdown": "Thanks you @anjum48, likewise, it was really great working with you.", "votes": null }, { "id": "1530922", "postDate": "10/01/2021 13:39:55", "content": "Thank you so much @theoviel. Even if we got our first gold medals together, for me it took quite longer than for you to become a competition GM. But I finally made it.
", "rawMarkdown": "Thank you so much @theoviel. Even if we got our first gold medals together, for me it took quite longer than for you to become a competition GM. But I finally made it.", "votes": null }, { "id": "1531209", "postDate": "10/01/2021 18:15:18", "content": "Thank you @anjum48 ! I had a great time working with you and I learned a lot from you!
", "rawMarkdown": "Thank you @anjum48 ! I had a great time working with you and I learned a lot from you!", "votes": null }, { "id": "1531244", "postDate": "10/01/2021 18:57:49", "content": "What a great write-up with great diagrams and insights, thanks for sharing! It seems again that 1D models where key in this competition.
\nCongratulations again on your 3rd place.
", "rawMarkdown": "What a great write-up with great diagrams and insights, thanks for sharing! It seems again that 1D models where key in this competition.\n\nCongratulations again on your 3rd place.", "votes": null }, { "id": "1531305", "postDate": "10/01/2021 20:14:52", "content": "Thanks so much. 1D models were really super stars in this competitions.
", "rawMarkdown": "Thanks so much. 1D models were really super stars in this competitions.", "votes": null }, { "id": "1531527", "postDate": "10/02/2021 05:18:54", "content": "Wow. Great solution. Congratulations 👏👏 and a second congratulations for the well deserved grand status @lafoss 👍🎉👏
", "rawMarkdown": "Wow. Great solution. Congratulations 👏👏 and a second congratulations for the well deserved grand status @lafoss 👍🎉👏", "votes": null }, { "id": "1531540", "postDate": "10/02/2021 05:31:25", "content": "Thank you so much!
", "rawMarkdown": "Thank you so much!", "votes": null }, { "id": "1531866", "postDate": "10/02/2021 12:58:36", "content": "Congratulations. I believe that proper use of domain knowledge and careful verification led to your results. Thanks for sharing your team's solution. And @iafoss, congratulations again on becoming competition GM!!
", "rawMarkdown": "Congratulations. I believe that proper use of domain knowledge and careful verification led to your results. Thanks for sharing your team's solution. And @iafoss, congratulations again on becoming competition GM!!", "votes": null }, { "id": "1531875", "postDate": "10/02/2021 13:06:11", "content": "Indeed, I wish I've given them more attention. 😄
", "rawMarkdown": "Indeed, I wish I've given them more attention. 😄", "votes": null }, { "id": "1532376", "postDate": "10/02/2021 22:21:52", "content": "@mahmoudhelmy957 What a great solution check it
", "rawMarkdown": "mahmoudhelmy957 What a great solution check it", "votes": null }, { "id": "1532486", "postDate": "10/03/2021 03:29:47", "content": "@naoism, Thank you so much. Congratulations to u too on getting Competition Master, it's an important milestone. I hope you can get your GM title soon!
", "rawMarkdown": "naoism, Thank you so much. Congratulations to u too on getting Competition Master, it's an important milestone. I hope you can get your GM title soon!", "votes": null }, { "id": "1532770", "postDate": "10/03/2021 11:25:11", "content": "@iafoss congrats yo your team and thank you for the write up. Congrats to you for becoming a GM. And congrats to @vincentwang25 and @richx86 for their first gold metals.
", "rawMarkdown": "iafoss congrats yo your team and thank you for the write up. Congrats to you for becoming a GM. And congrats to @vincentwang25 and @richx86 for their first gold metals.", "votes": null }, { "id": "1533056", "postDate": "10/03/2021 16:10:14", "content": "@yvonnef, thanks so much
", "rawMarkdown": "yvonnef, thanks so much", "votes": null }, { "id": "1533065", "postDate": "10/03/2021 16:18:29", "content": "@iafoss out of curiosity what tool did you to create the diagram? Thank you
", "rawMarkdown": "iafoss out of curiosity what tool did you to create the diagram? Thank you", "votes": null }, { "id": "1533143", "postDate": "10/03/2021 17:39:29", "content": "Just MS PowerPoint…
\nDuring my PhD I realized that it is nearly the best tool to create images for papers/proposals.
@iafoss ty. Wow !I never released power point could do that. I will have to give that a whirl.
", "rawMarkdown": "iafoss ty. Wow !I never released power point could do that. I will have to give that a whirl.", "votes": null }, { "id": "1533403", "postDate": "10/04/2021 01:55:09", "content": "@yvonnef Thank you very much!
", "rawMarkdown": "yvonnef Thank you very much!", "votes": null }, { "id": "1535922", "postDate": "10/06/2021 10:19:36", "content": "Congrats GM @iafoss ! You definitely deserve it💪
\nand also thanks to all teammates! It was such a fun experience for me!
Thank you, @anjum48. It was an honor for me to work with you 👍
", "rawMarkdown": "Thank you, @anjum48. It was an honor for me to work with you 👍", "votes": null }, { "id": "1536116", "postDate": "10/06/2021 13:28:31", "content": "Congratulations
", "rawMarkdown": "Congratulations", "votes": null }, { "id": "1536657", "postDate": "10/06/2021 22:41:26", "content": "Thanks
", "rawMarkdown": "Thanks", "votes": null }, { "id": "1536658", "postDate": "10/06/2021 22:42:53", "content": "Thank you so much @yamsam, it was my pleasure to work with u on this challenge!
", "rawMarkdown": "Thank you so much @yamsam, it was my pleasure to work with u on this challenge!", "votes": null }, { "id": "1543959", "postDate": "10/14/2021 01:31:23", "content": "Our code is cleaned and gathered here: https://github.com/VincentWang25/G2Net_GoGoGo . Hope you find it helpful and please let us know if you have any question :)
", "rawMarkdown": "Our code is cleaned and gathered here: https://github.com/VincentWang25/G2Net_GoGoGo . Hope you find it helpful and please let us know if you have any question :)", "votes": null }, { "id": "1544175", "postDate": "10/14/2021 05:58:26", "content": "Thanks for sharing the source code!
", "rawMarkdown": "Thanks for sharing the source code!", "votes": null }, { "id": "1548207", "postDate": "10/18/2021 04:38:40", "content": "you are welcome!
", "rawMarkdown": "you are welcome!", "votes": null }, { "id": "1559926", "postDate": "10/27/2021 08:23:51", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
Congratulations @iafoss on becoming competition GM!
", "votes": null, "replies": [ { "id": 1530218, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 03:17:25", "content": "Thanks so much @richx86 , it was a long journey for me but finally I reached this point. The time to post some datasets/random posts… just joking.
\nI was happy to work with you on this competition. It was really great teamwork, which at the end got rewarded.
I was happy and honored to work with you! Great teamwork indeed! 1+1+1+1+1>10 (not binary😊)
", "votes": null, "replies": [] }, { "id": 1530335, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 05:28:57", "content": "Definitely!
", "votes": null, "replies": [] } ] }, { "id": 1530205, "author_name": "vincentwang25", "author_url": "", "post_date": "10/01/2021 02:59:21", "content": "Thank you everyone! We will post our github repository here once it get cleaned and organized :D
", "votes": null, "replies": [] }, { "id": 1530217, "author_name": "zarif98sjs", "author_url": "", "post_date": "10/01/2021 03:17:13", "content": "Thanks for the writeup. Congrats on becoming GM ! Looks like we missed out on the 1D models, should have tried that at the end
", "votes": null, "replies": [ { "id": 1530228, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 03:31:36", "content": "Thanks so much. 1D models were really super effective here. Fortunately, ~1.5 weeks before the end of the competition we realized that proper architecture may easily give a quite high performance.
\nIt was quite a strange feeling when after ~12 hours of 1D architecture turning I got better performance than for 2D models after a month of experimenting with them… Architecture turning is really a cool thing. It was a way how I also got my first gold medal.
Thank you for the tips 😁 I actually started looking into 1D architectures in roughly the same time, in last week, but my final exams were going on and couldn't make time for the training :( Need to try this on some other competition.
", "votes": null, "replies": [] } ] }, { "id": 1530304, "author_name": "urvishp80", "author_url": "", "post_date": "10/01/2021 05:01:58", "content": "Congrats on becoming GM @iafoss.
", "votes": null, "replies": [ { "id": 1530333, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 05:28:17", "content": "Thank you so much @urvishp80. Unfortunately, we missed gold in the previous competition we worked togather, but eventually I received the last gold required for GM.
", "votes": null, "replies": [] }, { "id": 1530338, "author_name": "urvishp80", "author_url": "", "post_date": "10/01/2021 05:32:54", "content": "yeah @iafoss but I am very happy that we worked together and now you are GM. There was a time when I used to fork your kernels to start the competition like Bengali.Ai one. So, seeing you becoming GM is really inspiring for me.
", "votes": null, "replies": [] } ] }, { "id": 1530499, "author_name": "anjum48", "author_url": "", "post_date": "10/01/2021 07:42:43", "content": "Thank you @iafoss, @yamsam, @vincentwang25 & @richx86! This was one of the most fun competitions I've taken part in and I had a great time shooting for the top with you guys!
", "votes": null, "replies": [ { "id": 1530916, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 13:35:03", "content": "Thanks you @anjum48, likewise, it was really great working with you.
", "votes": null, "replies": [] }, { "id": 1531209, "author_name": "richx86", "author_url": "", "post_date": "10/01/2021 18:15:18", "content": "Thank you @anjum48 ! I had a great time working with you and I learned a lot from you!
", "votes": null, "replies": [] }, { "id": 1535927, "author_name": "yamsam", "author_url": "", "post_date": "10/06/2021 10:22:35", "content": "Thank you, @anjum48. It was an honor for me to work with you 👍
", "votes": null, "replies": [] } ] }, { "id": 1530575, "author_name": "theoviel", "author_url": "", "post_date": "10/01/2021 08:48:48", "content": "Congratz @iafoss for finally reaching competitions GM, and to your entire team for the impressive finish !
", "votes": null, "replies": [ { "id": 1530922, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 13:39:55", "content": "Thank you so much @theoviel. Even if we got our first gold medals together, for me it took quite longer than for you to become a competition GM. But I finally made it.
", "votes": null, "replies": [] } ] }, { "id": 1531244, "author_name": "yassinealouini", "author_url": "", "post_date": "10/01/2021 18:57:49", "content": "What a great write-up with great diagrams and insights, thanks for sharing! It seems again that 1D models where key in this competition.
\nCongratulations again on your 3rd place.
", "votes": null, "replies": [ { "id": 1531305, "author_name": "iafoss", "author_url": "", "post_date": "10/01/2021 20:14:52", "content": "Thanks so much. 1D models were really super stars in this competitions.
", "votes": null, "replies": [] }, { "id": 1531875, "author_name": "yassinealouini", "author_url": "", "post_date": "10/02/2021 13:06:11", "content": "Indeed, I wish I've given them more attention. 😄
", "votes": null, "replies": [] } ] }, { "id": 1531527, "author_name": "kalilurrahman", "author_url": "", "post_date": "10/02/2021 05:18:54", "content": "Wow. Great solution. Congratulations 👏👏 and a second congratulations for the well deserved grand status @lafoss 👍🎉👏
", "votes": null, "replies": [ { "id": 1531540, "author_name": "iafoss", "author_url": "", "post_date": "10/02/2021 05:31:25", "content": "Thank you so much!
", "votes": null, "replies": [] } ] }, { "id": 1531866, "author_name": "naoism", "author_url": "", "post_date": "10/02/2021 12:58:36", "content": "Congratulations. I believe that proper use of domain knowledge and careful verification led to your results. Thanks for sharing your team's solution. And @iafoss, congratulations again on becoming competition GM!!
", "votes": null, "replies": [ { "id": 1532486, "author_name": "iafoss", "author_url": "", "post_date": "10/03/2021 03:29:47", "content": "@naoism, Thank you so much. Congratulations to u too on getting Competition Master, it's an important milestone. I hope you can get your GM title soon!
", "votes": null, "replies": [] } ] }, { "id": 1532376, "author_name": "sharafcode", "author_url": "", "post_date": "10/02/2021 22:21:52", "content": "@mahmoudhelmy957 What a great solution check it
", "votes": null, "replies": [] }, { "id": 1532770, "author_name": "yvonnef", "author_url": "", "post_date": "10/03/2021 11:25:11", "content": "@iafoss congrats yo your team and thank you for the write up. Congrats to you for becoming a GM. And congrats to @vincentwang25 and @richx86 for their first gold metals.
", "votes": null, "replies": [ { "id": 1533056, "author_name": "iafoss", "author_url": "", "post_date": "10/03/2021 16:10:14", "content": "@yvonnef, thanks so much
", "votes": null, "replies": [] }, { "id": 1533065, "author_name": "yvonnef", "author_url": "", "post_date": "10/03/2021 16:18:29", "content": "@iafoss out of curiosity what tool did you to create the diagram? Thank you
", "votes": null, "replies": [] }, { "id": 1533143, "author_name": "iafoss", "author_url": "", "post_date": "10/03/2021 17:39:29", "content": "Just MS PowerPoint…
\nDuring my PhD I realized that it is nearly the best tool to create images for papers/proposals.
@iafoss ty. Wow !I never released power point could do that. I will have to give that a whirl.
", "votes": null, "replies": [] }, { "id": 1533403, "author_name": "richx86", "author_url": "", "post_date": "10/04/2021 01:55:09", "content": "@yvonnef Thank you very much!
", "votes": null, "replies": [] } ] }, { "id": 1535922, "author_name": "yamsam", "author_url": "", "post_date": "10/06/2021 10:19:36", "content": "Congrats GM @iafoss ! You definitely deserve it💪
\nand also thanks to all teammates! It was such a fun experience for me!
Thank you so much @yamsam, it was my pleasure to work with u on this challenge!
", "votes": null, "replies": [] } ] }, { "id": 1536116, "author_name": "adityakamal7", "author_url": "", "post_date": "10/06/2021 13:28:31", "content": "Congratulations
", "votes": null, "replies": [ { "id": 1536657, "author_name": "iafoss", "author_url": "", "post_date": "10/06/2021 22:41:26", "content": "Thanks
", "votes": null, "replies": [] } ] }, { "id": 1543959, "author_name": "vincentwang25", "author_url": "", "post_date": "10/14/2021 01:31:23", "content": "Our code is cleaned and gathered here: https://github.com/VincentWang25/G2Net_GoGoGo . Hope you find it helpful and please let us know if you have any question :)
", "votes": null, "replies": [ { "id": 1544175, "author_name": "yassinealouini", "author_url": "", "post_date": "10/14/2021 05:58:26", "content": "Thanks for sharing the source code!
", "votes": null, "replies": [] }, { "id": 1548207, "author_name": "vincentwang25", "author_url": "", "post_date": "10/18/2021 04:38:40", "content": "you are welcome!
", "votes": null, "replies": [] } ] }, { "id": 1559926, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 08:23:51", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris