Congrats to all who did well, esp the amazing top team. I can't wait to read solution writeups from all teams who want to share. You don't need to be in top 10 to share what you did. Don't be shy. We will share as well, maybe after GLR competitions are over. We'll see.
\nA little teaser still: we have no 1D model and no resnet34 models in our selected subs. Maybe we should pay more attention to what is being shared during the competition given some top teams claim that both were key!
\nAnd most important: I was lucky to team with @titericz and @onodera We come from three different cultures hence have quite different ideas. It proves 1+1+1 > 3.
", "messages": [ { "id": "1528782", "postDate": "09/30/2021 00:48:32", "content": "Congrats to all who did well, esp the amazing top team. I can't wait to read solution writeups from all teams who want to share. You don't need to be in top 10 to share what you did. Don't be shy. We will share as well, maybe after GLR competitions are over. We'll see.
\nA little teaser still: we have no 1D model and no resnet34 models in our selected subs. Maybe we should pay more attention to what is being shared during the competition given some top teams claim that both were key!
\nAnd most important: I was lucky to team with @titericz and @onodera We come from three different cultures hence have quite different ideas. It proves 1+1+1 > 3.
", "rawMarkdown": "Congrats to all who did well, esp the amazing top team. I can't wait to read solution writeups from all teams who want to share. You don't need to be in top 10 to share what you did. Don't be shy. We will share as well, maybe after GLR competitions are over. We'll see.\n\nA little teaser still: we have no 1D model and no resnet34 models in our selected subs. Maybe we should pay more attention to what is being shared during the competition given some top teams claim that both were key!\n\nAnd most important: I was lucky to team with @titericz and @onodera We come from three different cultures hence have quite different ideas. It proves 1+1+1 > 3.", "votes": null }, { "id": "1528834", "postDate": "09/30/2021 01:43:27", "content": "@cpmpml Congratulations on getting another gold! Great results without 1D model.
", "rawMarkdown": "cpmpml Congratulations on getting another gold! Great results without 1D model.", "votes": null }, { "id": "1529746", "postDate": "09/30/2021 16:32:54", "content": "Thanks!
\nCongrats on becoming GM! I knew it was a matter of time.
\nI also want to thank you for repeating that effnet b7 was silly and that smaller models were just fine, and that effnet were not the end of story. I think your posts helped many teams.
", "rawMarkdown": "Thanks!\n\nCongrats on becoming GM! I knew it was a matter of time.\n\nI also want to thank you for repeating that effnet b7 was silly and that smaller models were just fine, and that effnet were not the end of story. I think your posts helped many teams.", "votes": null }, { "id": "1529901", "postDate": "09/30/2021 18:36:19", "content": "Congratulations to all participants and teams who had won a medal or gained valuable learning experiences for themselves! 🎉You rocked it! ;-)
\nFor me this competition was more a like sprint than a marathon. I made most of my progress in the last three weeks and I can truly say that I have never learnt so much about Kaggle competitions in such a short time window. I joined this competition right from the start and shared EDA and ideas for signal processing. But then we had a severe flood that damaged almost all homes in our village. Me and my family had a lot of luck but it was an apocalyptic kind of experience that changed my view about life. It was the first time that I continued a competition without any kind of performance pressure. I just wanted to \"play Kaggle\" like I would play an adventure video game with lots of quests to be solved. 🤠In this kind of mood I started to cycle and iterate.
\nI started with the efficientnet and constant q transform ideas that can be found everywhere in the public notebooks. I tried out small image sizes and architectures as I wanted to keep the danger of overfitting as small as possible. I thought that such a hidden signal in noise must have a high danger of overfitting. But this turned out to be nonsense. More features and deeper model architectures turned out to work much better even though the gap between validation and train scores became larger.
After this nice lesson learnt I started to spend a few days more on the constant q transform. I tuned the hyperparameters bins per octave, fmax, filterscale and hop length and especially the filterscale close to 0.5 turned out to work great. I was really confused at that time and wanted to dive deeper into the theory of constant q transform and the meaning of all hyperparameters but with only 1-2 hours per day and only 3 weeks left I had to skip this part. As binary neutron star mergers became more visible with higher q values I tried out a set of different hyperparameters containing Q-values ranging from 3 to ~30. But it turned out to be a dead end when fitting next level models on oof with my large 5 fold models and large images. Now, I can say that I had spend too much time on this part. I was like paralysed to find an ensemble that is able to detect all kinds of sources for gravitational waves. But in the end the limited computing resources provided by Kaggle enforced me to try out something new. ;-)
Diving again trough the discussion forum I fell in love with the idea to run a 1dCNN on raw data with filters learnt themselves. No complicated hyperparameter tuning, much faster computation and the end-to-end idea were very appealing. I tried to built SampleCNN but I was not able to reach the same performance as the public notebook shared by @scaomath and I had to drop it as there were only 9 days left. So I forked it and started to tune the data itself. I tried it with raw data, with whitened and bandpassed data, only bandpassed as well as bandpassed and notched like you can find in the examples of GWpy (https://gwpy.github.io/docs/stable/examples/signal/gw150914.html). For the 1dCNN the bandpassed version (+normalized with means and stds in the public notebooks) worked best for me but for the 2dCNNs the bandpassed & notched (+ mu/std norm) worked even better. At the moment I have no good explanation why it's different.
From now on time was playing against me and I wanted to lift some of my ideas into the cloud but unfortunately GCP had no resources left and Kaggle was frozen as well. :-( I wanted to train on more data and I wanted to run more ideas in parallel (more different model architectures as well as more self-made 1dCNNs). I wanted to increase the diversity but without more computational power and with my tiny \"Kaggle-time\" per day this was hopeless.
Hence the last part of my adventure was blending ✨I felt like I'm a cook in the witch's kitchen who is looking for the optimal breeze of weights for each model. The 2dCNNs scored better in my CV as well as on the PLB so I decided to give them the highest proportion. In the end only a hand full of models were part of the blend and I was on the bottom of the bronze zone before I went to sleep.
The nightmare of the high scoring public notebook shared during the last hours let me drop roughly 15 places before the competition came to an end. This is sad but I'm not really frustrated. This competition provided many funny evenings and I learnt a lot that I can take with me about machine learning in general and for next Kaggle competitions. But I can really feel how sad it must be when you had been in the silver zone and dropped out because of that. I hope that Kaggle will find a solution for this competition.
\nI'm really curious and excited to jump into your solutions! Thank you for sharing your knowledge and making Kaggle to a good place for learning and having fun!
", "rawMarkdown": "Congratulations to all participants and teams who had won a medal or gained valuable learning experiences for themselves! 🎉You rocked it! ;-)\n\nFor me this competition was more a like sprint than a marathon. I made most of my progress in the last three weeks and I can truly say that I have never learnt so much about Kaggle competitions in such a short time window. I joined this competition right from the start and shared EDA and ideas for signal processing. But then we had a severe flood that damaged almost all homes in our village. Me and my family had a lot of luck but it was an apocalyptic kind of experience that changed my view about life. It was the first time that I continued a competition without any kind of performance pressure. I just wanted to \"play Kaggle\" like I would play an adventure video game with lots of quests to be solved. 🤠In this kind of mood I started to cycle and iterate. \n\n- I started with the efficientnet and constant q transform ideas that can be found everywhere in the public notebooks. I tried out small image sizes and architectures as I wanted to keep the danger of overfitting as small as possible. I thought that such a hidden signal in noise must have a high danger of overfitting. But this turned out to be nonsense. **More features and deeper model architectures turned out to work much better** even though the gap between validation and train scores became larger. \n\n- After this nice lesson learnt I started to spend a few days more on the constant q transform. I tuned the hyperparameters bins per octave, fmax, filterscale and hop length and especially the filterscale close to 0.5 turned out to work great. I was really confused at that time and wanted to dive deeper into the theory of constant q transform and the meaning of all hyperparameters but with only 1-2 hours per day and only 3 weeks left I had to skip this part. **As binary neutron star mergers became more visible with higher q values I tried out a set of different hyperparameters containing Q-values ranging from 3 to ~30. But it turned out to be a dead end** when fitting next level models on oof with my large 5 fold models and large images. Now, I can say that I had spend too much time on this part. I was like paralysed to find an ensemble that is able to detect all kinds of sources for gravitational waves. But in the end the limited computing resources provided by Kaggle enforced me to try out something new. ;-)\n\n- Diving again trough the discussion forum **I fell in love with the idea to run a 1dCNN on raw data with filters learnt themselves**. No complicated hyperparameter tuning, much faster computation and the end-to-end idea were very appealing. I tried to built SampleCNN but I was not able to reach the same performance as the public notebook shared by @scaomath and I had to drop it as there were only 9 days left. So I forked it and started to tune the data itself. I tried it with raw data, with whitened and bandpassed data, only bandpassed as well as bandpassed and notched like you can find in the examples of GWpy (https://gwpy.github.io/docs/stable/examples/signal/gw150914.html). **For the 1dCNN the bandpassed version (+normalized with means and stds in the public notebooks) worked best for me but for the 2dCNNs the bandpassed & notched (+ mu/std norm) worked even better**. At the moment I have no good explanation why it's different. \n\n- From now on time was playing against me and I wanted to lift some of my ideas into the cloud but unfortunately **GCP had no resources left and Kaggle was frozen as well.** :-( I wanted to train on more data and I wanted to run more ideas in parallel (more different model architectures as well as more self-made 1dCNNs). I wanted to increase the diversity but without more computational power and with my tiny \"Kaggle-time\" per day this was hopeless. \n\n- Hence the last part of my adventure was blending ✨I felt like I'm a cook in the witch's kitchen who is looking for the optimal breeze of weights for each model. The 2dCNNs scored better in my CV as well as on the PLB so I decided to give them the highest proportion. In the end only a hand full of models were part of the blend and I was on the bottom of the bronze zone before I went to sleep. \n\nThe nightmare of the high scoring public notebook shared during the last hours let me drop roughly 15 places before the competition came to an end. This is sad but I'm not really frustrated. This competition provided many funny evenings and I learnt a lot that I can take with me about machine learning in general and for next Kaggle competitions. But I can really feel how sad it must be when you had been in the silver zone and dropped out because of that. I hope that Kaggle will find a solution for this competition.\n\nI'm really curious and excited to jump into your solutions! Thank you for sharing your knowledge and making Kaggle to a good place for learning and having fun!", "votes": null }, { "id": "1529966", "postDate": "09/30/2021 19:53:27", "content": "Thank you so much, I finally reached it.
\nI just truly believe that competitions should not be based on just taking the largest possible model and training it. And I'm happy that I could help others to look into other aspects of the challenge. Here, as appeared at the end, proper preprocessing, alternative 1D approaches, and even synthetic data were much more important than the model size.
HI, this is worth a topic on its own. Hiding it as a comment will not give it the visibility it deserves. I suggest you repost as a topic.
", "rawMarkdown": "HI, this is worth a topic on its own. Hiding it as a comment will not give it the visibility it deserves. I suggest you repost as a topic.", "votes": null }, { "id": "1530245", "postDate": "10/01/2021 03:52:37", "content": "@cpmpml Congrats on your team for another gold. Your team were at the very top for so long. Those comments from @iafoss about smaller model/resolution was indeed very helpful and inspiring. It helped his future teammate(me), too. Who knows, the things you share in discussion may somehow benefit yourself someday. @cpmpml I am also really grateful for your sharing in discussions; they helped me very much.
", "rawMarkdown": "cpmpml Congrats on your team for another gold. Your team were at the very top for so long. Those comments from @iafoss about smaller model/resolution was indeed very helpful and inspiring. It helped his future teammate(me), too. Who knows, the things you share in discussion may somehow benefit yourself someday. @cpmpml I am also really grateful for your sharing in discussions; they helped me very much.", "votes": null }, { "id": "1530717", "postDate": "10/01/2021 11:07:09", "content": "@richx86 Indeed, sharing is always rewarding. People answers gives you back quite a lot in general.
", "rawMarkdown": "richx86 Indeed, sharing is always rewarding. People answers gives you back quite a lot in general.", "votes": null }, { "id": "1559898", "postDate": "10/27/2021 08:07:02", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
@cpmpml Congratulations on getting another gold! Great results without 1D model.
", "votes": null, "replies": [ { "id": 1529746, "author_name": "cpmpml", "author_url": "", "post_date": "09/30/2021 16:32:54", "content": "Thanks!
\nCongrats on becoming GM! I knew it was a matter of time.
\nI also want to thank you for repeating that effnet b7 was silly and that smaller models were just fine, and that effnet were not the end of story. I think your posts helped many teams.
", "votes": null, "replies": [] }, { "id": 1529966, "author_name": "iafoss", "author_url": "", "post_date": "09/30/2021 19:53:27", "content": "Thank you so much, I finally reached it.
\nI just truly believe that competitions should not be based on just taking the largest possible model and training it. And I'm happy that I could help others to look into other aspects of the challenge. Here, as appeared at the end, proper preprocessing, alternative 1D approaches, and even synthetic data were much more important than the model size.
@cpmpml Congrats on your team for another gold. Your team were at the very top for so long. Those comments from @iafoss about smaller model/resolution was indeed very helpful and inspiring. It helped his future teammate(me), too. Who knows, the things you share in discussion may somehow benefit yourself someday. @cpmpml I am also really grateful for your sharing in discussions; they helped me very much.
", "votes": null, "replies": [] }, { "id": 1530717, "author_name": "cpmpml", "author_url": "", "post_date": "10/01/2021 11:07:09", "content": "@richx86 Indeed, sharing is always rewarding. People answers gives you back quite a lot in general.
", "votes": null, "replies": [] } ] }, { "id": 1529901, "author_name": "allunia", "author_url": "", "post_date": "09/30/2021 18:36:19", "content": "Congratulations to all participants and teams who had won a medal or gained valuable learning experiences for themselves! 🎉You rocked it! ;-)
\nFor me this competition was more a like sprint than a marathon. I made most of my progress in the last three weeks and I can truly say that I have never learnt so much about Kaggle competitions in such a short time window. I joined this competition right from the start and shared EDA and ideas for signal processing. But then we had a severe flood that damaged almost all homes in our village. Me and my family had a lot of luck but it was an apocalyptic kind of experience that changed my view about life. It was the first time that I continued a competition without any kind of performance pressure. I just wanted to \"play Kaggle\" like I would play an adventure video game with lots of quests to be solved. 🤠In this kind of mood I started to cycle and iterate.
\nI started with the efficientnet and constant q transform ideas that can be found everywhere in the public notebooks. I tried out small image sizes and architectures as I wanted to keep the danger of overfitting as small as possible. I thought that such a hidden signal in noise must have a high danger of overfitting. But this turned out to be nonsense. More features and deeper model architectures turned out to work much better even though the gap between validation and train scores became larger.
After this nice lesson learnt I started to spend a few days more on the constant q transform. I tuned the hyperparameters bins per octave, fmax, filterscale and hop length and especially the filterscale close to 0.5 turned out to work great. I was really confused at that time and wanted to dive deeper into the theory of constant q transform and the meaning of all hyperparameters but with only 1-2 hours per day and only 3 weeks left I had to skip this part. As binary neutron star mergers became more visible with higher q values I tried out a set of different hyperparameters containing Q-values ranging from 3 to ~30. But it turned out to be a dead end when fitting next level models on oof with my large 5 fold models and large images. Now, I can say that I had spend too much time on this part. I was like paralysed to find an ensemble that is able to detect all kinds of sources for gravitational waves. But in the end the limited computing resources provided by Kaggle enforced me to try out something new. ;-)
Diving again trough the discussion forum I fell in love with the idea to run a 1dCNN on raw data with filters learnt themselves. No complicated hyperparameter tuning, much faster computation and the end-to-end idea were very appealing. I tried to built SampleCNN but I was not able to reach the same performance as the public notebook shared by @scaomath and I had to drop it as there were only 9 days left. So I forked it and started to tune the data itself. I tried it with raw data, with whitened and bandpassed data, only bandpassed as well as bandpassed and notched like you can find in the examples of GWpy (https://gwpy.github.io/docs/stable/examples/signal/gw150914.html). For the 1dCNN the bandpassed version (+normalized with means and stds in the public notebooks) worked best for me but for the 2dCNNs the bandpassed & notched (+ mu/std norm) worked even better. At the moment I have no good explanation why it's different.
From now on time was playing against me and I wanted to lift some of my ideas into the cloud but unfortunately GCP had no resources left and Kaggle was frozen as well. :-( I wanted to train on more data and I wanted to run more ideas in parallel (more different model architectures as well as more self-made 1dCNNs). I wanted to increase the diversity but without more computational power and with my tiny \"Kaggle-time\" per day this was hopeless.
Hence the last part of my adventure was blending ✨I felt like I'm a cook in the witch's kitchen who is looking for the optimal breeze of weights for each model. The 2dCNNs scored better in my CV as well as on the PLB so I decided to give them the highest proportion. In the end only a hand full of models were part of the blend and I was on the bottom of the bronze zone before I went to sleep.
The nightmare of the high scoring public notebook shared during the last hours let me drop roughly 15 places before the competition came to an end. This is sad but I'm not really frustrated. This competition provided many funny evenings and I learnt a lot that I can take with me about machine learning in general and for next Kaggle competitions. But I can really feel how sad it must be when you had been in the silver zone and dropped out because of that. I hope that Kaggle will find a solution for this competition.
\nI'm really curious and excited to jump into your solutions! Thank you for sharing your knowledge and making Kaggle to a good place for learning and having fun!
", "votes": null, "replies": [ { "id": 1529968, "author_name": "cpmpml", "author_url": "", "post_date": "09/30/2021 19:55:31", "content": "HI, this is worth a topic on its own. Hiding it as a comment will not give it the visibility it deserves. I suggest you repost as a topic.
", "votes": null, "replies": [] } ] }, { "id": 1559898, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 08:07:02", "content": "Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris