Heartiest congratulations to all the warriors and specially to the winners of this charged and electrifying competition:
\n\nI am eagerly waiting to learn about features used, feature engineering performed, stacking done and architecture used for RNN/CNN/NN networks to win this competition. I believed this competition required RNN chops but I am willing to be proven wrong with an insane new feature engineering method or architecture which furthers the learning for everyone involved in this competition.
\n\nMy learning's are divided into two components:
\n\nMy background in RNN/CNN: Zero.
\n\nI had zero background in those forms of NN architectures and this competition gave me a great platform to learn new architectures and how to perform Seq2Seq learning's.
\n\nInitial learning's:
\n\nRNN/CNN learnings:\n1. How CUDNNLSTM and CUDNNGRU are way better than plain LSTM and GRU implementations, initially at least. \n2. How to use dropout in above models? \n3. How to go back to LSTM and GRU from CUDA based implementations and try to optimize models based on underlying GPU's?\n4. How to create multiple RNN models and run them in parallel on two GPU's?\n5. How to use TensorBoard to understand more about my models?\n6. How to make sure that LSTM and GRU models accept multivariate data?
\n\nFinal notes on initial learning's, I learned a tonne from a lot of people and I will be taking these learning's in next competitions.
\n\nI will follow up with a detailed post on how LSTM and GRU played a part in my models and various feature engineering methods that I had thought about but could not try out in this competition.
\n\nThank you to the organizers of this competition.
", "messages": [ { "id": "496200", "postDate": "03/22/2019 01:01:21", "content": "Heartiest congratulations to all the warriors and specially to the winners of this charged and electrifying competition:
\n\nI am eagerly waiting to learn about features used, feature engineering performed, stacking done and architecture used for RNN/CNN/NN networks to win this competition. I believed this competition required RNN chops but I am willing to be proven wrong with an insane new feature engineering method or architecture which furthers the learning for everyone involved in this competition.
\n\nMy learning's are divided into two components:
\n\nMy background in RNN/CNN: Zero.
\n\nI had zero background in those forms of NN architectures and this competition gave me a great platform to learn new architectures and how to perform Seq2Seq learning's.
\n\nInitial learning's:
\n\nRNN/CNN learnings:\n1. How CUDNNLSTM and CUDNNGRU are way better than plain LSTM and GRU implementations, initially at least. \n2. How to use dropout in above models? \n3. How to go back to LSTM and GRU from CUDA based implementations and try to optimize models based on underlying GPU's?\n4. How to create multiple RNN models and run them in parallel on two GPU's?\n5. How to use TensorBoard to understand more about my models?\n6. How to make sure that LSTM and GRU models accept multivariate data?
\n\nFinal notes on initial learning's, I learned a tonne from a lot of people and I will be taking these learning's in next competitions.
\n\nI will follow up with a detailed post on how LSTM and GRU played a part in my models and various feature engineering methods that I had thought about but could not try out in this competition.
\n\nThank you to the organizers of this competition.
", "rawMarkdown": "Heartiest congratulations to all the warriors and specially to the winners of this charged and electrifying competition:\n\n1. Reza V\n2. MA\n3. oooooverfitting\n\nI am eagerly waiting to learn about features used, feature engineering performed, stacking done and architecture used for RNN/CNN/NN networks to win this competition. I believed this competition required RNN chops but I am willing to be proven wrong with an insane new feature engineering method or architecture which furthers the learning for everyone involved in this competition.\n\nMy learning's are divided into two components:\n\n1. Initial learning's\n2. Detailed learning's about the models\n\nMy background in RNN/CNN: Zero. \n\nI had zero background in those forms of NN architectures and this competition gave me a great platform to learn new architectures and how to perform Seq2Seq learning's. \n\nInitial learning's:\n\n1. How to perform feature engineering on power consumption data set?\n2. How to look at data from signals processing point of view?\n3. How to calculate various types of entropy (congratulations to original authors of those kernels as I knew nothing about this field and entropy calculations before I started the competition).\n4. How to study about LSTM and GRU and encoders-decoders network architectures?\n5. How electricity transmission works and how models that would be created in this competition can go a long way in helping transmission firms across the world?\n6. How Matthew's correlation coefficient works and how to use it as a model evaluation metric?\n\nRNN/CNN learnings:\n1. How CUDNNLSTM and CUDNNGRU are way better than plain LSTM and GRU implementations, initially at least. \n2. How to use dropout in above models? \n3. How to go back to LSTM and GRU from CUDA based implementations and try to optimize models based on underlying GPU's?\n4. How to create multiple RNN models and run them in parallel on two GPU's?\n5. How to use TensorBoard to understand more about my models?\n6. How to make sure that LSTM and GRU models accept multivariate data?\n\nFinal notes on initial learning's, I learned a tonne from a lot of people and I will be taking these learning's in next competitions. \n\nI will follow up with a detailed post on how LSTM and GRU played a part in my models and various feature engineering methods that I had thought about but could not try out in this competition.\n\nThank you to the organizers of this competition.", "votes": null }, { "id": "496309", "postDate": "03/22/2019 03:23:16", "content": "Hi Vikas, I can see that you have learned a lot here, thanks for sharing! Nice job and please keep it up :D
\n\nBTW, we accidentally have around the same score ;)
", "rawMarkdown": "Hi Vikas, I can see that you have learned a lot here, thanks for sharing! Nice job and please keep it up :D\n\nBTW, we accidentally have around the same score ;)", "votes": null }, { "id": "496323", "postDate": "03/22/2019 03:44:06", "content": "Thanks for the support, I definitely plan to carry over this knowledge into other challenges on this platform.
\n\nInteresting, I would love to see what did architecture did you use in your models.
", "rawMarkdown": "Thanks for the support, I definitely plan to carry over this knowledge into other challenges on this platform. \n\nInteresting, I would love to see what did architecture did you use in your models.", "votes": null }, { "id": "496355", "postDate": "03/22/2019 04:20:20", "content": "Hi Vikas, thanks for asking ... I tried many architectures, i.e. Multi-layers/Multi-head attention RNN, RCNN, CRNN, MobileNet with Signal Spectrogram as images ... (together with some unsuccessful feature extrations)
\n\nHowever, none of them seems to beat the baseline of the public kernel, so I just ended up to use the vanilla 2-layers Bi-LSTM ;p
", "rawMarkdown": "Hi Vikas, thanks for asking ... I tried many architectures, i.e. Multi-layers/Multi-head attention RNN, RCNN, CRNN, MobileNet with Signal Spectrogram as images ... (together with some unsuccessful feature extrations) \n\nHowever, none of them seems to beat the baseline of the public kernel, so I just ended up to use the vanilla 2-layers Bi-LSTM ;p", "votes": null }, { "id": "499699", "postDate": "03/25/2019 05:18:08", "content": "That's a great number of architectures for trying out. I am sure you had fun trying out those architectures.
\n\nThanks for sharing them.
", "rawMarkdown": "That's a great number of architectures for trying out. I am sure you had fun trying out those architectures. \n\nThanks for sharing them.", "votes": null } ], "comments": [ { "id": 496309, "author_name": "ratthachat", "author_url": "", "post_date": "03/22/2019 03:23:16", "content": "Hi Vikas, I can see that you have learned a lot here, thanks for sharing! Nice job and please keep it up :D
\n\nBTW, we accidentally have around the same score ;)
", "votes": null, "replies": [ { "id": 496323, "author_name": "vikasmalhotra08", "author_url": "", "post_date": "03/22/2019 03:44:06", "content": "Thanks for the support, I definitely plan to carry over this knowledge into other challenges on this platform.
\n\nInteresting, I would love to see what did architecture did you use in your models.
", "votes": null, "replies": [] }, { "id": 496355, "author_name": "ratthachat", "author_url": "", "post_date": "03/22/2019 04:20:20", "content": "Hi Vikas, thanks for asking ... I tried many architectures, i.e. Multi-layers/Multi-head attention RNN, RCNN, CRNN, MobileNet with Signal Spectrogram as images ... (together with some unsuccessful feature extrations)
\n\nHowever, none of them seems to beat the baseline of the public kernel, so I just ended up to use the vanilla 2-layers Bi-LSTM ;p
", "votes": null, "replies": [] }, { "id": 499699, "author_name": "vikasmalhotra08", "author_url": "", "post_date": "03/25/2019 05:18:08", "content": "That's a great number of architectures for trying out. I am sure you had fun trying out those architectures.
\n\nThanks for sharing them.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "496200": "Heartiest congratulations to all the warriors and specially to the winners of this charged and electrifying competition:\n\n1. Reza V\n2. MA\n3. oooooverfitting\n\nI am eagerly waiting to learn about features used, feature engineering performed, stacking done and architecture used for RNN/CNN/NN networks to win this competition. I believed this competition required RNN chops but I am willing to be proven wrong with an insane new feature engineering method or architecture which furthers the learning for everyone involved in this competition.\n\nMy learning's are divided into two components:\n\n1. Initial learning's\n2. Detailed learning's about the models\n\nMy background in RNN/CNN: Zero. \n\nI had zero background in those forms of NN architectures and this competition gave me a great platform to learn new architectures and how to perform Seq2Seq learning's. \n\nInitial learning's:\n\n1. How to perform feature engineering on power consumption data set?\n2. How to look at data from signals processing point of view?\n3. How to calculate various types of entropy (congratulations to original authors of those kernels as I knew nothing about this field and entropy calculations before I started the competition).\n4. How to study about LSTM and GRU and encoders-decoders network architectures?\n5. How electricity transmission works and how models that would be created in this competition can go a long way in helping transmission firms across the world?\n6. How Matthew's correlation coefficient works and how to use it as a model evaluation metric?\n\nRNN/CNN learnings:\n1. How CUDNNLSTM and CUDNNGRU are way better than plain LSTM and GRU implementations, initially at least. \n2. How to use dropout in above models? \n3. How to go back to LSTM and GRU from CUDA based implementations and try to optimize models based on underlying GPU's?\n4. How to create multiple RNN models and run them in parallel on two GPU's?\n5. How to use TensorBoard to understand more about my models?\n6. How to make sure that LSTM and GRU models accept multivariate data?\n\nFinal notes on initial learning's, I learned a tonne from a lot of people and I will be taking these learning's in next competitions. \n\nI will follow up with a detailed post on how LSTM and GRU played a part in my models and various feature engineering methods that I had thought about but could not try out in this competition.\n\nThank you to the organizers of this competition.", "496309": "Hi Vikas, I can see that you have learned a lot here, thanks for sharing! Nice job and please keep it up :D\n\nBTW, we accidentally have around the same score ;)", "496323": "Thanks for the support, I definitely plan to carry over this knowledge into other challenges on this platform. \n\nInteresting, I would love to see what did architecture did you use in your models.", "496355": "Hi Vikas, thanks for asking ... I tried many architectures, i.e. Multi-layers/Multi-head attention RNN, RCNN, CRNN, MobileNet with Signal Spectrogram as images ... (together with some unsuccessful feature extrations) \n\nHowever, none of them seems to beat the baseline of the public kernel, so I just ended up to use the vanilla 2-layers Bi-LSTM ;p", "499699": "That's a great number of architectures for trying out. I am sure you had fun trying out those architectures. \n\nThanks for sharing them." }, "source": "meta" }