I cannot recommend this tutorial enough
\n\nhttps://www.kaggle.com/thebrownviking20/intro-to-recurrent-neural-networks-lstm-gru
", "messages": [ { "id": "533676", "postDate": "05/19/2019 18:17:12", "content": "I cannot recommend this tutorial enough
\n\nhttps://www.kaggle.com/thebrownviking20/intro-to-recurrent-neural-networks-lstm-gru
", "rawMarkdown": "I cannot recommend this tutorial enough\n\nhttps://www.kaggle.com/thebrownviking20/intro-to-recurrent-neural-networks-lstm-gru", "votes": null }, { "id": "534208", "postDate": "05/20/2019 23:26:02", "content": "So as a beginner I have to ask (with regards to LTSM for this problem). LTSM predicts value given time, but for this competition we have to predict time (ttf) given value (acoustic data). How do we treat this type of problem using traditional time series prediction? What am I missing here? It seems like a reverse of a typical time series prediction.
", "rawMarkdown": "So as a beginner I have to ask (with regards to LTSM for this problem). LTSM predicts value given time, but for this competition we have to predict time (ttf) given value (acoustic data). How do we treat this type of problem using traditional time series prediction? What am I missing here? It seems like a reverse of a typical time series prediction.", "votes": null }, { "id": "534302", "postDate": "05/21/2019 03:34:12", "content": "LSTM is just a kind of \"machine learning\" model. A machine learning model can predict a \"label\", from a set of labeled past observed samples. Each sample has multiple features. Such as predicting a person's nationality (label) from his 5 features: age, skin tone, language, height, and weight. In this competition, we have samples as separate segments of signal. Each segment has a label, which is the end ttf value of that segment. Each signal segment has multiple features, which can be extracted from itself, such as mean, variance, FFT decomposition coefficients ... \nNext, LSTM (or RNN in general) is designed especially for features that can evolve in time. Imagine we have those 5 features (age, skin, language, height, weight) in 3 more different time steps, such as 1 year ago, 2 years ago, and 3 years ago (so 20 features in total), LSTM can exploit those time-dependent features and predict better. Of course all other machine learning models can be used with all those 20 features, but are not as good as RNN.\nHope this clarifies.
", "rawMarkdown": "LSTM is just a kind of \"machine learning\" model. A machine learning model can predict a \"label\", from a set of labeled past observed samples. Each sample has multiple features. Such as predicting a person's nationality (label) from his 5 features: age, skin tone, language, height, and weight. In this competition, we have samples as separate segments of signal. Each segment has a label, which is the end ttf value of that segment. Each signal segment has multiple features, which can be extracted from itself, such as mean, variance, FFT decomposition coefficients ... \nNext, LSTM (or RNN in general) is designed especially for features that can evolve in time. Imagine we have those 5 features (age, skin, language, height, weight) in 3 more different time steps, such as 1 year ago, 2 years ago, and 3 years ago (so 20 features in total), LSTM can exploit those time-dependent features and predict better. Of course all other machine learning models can be used with all those 20 features, but are not as good as RNN.\nHope this clarifies.", "votes": null }, { "id": "534346", "postDate": "05/21/2019 05:18:25", "content": "Cheers I was overthinking the problem, clarifies it well.
", "rawMarkdown": "Cheers I was overthinking the problem, clarifies it well.", "votes": null }, { "id": "534377", "postDate": "05/21/2019 06:37:29", "content": "I would generalize even further: LSTM (and RNN in general) are machine learning model specifically designed to capture patterns in data arranged in sequences, not necessarily time sequences. For instance, they are successfully used in text classification.
\n\nThey are not limited to classical classification/regression supervised learning either, as they can be used for text translation and generation.
", "rawMarkdown": "I would generalize even further: LSTM (and RNN in general) are machine learning model specifically designed to capture patterns in data arranged in *sequences*, not necessarily time sequences. For instance, they are successfully used in text classification.\n\nThey are not limited to classical classification/regression supervised learning either, as they can be used for text translation and generation.", "votes": null }, { "id": "534415", "postDate": "05/21/2019 08:00:16", "content": "https://www.kaggle.com/scirpus/unoriginal-lstm
", "rawMarkdown": "https://www.kaggle.com/scirpus/unoriginal-lstm", "votes": null }, { "id": "534429", "postDate": "05/21/2019 08:35:21", "content": "Thanks @scirpus very much appreciate the example!
", "rawMarkdown": "Thanks @scirpus very much appreciate the example!", "votes": null }, { "id": "534639", "postDate": "05/21/2019 15:39:04", "content": "planning on rewriting my LSTM from keras to pytorch. From what I understand, pytorch should run LSTMs faster. Anyone experienced with both see a speed up in pytorch on LSTM training time, in general? Thanks
", "rawMarkdown": "planning on rewriting my LSTM from keras to pytorch. From what I understand, pytorch should run LSTMs faster. Anyone experienced with both see a speed up in pytorch on LSTM training time, in general? Thanks", "votes": null }, { "id": "534783", "postDate": "05/21/2019 20:24:23", "content": "just use CuDNNLSTM in keras, it's much faster
", "rawMarkdown": "just use CuDNNLSTM in keras, it's much faster", "votes": null }, { "id": "534919", "postDate": "05/22/2019 03:54:31", "content": "why do you choose LSTM instead of GRU ? (I need to make this choice in my other project)
", "rawMarkdown": "why do you choose LSTM instead of GRU ? (I need to make this choice in my other project)", "votes": null }, { "id": "534947", "postDate": "05/22/2019 05:20:26", "content": "I do use CuDNNLSTM. From what I'm reading... the pytorch implementation is also CuDNN based so they should more or less take the same time to train. CuDNNGRU is not much faster than CuDNNLSTM in keras.
", "rawMarkdown": "I do use CuDNNLSTM. From what I'm reading... the pytorch implementation is also CuDNN based so they should more or less take the same time to train. CuDNNGRU is not much faster than CuDNNLSTM in keras.", "votes": null }, { "id": "535057", "postDate": "05/22/2019 08:08:43", "content": "@mchahhou - thanks for that I updated my LSTM kernel and the speed increased from 590s per epoch to 30s!! If I only learn this from the competition then I will be satisfied! ;)
", "rawMarkdown": "mchahhou - thanks for that I updated my LSTM kernel and the speed increased from 590s per epoch to 30s!! If I only learn this from the competition then I will be satisfied! ;)", "votes": null }, { "id": "535059", "postDate": "05/22/2019 08:18:15", "content": "@gideonvos - I updated it to use CUDNNLSTM which reduced run time from 30000s to just over 2000s
", "rawMarkdown": "gideonvos - I updated it to use CUDNNLSTM which reduced run time from 30000s to just over 2000s", "votes": null }, { "id": "535116", "postDate": "05/22/2019 10:32:05", "content": "Awesome, will check it out. My current R model just got me to #126 so still hanging onto it hoping it's not an over-fitting fluke.
", "rawMarkdown": "Awesome, will check it out. My current R model just got me to #126 so still hanging onto it hoping it's not an over-fitting fluke.", "votes": null }, { "id": "535123", "postDate": "05/22/2019 10:46:17", "content": "I doubt you will be alone. Once the competition ends I am sure you will have at least one that would be in the top 50! ;)
", "rawMarkdown": "I doubt you will be alone. Once the competition ends I am sure you will have at least one that would be in the top 50! ;)", "votes": null }, { "id": "535126", "postDate": "05/22/2019 10:47:36", "content": "I already use CuDNN and my training time per epoch is well over 10 minutes. I wish I could see that improvement on my end. Really limits the CV I can do.
", "rawMarkdown": "I already use CuDNN and my training time per epoch is well over 10 minutes. I wish I could see that improvement on my end. Really limits the CV I can do.", "votes": null }, { "id": "535384", "postDate": "05/22/2019 20:04:48", "content": "did you enable GPU in your kernel? Or maybe you are using LSTM on the raw data (150000 inputs)
", "rawMarkdown": "did you enable GPU in your kernel? Or maybe you are using LSTM on the raw data (150000 inputs)", "votes": null }, { "id": "535413", "postDate": "05/22/2019 21:20:36", "content": "If you use CuDNN without GPU enabled it throws error.
", "rawMarkdown": "If you use CuDNN without GPU enabled it throws error.", "votes": null }, { "id": "3194364", "postDate": "05/05/2025 18:39:05", "content": "Really needed some good tutorial for LSTM. This one so good thank you for this!!
", "rawMarkdown": "Really needed some good tutorial for LSTM. This one so good thank you for this!!", "votes": null } ], "comments": [ { "id": 3194364, "author_name": "prajwaljahagirdar", "author_url": "", "post_date": "05/05/2025 18:39:05", "content": "Really needed some good tutorial for LSTM. This one so good thank you for this!!
", "votes": null, "replies": [] }, { "id": 534208, "author_name": "gideonvos", "author_url": "", "post_date": "05/20/2019 23:26:02", "content": "So as a beginner I have to ask (with regards to LTSM for this problem). LTSM predicts value given time, but for this competition we have to predict time (ttf) given value (acoustic data). How do we treat this type of problem using traditional time series prediction? What am I missing here? It seems like a reverse of a typical time series prediction.
", "votes": null, "replies": [ { "id": 534302, "author_name": "khahuras", "author_url": "", "post_date": "05/21/2019 03:34:12", "content": "LSTM is just a kind of \"machine learning\" model. A machine learning model can predict a \"label\", from a set of labeled past observed samples. Each sample has multiple features. Such as predicting a person's nationality (label) from his 5 features: age, skin tone, language, height, and weight. In this competition, we have samples as separate segments of signal. Each segment has a label, which is the end ttf value of that segment. Each signal segment has multiple features, which can be extracted from itself, such as mean, variance, FFT decomposition coefficients ... \nNext, LSTM (or RNN in general) is designed especially for features that can evolve in time. Imagine we have those 5 features (age, skin, language, height, weight) in 3 more different time steps, such as 1 year ago, 2 years ago, and 3 years ago (so 20 features in total), LSTM can exploit those time-dependent features and predict better. Of course all other machine learning models can be used with all those 20 features, but are not as good as RNN.\nHope this clarifies.
", "votes": null, "replies": [] }, { "id": 534346, "author_name": "gideonvos", "author_url": "", "post_date": "05/21/2019 05:18:25", "content": "Cheers I was overthinking the problem, clarifies it well.
", "votes": null, "replies": [] }, { "id": 534377, "author_name": "stecasasso", "author_url": "", "post_date": "05/21/2019 06:37:29", "content": "I would generalize even further: LSTM (and RNN in general) are machine learning model specifically designed to capture patterns in data arranged in sequences, not necessarily time sequences. For instance, they are successfully used in text classification.
\n\nThey are not limited to classical classification/regression supervised learning either, as they can be used for text translation and generation.
", "votes": null, "replies": [] }, { "id": 534415, "author_name": "scirpus", "author_url": "", "post_date": "05/21/2019 08:00:16", "content": "https://www.kaggle.com/scirpus/unoriginal-lstm
", "votes": null, "replies": [] }, { "id": 534429, "author_name": "gideonvos", "author_url": "", "post_date": "05/21/2019 08:35:21", "content": "Thanks @scirpus very much appreciate the example!
", "votes": null, "replies": [] }, { "id": 535059, "author_name": "scirpus", "author_url": "", "post_date": "05/22/2019 08:18:15", "content": "@gideonvos - I updated it to use CUDNNLSTM which reduced run time from 30000s to just over 2000s
", "votes": null, "replies": [] }, { "id": 535116, "author_name": "gideonvos", "author_url": "", "post_date": "05/22/2019 10:32:05", "content": "Awesome, will check it out. My current R model just got me to #126 so still hanging onto it hoping it's not an over-fitting fluke.
", "votes": null, "replies": [] }, { "id": 535123, "author_name": "scirpus", "author_url": "", "post_date": "05/22/2019 10:46:17", "content": "I doubt you will be alone. Once the competition ends I am sure you will have at least one that would be in the top 50! ;)
", "votes": null, "replies": [] } ] }, { "id": 534639, "author_name": "teeyee314", "author_url": "", "post_date": "05/21/2019 15:39:04", "content": "planning on rewriting my LSTM from keras to pytorch. From what I understand, pytorch should run LSTMs faster. Anyone experienced with both see a speed up in pytorch on LSTM training time, in general? Thanks
", "votes": null, "replies": [ { "id": 534783, "author_name": "mchahhou", "author_url": "", "post_date": "05/21/2019 20:24:23", "content": "just use CuDNNLSTM in keras, it's much faster
", "votes": null, "replies": [] }, { "id": 534919, "author_name": "nyleve", "author_url": "", "post_date": "05/22/2019 03:54:31", "content": "why do you choose LSTM instead of GRU ? (I need to make this choice in my other project)
", "votes": null, "replies": [] }, { "id": 534947, "author_name": "teeyee314", "author_url": "", "post_date": "05/22/2019 05:20:26", "content": "I do use CuDNNLSTM. From what I'm reading... the pytorch implementation is also CuDNN based so they should more or less take the same time to train. CuDNNGRU is not much faster than CuDNNLSTM in keras.
", "votes": null, "replies": [] }, { "id": 535057, "author_name": "scirpus", "author_url": "", "post_date": "05/22/2019 08:08:43", "content": "@mchahhou - thanks for that I updated my LSTM kernel and the speed increased from 590s per epoch to 30s!! If I only learn this from the competition then I will be satisfied! ;)
", "votes": null, "replies": [] }, { "id": 535126, "author_name": "teeyee314", "author_url": "", "post_date": "05/22/2019 10:47:36", "content": "I already use CuDNN and my training time per epoch is well over 10 minutes. I wish I could see that improvement on my end. Really limits the CV I can do.
", "votes": null, "replies": [] }, { "id": 535384, "author_name": "mchahhou", "author_url": "", "post_date": "05/22/2019 20:04:48", "content": "did you enable GPU in your kernel? Or maybe you are using LSTM on the raw data (150000 inputs)
", "votes": null, "replies": [] }, { "id": 535413, "author_name": "teeyee314", "author_url": "", "post_date": "05/22/2019 21:20:36", "content": "If you use CuDNN without GPU enabled it throws error.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "533676": "I cannot recommend this tutorial enough\n\nhttps://www.kaggle.com/thebrownviking20/intro-to-recurrent-neural-networks-lstm-gru", "534208": "So as a beginner I have to ask (with regards to LTSM for this problem). LTSM predicts value given time, but for this competition we have to predict time (ttf) given value (acoustic data). How do we treat this type of problem using traditional time series prediction? What am I missing here? It seems like a reverse of a typical time series prediction.", "534302": "LSTM is just a kind of \"machine learning\" model. A machine learning model can predict a \"label\", from a set of labeled past observed samples. Each sample has multiple features. Such as predicting a person's nationality (label) from his 5 features: age, skin tone, language, height, and weight. In this competition, we have samples as separate segments of signal. Each segment has a label, which is the end ttf value of that segment. Each signal segment has multiple features, which can be extracted from itself, such as mean, variance, FFT decomposition coefficients ... \nNext, LSTM (or RNN in general) is designed especially for features that can evolve in time. Imagine we have those 5 features (age, skin, language, height, weight) in 3 more different time steps, such as 1 year ago, 2 years ago, and 3 years ago (so 20 features in total), LSTM can exploit those time-dependent features and predict better. Of course all other machine learning models can be used with all those 20 features, but are not as good as RNN.\nHope this clarifies.", "534346": "Cheers I was overthinking the problem, clarifies it well.", "534377": "I would generalize even further: LSTM (and RNN in general) are machine learning model specifically designed to capture patterns in data arranged in *sequences*, not necessarily time sequences. For instance, they are successfully used in text classification.\n\nThey are not limited to classical classification/regression supervised learning either, as they can be used for text translation and generation.", "534415": "https://www.kaggle.com/scirpus/unoriginal-lstm", "534429": "Thanks @scirpus very much appreciate the example!", "534639": "planning on rewriting my LSTM from keras to pytorch. From what I understand, pytorch should run LSTMs faster. Anyone experienced with both see a speed up in pytorch on LSTM training time, in general? Thanks", "534783": "just use CuDNNLSTM in keras, it's much faster", "534919": "why do you choose LSTM instead of GRU ? (I need to make this choice in my other project)", "534947": "I do use CuDNNLSTM. From what I'm reading... the pytorch implementation is also CuDNN based so they should more or less take the same time to train. CuDNNGRU is not much faster than CuDNNLSTM in keras.", "535057": "mchahhou - thanks for that I updated my LSTM kernel and the speed increased from 590s per epoch to 30s!! If I only learn this from the competition then I will be satisfied! ;)", "535059": "gideonvos - I updated it to use CUDNNLSTM which reduced run time from 30000s to just over 2000s", "535116": "Awesome, will check it out. My current R model just got me to #126 so still hanging onto it hoping it's not an over-fitting fluke.", "535123": "I doubt you will be alone. Once the competition ends I am sure you will have at least one that would be in the top 50! ;)", "535126": "I already use CuDNN and my training time per epoch is well over 10 minutes. I wish I could see that improvement on my end. Really limits the CV I can do.", "535384": "did you enable GPU in your kernel? Or maybe you are using LSTM on the raw data (150000 inputs)", "535413": "If you use CuDNN without GPU enabled it throws error.", "3194364": "Really needed some good tutorial for LSTM. This one so good thank you for this!!" }, "source": "meta" }