{ "id": 220659, "title": "My RNN finally catches up with LGB", "url": "/competitions/indoor-location-navigation/discussion/220659", "author_name": "Jiwei Liu", "post_date": "2021-02-19T05:51:11.059000", "votes": 63, "comment_count": 18, "views": 0, "content": "

RNN LB: 8.34
\nLGB LB: 8.33 (public benchmark)
\nAverage (RNN+LGB)/2 LB: 7.71

\n

Update:
\nRNN LB: 7.6
\nPrevious (RNN+LGB)/2 LB: 7.71
\nAverage LB: 7.1

\n

Update2:
\nRNN LB: 6.99
\nPrevious blend LB: 7.1
\nAverage LB: 6.86

\n

Update3:
\nRNN LB: 6.55
\nBlending with LGB (public benchmark) doesn't improve anymore.

\n

How is your deep learning model doing?

", "messages": [ { "id": 1209942, "postDate": "2021-02-19T05:51:11.060Z", "content": "

RNN LB: 8.34
\nLGB LB: 8.33 (public benchmark)
\nAverage (RNN+LGB)/2 LB: 7.71

\n

Update:
\nRNN LB: 7.6
\nPrevious (RNN+LGB)/2 LB: 7.71
\nAverage LB: 7.1

\n

Update2:
\nRNN LB: 6.99
\nPrevious blend LB: 7.1
\nAverage LB: 6.86

\n

Update3:
\nRNN LB: 6.55
\nBlending with LGB (public benchmark) doesn't improve anymore.

\n

How is your deep learning model doing?

", "rawMarkdown": "RNN LB: 8.34\nLGB LB: 8.33 ([public benchmark](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold))\nAverage (RNN+LGB)/2 LB: 7.71\n\nUpdate:\nRNN LB: 7.6\nPrevious (RNN+LGB)/2 LB: 7.71\nAverage LB: 7.1\n\nUpdate2:\nRNN LB: 6.99\nPrevious blend LB: 7.1\nAverage LB: 6.86\n\nUpdate3:\nRNN LB: 6.55\nBlending with LGB ([public benchmark](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold)) doesn't improve anymore.\n\nHow is your deep learning model doing?", "votes": 63 }, { "id": 1231330, "postDate": "2021-03-08T22:02:23.530Z", "content": "

Now I applied Transformer to my model and it works.

", "rawMarkdown": "Now I applied Transformer to my model and it works. ", "votes": 10, "replies": [ { "id": 1231491, "postDate": "2021-03-09T02:58:49.877Z", "content": "

Great! Does it outperform RNN?

", "rawMarkdown": "Great! Does it outperform RNN?", "votes": 3 }, { "id": 1231512, "postDate": "2021-03-09T03:27:01.750Z", "content": "

Yes!
\nThe LB rose around 6.8 to 6.3.
\nStill struggling with breaking 6 though.

", "rawMarkdown": "Yes!\nThe LB rose around 6.8 to 6.3. \nStill struggling with breaking 6 though. ", "votes": 6 }, { "id": 1231713, "postDate": "2021-03-09T07:20:27.217Z", "content": "

My Transformer outperforms LSTM too
\nStill not submitted though

", "rawMarkdown": "My Transformer outperforms LSTM too\nStill not submitted though ", "votes": 2 }, { "id": 1261558, "postDate": "2021-04-03T07:07:12.510Z", "content": "

Hi folks, what kind of architecture do you use for transformers? My implementation cannot catch up with LSTM. I've never used transformers before. Can someone give me an example?

", "rawMarkdown": "Hi folks, what kind of architecture do you use for transformers? My implementation cannot catch up with LSTM. I've never used transformers before. Can someone give me an example?", "votes": 3 } ] }, { "id": 1210597, "postDate": "2021-02-19T14:44:23.863Z", "content": "

if it can be done with LSTM, then it can be done by the transformer.
\nnow is still early, I will try this after other competitions in mid-march.

", "rawMarkdown": "if it can be done with LSTM, then it can be done by the transformer.\nnow is still early, I will try this after other competitions in mid-march.", "votes": 7 }, { "id": 1215024, "postDate": "2021-02-23T09:23:49.320Z", "content": "

Sounds like a good competition to join!

", "rawMarkdown": "Sounds like a good competition to join!", "votes": 6 }, { "id": 1210019, "postDate": "2021-02-19T06:51:47.960Z", "content": "

Great work! I will try nn models now.

", "rawMarkdown": "Great work! I will try nn models now.", "votes": 3 }, { "id": 1254803, "postDate": "2021-03-28T06:32:57.720Z", "content": "

I am trying to use RNN in my model for prediction but my training data is having unequal number of time stamps so i have to pass them one by one ,also i cant pass anything for validation data again due to unequal number of time stamps in each instance .Can anyone help how to approach this ?

", "rawMarkdown": "I am trying to use RNN in my model for prediction but my training data is having unequal number of time stamps so i have to pass them one by one ,also i cant pass anything for validation data again due to unequal number of time stamps in each instance .Can anyone help how to approach this ?", "votes": 1, "replies": [ { "id": 1262781, "postDate": "2021-04-04T17:55:30.863Z", "content": "

find the sequence/time appropiate dimension

", "rawMarkdown": "find the sequence/time appropiate dimension" } ] }, { "id": 1219951, "postDate": "2021-02-27T11:35:32.797Z", "content": "

Thanks for sharing your update info, great work! Really helpful

", "rawMarkdown": "Thanks for sharing your update info, great work! Really helpful", "votes": 1 }, { "id": 1210664, "postDate": "2021-02-19T15:37:09.513Z", "content": "

Congratulations!

", "rawMarkdown": "Congratulations! ", "votes": 1 }, { "id": 1229946, "postDate": "2021-03-07T17:51:54.700Z", "content": "

Awesome. Great job Jiwei !

", "rawMarkdown": "Awesome. Great job Jiwei !", "votes": 2, "replies": [ { "id": 1231517, "postDate": "2021-03-09T03:36:34.163Z", "content": "

Thank you! 😄

", "rawMarkdown": "Thank you! 😄", "votes": 1 } ] }, { "id": 1220429, "postDate": "2021-02-28T01:37:01.403Z", "content": "

Thanks for sharing your update info, great work!
\nDoes the RNN part only use wifi data? like lgb.

", "rawMarkdown": "Thanks for sharing your update info, great work! \nDoes the RNN part only use wifi data? like lgb.", "votes": 2, "replies": [ { "id": 1220619, "postDate": "2021-02-28T07:08:20.750Z", "content": "

I am using all the data. :)

", "rawMarkdown": "I am using all the data. :)", "votes": 2 } ] }, { "id": 1219621, "postDate": "2021-02-27T02:38:13.537Z", "content": "

It is fantastic RNN~=lgb

", "rawMarkdown": "It is fantastic RNN~=lgb", "votes": 2 }, { "id": 1244186, "postDate": "2021-03-18T18:44:26.423Z", "rawMarkdown": "", "votes": 4, "isDeleted": true } ], "comments": [ { "id": 1231330, "author_name": "Kouki", "author_url": "", "post_date": "2021-03-08T22:02:23.530000", "content": "

Now I applied Transformer to my model and it works.

", "votes": 10, "replies": [ { "id": 1231491, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2021-03-09T02:58:49.877000", "content": "

Great! Does it outperform RNN?

", "votes": 3, "replies": [] }, { "id": 1231512, "author_name": "Kouki", "author_url": "", "post_date": "2021-03-09T03:27:01.750000", "content": "

Yes!
\nThe LB rose around 6.8 to 6.3.
\nStill struggling with breaking 6 though.

", "votes": 6, "replies": [] }, { "id": 1231713, "author_name": "Serigne ", "author_url": "", "post_date": "2021-03-09T07:20:27.217000", "content": "

My Transformer outperforms LSTM too
\nStill not submitted though

", "votes": 2, "replies": [] }, { "id": 1261558, "author_name": "Anton Enns", "author_url": "", "post_date": "2021-04-03T07:07:12.510000", "content": "

Hi folks, what kind of architecture do you use for transformers? My implementation cannot catch up with LSTM. I've never used transformers before. Can someone give me an example?

", "votes": 3, "replies": [] } ] }, { "id": 1210597, "author_name": "hengck23", "author_url": "", "post_date": "2021-02-19T14:44:23.863000", "content": "

if it can be done with LSTM, then it can be done by the transformer.
\nnow is still early, I will try this after other competitions in mid-march.

", "votes": 7, "replies": [] }, { "id": 1215024, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2021-02-23T09:23:49.320000", "content": "

Sounds like a good competition to join!

", "votes": 6, "replies": [] }, { "id": 1210019, "author_name": "Ethan", "author_url": "", "post_date": "2021-02-19T06:51:47.960000", "content": "

Great work! I will try nn models now.

", "votes": 3, "replies": [] }, { "id": 1254803, "author_name": "Devansh Chowdhury", "author_url": "", "post_date": "2021-03-28T06:32:57.720000", "content": "

I am trying to use RNN in my model for prediction but my training data is having unequal number of time stamps so i have to pass them one by one ,also i cant pass anything for validation data again due to unequal number of time stamps in each instance .Can anyone help how to approach this ?

", "votes": 1, "replies": [ { "id": 1262781, "author_name": "EnricRovira", "author_url": "", "post_date": "2021-04-04T17:55:30.863000", "content": "

find the sequence/time appropiate dimension

", "votes": 0, "replies": [] } ] }, { "id": 1219951, "author_name": "GuanLong Lyu", "author_url": "", "post_date": "2021-02-27T11:35:32.797000", "content": "

Thanks for sharing your update info, great work! Really helpful

", "votes": 1, "replies": [] }, { "id": 1210664, "author_name": "Dongkyu Kim", "author_url": "", "post_date": "2021-02-19T15:37:09.513000", "content": "

Congratulations!

", "votes": 1, "replies": [] }, { "id": 1229946, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-03-07T17:51:54.700000", "content": "

Awesome. Great job Jiwei !

", "votes": 2, "replies": [ { "id": 1231517, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2021-03-09T03:36:34.163000", "content": "

Thank you! 😄

", "votes": 1, "replies": [] } ] }, { "id": 1220429, "author_name": "Chunyu Wei", "author_url": "", "post_date": "2021-02-28T01:37:01.403000", "content": "

Thanks for sharing your update info, great work!
\nDoes the RNN part only use wifi data? like lgb.

", "votes": 2, "replies": [ { "id": 1220619, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2021-02-28T07:08:20.750000", "content": "

I am using all the data. :)

", "votes": 2, "replies": [] } ] }, { "id": 1219621, "author_name": "Yoshiyasu", "author_url": "", "post_date": "2021-02-27T02:38:13.537000", "content": "

It is fantastic RNN~=lgb

", "votes": 2, "replies": [] }, { "id": 1244186, "author_name": "", "author_url": "", "post_date": "2021-03-18T18:44:26.423000", "content": "", "votes": 4, "replies": [] } ], "raw_markdown_by_id": { "1209942": "RNN LB: 8.34\nLGB LB: 8.33 ([public benchmark](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold))\nAverage (RNN+LGB)/2 LB: 7.71\n\nUpdate:\nRNN LB: 7.6\nPrevious (RNN+LGB)/2 LB: 7.71\nAverage LB: 7.1\n\nUpdate2:\nRNN LB: 6.99\nPrevious blend LB: 7.1\nAverage LB: 6.86\n\nUpdate3:\nRNN LB: 6.55\nBlending with LGB ([public benchmark](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold)) doesn't improve anymore.\n\nHow is your deep learning model doing?", "1231330": "Now I applied Transformer to my model and it works. ", "1210597": "if it can be done with LSTM, then it can be done by the transformer.\nnow is still early, I will try this after other competitions in mid-march.", "1215024": "Sounds like a good competition to join!", "1210019": "Great work! I will try nn models now.", "1254803": "I am trying to use RNN in my model for prediction but my training data is having unequal number of time stamps so i have to pass them one by one ,also i cant pass anything for validation data again due to unequal number of time stamps in each instance .Can anyone help how to approach this ?", "1219951": "Thanks for sharing your update info, great work! Really helpful", "1210664": "Congratulations! ", "1229946": "Awesome. Great job Jiwei !", "1220429": "Thanks for sharing your update info, great work! \nDoes the RNN part only use wifi data? like lgb.", "1219621": "It is fantastic RNN~=lgb", "1244186": "" } }