Forgive me for my ignorance, but I am trying to understand if an attention-based neural network adaptation would work for this type of sequence data as well. It's obviously effective in NLP, but I wonder why there is little talk about this approach in this setting. Would anybody be able to shed light on this?
", "messages": [ { "id": "524876", "postDate": "04/29/2019 17:25:05", "content": "Forgive me for my ignorance, but I am trying to understand if an attention-based neural network adaptation would work for this type of sequence data as well. It's obviously effective in NLP, but I wonder why there is little talk about this approach in this setting. Would anybody be able to shed light on this?
", "rawMarkdown": "Forgive me for my ignorance, but I am trying to understand if an attention-based neural network adaptation would work for this type of sequence data as well. It's obviously effective in NLP, but I wonder why there is little talk about this approach in this setting. Would anybody be able to shed light on this?", "votes": null }, { "id": "524963", "postDate": "04/29/2019 21:20:32", "content": "My opinion: in this very noisy data, overall statistics are important, rather than fine details at particular locations. Attention is supposed to be good at finding the latter.\nYou never know for sure if you don't try it, though.
", "rawMarkdown": "My opinion: in this very noisy data, overall statistics are important, rather than fine details at particular locations. Attention is supposed to be good at finding the latter.\nYou never know for sure if you don't try it, though.", "votes": null }, { "id": "524967", "postDate": "04/29/2019 21:38:44", "content": "Thank you 😄
", "rawMarkdown": "Thank you 😄", "votes": null }, { "id": "525195", "postDate": "04/30/2019 11:20:11", "content": "Since train data is shuffled (not continuous), i think Attention mechanisms won't do the trick.\nBut I tried one anyway during experiments with my simple NN - it didn't improve the performance, as I expected, but also it didn't make things much worse. All changes was about 1.53 +- 0.01 on LB.
", "rawMarkdown": "Since train data is shuffled (not continuous), i think Attention mechanisms won't do the trick.\nBut I tried one anyway during experiments with my simple NN - it didn't improve the performance, as I expected, but also it didn't make things much worse. All changes was about 1.53 +- 0.01 on LB.", "votes": null }, { "id": "525384", "postDate": "04/30/2019 21:46:12", "content": "Thanks for your reply. Do you happen to have the code publicly available somewhere? I'd be happy to take a look if you did 😄 Thank you regardless.
", "rawMarkdown": "Thanks for your reply. Do you happen to have the code publicly available somewhere? I'd be happy to take a look if you did 😄 Thank you regardless.", "votes": null }, { "id": "525394", "postDate": "04/30/2019 21:58:28", "content": "Sorry, but no, it's buried in 9000 versions of my private kernel
", "rawMarkdown": "Sorry, but no, it's buried in 9000 versions of my private kernel", "votes": null }, { "id": "525421", "postDate": "04/30/2019 23:40:45", "content": "No worries, thanks and good luck!
", "rawMarkdown": "No worries, thanks and good luck!", "votes": null }, { "id": "525543", "postDate": "05/01/2019 08:49:46", "content": "Train data is NOT shuffled. It is continuous.
", "rawMarkdown": "Train data is NOT shuffled. It is continuous.", "votes": null } ], "comments": [ { "id": 524963, "author_name": "redstr", "author_url": "", "post_date": "04/29/2019 21:20:32", "content": "My opinion: in this very noisy data, overall statistics are important, rather than fine details at particular locations. Attention is supposed to be good at finding the latter.\nYou never know for sure if you don't try it, though.
", "votes": null, "replies": [ { "id": 524967, "author_name": "simonplovyt", "author_url": "", "post_date": "04/29/2019 21:38:44", "content": "Thank you 😄
", "votes": null, "replies": [] } ] }, { "id": 525195, "author_name": "stanislavblinov", "author_url": "", "post_date": "04/30/2019 11:20:11", "content": "Since train data is shuffled (not continuous), i think Attention mechanisms won't do the trick.\nBut I tried one anyway during experiments with my simple NN - it didn't improve the performance, as I expected, but also it didn't make things much worse. All changes was about 1.53 +- 0.01 on LB.
", "votes": null, "replies": [ { "id": 525384, "author_name": "simonplovyt", "author_url": "", "post_date": "04/30/2019 21:46:12", "content": "Thanks for your reply. Do you happen to have the code publicly available somewhere? I'd be happy to take a look if you did 😄 Thank you regardless.
", "votes": null, "replies": [] }, { "id": 525394, "author_name": "stanislavblinov", "author_url": "", "post_date": "04/30/2019 21:58:28", "content": "Sorry, but no, it's buried in 9000 versions of my private kernel
", "votes": null, "replies": [] }, { "id": 525421, "author_name": "simonplovyt", "author_url": "", "post_date": "04/30/2019 23:40:45", "content": "No worries, thanks and good luck!
", "votes": null, "replies": [] }, { "id": 525543, "author_name": "mhviraf", "author_url": "", "post_date": "05/01/2019 08:49:46", "content": "Train data is NOT shuffled. It is continuous.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "524876": "Forgive me for my ignorance, but I am trying to understand if an attention-based neural network adaptation would work for this type of sequence data as well. It's obviously effective in NLP, but I wonder why there is little talk about this approach in this setting. Would anybody be able to shed light on this?", "524963": "My opinion: in this very noisy data, overall statistics are important, rather than fine details at particular locations. Attention is supposed to be good at finding the latter.\nYou never know for sure if you don't try it, though.", "524967": "Thank you 😄", "525195": "Since train data is shuffled (not continuous), i think Attention mechanisms won't do the trick.\nBut I tried one anyway during experiments with my simple NN - it didn't improve the performance, as I expected, but also it didn't make things much worse. All changes was about 1.53 +- 0.01 on LB.", "525384": "Thanks for your reply. Do you happen to have the code publicly available somewhere? I'd be happy to take a look if you did 😄 Thank you regardless.", "525394": "Sorry, but no, it's buried in 9000 versions of my private kernel", "525421": "No worries, thanks and good luck!", "525543": "Train data is NOT shuffled. It is continuous." }, "source": "meta" }