How to get 0.55+?Is there any base tricks we don't notice?
", "messages": [ { "id": 431980, "postDate": "2018-12-03T08:08:26.463Z", "content": "Attention, threshold selection and special loss for handling imbalanced data distribution may be helpful althogh i hadn't try it. The only thing i have tried in the training process is seeking for power from buddha and god, and its very helpful. May the force with you, young man.
", "rawMarkdown": "Attention, threshold selection and special loss for handling imbalanced data distribution may be helpful althogh i hadn't try it. The only thing i have tried in the training process is seeking for power from buddha and god, and its very helpful. May the force with you, young man.", "votes": 7, "replies": [ { "id": 432031, "postDate": "2018-12-03T09:34:07.003Z", "content": "You meant RNGesus? ;)
", "rawMarkdown": "You meant RNGesus? ;)", "votes": 2 }, { "id": 432122, "postDate": "2018-12-03T12:38:28.267Z", "content": "Yes, and training deeplearning models is just like playing hearthstone, lucky means everything;)
", "rawMarkdown": "Yes, and training deeplearning models is just like playing hearthstone, lucky means everything;)", "votes": 2 }, { "id": 432530, "postDate": "2018-12-04T02:15:06.810Z", "content": "Thanks a lot,I will try the your tricks
", "rawMarkdown": "Thanks a lot,I will try the your tricks" }, { "id": 432692, "postDate": "2018-12-04T07:42:04.393Z", "content": "actually, i have a master yoda on my table, but no luck today.\nmaybe i'll try Darth Vader tomorrow.
", "rawMarkdown": "actually, i have a master yoda on my table, but no luck today.\nmaybe i'll try Darth Vader tomorrow." }, { "id": 432715, "postDate": "2018-12-04T08:36:18.107Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 432738, "postDate": "2018-12-04T09:10:43.237Z", "content": "Thanks a lot ,I added attention layers to my baseline for single model,and get 0.05 updated .
", "rawMarkdown": "Thanks a lot ,I added attention layers to my baseline for single model,and get 0.05 updated ." }, { "id": 433024, "postDate": "2018-12-04T15:18:21.033Z", "content": "Hi \nCan you share how to add attention layers?\nThanks
", "rawMarkdown": "Hi \nCan you share how to add attention layers?\nThanks", "votes": 1 }, { "id": 434182, "postDate": "2018-12-06T02:50:09.417Z", "content": "Hi \nCan you share how to add attention layers?\nThanks
", "rawMarkdown": "Hi \nCan you share how to add attention layers?\nThanks" }, { "id": 451677, "postDate": "2019-01-07T13:26:52.097Z", "content": "\n\nFor one, congrats on your baseline kernel. Seems that many have picked it up and are doing really well with it. I have also benefitted from basing some of my code off of your repo so thanks!
\n\nI started a thread on how to implement attention and did not receive any replies, do you have any code anywhere or links where attention is implemented in PyTorch or it is explained clearly enough that I could try my own implementation? Most of the papers seem to involve complex RNN attention networks which are confusing. The idea seems simple enough, but I have yet to see a simple implementation.
\n\nGood luck in the competition!
", "rawMarkdown": "@spytensor\n\nFor one, congrats on your baseline kernel. Seems that many have picked it up and are doing really well with it. I have also benefitted from basing some of my code off of your repo so thanks!\n\nI started a thread on how to implement attention and did not receive any replies, do you have any code anywhere or links where attention is implemented in PyTorch or it is explained clearly enough that I could try my own implementation? Most of the papers seem to involve complex RNN attention networks which are confusing. The idea seems simple enough, but I have yet to see a simple implementation. \n\nGood luck in the competition!" }, { "id": 451972, "postDate": "2019-01-08T01:46:52.267Z", "content": "I refered to this paper “Attention is all your need” and repo attention-is-all-you-need-pytorch.But ,if you want to get better score ,please check the extra data and do data balancing.
", "rawMarkdown": "I refered to this paper “Attention is all your need” and repo [attention-is-all-you-need-pytorch](https://github.com/jadore801120/attention-is-all-you-need-pytorch).But ,if you want to get better score ,please check the extra data and do data balancing. ", "votes": 1 }, { "id": 452050, "postDate": "2019-01-08T05:39:11.983Z", "content": "Thanks for the quick response I'll check it out.
", "rawMarkdown": "Thanks for the quick response I'll check it out. " } ] }, { "id": 428822, "postDate": "2018-11-28T00:07:00.987Z", "content": "How to get 0.55+?Is there any base tricks we don't notice?
", "rawMarkdown": "How to get 0.55+?Is there any base tricks we don't notice?", "votes": 2 }, { "id": 428853, "postDate": "2018-11-28T01:18:48.400Z", "content": "真的搞不懂，为什么差距这么大，不知道，那些前几名的哪里有了大突破
", "rawMarkdown": "真的搞不懂，为什么差距这么大，不知道，那些前几名的哪里有了大突破", "votes": -7, "replies": [ { "id": 430494, "postDate": "2018-11-30T13:21:54.930Z", "content": "hhhhh
", "rawMarkdown": "hhhhh" }, { "id": 430884, "postDate": "2018-12-01T05:40:20.223Z", "content": "google translate -> I really don't understand why the gap is so big, I don't know, those big names have a big breakthrough.
", "rawMarkdown": "google translate -> I really don't understand why the gap is so big, I don't know, those big names have a big breakthrough." }, { "id": 431181, "postDate": "2018-12-01T18:40:28.837Z", "content": "\"those big names have a big breakthrough\" -> \"the top scoring team must have had a huge breakthrough somewhere\" :)
", "rawMarkdown": "\"those big names have a big breakthrough\" -> \"the top scoring team must have had a huge breakthrough somewhere\" :)" } ] }, { "id": 429302, "postDate": "2018-11-28T16:53:05.280Z", "content": "check the external data thread
", "rawMarkdown": "check the external data thread", "replies": [ { "id": 431483, "postDate": "2018-12-02T11:21:18.253Z", "content": "I use the external data which in the kaggle kernel, but not improve my result.
", "rawMarkdown": "I use the external data which in the kaggle kernel, but not improve my result. " } ] } ], "comments": [ { "id": 431980, "author_name": "Hogger", "author_url": "", "post_date": "2018-12-03T08:08:26.463000", "content": "Attention, threshold selection and special loss for handling imbalanced data distribution may be helpful althogh i hadn't try it. The only thing i have tried in the training process is seeking for power from buddha and god, and its very helpful. May the force with you, young man.
", "votes": 7, "replies": [ { "id": 432031, "author_name": "Kaggoo", "author_url": "", "post_date": "2018-12-03T09:34:07.003000", "content": "You meant RNGesus? ;)
", "votes": 2, "replies": [] }, { "id": 432122, "author_name": "Hogger", "author_url": "", "post_date": "2018-12-03T12:38:28.267000", "content": "Yes, and training deeplearning models is just like playing hearthstone, lucky means everything;)
", "votes": 2, "replies": [] }, { "id": 432530, "author_name": "Spytensor", "author_url": "", "post_date": "2018-12-04T02:15:06.810000", "content": "Thanks a lot,I will try the your tricks
", "votes": 0, "replies": [] }, { "id": 432692, "author_name": "irving", "author_url": "", "post_date": "2018-12-04T07:42:04.393000", "content": "actually, i have a master yoda on my table, but no luck today.\nmaybe i'll try Darth Vader tomorrow.
", "votes": 0, "replies": [] }, { "id": 432715, "author_name": "", "author_url": "", "post_date": "2018-12-04T08:36:18.107000", "content": "", "votes": -1, "replies": [] }, { "id": 432738, "author_name": "Spytensor", "author_url": "", "post_date": "2018-12-04T09:10:43.237000", "content": "Thanks a lot ,I added attention layers to my baseline for single model,and get 0.05 updated .
", "votes": 0, "replies": [] }, { "id": 433024, "author_name": "excllent123", "author_url": "", "post_date": "2018-12-04T15:18:21.033000", "content": "Hi \nCan you share how to add attention layers?\nThanks
", "votes": 1, "replies": [] }, { "id": 434182, "author_name": "Enzo", "author_url": "", "post_date": "2018-12-06T02:50:09.417000", "content": "Hi \nCan you share how to add attention layers?\nThanks
", "votes": 0, "replies": [] }, { "id": 451677, "author_name": "David Wagner", "author_url": "", "post_date": "2019-01-07T13:26:52.097000", "content": "\n\nFor one, congrats on your baseline kernel. Seems that many have picked it up and are doing really well with it. I have also benefitted from basing some of my code off of your repo so thanks!
\n\nI started a thread on how to implement attention and did not receive any replies, do you have any code anywhere or links where attention is implemented in PyTorch or it is explained clearly enough that I could try my own implementation? Most of the papers seem to involve complex RNN attention networks which are confusing. The idea seems simple enough, but I have yet to see a simple implementation.
\n\nGood luck in the competition!
", "votes": 0, "replies": [] }, { "id": 451972, "author_name": "Spytensor", "author_url": "", "post_date": "2019-01-08T01:46:52.267000", "content": "I refered to this paper “Attention is all your need” and repo attention-is-all-you-need-pytorch.But ,if you want to get better score ,please check the extra data and do data balancing.
", "votes": 1, "replies": [] }, { "id": 452050, "author_name": "David Wagner", "author_url": "", "post_date": "2019-01-08T05:39:11.983000", "content": "Thanks for the quick response I'll check it out.
", "votes": 0, "replies": [] } ] }, { "id": 428853, "author_name": "willer", "author_url": "", "post_date": "2018-11-28T01:18:48.400000", "content": "真的搞不懂，为什么差距这么大，不知道，那些前几名的哪里有了大突破
", "votes": -7, "replies": [ { "id": 430494, "author_name": "Kevin Zheng", "author_url": "", "post_date": "2018-11-30T13:21:54.930000", "content": "hhhhh
", "votes": 0, "replies": [] }, { "id": 430884, "author_name": "Dieter", "author_url": "", "post_date": "2018-12-01T05:40:20.223000", "content": "google translate -> I really don't understand why the gap is so big, I don't know, those big names have a big breakthrough.
", "votes": 0, "replies": [] }, { "id": 431181, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2018-12-01T18:40:28.837000", "content": "\"those big names have a big breakthrough\" -> \"the top scoring team must have had a huge breakthrough somewhere\" :)
", "votes": 0, "replies": [] } ] }, { "id": 429302, "author_name": "Zero", "author_url": "", "post_date": "2018-11-28T16:53:05.280000", "content": "check the external data thread
", "votes": 0, "replies": [ { "id": 431483, "author_name": "wchzh", "author_url": "", "post_date": "2018-12-02T11:21:18.253000", "content": "I use the external data which in the kaggle kernel, but not improve my result.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "431980": "Attention, threshold selection and special loss for handling imbalanced data distribution may be helpful althogh i hadn't try it. The only thing i have tried in the training process is seeking for power from buddha and god, and its very helpful. May the force with you, young man.", "428822": "How to get 0.55+?Is there any base tricks we don't notice?", "428853": "真的搞不懂，为什么差距这么大，不知道，那些前几名的哪里有了大突破", "429302": "check the external data thread" } }