I finetuned a BERT model outside of kaggle kernel on 30% of training data. Results were so impressive, I couldn't resist to test them. I did it with purely educational purpose, notebook describing it: https://www.kaggle.com/sergeykalutsky/introducing-bert-with-tensorflow?scriptVersionId=7346408\nI want to participate in the competition, but it's really annoying not to see my real place, because of artifitially high score. \nI would also argue, it would be great to add model weights and necessary dependencies to the competition. You don't need that much computational power or data to get good results from finetuning it. BERT has Apache 2.0 license as all of the given embeddings and not an official Google product.\nI apologize for inconveniences.
", "messages": [ { "id": "420402", "postDate": "11/13/2018 15:23:19", "content": "I finetuned a BERT model outside of kaggle kernel on 30% of training data. Results were so impressive, I couldn't resist to test them. I did it with purely educational purpose, notebook describing it: https://www.kaggle.com/sergeykalutsky/introducing-bert-with-tensorflow?scriptVersionId=7346408\nI want to participate in the competition, but it's really annoying not to see my real place, because of artifitially high score. \nI would also argue, it would be great to add model weights and necessary dependencies to the competition. You don't need that much computational power or data to get good results from finetuning it. BERT has Apache 2.0 license as all of the given embeddings and not an official Google product.\nI apologize for inconveniences.
", "rawMarkdown": "I finetuned a BERT model outside of kaggle kernel on 30% of training data. Results were so impressive, I couldn't resist to test them. I did it with purely educational purpose, notebook describing it: https://www.kaggle.com/sergeykalutsky/introducing-bert-with-tensorflow?scriptVersionId=7346408\nI want to participate in the competition, but it's really annoying not to see my real place, because of artifitially high score. \nI would also argue, it would be great to add model weights and necessary dependencies to the competition. You don't need that much computational power or data to get good results from finetuning it. BERT has Apache 2.0 license as all of the given embeddings and not an official Google product.\nI apologize for inconveniences.", "votes": null }, { "id": "420496", "postDate": "11/13/2018 18:07:33", "content": "How did you think bypassing the kernels constraints could be a positive thing for this competition and for the overwhelming high percentage of the community that follow the rules?
", "rawMarkdown": "How did you think bypassing the kernels constraints could be a positive thing for this competition and for the overwhelming high percentage of the community that follow the rules?", "votes": null }, { "id": "420498", "postDate": "11/13/2018 18:10:38", "content": "I know that was really foolish of me. I'm sorry, that was a mistake it won'r happen again.
", "rawMarkdown": "I know that was really foolish of me. I'm sorry, that was a mistake it won'r happen again.", "votes": null }, { "id": "420507", "postDate": "11/13/2018 18:39:07", "content": "Hi Sergey -
\n\nI appreciate the very informative kernel about BERT. I also appreciate you being up front about this. We almost never, though, make administrative leaderboard updates during a competition (for a number of reasons). Since the competition is still early and there's still plenty of upward mobility, I recommend working on improving your score to beat your current position. In a worst case scenario, the situation will sort itself out during stage 2.
", "rawMarkdown": "Hi Sergey -\n\nI appreciate the very informative kernel about BERT. I also appreciate you being up front about this. We almost never, though, make administrative leaderboard updates during a competition (for a number of reasons). Since the competition is still early and there's still plenty of upward mobility, I recommend working on improving your score to beat your current position. In a worst case scenario, the situation will sort itself out during stage 2.", "votes": null }, { "id": "420511", "postDate": "11/13/2018 18:43:23", "content": "Thanks for the answer! I appreciate the understanding, and I've learned my lesson.
", "rawMarkdown": "Thanks for the answer! I appreciate the understanding, and I've learned my lesson.", "votes": null } ], "comments": [ { "id": 420496, "author_name": "inversion", "author_url": "", "post_date": "11/13/2018 18:07:33", "content": "How did you think bypassing the kernels constraints could be a positive thing for this competition and for the overwhelming high percentage of the community that follow the rules?
", "votes": null, "replies": [ { "id": 420498, "author_name": "sergeykalutsky", "author_url": "", "post_date": "11/13/2018 18:10:38", "content": "I know that was really foolish of me. I'm sorry, that was a mistake it won'r happen again.
", "votes": null, "replies": [] } ] }, { "id": 420507, "author_name": "inversion", "author_url": "", "post_date": "11/13/2018 18:39:07", "content": "Hi Sergey -
\n\nI appreciate the very informative kernel about BERT. I also appreciate you being up front about this. We almost never, though, make administrative leaderboard updates during a competition (for a number of reasons). Since the competition is still early and there's still plenty of upward mobility, I recommend working on improving your score to beat your current position. In a worst case scenario, the situation will sort itself out during stage 2.
", "votes": null, "replies": [ { "id": 420511, "author_name": "sergeykalutsky", "author_url": "", "post_date": "11/13/2018 18:43:23", "content": "Thanks for the answer! I appreciate the understanding, and I've learned my lesson.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "420402": "I finetuned a BERT model outside of kaggle kernel on 30% of training data. Results were so impressive, I couldn't resist to test them. I did it with purely educational purpose, notebook describing it: https://www.kaggle.com/sergeykalutsky/introducing-bert-with-tensorflow?scriptVersionId=7346408\nI want to participate in the competition, but it's really annoying not to see my real place, because of artifitially high score. \nI would also argue, it would be great to add model weights and necessary dependencies to the competition. You don't need that much computational power or data to get good results from finetuning it. BERT has Apache 2.0 license as all of the given embeddings and not an official Google product.\nI apologize for inconveniences.", "420496": "How did you think bypassing the kernels constraints could be a positive thing for this competition and for the overwhelming high percentage of the community that follow the rules?", "420498": "I know that was really foolish of me. I'm sorry, that was a mistake it won'r happen again.", "420507": "Hi Sergey -\n\nI appreciate the very informative kernel about BERT. I also appreciate you being up front about this. We almost never, though, make administrative leaderboard updates during a competition (for a number of reasons). Since the competition is still early and there's still plenty of upward mobility, I recommend working on improving your score to beat your current position. In a worst case scenario, the situation will sort itself out during stage 2.", "420511": "Thanks for the answer! I appreciate the understanding, and I've learned my lesson." }, "source": "meta" }