After finishing the Tweet Sentiment Extraction competition (and learning a ton about NLP and TPUs), I have decided to enter this competition over the last few days of its timeline. This wasn't very wise of me and I should have entered earlier. That being said, here are the few things I have learned:
\n\nThat's it, time to do some computer vision now. ;)
", "messages": [ { "id": "900538", "postDate": "06/24/2020 21:58:29", "content": "After finishing the Tweet Sentiment Extraction competition (and learning a ton about NLP and TPUs), I have decided to enter this competition over the last few days of its timeline. This wasn't very wise of me and I should have entered earlier. That being said, here are the few things I have learned:
\n\nThat's it, time to do some computer vision now. ;)
", "rawMarkdown": "After finishing the [**Tweet Sentiment Extraction**](https://www.kaggle.com/c/tweet-sentiment-extraction) competition (and learning a ton about NLP and TPUs), I have decided to enter this competition over the last few days of its timeline. This wasn't very wise of me and I should have entered earlier. That being said, here are the few things I have learned: \n\n\n- There are many BERT language flavors: [FlauBERT](https://huggingface.co/transformers/model_doc/flaubert.html), [BERTurk](https://github.com/stefan-it/turkish-bert), [BETO](https://github.com/dccuchile/beto), and so on.\n- [**XLM-RoBERTa**](https://huggingface.co/transformers/model_doc/xlmroberta.html) performs very well on multi-lingual/cross-lingual datasets\n- Working with **TPUs** is (still) hard!\n- **Public LB validation** is a thing, especially when the test dataset is different to the train dataset and the public/private split is random (thus making LB probing viable)\n- **Post-processing** seems to be one of the winning strategies in a lot of top solutions. I should learn how to do it better and spot those patterns if any. \n\nThat's it, time to do some computer vision now. ;)", "votes": null }, { "id": "912468", "postDate": "07/02/2020 13:48:46", "content": "that doesnt sound like a few things. getting creative is a skill that one can learn w/ time, e.g. the 1st place's finetuning on the public LB. next step is to practice these bullet-points as the one thing I have learnt over the course of a few years of self-education is that learning theory w/ no practice is vain.
\n\nI have learnt to try many ideas as this is a very engineering discipline, like physics 😊
", "rawMarkdown": "that doesnt sound like a few things. getting creative is a skill that one can learn w/ time, e.g. the 1st place's finetuning on the public LB. next step is to practice these bullet-points as the one thing I have learnt over the course of a few years of self-education is that learning theory w/ no practice is vain.\n\nI have learnt to try many ideas as this is a very engineering discipline, like physics 😊", "votes": null } ], "comments": [ { "id": 912468, "author_name": "dronych", "author_url": "", "post_date": "07/02/2020 13:48:46", "content": "that doesnt sound like a few things. getting creative is a skill that one can learn w/ time, e.g. the 1st place's finetuning on the public LB. next step is to practice these bullet-points as the one thing I have learnt over the course of a few years of self-education is that learning theory w/ no practice is vain.
\n\nI have learnt to try many ideas as this is a very engineering discipline, like physics 😊
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "900538": "After finishing the [**Tweet Sentiment Extraction**](https://www.kaggle.com/c/tweet-sentiment-extraction) competition (and learning a ton about NLP and TPUs), I have decided to enter this competition over the last few days of its timeline. This wasn't very wise of me and I should have entered earlier. That being said, here are the few things I have learned: \n\n\n- There are many BERT language flavors: [FlauBERT](https://huggingface.co/transformers/model_doc/flaubert.html), [BERTurk](https://github.com/stefan-it/turkish-bert), [BETO](https://github.com/dccuchile/beto), and so on.\n- [**XLM-RoBERTa**](https://huggingface.co/transformers/model_doc/xlmroberta.html) performs very well on multi-lingual/cross-lingual datasets\n- Working with **TPUs** is (still) hard!\n- **Public LB validation** is a thing, especially when the test dataset is different to the train dataset and the public/private split is random (thus making LB probing viable)\n- **Post-processing** seems to be one of the winning strategies in a lot of top solutions. I should learn how to do it better and spot those patterns if any. \n\nThat's it, time to do some computer vision now. ;)", "912468": "that doesnt sound like a few things. getting creative is a skill that one can learn w/ time, e.g. the 1st place's finetuning on the public LB. next step is to practice these bullet-points as the one thing I have learnt over the course of a few years of self-education is that learning theory w/ no practice is vain.\n\nI have learnt to try many ideas as this is a very engineering discipline, like physics 😊" }, "source": "meta" }