Here is my solution. Simple one directional GRU, with some effective preprocessing for OOV words. No spell correction or anything. I have used 5 fold cross validation with exponential moving average ( which is not necessary ). Code is bit clumsy and keras lovers, don't find it happy as it is in pure tensorflow :-) . If any doubts, please do comment. Hope it will be useful.
\n\nhttps://www.kaggle.com/s4sarath/cudnngru-best-final
\n\n1.) I have preprocessed by word basis. Normally, I will split by space and check if a word is present in the vocab. If present, I will add it to my train embedding else I will preprocess only that word and cache it to a dictionary. For eg: goood might not be in glove or fasttext, then I will preprocess (a lot of sub and small functions are there) will give me good which is in glove. So, I will add it to mapping_dict = {goood : good}. Then, next time goood is appearing, I dont have to preprocess it, I will first look at mapping_dict and so on. My final vocab size was ```192000
2.) I have used GRU one side, with 256 dimensional embeddings. Cross validation of 5 fold, where each folds have different epochs, I used [5,5,5,4,4]. Used Adam Optimizer for minimzing the objective.
3.) I have used meta embeddings Glove+Paragram as per @shijuan's Kernel
4.) Used tensorflow,CudnnGRU, it is very fast and max len is 150 for a sentence.
Pretty much that's it.
", "messages": [ { "id": "471169", "postDate": "02/14/2019 05:29:56", "content": "Here is my solution. Simple one directional GRU, with some effective preprocessing for OOV words. No spell correction or anything. I have used 5 fold cross validation with exponential moving average ( which is not necessary ). Code is bit clumsy and keras lovers, don't find it happy as it is in pure tensorflow :-) . If any doubts, please do comment. Hope it will be useful.
\n\nhttps://www.kaggle.com/s4sarath/cudnngru-best-final
\n\n1.) I have preprocessed by word basis. Normally, I will split by space and check if a word is present in the vocab. If present, I will add it to my train embedding else I will preprocess only that word and cache it to a dictionary. For eg: goood might not be in glove or fasttext, then I will preprocess (a lot of sub and small functions are there) will give me good which is in glove. So, I will add it to mapping_dict = {goood : good}. Then, next time goood is appearing, I dont have to preprocess it, I will first look at mapping_dict and so on. My final vocab size was ```192000
2.) I have used GRU one side, with 256 dimensional embeddings. Cross validation of 5 fold, where each folds have different epochs, I used [5,5,5,4,4]. Used Adam Optimizer for minimzing the objective.
3.) I have used meta embeddings Glove+Paragram as per @shijuan's Kernel
4.) Used tensorflow,CudnnGRU, it is very fast and max len is 150 for a sentence.
Pretty much that's it.
", "rawMarkdown": "Here is my solution. Simple one directional GRU, with some effective preprocessing for OOV words. No spell correction or anything. I have used 5 fold cross validation with exponential moving average ( which is not necessary ). Code is bit clumsy and keras lovers, don't find it happy as it is in pure tensorflow :-) . If any doubts, please do comment. Hope it will be useful.\n\nhttps://www.kaggle.com/s4sarath/cudnngru-best-final\n\n1.) I have ```preprocessed by word basis```. Normally, I will split by space and check if a word is present in the vocab. If present, I will add it to my train embedding else I will preprocess only that word and cache it to a dictionary. For eg: ```goood``` might not be in glove or fasttext, then I will preprocess (a lot of sub and small functions are there) will give me ```good``` which is in glove. So, I will add it to mapping_dict = {```goood``` : ```good```}. Then, next time ```goood``` is appearing, I dont have to preprocess it, I will first look at ```mapping_dict``` and so on. My final vocab size was ```192000\n\n2.) I have used ```GRU``` one side, with ```256``` dimensional embeddings. ```Cross validation``` of ```5``` fold, where each folds have different epochs, I used ```[5,5,5,4,4]```. Used ```Adam Optimizer``` for minimzing the objective.\n\n3.) I have used ```meta embeddings``` ```Glove+Paragram``` as per ```@shijuan's``` Kernel\n\n4.) Used tensorflow,``` CudnnGRU```, it is very fast and max len is ```150 for a sentence. ```\n\nPretty much that's it.", "votes": null }, { "id": "471183", "postDate": "02/14/2019 06:01:20", "content": "Why it should be downvoted?
", "rawMarkdown": "Why it should be downvoted?", "votes": null }, { "id": "471193", "postDate": "02/14/2019 06:14:18", "content": "??
", "rawMarkdown": "??", "votes": null }, { "id": "471200", "postDate": "02/14/2019 06:20:33", "content": "I think people downvoted from frustration after shakeup. Do not strain about this. You did a great job. Thank you for sharing.
\n\nBut formally, of course, you could write more about how you came to your result:)
", "rawMarkdown": "I think people downvoted from frustration after shakeup. Do not strain about this. You did a great job. Thank you for sharing.\n\nBut formally, of course, you could write more about how you came to your result:)", "votes": null }, { "id": "471477", "postDate": "02/14/2019 14:15:00", "content": "Hey Sarath, I just know the result!\nCongratulation for your great performance here!
", "rawMarkdown": "Hey Sarath, I just know the result!\nCongratulation for your great performance here!", "votes": null }, { "id": "471856", "postDate": "02/15/2019 01:44:33", "content": "Thanks @Neuron Engineer. Glad that you also did well. I had a great time learning new things.
", "rawMarkdown": "Thanks @Neuron Engineer. Glad that you also did well. I had a great time learning new things.", "votes": null } ], "comments": [ { "id": 471183, "author_name": "s4sarath", "author_url": "", "post_date": "02/14/2019 06:01:20", "content": "Why it should be downvoted?
", "votes": null, "replies": [ { "id": 471200, "author_name": "rooshroosh", "author_url": "", "post_date": "02/14/2019 06:20:33", "content": "I think people downvoted from frustration after shakeup. Do not strain about this. You did a great job. Thank you for sharing.
\n\nBut formally, of course, you could write more about how you came to your result:)
", "votes": null, "replies": [] } ] }, { "id": 471193, "author_name": "s4sarath", "author_url": "", "post_date": "02/14/2019 06:14:18", "content": "??
", "votes": null, "replies": [] }, { "id": 471477, "author_name": "ratthachat", "author_url": "", "post_date": "02/14/2019 14:15:00", "content": "Hey Sarath, I just know the result!\nCongratulation for your great performance here!
", "votes": null, "replies": [ { "id": 471856, "author_name": "s4sarath", "author_url": "", "post_date": "02/15/2019 01:44:33", "content": "Thanks @Neuron Engineer. Glad that you also did well. I had a great time learning new things.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "471169": "Here is my solution. Simple one directional GRU, with some effective preprocessing for OOV words. No spell correction or anything. I have used 5 fold cross validation with exponential moving average ( which is not necessary ). Code is bit clumsy and keras lovers, don't find it happy as it is in pure tensorflow :-) . If any doubts, please do comment. Hope it will be useful.\n\nhttps://www.kaggle.com/s4sarath/cudnngru-best-final\n\n1.) I have ```preprocessed by word basis```. Normally, I will split by space and check if a word is present in the vocab. If present, I will add it to my train embedding else I will preprocess only that word and cache it to a dictionary. For eg: ```goood``` might not be in glove or fasttext, then I will preprocess (a lot of sub and small functions are there) will give me ```good``` which is in glove. So, I will add it to mapping_dict = {```goood``` : ```good```}. Then, next time ```goood``` is appearing, I dont have to preprocess it, I will first look at ```mapping_dict``` and so on. My final vocab size was ```192000\n\n2.) I have used ```GRU``` one side, with ```256``` dimensional embeddings. ```Cross validation``` of ```5``` fold, where each folds have different epochs, I used ```[5,5,5,4,4]```. Used ```Adam Optimizer``` for minimzing the objective.\n\n3.) I have used ```meta embeddings``` ```Glove+Paragram``` as per ```@shijuan's``` Kernel\n\n4.) Used tensorflow,``` CudnnGRU```, it is very fast and max len is ```150 for a sentence. ```\n\nPretty much that's it.", "471183": "Why it should be downvoted?", "471193": "??", "471200": "I think people downvoted from frustration after shakeup. Do not strain about this. You did a great job. Thank you for sharing.\n\nBut formally, of course, you could write more about how you came to your result:)", "471477": "Hey Sarath, I just know the result!\nCongratulation for your great performance here!", "471856": "Thanks @Neuron Engineer. Glad that you also did well. I had a great time learning new things." }, "source": "meta" }