I added a kernel for showing how to train a Word2Vec model from train_active.csv and another one for showing how to use a self-trained model and to compare with using pre-trained Fasttext embeddings
\n\nBottom line is that self-trained embeddings using the text in train_active.csv might significantly improve your model.
", "messages": [ { "id": "328226", "postDate": "05/13/2018 18:13:06", "content": "I added a kernel for showing how to train a Word2Vec model from train_active.csv and another one for showing how to use a self-trained model and to compare with using pre-trained Fasttext embeddings
\n\nBottom line is that self-trained embeddings using the text in train_active.csv might significantly improve your model.
", "rawMarkdown": "I added a kernel for showing [how to train a Word2Vec model][1] from train_active.csv and another one for showing how to [use a self-trained model][2] and to compare with using [pre-trained Fasttext embeddings][3]\n\nBottom line is that self-trained embeddings using the text in train_active.csv might significantly improve your model. \n\n\n [1]: https://www.kaggle.com/christofhenkel/using-train-active-for-training-word-embeddings\n [2]: https://www.kaggle.com/christofhenkel/self-trained-embeddings-starter-only-description\n [3]: https://www.kaggle.com/christofhenkel/fasttext-starter-description-only", "votes": null }, { "id": "328334", "postDate": "05/14/2018 02:17:28", "content": "So there is one question. How to jointly train word embed with other features. It seems hard to combine NN with Lightgbm.
\n\nMaybe one should train a base NN without embedding and plugin the embeddings.
", "rawMarkdown": "So there is one question. How to jointly train word embed with other features. It seems hard to combine NN with Lightgbm.\n\nMaybe one should train a base NN without embedding and plugin the embeddings.", "votes": null }, { "id": "328406", "postDate": "05/14/2018 08:27:24", "content": "Thank you very much Dieter
\n\nSelf-trained embeddings gave me some improvement on CV ( I was using Fasttext trained on Wiki before )
", "rawMarkdown": "Thank you very much Dieter\n\nSelf-trained embeddings gave me some improvement on CV ( I was using Fasttext trained on Wiki before )", "votes": null }, { "id": "328568", "postDate": "05/14/2018 16:10:08", "content": "\n\n\nMaybe one should train a base NN without embedding and plugin the embeddings
\n
Sorry, I don´t get your idea. Could you explain a bit more?
", "rawMarkdown": "> Maybe one should train a base NN without embedding and plugin the embeddings\n\nSorry, I don´t get your idea. Could you explain a bit more?", "votes": null }, { "id": "328573", "postDate": "05/14/2018 16:24:48", "content": "The previous competition, which contains no deep learning features(image, text), shows NN is comparable to LGB based model.
\n\nSo training a NN model with origin features and add the embedding at some or other NN layer.\nA lot of things to try, but a good base NN model is the first step.
\n\nSorry for my English.
", "rawMarkdown": "The previous [competition](https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56262), which contains no deep learning features(image, text), shows NN is comparable to LGB based model.\n\nSo training a NN model with origin features and add the embedding at some or other NN layer.\nA lot of things to try, but a good base NN model is the first step.\n\nSorry for my English.", "votes": null }, { "id": "328902", "postDate": "05/15/2018 10:33:20", "content": "They are not \"comparable\"..
\n\nNN models are way stronger when the data include (lot of) text features and/or images ...(unless you 're ready to do lot of feature engeneering and tuning for LGB ^^)
", "rawMarkdown": "They are not \"comparable\"..\n\nNN models are way stronger when the data include (lot of) text features and/or images ...(unless you 're ready to do lot of feature engeneering and tuning for LGB ^^)", "votes": null }, { "id": "328931", "postDate": "05/15/2018 11:52:13", "content": "Congrats.
\nUnfortunately, It does not work for me :D
good, thank you very much.
", "rawMarkdown": "good, thank you very much.", "votes": null } ], "comments": [ { "id": 328334, "author_name": "liujilong", "author_url": "", "post_date": "05/14/2018 02:17:28", "content": "So there is one question. How to jointly train word embed with other features. It seems hard to combine NN with Lightgbm.
\n\nMaybe one should train a base NN without embedding and plugin the embeddings.
", "votes": null, "replies": [ { "id": 328568, "author_name": "christofhenkel", "author_url": "", "post_date": "05/14/2018 16:10:08", "content": "\n\n\nMaybe one should train a base NN without embedding and plugin the embeddings
\n
Sorry, I don´t get your idea. Could you explain a bit more?
", "votes": null, "replies": [] }, { "id": 328573, "author_name": "liujilong", "author_url": "", "post_date": "05/14/2018 16:24:48", "content": "The previous competition, which contains no deep learning features(image, text), shows NN is comparable to LGB based model.
\n\nSo training a NN model with origin features and add the embedding at some or other NN layer.\nA lot of things to try, but a good base NN model is the first step.
\n\nSorry for my English.
", "votes": null, "replies": [] }, { "id": 328902, "author_name": "serigne", "author_url": "", "post_date": "05/15/2018 10:33:20", "content": "They are not \"comparable\"..
\n\nNN models are way stronger when the data include (lot of) text features and/or images ...(unless you 're ready to do lot of feature engeneering and tuning for LGB ^^)
", "votes": null, "replies": [] } ] }, { "id": 328406, "author_name": "serigne", "author_url": "", "post_date": "05/14/2018 08:27:24", "content": "Thank you very much Dieter
\n\nSelf-trained embeddings gave me some improvement on CV ( I was using Fasttext trained on Wiki before )
", "votes": null, "replies": [ { "id": 328931, "author_name": "ngxbac", "author_url": "", "post_date": "05/15/2018 11:52:13", "content": "Congrats.
\nUnfortunately, It does not work for me :D
good, thank you very much.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "328226": "I added a kernel for showing [how to train a Word2Vec model][1] from train_active.csv and another one for showing how to [use a self-trained model][2] and to compare with using [pre-trained Fasttext embeddings][3]\n\nBottom line is that self-trained embeddings using the text in train_active.csv might significantly improve your model. \n\n\n [1]: https://www.kaggle.com/christofhenkel/using-train-active-for-training-word-embeddings\n [2]: https://www.kaggle.com/christofhenkel/self-trained-embeddings-starter-only-description\n [3]: https://www.kaggle.com/christofhenkel/fasttext-starter-description-only", "328334": "So there is one question. How to jointly train word embed with other features. It seems hard to combine NN with Lightgbm.\n\nMaybe one should train a base NN without embedding and plugin the embeddings.", "328406": "Thank you very much Dieter\n\nSelf-trained embeddings gave me some improvement on CV ( I was using Fasttext trained on Wiki before )", "328568": "> Maybe one should train a base NN without embedding and plugin the embeddings\n\nSorry, I don´t get your idea. Could you explain a bit more?", "328573": "The previous [competition](https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56262), which contains no deep learning features(image, text), shows NN is comparable to LGB based model.\n\nSo training a NN model with origin features and add the embedding at some or other NN layer.\nA lot of things to try, but a good base NN model is the first step.\n\nSorry for my English.", "328902": "They are not \"comparable\"..\n\nNN models are way stronger when the data include (lot of) text features and/or images ...(unless you 're ready to do lot of feature engeneering and tuning for LGB ^^)", "328931": "Congrats. \nUnfortunately, It does not work for me :D", "331308": "good, thank you very much." }, "source": "meta" }