If you find long posts boring then there is a TLDR below ;)
\nThis competition's discussion forum is not that active maybe because @hengck23 and @cdeotte are not participating and @cpmpml has decided to share everything after the competition..
\nHowever, I will try to share whatever I can to keep this competition's forum active.
So, most of us know that Ensembles will win us this competition. No one is going to use a single model as their final submission.
\nBut the question is which models should I pick for ensemble? I am sure many of us have this question and we just try to play around with different models until we get a good score.
\nAlthough, playing around and trying different combinations of models is fine but it becomes a headache as your number of models increase. You need some strategy….
Here is Tilii's post that helped me a lot to decide which models to pick for my ensemble.
\nOkay now as we have an idea of which models to use but how should I combine them???
\nShould I average them all? Should I do a weighted average? Should I do a voting ensemble?
I am not sure if this post will answer all your questions as I also have not gone through it fully, but it is a really good starting point.
\nIf I find something more useful I will update it here and if you have any great resources to share please share it in the comments.
TLDR: Please read the Topic's title xD
", "messages": [ { "id": "1298600", "postDate": "05/09/2021 03:55:25", "content": "If you find long posts boring then there is a TLDR below ;)
\nThis competition's discussion forum is not that active maybe because @hengck23 and @cdeotte are not participating and @cpmpml has decided to share everything after the competition..
\nHowever, I will try to share whatever I can to keep this competition's forum active.
So, most of us know that Ensembles will win us this competition. No one is going to use a single model as their final submission.
\nBut the question is which models should I pick for ensemble? I am sure many of us have this question and we just try to play around with different models until we get a good score.
\nAlthough, playing around and trying different combinations of models is fine but it becomes a headache as your number of models increase. You need some strategy….
Here is Tilii's post that helped me a lot to decide which models to pick for my ensemble.
\nOkay now as we have an idea of which models to use but how should I combine them???
\nShould I average them all? Should I do a weighted average? Should I do a voting ensemble?
I am not sure if this post will answer all your questions as I also have not gone through it fully, but it is a really good starting point.
\nIf I find something more useful I will update it here and if you have any great resources to share please share it in the comments.
TLDR: Please read the Topic's title xD
", "rawMarkdown": "If you find long posts boring then there is a TLDR below ;) \nThis competition's discussion forum is not that active maybe because @hengck23 and @cdeotte are not participating and @cpmpml has decided to share everything after the competition.. \nHowever, I will try to share whatever I can to keep this competition's forum active.\n \nSo, most of us know that Ensembles will win us this competition. No one is going to use a single model as their final submission. \nBut the question is which models should I pick for ensemble? I am sure many of us have this question and we just try to play around with different models until we get a good score. \nAlthough, playing around and trying different combinations of models is fine but it becomes a headache as your number of models increase. You need some strategy.... \n\nHere is [Tilii's post](https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/discussion/51058) that helped me a lot to decide which models to pick for my ensemble.\n\nOkay now as we have an idea of which models to use but how should I combine them???\nShould I average them all? Should I do a weighted average? Should I do a voting ensemble? \n\nI am not sure if [this post](https://mlwave.com/kaggle-ensembling-guide/) will answer all your questions as I also have not gone through it fully, but it is a really good starting point.\nIf I find something more useful I will update it here and if you have any great resources to share please share it in the comments.\n\nTLDR: Please read the Topic's title xD", "votes": null }, { "id": "1298606", "postDate": "05/09/2021 04:02:53", "content": "Early in a competition, i suggest that you focus on one model and make it as accurate as possible. Late in the competition, you can take your one model and change stuff to make other models for ensemble. Or you can team up and ensemble your model with other Kagglers.
", "rawMarkdown": "Early in a competition, i suggest that you focus on one model and make it as accurate as possible. Late in the competition, you can take your one model and change stuff to make other models for ensemble. Or you can team up and ensemble your model with other Kagglers.", "votes": null }, { "id": "1298850", "postDate": "05/09/2021 09:15:37", "content": "I'll give my favorite answer: trust your CV. Of course, this assumes that one can find a reliable CV setting which may not be easy here.
\nIf CV is not very reliable then I use the LB as yet either fold.
\nThen you can select models or ensembles based on a combination of CV and LB scores.
\nWhatever you chose, don't be greedy as it leads to overfitting. I said more about it in Some tips to avoid overfitting.
\nIMHO, the forum isn't very active because no one has shared a competitive public notebook. As a result, people who made significant progress see the value of what they have and they are less likely to share.
", "rawMarkdown": "I'll give my favorite answer: trust your CV. Of course, this assumes that one can find a reliable CV setting which may not be easy here.\n\nIf CV is not very reliable then I use the LB as yet either fold. \n\nThen you can select models or ensembles based on a combination of CV and LB scores.\n\nWhatever you chose, don't be greedy as it leads to overfitting. I said more about it in [Some tips to avoid overfitting](https://www.kaggle.com/c/lish-moa/discussion/196913).\n\nIMHO, the forum isn't very active because no one has shared a competitive public notebook. As a result, people who made significant progress see the value of what they have and they are less likely to share.", "votes": null } ], "comments": [ { "id": 1298606, "author_name": "cdeotte", "author_url": "", "post_date": "05/09/2021 04:02:53", "content": "Early in a competition, i suggest that you focus on one model and make it as accurate as possible. Late in the competition, you can take your one model and change stuff to make other models for ensemble. Or you can team up and ensemble your model with other Kagglers.
", "votes": null, "replies": [] }, { "id": 1298850, "author_name": "cpmpml", "author_url": "", "post_date": "05/09/2021 09:15:37", "content": "I'll give my favorite answer: trust your CV. Of course, this assumes that one can find a reliable CV setting which may not be easy here.
\nIf CV is not very reliable then I use the LB as yet either fold.
\nThen you can select models or ensembles based on a combination of CV and LB scores.
\nWhatever you chose, don't be greedy as it leads to overfitting. I said more about it in Some tips to avoid overfitting.
\nIMHO, the forum isn't very active because no one has shared a competitive public notebook. As a result, people who made significant progress see the value of what they have and they are less likely to share.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1298600": "If you find long posts boring then there is a TLDR below ;) \nThis competition's discussion forum is not that active maybe because @hengck23 and @cdeotte are not participating and @cpmpml has decided to share everything after the competition.. \nHowever, I will try to share whatever I can to keep this competition's forum active.\n \nSo, most of us know that Ensembles will win us this competition. No one is going to use a single model as their final submission. \nBut the question is which models should I pick for ensemble? I am sure many of us have this question and we just try to play around with different models until we get a good score. \nAlthough, playing around and trying different combinations of models is fine but it becomes a headache as your number of models increase. You need some strategy.... \n\nHere is [Tilii's post](https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/discussion/51058) that helped me a lot to decide which models to pick for my ensemble.\n\nOkay now as we have an idea of which models to use but how should I combine them???\nShould I average them all? Should I do a weighted average? Should I do a voting ensemble? \n\nI am not sure if [this post](https://mlwave.com/kaggle-ensembling-guide/) will answer all your questions as I also have not gone through it fully, but it is a really good starting point.\nIf I find something more useful I will update it here and if you have any great resources to share please share it in the comments.\n\nTLDR: Please read the Topic's title xD", "1298606": "Early in a competition, i suggest that you focus on one model and make it as accurate as possible. Late in the competition, you can take your one model and change stuff to make other models for ensemble. Or you can team up and ensemble your model with other Kagglers.", "1298850": "I'll give my favorite answer: trust your CV. Of course, this assumes that one can find a reliable CV setting which may not be easy here.\n\nIf CV is not very reliable then I use the LB as yet either fold. \n\nThen you can select models or ensembles based on a combination of CV and LB scores.\n\nWhatever you chose, don't be greedy as it leads to overfitting. I said more about it in [Some tips to avoid overfitting](https://www.kaggle.com/c/lish-moa/discussion/196913).\n\nIMHO, the forum isn't very active because no one has shared a competitive public notebook. As a result, people who made significant progress see the value of what they have and they are less likely to share." }, "source": "meta" }