Congrats and thank you all. It's a great competition. I leaned so much from all the great shared notebooks.
\nWhile from the notebooks and discussions there has some common points, more and more features or more model to ensemble.
\nI know that more features will be helpful for digging unseen feature.
\nBut why ensemble model is more precise not the opposite?
\n(Of course, the model to be ensembled is trained well)
It seems that the unstable predicted samples from one model will be corrected by other model not the opposite.
\nCan anyone explain this in depth?
\nThanks!
", "messages": [ { "id": "1916627", "postDate": "08/28/2022 03:36:34", "content": "Congrats and thank you all. It's a great competition. I leaned so much from all the great shared notebooks.
\nWhile from the notebooks and discussions there has some common points, more and more features or more model to ensemble.
\nI know that more features will be helpful for digging unseen feature.
\nBut why ensemble model is more precise not the opposite?
\n(Of course, the model to be ensembled is trained well)
It seems that the unstable predicted samples from one model will be corrected by other model not the opposite.
\nCan anyone explain this in depth?
\nThanks!
", "rawMarkdown": "Congrats and thank you all. It's a great competition. I leaned so much from all the great shared notebooks. \n\nWhile from the notebooks and discussions there has some common points, more and more features or more model to ensemble.\n\nI know that more features will be helpful for digging unseen feature. \n\n**But why ensemble model is more precise not the opposite?**\n(Of course, the model to be ensembled is trained well)\n\nIt seems that the unstable predicted samples from one model will be corrected by other model not the opposite.\n\nCan anyone explain this in depth?\n\nThanks!", "votes": null }, { "id": "1916648", "postDate": "08/28/2022 04:05:35", "content": "Awhile back, Carl linked to a great ensembling guide, it covers this question pretty well in the first section. https://www.kaggle.com/competitions/amex-default-prediction/discussion/332729
", "rawMarkdown": "Awhile back, Carl linked to a great ensembling guide, it covers this question pretty well in the first section. https://www.kaggle.com/competitions/amex-default-prediction/discussion/332729", "votes": null }, { "id": "1916688", "postDate": "08/28/2022 05:28:23", "content": "Great! Thank you very much.
", "rawMarkdown": "Great! Thank you very much.", "votes": null }, { "id": "1917410", "postDate": "08/28/2022 17:47:43", "content": "A simple explanation is that models that complement each other will make a good ensemble, rather than models that agree with each other. That's why diversity is always emphasized when creating ensembles.
\nA longer explanation is here, and beware that I am sending you to one of my previous posts.
", "rawMarkdown": "A simple explanation is that models that complement each other will make a good ensemble, rather than models that agree with each other. That's why diversity is always emphasized when creating ensembles.\n\nA longer explanation is [**here**](https://www.kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge/discussion/51058), and beware that I am sending you to one of my previous posts.", "votes": null }, { "id": "1917660", "postDate": "08/29/2022 00:47:14", "content": "If two models are very similar then if the first model has score 0.798 and the second model has score 0.796. Then the ensemble will result in the middle at 0.797.
\nWhen two models are different, then the result magically boosts both! If the first model has score 0.798 and the second model has score 0.796, then the ensemble will result in 0.799 or 0.800!
\nSuccessful ensemble is about achieving model diversity!
", "rawMarkdown": "If two models are very similar then if the first model has score 0.798 and the second model has score 0.796. Then the ensemble will result in the middle at 0.797.\n\nWhen two models are different, then the result magically boosts both! If the first model has score 0.798 and the second model has score 0.796, then the ensemble will result in 0.799 or 0.800!\n\nSuccessful ensemble is about achieving model diversity!", "votes": null }, { "id": "1920796", "postDate": "08/31/2022 12:15:08", "content": "In short, It extends the hypotheses space.
\nThere is a requirement that is the models are diverse and independent models.
\nSo the generalization ability will be promoted.
Think of any ensemble as a single random forest classifier with n_estimators=N, where N is the number of models you put into ensemble.
Many high profile Kagglers already answered (providing excelent insights) and, happily autoreferential, their answers complete each other, so that the ensemble of the answers has even better value than each of them separately.
", "rawMarkdown": "Many high profile Kagglers already answered (providing excelent insights) and, happily autoreferential, their answers complete each other, so that the ensemble of the answers has even better value than each of them separately.", "votes": null } ], "comments": [ { "id": 1916648, "author_name": "roberthatch", "author_url": "", "post_date": "08/28/2022 04:05:35", "content": "Awhile back, Carl linked to a great ensembling guide, it covers this question pretty well in the first section. https://www.kaggle.com/competitions/amex-default-prediction/discussion/332729
", "votes": null, "replies": [ { "id": 1916688, "author_name": "itisliukun", "author_url": "", "post_date": "08/28/2022 05:28:23", "content": "Great! Thank you very much.
", "votes": null, "replies": [] } ] }, { "id": 1917410, "author_name": "tilii7", "author_url": "", "post_date": "08/28/2022 17:47:43", "content": "A simple explanation is that models that complement each other will make a good ensemble, rather than models that agree with each other. That's why diversity is always emphasized when creating ensembles.
\nA longer explanation is here, and beware that I am sending you to one of my previous posts.
", "votes": null, "replies": [] }, { "id": 1917660, "author_name": "cdeotte", "author_url": "", "post_date": "08/29/2022 00:47:14", "content": "If two models are very similar then if the first model has score 0.798 and the second model has score 0.796. Then the ensemble will result in the middle at 0.797.
\nWhen two models are different, then the result magically boosts both! If the first model has score 0.798 and the second model has score 0.796, then the ensemble will result in 0.799 or 0.800!
\nSuccessful ensemble is about achieving model diversity!
", "votes": null, "replies": [] }, { "id": 1920796, "author_name": "zhehaoliang", "author_url": "", "post_date": "08/31/2022 12:15:08", "content": "In short, It extends the hypotheses space.
\nThere is a requirement that is the models are diverse and independent models.
\nSo the generalization ability will be promoted.
Think of any ensemble as a single random forest classifier with n_estimators=N, where N is the number of models you put into ensemble.
Many high profile Kagglers already answered (providing excelent insights) and, happily autoreferential, their answers complete each other, so that the ensemble of the answers has even better value than each of them separately.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1916627": "Congrats and thank you all. It's a great competition. I leaned so much from all the great shared notebooks. \n\nWhile from the notebooks and discussions there has some common points, more and more features or more model to ensemble.\n\nI know that more features will be helpful for digging unseen feature. \n\n**But why ensemble model is more precise not the opposite?**\n(Of course, the model to be ensembled is trained well)\n\nIt seems that the unstable predicted samples from one model will be corrected by other model not the opposite.\n\nCan anyone explain this in depth?\n\nThanks!", "1916648": "Awhile back, Carl linked to a great ensembling guide, it covers this question pretty well in the first section. https://www.kaggle.com/competitions/amex-default-prediction/discussion/332729", "1916688": "Great! Thank you very much.", "1917410": "A simple explanation is that models that complement each other will make a good ensemble, rather than models that agree with each other. That's why diversity is always emphasized when creating ensembles.\n\nA longer explanation is [**here**](https://www.kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge/discussion/51058), and beware that I am sending you to one of my previous posts.", "1917660": "If two models are very similar then if the first model has score 0.798 and the second model has score 0.796. Then the ensemble will result in the middle at 0.797.\n\nWhen two models are different, then the result magically boosts both! If the first model has score 0.798 and the second model has score 0.796, then the ensemble will result in 0.799 or 0.800!\n\nSuccessful ensemble is about achieving model diversity!", "1920796": "In short, It extends the hypotheses space.\nThere is a requirement that is the models are diverse and independent models.\nSo the generalization ability will be promoted.", "1920800": "Think of any ensemble as a single random forest classifier with `n_estimators=N`, where `N` is the number of models you put into ensemble.", "1922275": "Many high profile Kagglers already answered (providing excelent insights) and, happily autoreferential, their answers complete each other, so that the ensemble of the answers has even better value than each of them separately." }, "source": "meta" }