{ "id": 219758, "title": "Is there a standard for determining the ensemble ratio?", "url": "/competitions/cassava-leaf-disease-classification/discussion/219758", "author_name": "", "post_date": "2021-02-16T09:34:55.627757200Z", "votes": 2, "comment_count": 4, "views": 0, "content": "

I implement three models and experimentally extract the best combination.

\n

However, it is difficult to make all attempts due to the limited number of submissions per day.

\n

Is there a standard for determining the ensemble ratio?

", "messages": [ { "id": "1204685", "postDate": "02/16/2021 09:34:55", "content": "

I implement three models and experimentally extract the best combination.

\n

However, it is difficult to make all attempts due to the limited number of submissions per day.

\n

Is there a standard for determining the ensemble ratio?

", "rawMarkdown": "I implement three models and experimentally extract the best combination.\n\nHowever, it is difficult to make all attempts due to the limited number of submissions per day.\n\nIs there a standard for determining the ensemble ratio?", "votes": null }, { "id": "1204789", "postDate": "02/16/2021 10:47:59", "content": "

If you have all the OOF predictions of the four models and every one of them uses the same folds (i.e. same seed in KFold), you can ensemble using the procedure explained here called hill climbing.

", "rawMarkdown": "If you have all the OOF predictions of the four models and every one of them uses the same folds (i.e. same seed in KFold), you can ensemble using the procedure explained [here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614) called `hill climbing`.", "votes": null }, { "id": "1205714", "postDate": "02/16/2021 22:55:20", "content": "

You could just train a blending model, which does the \"weighting\" for you see: https://medium.com/ml-research-lab/stacking-ensemble-meta-algorithms-for-improve-predictions-f4b4cf3b9237

", "rawMarkdown": "You could just train a blending model, which does the \"weighting\" for you see: https://medium.com/ml-research-lab/stacking-ensemble-meta-algorithms-for-improve-predictions-f4b4cf3b9237", "votes": null }, { "id": "1207592", "postDate": "02/18/2021 00:02:37", "content": "

Common practice is to train another blending model based on the CV predictions to determine the linear weights between each model. Then apply that weights to the final predictions.
\nOr, simple average between models sometimes work pretty good as well.

", "rawMarkdown": "Common practice is to train another blending model based on the CV predictions to determine the linear weights between each model. Then apply that weights to the final predictions.\nOr, simple average between models sometimes work pretty good as well.", "votes": null }, { "id": "1208252", "postDate": "02/18/2021 08:16:27", "content": "

François Chollet(Creator of Keras) given a way as :
\nA smarter way to ensemble classifiers is to do a weighted average, where the weights are learned on the validation data typically, the better classifiers are given a higher weight, and the worse classifiers are given a lower weight. To search for a good set of ensembling weights, you can use random search or a simple optimization algorithm such as Nelder-Mead:

\n
preds_a = model_a.predict(x_val)\npreds_b = model_b.predict(x_val)\npreds_c = model_c.predict(x_val)\npreds_d = model_d.predict(x_val)\n
\n

These weights (0.5, 0.25,0.1, 0.15) are assumed to be learned empirically.
\nfinal_preds = 0.5 * preds_a + 0.25 * preds_b + 0.1 * preds_c + 0.15 * preds_d

", "rawMarkdown": "François Chollet(Creator of Keras) given a way as : \nA smarter way to ensemble classifiers is to do a weighted average, where the weights are learned on the validation data typically, the better classifiers are given a higher weight, and the worse classifiers are given a lower weight. To search for a good set of ensembling weights, you can use random search or a simple optimization algorithm such as Nelder-Mead:\n\n```\npreds_a = model_a.predict(x_val)\npreds_b = model_b.predict(x_val)\npreds_c = model_c.predict(x_val)\npreds_d = model_d.predict(x_val)\n```\n\nThese weights (0.5, 0.25,0.1, 0.15) are assumed to be learned empirically.\n`final_preds = 0.5 * preds_a + 0.25 * preds_b + 0.1 * preds_c + 0.15 * preds_d`", "votes": null } ], "comments": [ { "id": 1204789, "author_name": "mviola", "author_url": "", "post_date": "02/16/2021 10:47:59", "content": "

If you have all the OOF predictions of the four models and every one of them uses the same folds (i.e. same seed in KFold), you can ensemble using the procedure explained here called hill climbing.

", "votes": null, "replies": [] }, { "id": 1205714, "author_name": "aliabdin1", "author_url": "", "post_date": "02/16/2021 22:55:20", "content": "

You could just train a blending model, which does the \"weighting\" for you see: https://medium.com/ml-research-lab/stacking-ensemble-meta-algorithms-for-improve-predictions-f4b4cf3b9237

", "votes": null, "replies": [] }, { "id": 1207592, "author_name": "louis925", "author_url": "", "post_date": "02/18/2021 00:02:37", "content": "

Common practice is to train another blending model based on the CV predictions to determine the linear weights between each model. Then apply that weights to the final predictions.
\nOr, simple average between models sometimes work pretty good as well.

", "votes": null, "replies": [] }, { "id": 1208252, "author_name": "vickygoyal", "author_url": "", "post_date": "02/18/2021 08:16:27", "content": "

François Chollet(Creator of Keras) given a way as :
\nA smarter way to ensemble classifiers is to do a weighted average, where the weights are learned on the validation data typically, the better classifiers are given a higher weight, and the worse classifiers are given a lower weight. To search for a good set of ensembling weights, you can use random search or a simple optimization algorithm such as Nelder-Mead:

\n
preds_a = model_a.predict(x_val)\npreds_b = model_b.predict(x_val)\npreds_c = model_c.predict(x_val)\npreds_d = model_d.predict(x_val)\n
\n

These weights (0.5, 0.25,0.1, 0.15) are assumed to be learned empirically.
\nfinal_preds = 0.5 * preds_a + 0.25 * preds_b + 0.1 * preds_c + 0.15 * preds_d

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1204685": "I implement three models and experimentally extract the best combination.\n\nHowever, it is difficult to make all attempts due to the limited number of submissions per day.\n\nIs there a standard for determining the ensemble ratio?", "1204789": "If you have all the OOF predictions of the four models and every one of them uses the same folds (i.e. same seed in KFold), you can ensemble using the procedure explained [here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614) called `hill climbing`.", "1205714": "You could just train a blending model, which does the \"weighting\" for you see: https://medium.com/ml-research-lab/stacking-ensemble-meta-algorithms-for-improve-predictions-f4b4cf3b9237", "1207592": "Common practice is to train another blending model based on the CV predictions to determine the linear weights between each model. Then apply that weights to the final predictions.\nOr, simple average between models sometimes work pretty good as well.", "1208252": "François Chollet(Creator of Keras) given a way as : \nA smarter way to ensemble classifiers is to do a weighted average, where the weights are learned on the validation data typically, the better classifiers are given a higher weight, and the worse classifiers are given a lower weight. To search for a good set of ensembling weights, you can use random search or a simple optimization algorithm such as Nelder-Mead:\n\n```\npreds_a = model_a.predict(x_val)\npreds_b = model_b.predict(x_val)\npreds_c = model_c.predict(x_val)\npreds_d = model_d.predict(x_val)\n```\n\nThese weights (0.5, 0.25,0.1, 0.15) are assumed to be learned empirically.\n`final_preds = 0.5 * preds_a + 0.25 * preds_b + 0.1 * preds_c + 0.15 * preds_d`" }, "source": "meta" }