{ "id": 475712, "title": "Why didn't Adaboost work well in this competition ? Any help or idea.", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/475712", "author_name": "", "post_date": "2024-02-09T13:31:04.910951700Z", "votes": 2, "comment_count": 5, "views": 0, "content": "

I submitted my code like following.
\nhttps://www.kaggle.com/code/takumimukaiyama/adaboost-baseline

\n

LB Score is 0.
\nBecause adaboost cannot predict probability well (predicted probabilities are under 10^(-30)).
\nWhy didn't Adaboost work well ?

\n

The LB Score is 0.548 for the LGBM baseline using the same data processing method and cross-validation like below.
\nhttps://www.kaggle.com/code/takumimukaiyama/lgbclassifier-baseline?scriptVersionId=162185116

\n

If you have any idea or knowledge, please write here !

", "messages": [ { "id": "2644433", "postDate": "02/09/2024 13:31:04", "content": "

I submitted my code like following.
\nhttps://www.kaggle.com/code/takumimukaiyama/adaboost-baseline

\n

LB Score is 0.
\nBecause adaboost cannot predict probability well (predicted probabilities are under 10^(-30)).
\nWhy didn't Adaboost work well ?

\n

The LB Score is 0.548 for the LGBM baseline using the same data processing method and cross-validation like below.
\nhttps://www.kaggle.com/code/takumimukaiyama/lgbclassifier-baseline?scriptVersionId=162185116

\n

If you have any idea or knowledge, please write here !

", "rawMarkdown": "I submitted my code like following.\nhttps://www.kaggle.com/code/takumimukaiyama/adaboost-baseline\n\nLB Score is 0.\nBecause adaboost cannot predict probability well (predicted probabilities are under 10^(-30)).\nWhy didn't Adaboost work well ? \n\nThe LB Score is 0.548 for the LGBM baseline using the same data processing method and cross-validation like below.\nhttps://www.kaggle.com/code/takumimukaiyama/lgbclassifier-baseline?scriptVersionId=162185116\n\n\nIf you have any idea or knowledge, please write here !", "votes": null }, { "id": "2644888", "postDate": "02/09/2024 18:34:03", "content": "

The link you provided throws a 404 error. Either it is not there, or you didn't make the notebook public.

\n

There are all kinds of reasons why a particular approach doesn't work. It can be a coding error, poor data preparation, and in some cases certain types of modeling approaches intrinsically work better or worse. Neural networks will generally do better with images, while gradient boosting machines will generally work with tabular data such as in this competition. Rather than being married to a particular approach, I think it is important to be flexible and willing to use with whatever is most appropriate. I don't know how well AdaBoost works with regard to this dataset, but generally speaking there are better boosting methods out there.

", "rawMarkdown": "The link you provided throws a 404 error. Either it is not there, or you didn't make the notebook public.\n\nThere are all kinds of reasons why a particular approach doesn't work. It can be a coding error, poor data preparation, and in some cases certain types of modeling approaches intrinsically work better or worse. Neural networks will generally do better with images, while gradient boosting machines will generally work with tabular data such as in this competition. Rather than being married to a particular approach, I think it is important to be flexible and willing to use with whatever is most appropriate. I don't know how well AdaBoost works with regard to this dataset, but generally speaking there are better boosting methods out there.", "votes": null }, { "id": "2645211", "postDate": "02/10/2024 04:21:17", "content": "

Thank you for your great tips.
\nI thought adaboost is not appropriate in this competition object.
\nI will try other methods!

", "rawMarkdown": "Thank you for your great tips.\nI thought adaboost is not appropriate in this competition object.\nI will try other methods!", "votes": null }, { "id": "2645767", "postDate": "02/10/2024 13:01:34", "content": "

gbt = gradient boosting trees
\ngbm = gradient-boosting machine

\n

isn't this a variant of adaboost?
\n(or they all belong the the boosting family)

", "rawMarkdown": "gbt = gradient boosting trees\ngbm = gradient-boosting machine\n\nisn't this a variant of adaboost?\n(or they all belong the the boosting family)", "votes": null }, { "id": "2649326", "postDate": "02/12/2024 18:54:55", "content": "
\n

isn't this a variant of adaboost?

\n
\n

All of them belong to the same general family of classifiers, yet that doesn't mean they are equally effective. Not many people I know would argue that sklearn's implementation of gradient boosting is as good as XGBoost or LightGBM, even though all three belong to the same group.

", "rawMarkdown": "> isn't this a variant of adaboost?\n\nAll of them belong to the same general family of classifiers, yet that doesn't mean they are equally effective. Not many people I know would argue that [**sklearn's implementation of gradient boosting**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html) is as good as XGBoost or LightGBM, even though all three belong to the same group.", "votes": null }, { "id": "2649485", "postDate": "02/12/2024 21:31:03", "content": "

I just quickly glanced at your notebook and noticed that you fillna with -9999 in X_train, but fillna with 0 in the X_test. There is a lot of missing data in this competition, so maybe more consistent fillna would help?

", "rawMarkdown": "I just quickly glanced at your notebook and noticed that you fillna with -9999 in X_train, but fillna with 0 in the X_test. There is a lot of missing data in this competition, so maybe more consistent fillna would help?", "votes": null } ], "comments": [ { "id": 2644888, "author_name": "tilii7", "author_url": "", "post_date": "02/09/2024 18:34:03", "content": "

The link you provided throws a 404 error. Either it is not there, or you didn't make the notebook public.

\n

There are all kinds of reasons why a particular approach doesn't work. It can be a coding error, poor data preparation, and in some cases certain types of modeling approaches intrinsically work better or worse. Neural networks will generally do better with images, while gradient boosting machines will generally work with tabular data such as in this competition. Rather than being married to a particular approach, I think it is important to be flexible and willing to use with whatever is most appropriate. I don't know how well AdaBoost works with regard to this dataset, but generally speaking there are better boosting methods out there.

", "votes": null, "replies": [ { "id": 2645211, "author_name": "takumimukaiyama", "author_url": "", "post_date": "02/10/2024 04:21:17", "content": "

Thank you for your great tips.
\nI thought adaboost is not appropriate in this competition object.
\nI will try other methods!

", "votes": null, "replies": [ { "id": 2645767, "author_name": "hengck23", "author_url": "", "post_date": "02/10/2024 13:01:34", "content": "

gbt = gradient boosting trees
\ngbm = gradient-boosting machine

\n

isn't this a variant of adaboost?
\n(or they all belong the the boosting family)

", "votes": null, "replies": [ { "id": 2649326, "author_name": "tilii7", "author_url": "", "post_date": "02/12/2024 18:54:55", "content": "
\n

isn't this a variant of adaboost?

\n
\n

All of them belong to the same general family of classifiers, yet that doesn't mean they are equally effective. Not many people I know would argue that sklearn's implementation of gradient boosting is as good as XGBoost or LightGBM, even though all three belong to the same group.

", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2649485, "author_name": "ryancaldwell", "author_url": "", "post_date": "02/12/2024 21:31:03", "content": "

I just quickly glanced at your notebook and noticed that you fillna with -9999 in X_train, but fillna with 0 in the X_test. There is a lot of missing data in this competition, so maybe more consistent fillna would help?

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2644433": "I submitted my code like following.\nhttps://www.kaggle.com/code/takumimukaiyama/adaboost-baseline\n\nLB Score is 0.\nBecause adaboost cannot predict probability well (predicted probabilities are under 10^(-30)).\nWhy didn't Adaboost work well ? \n\nThe LB Score is 0.548 for the LGBM baseline using the same data processing method and cross-validation like below.\nhttps://www.kaggle.com/code/takumimukaiyama/lgbclassifier-baseline?scriptVersionId=162185116\n\n\nIf you have any idea or knowledge, please write here !", "2644888": "The link you provided throws a 404 error. Either it is not there, or you didn't make the notebook public.\n\nThere are all kinds of reasons why a particular approach doesn't work. It can be a coding error, poor data preparation, and in some cases certain types of modeling approaches intrinsically work better or worse. Neural networks will generally do better with images, while gradient boosting machines will generally work with tabular data such as in this competition. Rather than being married to a particular approach, I think it is important to be flexible and willing to use with whatever is most appropriate. I don't know how well AdaBoost works with regard to this dataset, but generally speaking there are better boosting methods out there.", "2645211": "Thank you for your great tips.\nI thought adaboost is not appropriate in this competition object.\nI will try other methods!", "2645767": "gbt = gradient boosting trees\ngbm = gradient-boosting machine\n\nisn't this a variant of adaboost?\n(or they all belong the the boosting family)", "2649326": "> isn't this a variant of adaboost?\n\nAll of them belong to the same general family of classifiers, yet that doesn't mean they are equally effective. Not many people I know would argue that [**sklearn's implementation of gradient boosting**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html) is as good as XGBoost or LightGBM, even though all three belong to the same group.", "2649485": "I just quickly glanced at your notebook and noticed that you fillna with -9999 in X_train, but fillna with 0 in the X_test. There is a lot of missing data in this competition, so maybe more consistent fillna would help?" }, "source": "meta" }