As the competition concludes in one month, I'm starting to consider my approach for model selection and ensembling. As someone new to competitions, I'm wondering how to determine if a single model's LB score is high enough to start focusing on ensembling.
\nIn other words, what's the best single model LB score among other Kagglers?
\nAny insights would be highly appreciated!
", "messages": [ { "id": 2768039, "postDate": "2024-04-22T16:26:59.827Z", "content": "As the competition concludes in one month, I'm starting to consider my approach for model selection and ensembling. As someone new to competitions, I'm wondering how to determine if a single model's LB score is high enough to start focusing on ensembling.
\nIn other words, what's the best single model LB score among other Kagglers?
\nAny insights would be highly appreciated!
", "rawMarkdown": "As the competition concludes in one month, I'm starting to consider my approach for model selection and ensembling. As someone new to competitions, I'm wondering how to determine if a single model's LB score is high enough to start focusing on ensembling.\n\nIn other words, what's the best single model LB score among other Kagglers?\n \nAny insights would be highly appreciated!", "votes": 15 }, { "id": 2768178, "postDate": "2024-04-22T18:06:57.513Z", "content": "lgbm 5 fold:
\nlocal CV: 0.688, Public Lb: 0.584
CatBoost 5 folds:
\nCV: 0.689, LB: 0.592
Single Catboost: 0.571
\nSingle LGBM: 0.571
\nSingle Catboost + Single LGBM: 0.585
Catboost 5 fold:
\nPublic LB: 0.582
How to ensemble can get better score？ I had tried the share code to ensemble, but only got 0.585.
", "rawMarkdown": "Catboost 5 fold:\nPublic LB: 0.582\n\nHow to ensemble can get better score？ I had tried the share code to ensemble, but only got 0.585.", "votes": 1 }, { "id": 2768249, "postDate": "2024-04-22T18:37:08.083Z", "content": "lgbm 10 folds
\nCV: 0.698 (gini stability)
\nLB: 0.572
What about xgboost? Has anyone achieved a decent result?
", "rawMarkdown": "What about xgboost? Has anyone achieved a decent result?", "replies": [ { "id": 2770357, "postDate": "2024-04-23T19:49:10.490Z", "content": "I have not tried it, but judging from other public notebooks, it does not meet expectations.
", "rawMarkdown": "I have not tried it, but judging from other public notebooks, it does not meet expectations. ", "votes": 1 }, { "id": 2771536, "postDate": "2024-04-24T09:29:44.003Z", "content": "XGBoost gave me OOM error when I tried it (the amount of features probably didn't help). Maybe others have experienced the same thing, which is why nobody bother to publish a notebook with it.
", "rawMarkdown": "XGBoost gave me OOM error when I tried it (the amount of features probably didn't help). Maybe others have experienced the same thing, which is why nobody bother to publish a notebook with it.", "votes": 1, "replies": [ { "id": 2773572, "postDate": "2024-04-24T19:38:33.883Z", "content": "XGB goes OOM when training on several folds if the dataset is big, but there is always possible to train it externally if the hardware permits or to train one fold, save the model, then another fold and so on and at the end to load all the pretrained models into a list and predict with them
", "rawMarkdown": "XGB goes OOM when training on several folds if the dataset is big, but there is always possible to train it externally if the hardware permits or to train one fold, save the model, then another fold and so on and at the end to load all the pretrained models into a list and predict with them " } ] }, { "id": 2771555, "postDate": "2024-04-24T09:40:58.213Z", "content": "I have tried, but xgboost is worse than catboost and lgbm (with the same features set)
", "rawMarkdown": "I have tried, but xgboost is worse than catboost and lgbm (with the same features set)", "votes": 1, "replies": [ { "id": 2773562, "postDate": "2024-04-24T19:33:16.240Z", "content": "Depends how well tuned were the models too
", "rawMarkdown": "Depends how well tuned were the models too", "votes": 1 } ] }, { "id": 2771981, "postDate": "2024-04-24T13:33:40.473Z", "content": "Single XGB on full data (no folds) - LB 0.583
", "rawMarkdown": "Single XGB on full data (no folds) - LB 0.583", "votes": 3, "replies": [ { "id": 2772068, "postDate": "2024-04-24T14:11:35.603Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2773561, "postDate": "2024-04-24T19:31:07.883Z", "content": "Yes, but all these results doesn't say much because it all depends of the quantity and the quality of the data we train on. This score for example was made on ~850 features dataset, but again it depends what aggregations were used and so on.
", "rawMarkdown": "Yes, but all these results doesn't say much because it all depends of the quantity and the quality of the data we train on. This score for example was made on ~850 features dataset, but again it depends what aggregations were used and so on.", "votes": 1 }, { "id": 2773621, "postDate": "2024-04-24T20:01:49.653Z", "content": "Have you compared XGB with LGB on the same data?
", "rawMarkdown": "Have you compared XGB with LGB on the same data?" }, { "id": 2773675, "postDate": "2024-04-24T20:43:47.890Z", "content": "yes, XGB gave better results in my cases, but it depends on the data we a training on, how it was processed / aggregated
", "rawMarkdown": "yes, XGB gave better results in my cases, but it depends on the data we a training on, how it was processed / aggregated", "votes": 1 } ] } ] }, { "id": 2768693, "postDate": "2024-04-23T01:54:16.453Z", "content": "lgbm 5 folds
\nCV: 0.694, LB 0.574
catboost 5 folds
\ngini stab: 0.687309 LB 0.580
\nlgbm 5 folds
\ngini stab: 0.6944810 LB 0.575
lgbm 5 fold:
\nlocal CV: 0.688, Public Lb: 0.584
CatBoost 5 folds:
\nCV: 0.689, LB: 0.592
Single Catboost: 0.571
\nSingle LGBM: 0.571
\nSingle Catboost + Single LGBM: 0.585
Catboost 5 fold:
\nPublic LB: 0.582
How to ensemble can get better score？ I had tried the share code to ensemble, but only got 0.585.
", "votes": 1, "replies": [] }, { "id": 2768249, "author_name": "Dmitry Uarov", "author_url": "", "post_date": "2024-04-22T18:37:08.083000", "content": "lgbm 10 folds
\nCV: 0.698 (gini stability)
\nLB: 0.572
What about xgboost? Has anyone achieved a decent result?
", "votes": 0, "replies": [ { "id": 2770357, "author_name": "Andreas Bisiadis", "author_url": "", "post_date": "2024-04-23T19:49:10.490000", "content": "I have not tried it, but judging from other public notebooks, it does not meet expectations.
", "votes": 1, "replies": [] }, { "id": 2771536, "author_name": "Iqbal Syah Akbar", "author_url": "", "post_date": "2024-04-24T09:29:44.003000", "content": "XGBoost gave me OOM error when I tried it (the amount of features probably didn't help). Maybe others have experienced the same thing, which is why nobody bother to publish a notebook with it.
", "votes": 1, "replies": [ { "id": 2773572, "author_name": "Danu A.", "author_url": "", "post_date": "2024-04-24T19:38:33.883000", "content": "XGB goes OOM when training on several folds if the dataset is big, but there is always possible to train it externally if the hardware permits or to train one fold, save the model, then another fold and so on and at the end to load all the pretrained models into a list and predict with them
", "votes": 0, "replies": [] } ] }, { "id": 2771555, "author_name": "Ts", "author_url": "", "post_date": "2024-04-24T09:40:58.213000", "content": "I have tried, but xgboost is worse than catboost and lgbm (with the same features set)
", "votes": 1, "replies": [ { "id": 2773562, "author_name": "Danu A.", "author_url": "", "post_date": "2024-04-24T19:33:16.240000", "content": "Depends how well tuned were the models too
", "votes": 1, "replies": [] } ] }, { "id": 2771981, "author_name": "Danu A.", "author_url": "", "post_date": "2024-04-24T13:33:40.473000", "content": "Single XGB on full data (no folds) - LB 0.583
", "votes": 3, "replies": [ { "id": 2772068, "author_name": "", "author_url": "", "post_date": "2024-04-24T14:11:35.603000", "content": "", "votes": 0, "replies": [] }, { "id": 2773561, "author_name": "Danu A.", "author_url": "", "post_date": "2024-04-24T19:31:07.883000", "content": "Yes, but all these results doesn't say much because it all depends of the quantity and the quality of the data we train on. This score for example was made on ~850 features dataset, but again it depends what aggregations were used and so on.
", "votes": 1, "replies": [] }, { "id": 2773621, "author_name": "Evgeniia Grigoreva", "author_url": "", "post_date": "2024-04-24T20:01:49.653000", "content": "Have you compared XGB with LGB on the same data?
", "votes": 0, "replies": [] }, { "id": 2773675, "author_name": "Danu A.", "author_url": "", "post_date": "2024-04-24T20:43:47.890000", "content": "yes, XGB gave better results in my cases, but it depends on the data we a training on, how it was processed / aggregated
", "votes": 1, "replies": [] } ] } ] }, { "id": 2768693, "author_name": "", "author_url": "", "post_date": "2024-04-23T01:54:16.453000", "content": "lgbm 5 folds
\nCV: 0.694, LB 0.574
catboost 5 folds
\ngini stab: 0.687309 LB 0.580
\nlgbm 5 folds
\ngini stab: 0.6944810 LB 0.575