I was looking for how to use all training data provided in competition without having out of memory problem. Found out some features provided by LightGBM.
\nLightGBM offers two options for continue training:
\nRefit: Uses existing tree structures to fit the new data. It doesn't add new trees and keeps model structure of a trained model. Only updates leaf counts and leaf values with new data.
Train: with init_model parameter continued training will add new trees to the existing trained model with new data.
My notebook which I tried 'train' method with LightGBM, using all 10 train datasets here: https://www.kaggle.com/code/landfallmotto/tps-oct-22-continue-training-method-lightgbm
\nAlso XGboost offers a similar method to LightGBM train called XGBoost train with xgb_model parameter.
\nPlease comment any feedbacks, alternative methods and ideas.
", "messages": [ { "id": "1967773", "postDate": "10/02/2022 17:10:44", "content": "I was looking for how to use all training data provided in competition without having out of memory problem. Found out some features provided by LightGBM.
\nLightGBM offers two options for continue training:
\nRefit: Uses existing tree structures to fit the new data. It doesn't add new trees and keeps model structure of a trained model. Only updates leaf counts and leaf values with new data.
Train: with init_model parameter continued training will add new trees to the existing trained model with new data.
My notebook which I tried 'train' method with LightGBM, using all 10 train datasets here: https://www.kaggle.com/code/landfallmotto/tps-oct-22-continue-training-method-lightgbm
\nAlso XGboost offers a similar method to LightGBM train called XGBoost train with xgb_model parameter.
\nPlease comment any feedbacks, alternative methods and ideas.
", "rawMarkdown": "I was looking for how to use all training data provided in competition without having out of memory problem. Found out some features provided by LightGBM.\n\nLightGBM offers two options for continue training:\n\n- **Refit:** Uses existing tree structures to fit the new data. It doesn't add new trees and keeps model structure of a trained model. Only updates leaf counts and leaf values with new data. \n\n- **Train:** with init_model parameter continued training will add new trees to the existing trained model with new data. \n\n[LightGBM Train Doc link](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.train.html)\n\nMy notebook which I tried 'train' method with LightGBM, using all 10 train datasets here: [https://www.kaggle.com/code/landfallmotto/tps-oct-22-continue-training-method-lightgbm](https://www.kaggle.com/code/landfallmotto/tps-oct-22-continue-training-method-lightgbm)\n\n\nAlso XGboost offers a similar method to LightGBM train called XGBoost train with xgb_model parameter.\n\n- **xgb_model** (Optional[Union[str, PathLike, Booster, bytearray]]) – Xgb model to be loaded before training (allows training continuation).\n\n[XGBoost Learning API Doc link](https://xgboost.readthedocs.io/en/stable/python/python_api.html#module-xgboost.training)\n\n\nPlease comment any feedbacks, alternative methods and ideas.", "votes": null }, { "id": "1967802", "postDate": "10/02/2022 17:35:14", "content": "Great observations @landfallmotto, I think this will be extremely useful for people using batch ML models.
\nMany thanks!
Hi @landfallmotto Feel free to check out my notebooks this one and this one where we use boosting training for the gradient boosting. We manage to show that the results are as good as or better than using all the data at once. Thanks.
", "rawMarkdown": "Hi @landfallmotto Feel free to check out my notebooks [this one](https://www.kaggle.com/code/stautxie/tabular-playground-oct-2022-baseline) and [this one](https://www.kaggle.com/code/stautxie/tabular-playground-oct-2022-catboost) where we use boosting training for the gradient boosting. We manage to show that the results are as good as or better than using all the data at once. Thanks.", "votes": null }, { "id": "1969041", "postDate": "10/03/2022 10:10:29", "content": "This topic has covered my concept in machine learning keep it going bro and i will be glad if you will also check some of my notebooks.
", "rawMarkdown": "This topic has covered my concept in machine learning keep it going bro and i will be glad if you will also check some of my notebooks.", "votes": null }, { "id": "1971263", "postDate": "10/04/2022 13:55:14", "content": "Continues training, a nice topic for this month.
\nValid_0's binary_logloss is always the same 0.185145. No improvement with additional trainings?
", "rawMarkdown": "Continues training, a nice topic for this month.\n\nValid_0's binary_logloss is always the same 0.185145. No improvement with additional trainings?", "votes": null } ], "comments": [ { "id": 1967802, "author_name": "ravi20076", "author_url": "", "post_date": "10/02/2022 17:35:14", "content": "Great observations @landfallmotto, I think this will be extremely useful for people using batch ML models.
\nMany thanks!
Hi @landfallmotto Feel free to check out my notebooks this one and this one where we use boosting training for the gradient boosting. We manage to show that the results are as good as or better than using all the data at once. Thanks.
", "votes": null, "replies": [] }, { "id": 1969041, "author_name": "faisaljanjua0555", "author_url": "", "post_date": "10/03/2022 10:10:29", "content": "This topic has covered my concept in machine learning keep it going bro and i will be glad if you will also check some of my notebooks.
", "votes": null, "replies": [] }, { "id": 1971263, "author_name": "stevesimons", "author_url": "", "post_date": "10/04/2022 13:55:14", "content": "Continues training, a nice topic for this month.
\nValid_0's binary_logloss is always the same 0.185145. No improvement with additional trainings?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1967773": "I was looking for how to use all training data provided in competition without having out of memory problem. Found out some features provided by LightGBM.\n\nLightGBM offers two options for continue training:\n\n- **Refit:** Uses existing tree structures to fit the new data. It doesn't add new trees and keeps model structure of a trained model. Only updates leaf counts and leaf values with new data. \n\n- **Train:** with init_model parameter continued training will add new trees to the existing trained model with new data. \n\n[LightGBM Train Doc link](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.train.html)\n\nMy notebook which I tried 'train' method with LightGBM, using all 10 train datasets here: [https://www.kaggle.com/code/landfallmotto/tps-oct-22-continue-training-method-lightgbm](https://www.kaggle.com/code/landfallmotto/tps-oct-22-continue-training-method-lightgbm)\n\n\nAlso XGboost offers a similar method to LightGBM train called XGBoost train with xgb_model parameter.\n\n- **xgb_model** (Optional[Union[str, PathLike, Booster, bytearray]]) – Xgb model to be loaded before training (allows training continuation).\n\n[XGBoost Learning API Doc link](https://xgboost.readthedocs.io/en/stable/python/python_api.html#module-xgboost.training)\n\n\nPlease comment any feedbacks, alternative methods and ideas.", "1967802": "Great observations @landfallmotto, I think this will be extremely useful for people using batch ML models. \nMany thanks!", "1968116": "Hi @landfallmotto Feel free to check out my notebooks [this one](https://www.kaggle.com/code/stautxie/tabular-playground-oct-2022-baseline) and [this one](https://www.kaggle.com/code/stautxie/tabular-playground-oct-2022-catboost) where we use boosting training for the gradient boosting. We manage to show that the results are as good as or better than using all the data at once. Thanks.", "1969041": "This topic has covered my concept in machine learning keep it going bro and i will be glad if you will also check some of my notebooks.", "1971263": "Continues training, a nice topic for this month.\n\nValid_0's binary_logloss is always the same 0.185145. No improvement with additional trainings?" }, "source": "meta" }