Hi. Guys. This is my first kaggle competition. I'm using a lightGBM model on 40M train data. When I increase the size of the dataset, I don't know in which way i should change the hypterparamters in the model. Of course I can try Bayesian hypterparameter tuning, but it would be helpful if I can narrow the search space.
\n\nAny advice from your guys will be approciated. Thank you
", "messages": [ { "id": "322761", "postDate": "05/03/2018 15:54:21", "content": "Hi. Guys. This is my first kaggle competition. I'm using a lightGBM model on 40M train data. When I increase the size of the dataset, I don't know in which way i should change the hypterparamters in the model. Of course I can try Bayesian hypterparameter tuning, but it would be helpful if I can narrow the search space.
\n\nAny advice from your guys will be approciated. Thank you
", "rawMarkdown": "Hi. Guys. This is my first kaggle competition. I'm using a lightGBM model on 40M train data. When I increase the size of the dataset, I don't know in which way i should change the hypterparamters in the model. Of course I can try Bayesian hypterparameter tuning, but it would be helpful if I can narrow the search space. \n\nAny advice from your guys will be approciated. Thank you", "votes": null }, { "id": "322842", "postDate": "05/03/2018 18:46:07", "content": "Change one parameter and keep others constant and test its result . Repeat this for the parameters you want to change.
", "rawMarkdown": "Change one parameter and keep others constant and test its result . Repeat this for the parameters you want to change.", "votes": null }, { "id": "322844", "postDate": "05/03/2018 19:05:11", "content": "+start sooner +- 10 days before the due date, because it takes time.
", "rawMarkdown": "start sooner +- 10 days before the due date, because it takes time.", "votes": null }, { "id": "322901", "postDate": "05/03/2018 22:02:49", "content": "Hi HuaGo\nFor Lightgbm, earlty stopping is a good start. Then increase also the number of iteration
", "rawMarkdown": "Hi [HuaGo](https://www.kaggle.com/huaguo)\nFor Lightgbm, earlty stopping is a good start. Then increase also the number of iteration", "votes": null }, { "id": "323092", "postDate": "05/04/2018 11:13:24", "content": "I use BayesianOptimization to tune the hyperparameters, let it run for a day and take the best parameters found
", "rawMarkdown": "I use BayesianOptimization to tune the hyperparameters, let it run for a day and take the best parameters found", "votes": null }, { "id": "323101", "postDate": "05/04/2018 11:45:27", "content": "Hi HuaGuo for lightgbm the two most important parameters are \nearly_stopping_rounds\nput it at least to 50\nand number of iterations\nnum_boost_round\nput it to 1000 or even more if you run on your own server 2000.
", "rawMarkdown": "Hi [HuaGuo](https://www.kaggle.com/huaguo) for lightgbm the two most important parameters are \nearly_stopping_rounds\nput it at least to 50\nand number of iterations\nnum_boost_round\nput it to 1000 or even more if you run on your own server 2000.", "votes": null }, { "id": "323128", "postDate": "05/04/2018 12:41:02", "content": "I have some doubts on that . for example i got parameter x with value y as the best , but turns out - another value is better.
", "rawMarkdown": "I have some doubts on that . for example i got parameter x with value y as the best , but turns out - another value is better.", "votes": null }, { "id": "323248", "postDate": "05/04/2018 17:34:55", "content": "Bayesian Optimization is an algorithm in its own with hyperparameters of its own, i ran it with multiple configurations to find the best one, and still - i get weird behaviours from it.\nI use it partially as a RandomSearch algorithm, it just runs the model with lots of different parameters and i just took the best one it found within the time i gave it - maybe it needs a few dozens of runs to find a better configuration, but i don't have enough time for that =)
", "rawMarkdown": "Bayesian Optimization is an algorithm in its own with hyperparameters of its own, i ran it with multiple configurations to find the best one, and still - i get weird behaviours from it.\nI use it partially as a RandomSearch algorithm, it just runs the model with lots of different parameters and i just took the best one it found within the time i gave it - maybe it needs a few dozens of runs to find a better configuration, but i don't have enough time for that =)", "votes": null }, { "id": "323427", "postDate": "05/05/2018 06:00:06", "content": "Please see this thread
\n\nhttps://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56056
", "rawMarkdown": "Please see this thread\n\nhttps://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56056", "votes": null } ], "comments": [ { "id": 322842, "author_name": "mayanksoni", "author_url": "", "post_date": "05/03/2018 18:46:07", "content": "Change one parameter and keep others constant and test its result . Repeat this for the parameters you want to change.
", "votes": null, "replies": [ { "id": 322844, "author_name": "mrbeer", "author_url": "", "post_date": "05/03/2018 19:05:11", "content": "+start sooner +- 10 days before the due date, because it takes time.
", "votes": null, "replies": [] } ] }, { "id": 322901, "author_name": "ericbenhamou", "author_url": "", "post_date": "05/03/2018 22:02:49", "content": "Hi HuaGo\nFor Lightgbm, earlty stopping is a good start. Then increase also the number of iteration
", "votes": null, "replies": [] }, { "id": 323092, "author_name": "tpthegreat", "author_url": "", "post_date": "05/04/2018 11:13:24", "content": "I use BayesianOptimization to tune the hyperparameters, let it run for a day and take the best parameters found
", "votes": null, "replies": [ { "id": 323128, "author_name": "mayanksoni", "author_url": "", "post_date": "05/04/2018 12:41:02", "content": "I have some doubts on that . for example i got parameter x with value y as the best , but turns out - another value is better.
", "votes": null, "replies": [] }, { "id": 323248, "author_name": "tpthegreat", "author_url": "", "post_date": "05/04/2018 17:34:55", "content": "Bayesian Optimization is an algorithm in its own with hyperparameters of its own, i ran it with multiple configurations to find the best one, and still - i get weird behaviours from it.\nI use it partially as a RandomSearch algorithm, it just runs the model with lots of different parameters and i just took the best one it found within the time i gave it - maybe it needs a few dozens of runs to find a better configuration, but i don't have enough time for that =)
", "votes": null, "replies": [] } ] }, { "id": 323101, "author_name": "ericbenhamou", "author_url": "", "post_date": "05/04/2018 11:45:27", "content": "Hi HuaGuo for lightgbm the two most important parameters are \nearly_stopping_rounds\nput it at least to 50\nand number of iterations\nnum_boost_round\nput it to 1000 or even more if you run on your own server 2000.
", "votes": null, "replies": [] }, { "id": 323427, "author_name": "mpsampat", "author_url": "", "post_date": "05/05/2018 06:00:06", "content": "Please see this thread
\n\nhttps://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56056
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "322761": "Hi. Guys. This is my first kaggle competition. I'm using a lightGBM model on 40M train data. When I increase the size of the dataset, I don't know in which way i should change the hypterparamters in the model. Of course I can try Bayesian hypterparameter tuning, but it would be helpful if I can narrow the search space. \n\nAny advice from your guys will be approciated. Thank you", "322842": "Change one parameter and keep others constant and test its result . Repeat this for the parameters you want to change.", "322844": "start sooner +- 10 days before the due date, because it takes time.", "322901": "Hi [HuaGo](https://www.kaggle.com/huaguo)\nFor Lightgbm, earlty stopping is a good start. Then increase also the number of iteration", "323092": "I use BayesianOptimization to tune the hyperparameters, let it run for a day and take the best parameters found", "323101": "Hi [HuaGuo](https://www.kaggle.com/huaguo) for lightgbm the two most important parameters are \nearly_stopping_rounds\nput it at least to 50\nand number of iterations\nnum_boost_round\nput it to 1000 or even more if you run on your own server 2000.", "323128": "I have some doubts on that . for example i got parameter x with value y as the best , but turns out - another value is better.", "323248": "Bayesian Optimization is an algorithm in its own with hyperparameters of its own, i ran it with multiple configurations to find the best one, and still - i get weird behaviours from it.\nI use it partially as a RandomSearch algorithm, it just runs the model with lots of different parameters and i just took the best one it found within the time i gave it - maybe it needs a few dozens of runs to find a better configuration, but i don't have enough time for that =)", "323427": "Please see this thread\n\nhttps://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/56056" }, "source": "meta" }