Hi, Kaggler, I am tring the LightGBM to train model for this competition.\nBut the CPU version is very slow. I try to use the GPU version of LightGBM, \nBut the lightgbm told me that the’bin size cannot run on GPU’.\nIs there anyone use lightGBM with GPU successful?\nHow can I speed up the training speed?\nThank you !
", "messages": [ { "id": "301818", "postDate": "03/23/2018 09:56:57", "content": "Hi, Kaggler, I am tring the LightGBM to train model for this competition.\nBut the CPU version is very slow. I try to use the GPU version of LightGBM, \nBut the lightgbm told me that the’bin size cannot run on GPU’.\nIs there anyone use lightGBM with GPU successful?\nHow can I speed up the training speed?\nThank you !
", "rawMarkdown": "Hi, Kaggler, I am tring the LightGBM to train model for this competition.\nBut the CPU version is very slow. I try to use the GPU version of LightGBM, \nBut the lightgbm told me that the’bin size cannot run on GPU’.\nIs there anyone use lightGBM with GPU successful?\nHow can I speed up the training speed?\nThank you !", "votes": null }, { "id": "301830", "postDate": "03/23/2018 10:33:45", "content": "You might wanna look into this, it is related to bin size error.
", "rawMarkdown": "You might wanna look into [this][1], it is related to bin size error.\n\n\n [1]: https://www.kaggle.com/c/instacart-market-basket-analysis/discussion/37836#300119", "votes": null }, { "id": "301836", "postDate": "03/23/2018 10:46:05", "content": "I often refer to their github issues page for parameters tuning and troubleshooting. btw, I use CPU version which supports categorical features.
\n\nGPU version seems to have some issues as mentioned here .
", "rawMarkdown": "I often refer to their github [issues][1] page for parameters tuning and troubleshooting. btw, I use CPU version which supports categorical features. \n\nGPU version seems to have some issues as mentioned [here][1] . \n\n\n [1]: https://github.com/Microsoft/LightGBM/issues/1116", "votes": null }, { "id": "302493", "postDate": "03/24/2018 06:59:56", "content": "thank you for your help. i will try to tune off categorical variables.
", "rawMarkdown": "thank you for your help. i will try to tune off categorical variables.", "votes": null }, { "id": "302494", "postDate": "03/24/2018 07:04:25", "content": "thank you for your answer. this issues seems like \"GPU version cannot support categorical features with high cardinality.\" problem.\nalso that thread told the solution is \"You can fix it by split one categorical feature into multi categorical features.\" i will try to this way. thank you !
", "rawMarkdown": "thank you for your answer. this issues seems like \"GPU version cannot support categorical features with high cardinality.\" problem.\nalso that thread told the solution is \"You can fix it by split one categorical feature into multi categorical features.\" i will try to this way. thank you !", "votes": null }, { "id": "303441", "postDate": "03/26/2018 07:29:53", "content": "I found the GPU version is slower,so I turned to CPU version
", "rawMarkdown": "I found the GPU version is slower,so I turned to CPU version", "votes": null }, { "id": "303756", "postDate": "03/26/2018 16:15:38", "content": "I think LightGBM uses CPU for computation on sparse features. Also, there's also a lot of overhead to using GPU compute when the dataset is this large, so you want probably 8-12 CPU cores per GPU.
", "rawMarkdown": "I think LightGBM uses CPU for computation on sparse features. Also, there's also a lot of overhead to using GPU compute when the dataset is this large, so you want probably 8-12 CPU cores per GPU.", "votes": null }, { "id": "311950", "postDate": "04/11/2018 00:52:14", "content": "LGBM seems to run ~3sec faster / tree for me using GPU than CPU. Average forest size is ~500 so that's about 25minutes per fold right there. I have a 3.4GHz 4-core machine and a 1080 gtx (non-ti).
\n\nFew questions for y'all:
\n\nnthread -- how does this param affect GPU training?max_bin; is that accurate? Or does it handle cont vars differently internally by using some additional mechanisms? Categorical and cont vars are very different beasts so I cant imagine it only being the former, but what I've seen so far of the docs make it seem that way.Any lgbm masters out there? Pokes kaz
", "rawMarkdown": "LGBM seems to run ~3sec faster / tree for me using GPU than CPU. Average forest size is ~500 so that's about 25minutes per fold right there. I have a 3.4GHz 4-core machine and a 1080 gtx (non-ti).\n\nFew questions for y'all:\n\n - `nthread` -- how does this param affect GPU training?\n - \"So you want probably 8-12 CPU cores per GPU\" what do you mean by this? I've noticed even with gpu training, ~4 of my cpu cores will be at 100% and nvidia-smi reports 0% usage but gpu ram would reflect the size of my dataset. What exactly is going on? I wish the lgbm docs were more verbose in that regard...\n - My understanding is that lgbm descritizes all continuous vars by `max_bin`; is that accurate? Or does it handle cont vars differently internally by using some additional mechanisms? Categorical and cont vars are very different beasts so I cant imagine it only being the former, but what I've seen so far of the docs make it seem that way.\n\nAny lgbm masters out there? Pokes kaz", "votes": null } ], "comments": [ { "id": 301830, "author_name": "sohaibomar", "author_url": "", "post_date": "03/23/2018 10:33:45", "content": "You might wanna look into this, it is related to bin size error.
", "votes": null, "replies": [ { "id": 302493, "author_name": "jiehunt", "author_url": "", "post_date": "03/24/2018 06:59:56", "content": "thank you for your help. i will try to tune off categorical variables.
", "votes": null, "replies": [] } ] }, { "id": 301836, "author_name": "pranav84", "author_url": "", "post_date": "03/23/2018 10:46:05", "content": "I often refer to their github issues page for parameters tuning and troubleshooting. btw, I use CPU version which supports categorical features.
\n\nGPU version seems to have some issues as mentioned here .
", "votes": null, "replies": [ { "id": 302494, "author_name": "jiehunt", "author_url": "", "post_date": "03/24/2018 07:04:25", "content": "thank you for your answer. this issues seems like \"GPU version cannot support categorical features with high cardinality.\" problem.\nalso that thread told the solution is \"You can fix it by split one categorical feature into multi categorical features.\" i will try to this way. thank you !
", "votes": null, "replies": [] } ] }, { "id": 303441, "author_name": "lccever", "author_url": "", "post_date": "03/26/2018 07:29:53", "content": "I found the GPU version is slower,so I turned to CPU version
", "votes": null, "replies": [ { "id": 303756, "author_name": "stevenknguyen", "author_url": "", "post_date": "03/26/2018 16:15:38", "content": "I think LightGBM uses CPU for computation on sparse features. Also, there's also a lot of overhead to using GPU compute when the dataset is this large, so you want probably 8-12 CPU cores per GPU.
", "votes": null, "replies": [] }, { "id": 311950, "author_name": "authman", "author_url": "", "post_date": "04/11/2018 00:52:14", "content": "LGBM seems to run ~3sec faster / tree for me using GPU than CPU. Average forest size is ~500 so that's about 25minutes per fold right there. I have a 3.4GHz 4-core machine and a 1080 gtx (non-ti).
\n\nFew questions for y'all:
\n\nnthread -- how does this param affect GPU training?max_bin; is that accurate? Or does it handle cont vars differently internally by using some additional mechanisms? Categorical and cont vars are very different beasts so I cant imagine it only being the former, but what I've seen so far of the docs make it seem that way.Any lgbm masters out there? Pokes kaz
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "301818": "Hi, Kaggler, I am tring the LightGBM to train model for this competition.\nBut the CPU version is very slow. I try to use the GPU version of LightGBM, \nBut the lightgbm told me that the’bin size cannot run on GPU’.\nIs there anyone use lightGBM with GPU successful?\nHow can I speed up the training speed?\nThank you !", "301830": "You might wanna look into [this][1], it is related to bin size error.\n\n\n [1]: https://www.kaggle.com/c/instacart-market-basket-analysis/discussion/37836#300119", "301836": "I often refer to their github [issues][1] page for parameters tuning and troubleshooting. btw, I use CPU version which supports categorical features. \n\nGPU version seems to have some issues as mentioned [here][1] . \n\n\n [1]: https://github.com/Microsoft/LightGBM/issues/1116", "302493": "thank you for your help. i will try to tune off categorical variables.", "302494": "thank you for your answer. this issues seems like \"GPU version cannot support categorical features with high cardinality.\" problem.\nalso that thread told the solution is \"You can fix it by split one categorical feature into multi categorical features.\" i will try to this way. thank you !", "303441": "I found the GPU version is slower,so I turned to CPU version", "303756": "I think LightGBM uses CPU for computation on sparse features. Also, there's also a lot of overhead to using GPU compute when the dataset is this large, so you want probably 8-12 CPU cores per GPU.", "311950": "LGBM seems to run ~3sec faster / tree for me using GPU than CPU. Average forest size is ~500 so that's about 25minutes per fold right there. I have a 3.4GHz 4-core machine and a 1080 gtx (non-ti).\n\nFew questions for y'all:\n\n - `nthread` -- how does this param affect GPU training?\n - \"So you want probably 8-12 CPU cores per GPU\" what do you mean by this? I've noticed even with gpu training, ~4 of my cpu cores will be at 100% and nvidia-smi reports 0% usage but gpu ram would reflect the size of my dataset. What exactly is going on? I wish the lgbm docs were more verbose in that regard...\n - My understanding is that lgbm descritizes all continuous vars by `max_bin`; is that accurate? Or does it handle cont vars differently internally by using some additional mechanisms? Categorical and cont vars are very different beasts so I cant imagine it only being the former, but what I've seen so far of the docs make it seem that way.\n\nAny lgbm masters out there? Pokes kaz" }, "source": "meta" }