I'd followed some guides on Kaggle about how to use GPU for lightgbm, but when I run the command:
\n!rm -r /opt/conda/lib/python3.6/site-packages/lightgbm\n!git clone --recursive https://github.com/Microsoft/LightGBM\n!apt-get install -y -qq libboost-all-dev\n%cd LightGBM\n!rm -r build\n!mkdir build\n%cd build\n!cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..\n!make -j$(nproc)\n%cd LightGBM/python-package/;python3 setup.py install --precompile\n!mkdir -p /etc/OpenCL/vendors && echo \"libnvidia-opencl.so.1\" > /etc/OpenCL/vendors/nvidia.icd\n!rm -r LightGBM
I encounter this error while cmake part:
\n/kaggle/working/LightGBM/compute/include/boost/compute/cl.hpp:19:10: fatal error: CL/cl.h: No such file or directory
\n #include
\n ^~~~
\ncompilation terminated.
\nCMakeFiles/_lightgbm.dir/build.make:686: recipe for target 'CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o' failed
How could I solve the problem? Is there any useful tutorials about install lightgbm gpu ?
", "messages": [ { "id": "1085721", "postDate": "11/21/2020 06:32:27", "content": "I'd followed some guides on Kaggle about how to use GPU for lightgbm, but when I run the command:
\n!rm -r /opt/conda/lib/python3.6/site-packages/lightgbm\n!git clone --recursive https://github.com/Microsoft/LightGBM\n!apt-get install -y -qq libboost-all-dev\n%cd LightGBM\n!rm -r build\n!mkdir build\n%cd build\n!cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..\n!make -j$(nproc)\n%cd LightGBM/python-package/;python3 setup.py install --precompile\n!mkdir -p /etc/OpenCL/vendors && echo \"libnvidia-opencl.so.1\" > /etc/OpenCL/vendors/nvidia.icd\n!rm -r LightGBM
I encounter this error while cmake part:
\n/kaggle/working/LightGBM/compute/include/boost/compute/cl.hpp:19:10: fatal error: CL/cl.h: No such file or directory
\n #include
\n ^~~~
\ncompilation terminated.
\nCMakeFiles/_lightgbm.dir/build.make:686: recipe for target 'CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o' failed
How could I solve the problem? Is there any useful tutorials about install lightgbm gpu ?
", "rawMarkdown": "I'd followed some guides on Kaggle about how to use GPU for lightgbm, but when I run the command:\n`!rm -r /opt/conda/lib/python3.6/site-packages/lightgbm\n!git clone --recursive https://github.com/Microsoft/LightGBM\n!apt-get install -y -qq libboost-all-dev\n%cd LightGBM\n!rm -r build\n!mkdir build\n%cd build\n!cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..\n!make -j$(nproc)\n%cd LightGBM/python-package/;python3 setup.py install --precompile\n!mkdir -p /etc/OpenCL/vendors && echo \"libnvidia-opencl.so.1\" > /etc/OpenCL/vendors/nvidia.icd\n!rm -r LightGBM`\n\nI encounter this error while cmake part:\n\n/kaggle/working/LightGBM/compute/include/boost/compute/cl.hpp:19:10: fatal error: CL/cl.h: No such file or directory\n #includeHave battled for two years tyring to get local PC running GPU version of lightgbm.
\nBut lots of discussion over many different competitions suggest that the GPU version does not add that much speed to most models.
", "rawMarkdown": "Have battled for two years tyring to get local PC running GPU version of lightgbm. \n\nBut lots of discussion over many different competitions suggest that the GPU version does not add that much speed to most models.", "votes": null }, { "id": "1086923", "postDate": "11/22/2020 07:17:26", "content": "Thank you for your response, could I ask does cpu satisfied your needs for kaggle competitions or other data science project? The most frequent model I run is sklearn and lightgbm, but both could only run on CPU right now, and lightgbm being an effective general model run too slow on my Predict Future Sales project, 30 minutes for 2000 iterations. Now I'm considering if I should try tensorflow wigh GPU, but sklearn and lightgbm is my prior choice for data modelling, waiting for 30 minutes for an non perfect model is really disturbing.
", "rawMarkdown": "Thank you for your response, could I ask does cpu satisfied your needs for kaggle competitions or other data science project? The most frequent model I run is sklearn and lightgbm, but both could only run on CPU right now, and lightgbm being an effective general model run too slow on my Predict Future Sales project, 30 minutes for 2000 iterations. Now I'm considering if I should try tensorflow wigh GPU, but sklearn and lightgbm is my prior choice for data modelling, waiting for 30 minutes for an non perfect model is really disturbing.", "votes": null }, { "id": "1087503", "postDate": "11/22/2020 19:36:54", "content": "On my local machines I almost always use catboost rather than lgbm - with a simple parameter (task=GPU) catboost will use my dual GPU's with no fuss!
", "rawMarkdown": "On my local machines I almost always use catboost rather than lgbm - with a simple parameter (task=GPU) catboost will use my dual GPU's with no fuss!", "votes": null }, { "id": "1087940", "postDate": "11/23/2020 07:16:29", "content": "Thanks for recommendation, I'm usually working on numerical data, I hope catboost would be applicable. In that way, it seems most of the models don't support GPU speed up now: sklearn, lightgbm…, to bad for that.
", "rawMarkdown": "Thanks for recommendation, I'm usually working on numerical data, I hope catboost would be applicable. In that way, it seems most of the models don't support GPU speed up now: sklearn, lightgbm..., to bad for that.", "votes": null }, { "id": "1089032", "postDate": "11/24/2020 06:38:18", "content": "lightgbm and catboost both work almost the same for tabular numerical data. The current tensorflow version also working decently with my dual GPU's on this competion and the datasets I have created - a few more code snips needed compared to catboost but tf NN also working fine for this dataset. If you check out a few of the shared catboost kernels from this competition you can see catboost are pretty easy to get going. The parameters a bit more confusing but default settings get the job done - only when you need that extra 0.01 have I needed to tune.
", "rawMarkdown": "lightgbm and catboost both work almost the same for tabular numerical data. The current tensorflow version also working decently with my dual GPU's on this competion and the datasets I have created - a few more code snips needed compared to catboost but tf NN also working fine for this dataset. If you check out a few of the shared catboost kernels from this competition you can see catboost are pretty easy to get going. The parameters a bit more confusing but default settings get the job done - only when you need that extra 0.01 have I needed to tune.", "votes": null } ], "comments": [ { "id": 1086906, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/22/2020 07:06:01", "content": "Have battled for two years tyring to get local PC running GPU version of lightgbm.
\nBut lots of discussion over many different competitions suggest that the GPU version does not add that much speed to most models.
", "votes": null, "replies": [ { "id": 1086923, "author_name": "laurencelin", "author_url": "", "post_date": "11/22/2020 07:17:26", "content": "Thank you for your response, could I ask does cpu satisfied your needs for kaggle competitions or other data science project? The most frequent model I run is sklearn and lightgbm, but both could only run on CPU right now, and lightgbm being an effective general model run too slow on my Predict Future Sales project, 30 minutes for 2000 iterations. Now I'm considering if I should try tensorflow wigh GPU, but sklearn and lightgbm is my prior choice for data modelling, waiting for 30 minutes for an non perfect model is really disturbing.
", "votes": null, "replies": [] }, { "id": 1087503, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/22/2020 19:36:54", "content": "On my local machines I almost always use catboost rather than lgbm - with a simple parameter (task=GPU) catboost will use my dual GPU's with no fuss!
", "votes": null, "replies": [] }, { "id": 1087940, "author_name": "laurencelin", "author_url": "", "post_date": "11/23/2020 07:16:29", "content": "Thanks for recommendation, I'm usually working on numerical data, I hope catboost would be applicable. In that way, it seems most of the models don't support GPU speed up now: sklearn, lightgbm…, to bad for that.
", "votes": null, "replies": [] }, { "id": 1089032, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/24/2020 06:38:18", "content": "lightgbm and catboost both work almost the same for tabular numerical data. The current tensorflow version also working decently with my dual GPU's on this competion and the datasets I have created - a few more code snips needed compared to catboost but tf NN also working fine for this dataset. If you check out a few of the shared catboost kernels from this competition you can see catboost are pretty easy to get going. The parameters a bit more confusing but default settings get the job done - only when you need that extra 0.01 have I needed to tune.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1085721": "I'd followed some guides on Kaggle about how to use GPU for lightgbm, but when I run the command:\n`!rm -r /opt/conda/lib/python3.6/site-packages/lightgbm\n!git clone --recursive https://github.com/Microsoft/LightGBM\n!apt-get install -y -qq libboost-all-dev\n%cd LightGBM\n!rm -r build\n!mkdir build\n%cd build\n!cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..\n!make -j$(nproc)\n%cd LightGBM/python-package/;python3 setup.py install --precompile\n!mkdir -p /etc/OpenCL/vendors && echo \"libnvidia-opencl.so.1\" > /etc/OpenCL/vendors/nvidia.icd\n!rm -r LightGBM`\n\nI encounter this error while cmake part:\n\n/kaggle/working/LightGBM/compute/include/boost/compute/cl.hpp:19:10: fatal error: CL/cl.h: No such file or directory\n #include