{
  "id": 54016,
  "title": "Installing LightGBM GPU on Kaggle Kernels",
  "url": "/competitions/talkingdata-adtracking-fraud-detection/discussion/54016",
  "author_name": "Nathan Cohen",
  "post_date": "2018-04-08T11:37:03.443000",
  "votes": 8,
  "comment_count": 4,
  "views": 0,
  "content": "<p>The <strong>holy GPUs</strong> are now available to us on Kaggle Kernels and that's great.\nSome libraries come with GPU support already installed on the Kernels but as much as I've tried, I haven't been able to use ( or install ) the LightGBM GPU version.\nKeep in mind this is all \"experimental\" as LightGBM GPU doesn't run as smoothly on NVIDIA Kepler architectures ( from the lgbm docs ) and that is what we have access to so far ( K80s ).</p>\n\n<p>A lot of people, including myself are using lightgbm on this problem and I think it would be interesting to experiment with the GPU version.</p>\n\n<p>To launch GPU on the Kernel, simply go into the Settings tab and turn on the GPU, it'll relauch the Kernel on a GPU</p>\n\n<p>Here is a list of what I have tried so far :</p>\n\n<ul>\n<li><p>Running <code>lightgbm --install-option=--gpu</code> in the Settings tab where the pip install is and it came out with an error code ( probably just impossible to use --install-option in this setup).</p></li>\n<li><p>Running <code>!pip install lightgbm --install-option=--gpu</code> in the Kernel itself, and it ran properly, but after adding the parameter  <code>'device' : 'gpu'</code> in my parameters list and trying to train the lgb I got an error : <br>\n<code>LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'</code> showing that installation did <strong>not</strong> work properly.</p></li>\n<li><p>Running <code>!apt-get -y install --no-install-recommends nvidia-375 <br>\n!apt-get -y install --no-install-recommends nvidia-opencl-icd-375 nvidia-opencl-dev opencl-headers <br>\n!apt-get -y install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev <br>\n!pip3 install -y lightgbm --install-option=--gpu</code> <br>\nAs the doc advises to create the needed OpenCL and Boost, but no luck here again, I got the same error...</p></li>\n</ul>\n\n<p>Let's use this thread to share what we've done regarding this installation and hopefully come up with a solution !</p>",
  "messages": [
    {
      "id": 310718,
      "postDate": "2018-04-08T11:37:03.443Z",
      "content": "<p>The <strong>holy GPUs</strong> are now available to us on Kaggle Kernels and that's great.\nSome libraries come with GPU support already installed on the Kernels but as much as I've tried, I haven't been able to use ( or install ) the LightGBM GPU version.\nKeep in mind this is all \"experimental\" as LightGBM GPU doesn't run as smoothly on NVIDIA Kepler architectures ( from the lgbm docs ) and that is what we have access to so far ( K80s ).</p>\n\n<p>A lot of people, including myself are using lightgbm on this problem and I think it would be interesting to experiment with the GPU version.</p>\n\n<p>To launch GPU on the Kernel, simply go into the Settings tab and turn on the GPU, it'll relauch the Kernel on a GPU</p>\n\n<p>Here is a list of what I have tried so far :</p>\n\n<ul>\n<li><p>Running <code>lightgbm --install-option=--gpu</code> in the Settings tab where the pip install is and it came out with an error code ( probably just impossible to use --install-option in this setup).</p></li>\n<li><p>Running <code>!pip install lightgbm --install-option=--gpu</code> in the Kernel itself, and it ran properly, but after adding the parameter  <code>'device' : 'gpu'</code> in my parameters list and trying to train the lgb I got an error : <br>\n<code>LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'</code> showing that installation did <strong>not</strong> work properly.</p></li>\n<li><p>Running <code>!apt-get -y install --no-install-recommends nvidia-375 <br>\n!apt-get -y install --no-install-recommends nvidia-opencl-icd-375 nvidia-opencl-dev opencl-headers <br>\n!apt-get -y install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev <br>\n!pip3 install -y lightgbm --install-option=--gpu</code> <br>\nAs the doc advises to create the needed OpenCL and Boost, but no luck here again, I got the same error...</p></li>\n</ul>\n\n<p>Let's use this thread to share what we've done regarding this installation and hopefully come up with a solution !</p>",
      "rawMarkdown": "The **holy GPUs** are now available to us on Kaggle Kernels and that's great.\nSome libraries come with GPU support already installed on the Kernels but as much as I've tried, I haven't been able to use ( or install ) the LightGBM GPU version.\nKeep in mind this is all \"experimental\" as LightGBM GPU doesn't run as smoothly on NVIDIA Kepler architectures ( from the lgbm docs ) and that is what we have access to so far ( K80s ).\n\nA lot of people, including myself are using lightgbm on this problem and I think it would be interesting to experiment with the GPU version.\n\nTo launch GPU on the Kernel, simply go into the Settings tab and turn on the GPU, it'll relauch the Kernel on a GPU\n\nHere is a list of what I have tried so far :\n\n* Running `lightgbm --install-option=--gpu` in the Settings tab where the pip install is and it came out with an error code ( probably just impossible to use --install-option in this setup).\n\n* Running `!pip install lightgbm --install-option=--gpu` in the Kernel itself, and it ran properly, but after adding the parameter  `'device' : 'gpu'` in my parameters list and trying to train the lgb I got an error :  \n `LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'` showing that installation did **not** work properly.\n\n* Running `!apt-get -y install --no-install-recommends nvidia-375  \n!apt-get -y install --no-install-recommends nvidia-opencl-icd-375 nvidia-opencl-dev opencl-headers  \n!apt-get -y install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev  \n!pip3 install -y lightgbm --install-option=--gpu`  \nAs the doc advises to create the needed OpenCL and Boost, but no luck here again, I got the same error...\n\nLet's use this thread to share what we've done regarding this installation and hopefully come up with a solution !\n",
      "votes": 8
    },
    {
      "id": 456209,
      "postDate": "2019-01-15T11:01:40.247Z",
      "content": "<p>I raised an issue here:</p>\n\n<p><a href=\"https://www.kaggle.com/product-feedback/77663\">https://www.kaggle.com/product-feedback/77663</a></p>",
      "rawMarkdown": "I raised an issue here:\n\nhttps://www.kaggle.com/product-feedback/77663",
      "votes": 1
    },
    {
      "id": 477612,
      "postDate": "2019-02-25T01:03:06.430Z",
      "content": "<p>I've made an example of using LightGBM with the Kaggle-provided GPU here:\n<a href=\"https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm\">https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm</a></p>",
      "rawMarkdown": "I've made an example of using LightGBM with the Kaggle-provided GPU here:\nhttps://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm"
    },
    {
      "id": 396553,
      "postDate": "2018-10-01T01:59:49.827Z",
      "content": "<p>Any update on this ?</p>",
      "rawMarkdown": "Any update on this ?"
    },
    {
      "id": 404102,
      "postDate": "2018-10-15T08:35:21.187Z",
      "content": "<p>Hi Nathan, have you find out how to use lightGBM gpu on Kaggle kernel yet?</p>",
      "rawMarkdown": "Hi Nathan, have you find out how to use lightGBM gpu on Kaggle kernel yet?",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 456209,
      "author_name": "Dieter",
      "author_url": "",
      "post_date": "2019-01-15T11:01:40.247000",
      "content": "<p>I raised an issue here:</p>\n\n<p><a href=\"https://www.kaggle.com/product-feedback/77663\">https://www.kaggle.com/product-feedback/77663</a></p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 477612,
      "author_name": "Vinh Nguyen",
      "author_url": "",
      "post_date": "2019-02-25T01:03:06.430000",
      "content": "<p>I've made an example of using LightGBM with the Kaggle-provided GPU here:\n<a href=\"https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm\">https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 396553,
      "author_name": "xaviermaxime",
      "author_url": "",
      "post_date": "2018-10-01T01:59:49.827000",
      "content": "<p>Any update on this ?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 404102,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-10-15T08:35:21.187000",
      "content": "<p>Hi Nathan, have you find out how to use lightGBM gpu on Kaggle kernel yet?</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "310718": "The **holy GPUs** are now available to us on Kaggle Kernels and that's great.\nSome libraries come with GPU support already installed on the Kernels but as much as I've tried, I haven't been able to use ( or install ) the LightGBM GPU version.\nKeep in mind this is all \"experimental\" as LightGBM GPU doesn't run as smoothly on NVIDIA Kepler architectures ( from the lgbm docs ) and that is what we have access to so far ( K80s ).\n\nA lot of people, including myself are using lightgbm on this problem and I think it would be interesting to experiment with the GPU version.\n\nTo launch GPU on the Kernel, simply go into the Settings tab and turn on the GPU, it'll relauch the Kernel on a GPU\n\nHere is a list of what I have tried so far :\n\n* Running `lightgbm --install-option=--gpu` in the Settings tab where the pip install is and it came out with an error code ( probably just impossible to use --install-option in this setup).\n\n* Running `!pip install lightgbm --install-option=--gpu` in the Kernel itself, and it ran properly, but after adding the parameter  `'device' : 'gpu'` in my parameters list and trying to train the lgb I got an error :  \n `LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'` showing that installation did **not** work properly.\n\n* Running `!apt-get -y install --no-install-recommends nvidia-375  \n!apt-get -y install --no-install-recommends nvidia-opencl-icd-375 nvidia-opencl-dev opencl-headers  \n!apt-get -y install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev  \n!pip3 install -y lightgbm --install-option=--gpu`  \nAs the doc advises to create the needed OpenCL and Boost, but no luck here again, I got the same error...\n\nLet's use this thread to share what we've done regarding this installation and hopefully come up with a solution !\n",
    "456209": "I raised an issue here:\n\nhttps://www.kaggle.com/product-feedback/77663",
    "477612": "I've made an example of using LightGBM with the Kaggle-provided GPU here:\nhttps://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm",
    "396553": "Any update on this ?",
    "404102": "Hi Nathan, have you find out how to use lightGBM gpu on Kaggle kernel yet?"
  }
}