{
  "id": 498970,
  "title": "Initializing the GPU to good effect in your LGBM models",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/498970",
  "author_name": "",
  "post_date": "2024-04-30T07:10:56.165733900Z",
  "votes": 34,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello readers,</p>\n<p>Hope you are doing well. I wish to highlight a small trick here that may be used to harness the capability of the GPU to great effect in your GBM models, especially in <strong>LightGBM</strong>. </p>\n<h3>LightGBM device parameter</h3>\n<p>LGBM 4.1, 4.2 and 4.3 has a parameter named <strong>device_type</strong> that supports 3 entries -</p>\n<ol>\n<li><strong>cpu</strong> - this is by default and initializes the usage of the traditional cpu settings</li>\n<li><strong>gpu</strong> - this is commonly used to invoke the gpu - this was traditionally used for gpu models from legacy versions. This is a general parameter, applicable for all GPU types, including cuda and other devices </li>\n<li><strong>cuda</strong> - this is a relatively new addition available lately in these releases. It explicitly mentions to LGBM that the GPU used supports <strong>cuda</strong>. This offers a better training speed than the <strong>gpu</strong> option. I have experienced this in other endeavors and can confirm from my experiments</li>\n</ol>\n<p>Additionally, using the parameter <strong>gpu_use_dp</strong> for <strong>cuda based GPUs</strong> uses the 64-bit floats. This slows down the training process but the results are more likely than the non-usage of this parameter to be reproducible. </p>\n<p>Some additional references from the LightGBM for this endeavor- </p>\n<ul>\n<li><a href=\"https://lightgbm.readthedocs.io/en/latest/GPU-Targets.html#query-opencl-devices-in-your-system\" target=\"_blank\">https://lightgbm.readthedocs.io/en/latest/GPU-Targets.html#query-opencl-devices-in-your-system</a></li>\n<li><a href=\"https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html\" target=\"_blank\">https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html</a></li>\n</ul>\n<p>Wishing you the best for the competition! Happy learning and regards!</p>",
  "messages": [
    {
      "id": "2784256",
      "postDate": "04/30/2024 07:10:56",
      "content": "<p>Hello readers,</p>\n<p>Hope you are doing well. I wish to highlight a small trick here that may be used to harness the capability of the GPU to great effect in your GBM models, especially in <strong>LightGBM</strong>. </p>\n<h3>LightGBM device parameter</h3>\n<p>LGBM 4.1, 4.2 and 4.3 has a parameter named <strong>device_type</strong> that supports 3 entries -</p>\n<ol>\n<li><strong>cpu</strong> - this is by default and initializes the usage of the traditional cpu settings</li>\n<li><strong>gpu</strong> - this is commonly used to invoke the gpu - this was traditionally used for gpu models from legacy versions. This is a general parameter, applicable for all GPU types, including cuda and other devices </li>\n<li><strong>cuda</strong> - this is a relatively new addition available lately in these releases. It explicitly mentions to LGBM that the GPU used supports <strong>cuda</strong>. This offers a better training speed than the <strong>gpu</strong> option. I have experienced this in other endeavors and can confirm from my experiments</li>\n</ol>\n<p>Additionally, using the parameter <strong>gpu_use_dp</strong> for <strong>cuda based GPUs</strong> uses the 64-bit floats. This slows down the training process but the results are more likely than the non-usage of this parameter to be reproducible. </p>\n<p>Some additional references from the LightGBM for this endeavor- </p>\n<ul>\n<li><a href=\"https://lightgbm.readthedocs.io/en/latest/GPU-Targets.html#query-opencl-devices-in-your-system\" target=\"_blank\">https://lightgbm.readthedocs.io/en/latest/GPU-Targets.html#query-opencl-devices-in-your-system</a></li>\n<li><a href=\"https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html\" target=\"_blank\">https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html</a></li>\n</ul>\n<p>Wishing you the best for the competition! Happy learning and regards!</p>",
      "rawMarkdown": "Hello readers,\n\nHope you are doing well. I wish to highlight a small trick here that may be used to harness the capability of the GPU to great effect in your GBM models, especially in **LightGBM**. \n\n### LightGBM device parameter\nLGBM 4.1, 4.2 and 4.3 has a parameter named **device_type** that supports 3 entries -\n1. **cpu** - this is by default and initializes the usage of the traditional cpu settings\n2. **gpu** - this is commonly used to invoke the gpu - this was traditionally used for gpu models from legacy versions. This is a general parameter, applicable for all GPU types, including cuda and other devices \n3. **cuda** - this is a relatively new addition available lately in these releases. It explicitly mentions to LGBM that the GPU used supports **cuda**. This offers a better training speed than the **gpu** option. I have experienced this in other endeavors and can confirm from my experiments\n\nAdditionally, using the parameter **gpu_use_dp** for **cuda based GPUs** uses the 64-bit floats. This slows down the training process but the results are more likely than the non-usage of this parameter to be reproducible. \n\nSome additional references from the LightGBM for this endeavor- \n- https://lightgbm.readthedocs.io/en/latest/GPU-Targets.html#query-opencl-devices-in-your-system\n- https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html\n\nWishing you the best for the competition! Happy learning and regards!",
      "votes": null
    },
    {
      "id": "2797848",
      "postDate": "05/07/2024 01:22:15",
      "content": "<p>Thank you sir.</p>",
      "rawMarkdown": "Thank you sir.",
      "votes": null
    },
    {
      "id": "2798037",
      "postDate": "05/07/2024 04:49:10",
      "content": "<p>Thanks for sharing this info. </p>",
      "rawMarkdown": "Thanks for sharing this info.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2797848,
      "author_name": "infinitysherlock",
      "author_url": "",
      "post_date": "05/07/2024 01:22:15",
      "content": "<p>Thank you sir.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2798037,
      "author_name": "diwenzhou",
      "author_url": "",
      "post_date": "05/07/2024 04:49:10",
      "content": "<p>Thanks for sharing this info. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2784256": "Hello readers,\n\nHope you are doing well. I wish to highlight a small trick here that may be used to harness the capability of the GPU to great effect in your GBM models, especially in **LightGBM**. \n\n### LightGBM device parameter\nLGBM 4.1, 4.2 and 4.3 has a parameter named **device_type** that supports 3 entries -\n1. **cpu** - this is by default and initializes the usage of the traditional cpu settings\n2. **gpu** - this is commonly used to invoke the gpu - this was traditionally used for gpu models from legacy versions. This is a general parameter, applicable for all GPU types, including cuda and other devices \n3. **cuda** - this is a relatively new addition available lately in these releases. It explicitly mentions to LGBM that the GPU used supports **cuda**. This offers a better training speed than the **gpu** option. I have experienced this in other endeavors and can confirm from my experiments\n\nAdditionally, using the parameter **gpu_use_dp** for **cuda based GPUs** uses the 64-bit floats. This slows down the training process but the results are more likely than the non-usage of this parameter to be reproducible. \n\nSome additional references from the LightGBM for this endeavor- \n- https://lightgbm.readthedocs.io/en/latest/GPU-Targets.html#query-opencl-devices-in-your-system\n- https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html\n\nWishing you the best for the competition! Happy learning and regards!",
    "2797848": "Thank you sir.",
    "2798037": "Thanks for sharing this info."
  },
  "source": "meta"
}