{
  "id": 486740,
  "title": "[LightGBM] [Fatal] bin size 258 cannot run on GPU ",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/486740",
  "author_name": "",
  "post_date": "2024-03-26T07:03:35.806705800Z",
  "votes": 4,
  "comment_count": 8,
  "views": 0,
  "content": "<p>How can I solve this problem</p>",
  "messages": [
    {
      "id": "2716712",
      "postDate": "03/26/2024 07:03:35",
      "content": "<p>How can I solve this problem</p>",
      "rawMarkdown": "How can I solve this problem",
      "votes": null
    },
    {
      "id": "2716722",
      "postDate": "03/26/2024 07:13:46",
      "content": "<p>I also got this error, I think gpu has a binning limit of 255 in my opinion <a href=\"https://www.kaggle.com/zivanwan\" target=\"_blank\">@zivanwan</a> <br>\nRemove this and it will work </p>",
      "rawMarkdown": "I also got this error, I think gpu has a binning limit of 255 in my opinion @zivanwan \nRemove this and it will work",
      "votes": null
    },
    {
      "id": "2716726",
      "postDate": "03/26/2024 07:17:27",
      "content": "<p>When I encounter this problem, I solve it in two ways</p>\n<ol>\n<li>Reducing the number of categories in columns with too many categories</li>\n<li>If you are using lightgbm, set the \"min_data_in_bin\" parameter to 256 or higher</li>\n</ol>",
      "rawMarkdown": "When I encounter this problem, I solve it in two ways\n\n1. Reducing the number of categories in columns with too many categories\n2. If you are using lightgbm, set the \"min_data_in_bin\" parameter to 256 or higher",
      "votes": null
    },
    {
      "id": "2716747",
      "postDate": "03/26/2024 07:32:12",
      "content": "<p>Thank you very much!! I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher</p>",
      "rawMarkdown": "Thank you very much!! I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher",
      "votes": null
    },
    {
      "id": "2716749",
      "postDate": "03/26/2024 07:35:31",
      "content": "<p>Thank you very much!! I think lgb recognizes an integer type as a category, and I'm sure the number of values in each of my categories is less than 256.I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher.</p>",
      "rawMarkdown": "Thank you very much!! I think lgb recognizes an integer type as a category, and I'm sure the number of values in each of my categories is less than 256.I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher.",
      "votes": null
    },
    {
      "id": "2719298",
      "postDate": "03/27/2024 16:02:43",
      "content": "<p>Did you encounter this issue during the parameter tuning process? If so, you can add code like: if params['max_bin']&gt;255: params['max_bin']=255</p>",
      "rawMarkdown": "Did you encounter this issue during the parameter tuning process? If so, you can add code like: if params['max_bin']>255: params['max_bin']=255",
      "votes": null
    },
    {
      "id": "2719922",
      "postDate": "03/28/2024 02:52:04",
      "content": "<p>I haven't tried this approach, but it is possible to set the \"min_data_in_bin\" parameter to 256 or higher</p>",
      "rawMarkdown": "I haven't tried this approach, but it is possible to set the \"min_data_in_bin\" parameter to 256 or higher",
      "votes": null
    },
    {
      "id": "2722023",
      "postDate": "03/29/2024 11:05:26",
      "content": "<p>LGBM documents recommends max_bins=63 for better performance using GPU and it can resolve the error you get, but I think it will not work if you feed the model with high cardinality categoricals, but it should work if the categoricals are encoded before training. Give it a try.</p>\n<p><a href=\"https://lightgbm.readthedocs.io/en/latest/GPU-Performance.html#how-to-achieve-good-speedup-on-gpu\" target=\"_blank\">https://lightgbm.readthedocs.io/en/latest/GPU-Performance.html#how-to-achieve-good-speedup-on-gpu</a></p>",
      "rawMarkdown": "LGBM documents recommends max_bins=63 for better performance using GPU and it can resolve the error you get, but I think it will not work if you feed the model with high cardinality categoricals, but it should work if the categoricals are encoded before training. Give it a try.\n\nhttps://lightgbm.readthedocs.io/en/latest/GPU-Performance.html#how-to-achieve-good-speedup-on-gpu",
      "votes": null
    },
    {
      "id": "2738181",
      "postDate": "04/06/2024 06:57:41",
      "content": "<p>Thank you very much!! This is a robust and efficient solution。</p>",
      "rawMarkdown": "Thank you very much!! This is a robust and efficient solution。",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2716722,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "03/26/2024 07:13:46",
      "content": "<p>I also got this error, I think gpu has a binning limit of 255 in my opinion <a href=\"https://www.kaggle.com/zivanwan\" target=\"_blank\">@zivanwan</a> <br>\nRemove this and it will work </p>",
      "votes": null,
      "replies": [
        {
          "id": 2716749,
          "author_name": "zivanwan",
          "author_url": "",
          "post_date": "03/26/2024 07:35:31",
          "content": "<p>Thank you very much!! I think lgb recognizes an integer type as a category, and I'm sure the number of values in each of my categories is less than 256.I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2716726,
      "author_name": "greysky",
      "author_url": "",
      "post_date": "03/26/2024 07:17:27",
      "content": "<p>When I encounter this problem, I solve it in two ways</p>\n<ol>\n<li>Reducing the number of categories in columns with too many categories</li>\n<li>If you are using lightgbm, set the \"min_data_in_bin\" parameter to 256 or higher</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 2716747,
          "author_name": "zivanwan",
          "author_url": "",
          "post_date": "03/26/2024 07:32:12",
          "content": "<p>Thank you very much!! I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2719298,
      "author_name": "yangzshi68",
      "author_url": "",
      "post_date": "03/27/2024 16:02:43",
      "content": "<p>Did you encounter this issue during the parameter tuning process? If so, you can add code like: if params['max_bin']&gt;255: params['max_bin']=255</p>",
      "votes": null,
      "replies": [
        {
          "id": 2719922,
          "author_name": "zivanwan",
          "author_url": "",
          "post_date": "03/28/2024 02:52:04",
          "content": "<p>I haven't tried this approach, but it is possible to set the \"min_data_in_bin\" parameter to 256 or higher</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2722023,
      "author_name": "eu1234",
      "author_url": "",
      "post_date": "03/29/2024 11:05:26",
      "content": "<p>LGBM documents recommends max_bins=63 for better performance using GPU and it can resolve the error you get, but I think it will not work if you feed the model with high cardinality categoricals, but it should work if the categoricals are encoded before training. Give it a try.</p>\n<p><a href=\"https://lightgbm.readthedocs.io/en/latest/GPU-Performance.html#how-to-achieve-good-speedup-on-gpu\" target=\"_blank\">https://lightgbm.readthedocs.io/en/latest/GPU-Performance.html#how-to-achieve-good-speedup-on-gpu</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 2738181,
          "author_name": "zivanwan",
          "author_url": "",
          "post_date": "04/06/2024 06:57:41",
          "content": "<p>Thank you very much!! This is a robust and efficient solution。</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2716712": "How can I solve this problem",
    "2716722": "I also got this error, I think gpu has a binning limit of 255 in my opinion @zivanwan \nRemove this and it will work",
    "2716726": "When I encounter this problem, I solve it in two ways\n\n1. Reducing the number of categories in columns with too many categories\n2. If you are using lightgbm, set the \"min_data_in_bin\" parameter to 256 or higher",
    "2716747": "Thank you very much!! I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher",
    "2716749": "Thank you very much!! I think lgb recognizes an integer type as a category, and I'm sure the number of values in each of my categories is less than 256.I solved the problem setting the \"min_data_in_bin\" parameter to 256 or higher.",
    "2719298": "Did you encounter this issue during the parameter tuning process? If so, you can add code like: if params['max_bin']>255: params['max_bin']=255",
    "2719922": "I haven't tried this approach, but it is possible to set the \"min_data_in_bin\" parameter to 256 or higher",
    "2722023": "LGBM documents recommends max_bins=63 for better performance using GPU and it can resolve the error you get, but I think it will not work if you feed the model with high cardinality categoricals, but it should work if the categoricals are encoded before training. Give it a try.\n\nhttps://lightgbm.readthedocs.io/en/latest/GPU-Performance.html#how-to-achieve-good-speedup-on-gpu",
    "2738181": "Thank you very much!! This is a robust and efficient solution。"
  },
  "source": "meta"
}