{ "id": 194307, "title": "How to do LGBM with GPU?", "url": "/competitions/riiid-test-answer-prediction/discussion/194307", "author_name": "", "post_date": "2020-11-01T04:47:17.459638500Z", "votes": 7, "comment_count": 5, "views": 0, "content": "

UPDATE

\n

Self-solving
\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, X_trn.to_pandas() because training in LGBM seems not support cudf.

\n

If you don't take process of conversion,

\n
---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n
\n

And comparing xgboost with GPU, LGBM ones doesn't accelerate training speed so much in my quick observation. There are similar discussion in past competitions.

\n
\n

Seeing at Answer Correctness - RAPIDS crazy fast, I'm considering use of LGBM with GPU (cudf and cupy).

\n

Looking over this issue, it occurs to me three questions. Maybe newbie question..

\n
    \n

  1. Until now, there is no notebook about that and no discussion so far in this competition. It means LGBM with GPU isn't wise choice in the first place?

    2. \n

  2. If this choice is somewhat good, or at least not bad, do you know how to use it without internet connection? (This competition seems not to allow internet connection)
    \n(There is already a way to do with internet connection)

    3. \n

  3. In Answer Correctness - RAPIDS crazy fast, it seems there is no process of installing or setting for GPU ver of xgboost. This means xgboost has GPU setting inherently, unlike LGBM?? (and all you need to do in xgboost is just add hyper-parameter for switching GPU/CPU?)

    4. \n
\n

Thank you

", "messages": [ { "id": "1065939", "postDate": "11/01/2020 04:47:17", "content": "

UPDATE

\n

Self-solving
\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, X_trn.to_pandas() because training in LGBM seems not support cudf.

\n

If you don't take process of conversion,

\n
---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n
\n

And comparing xgboost with GPU, LGBM ones doesn't accelerate training speed so much in my quick observation. There are similar discussion in past competitions.

\n
\n

Seeing at Answer Correctness - RAPIDS crazy fast, I'm considering use of LGBM with GPU (cudf and cupy).

\n

Looking over this issue, it occurs to me three questions. Maybe newbie question..

\n
    \n

  1. Until now, there is no notebook about that and no discussion so far in this competition. It means LGBM with GPU isn't wise choice in the first place?

    2. \n

  2. If this choice is somewhat good, or at least not bad, do you know how to use it without internet connection? (This competition seems not to allow internet connection)
    \n(There is already a way to do with internet connection)

    3. \n

  3. In Answer Correctness - RAPIDS crazy fast, it seems there is no process of installing or setting for GPU ver of xgboost. This means xgboost has GPU setting inherently, unlike LGBM?? (and all you need to do in xgboost is just add hyper-parameter for switching GPU/CPU?)

    4. \n
\n

Thank you

", "rawMarkdown": "## UPDATE\nSelf-solving\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, `X_trn.to_pandas()` because training in LGBM seems not support cudf.\n\nIf you don't take process of conversion,\n```\n---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n```\n\nAnd comparing xgboost with GPU, LGBM ones doesn't accelerate training speed so much in my quick observation. There are [similar discussion in past competitions](https://www.kaggle.com/c/champs-scalar-coupling/discussion/103037).\n\n------------------------------------------------------------\nSeeing at [Answer Correctness - RAPIDS crazy fast](https://www.kaggle.com/andradaolteanu/answer-correctness-rapids-crazy-fast), I'm considering use of LGBM with GPU (cudf and cupy).\n\nLooking over this issue, it occurs to me three questions. Maybe newbie question..\n\n1. Until now, there is no notebook about that and no discussion so far in this competition. It means LGBM with GPU isn't wise choice in the first place?\n\n2. If this choice is somewhat good, or at least not bad, do you know how to use it without internet connection? (This competition seems not to allow internet connection)\n([There is already a way to do with internet connection](https://www.kaggle.com/kirankunapuli/ieee-fraud-lightgbm-with-gpu))\n\n3. In [Answer Correctness - RAPIDS crazy fast](https://www.kaggle.com/andradaolteanu/answer-correctness-rapids-crazy-fast), it seems there is no process of installing or setting for GPU ver of xgboost. This means xgboost has GPU setting inherently, unlike LGBM?? (and all you need to do in xgboost is just add hyper-parameter for switching GPU/CPU?)\n\nThank you", "votes": null }, { "id": "1066241", "postDate": "11/01/2020 14:26:13", "content": "

Try these Parameters,It may help

\n
   lg= LGBMClassifier( metric= 'auc',\n                       # GPU PARAMETERS #\n                       device = \"gpu\",\n                       gpu_device_id =0,\n                       max_bin = 63,\n                       gpu_platform_id=1,\n                       # GPU PARAMETERS #\n)\n
\n

These Links may also be helpful
\nLink1
\nLink2

", "rawMarkdown": "Try these Parameters,It may help \n```\n lg= LGBMClassifier( metric= 'auc',\n # GPU PARAMETERS #\n device = \"gpu\",\n gpu_device_id =0,\n max_bin = 63,\n gpu_platform_id=1,\n # GPU PARAMETERS #\n)\n```\n**These Links may also be helpful**\n[Link1](https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html)\n[Link2](https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm)", "votes": null }, { "id": "1068046", "postDate": "11/03/2020 02:50:41", "content": "

@vpkprasanna
\nThanks your reply

\n

I directly download lgbm from github and put into dataset in kaggle. (Because internet connection is not allowed in this competition) And Install in kaggle notebook was done.
\nHowever, LGBM with cudf seems to cause problems, saying

\n
---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n
\n

My guess is when it comes to training with LGBM, you cannot use cudf
\nI really want to know how to overcome inference problem in test data (2.5 million questions). Despite no feature engineering at all, submission takes at least more than 3 hours…

", "rawMarkdown": "vpkprasanna \nThanks your reply\n\nI directly download lgbm from github and put into dataset in kaggle. (Because internet connection is not allowed in this competition) And Install in kaggle notebook was done.\nHowever, LGBM with cudf seems to cause problems, saying \n\n```\n---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n```\n\nMy guess is when it comes to training with LGBM, you cannot use cudf\nI really want to know how to overcome inference problem in test data (2.5 million questions). Despite no feature engineering at all, submission takes at least more than 3 hours...", "votes": null }, { "id": "1072497", "postDate": "11/08/2020 10:27:53", "content": "

Self-solving
\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, X_trn.to_pandas() because LGBM seems not support cudf.

\n

And LGBM with GPU doesn't accelerate training speed so much, as many people said in past discussion. While xgboost is so speedy.

", "rawMarkdown": "Self-solving\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, `X_trn.to_pandas()` because LGBM seems not support cudf.\n\nAnd LGBM with GPU doesn't accelerate training speed so much, as many people said in past discussion. While xgboost is so speedy.", "votes": null }, { "id": "1104187", "postDate": "12/06/2020 17:30:42", "content": "

whats the speed improvement ratio? I heard lgbm doesnt gain alot from gpu, however I tested catboost gpu and it is much faster than cpu verion (almost 10x)

", "rawMarkdown": "whats the speed improvement ratio? I heard lgbm doesnt gain alot from gpu, however I tested catboost gpu and it is much faster than cpu verion (almost 10x)", "votes": null }, { "id": "1109754", "postDate": "12/12/2020 03:10:08", "content": "

@feriiiiiiiiii
\nI didn't compare speed ratio exactly so I cannot say how it is different. As far as I did, xgboost with GPU is significantly faster than one with CPU, but not so much about LGBM

", "rawMarkdown": "feriiiiiiiiii \nI didn't compare speed ratio exactly so I cannot say how it is different. As far as I did, xgboost with GPU is significantly faster than one with CPU, but not so much about LGBM", "votes": null } ], "comments": [ { "id": 1066241, "author_name": "vpkprasanna", "author_url": "", "post_date": "11/01/2020 14:26:13", "content": "

Try these Parameters,It may help

\n
   lg= LGBMClassifier( metric= 'auc',\n                       # GPU PARAMETERS #\n                       device = \"gpu\",\n                       gpu_device_id =0,\n                       max_bin = 63,\n                       gpu_platform_id=1,\n                       # GPU PARAMETERS #\n)\n
\n

These Links may also be helpful
\nLink1
\nLink2

", "votes": null, "replies": [ { "id": 1068046, "author_name": "ant3ng", "author_url": "", "post_date": "11/03/2020 02:50:41", "content": "

@vpkprasanna
\nThanks your reply

\n

I directly download lgbm from github and put into dataset in kaggle. (Because internet connection is not allowed in this competition) And Install in kaggle notebook was done.
\nHowever, LGBM with cudf seems to cause problems, saying

\n
---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n
\n

My guess is when it comes to training with LGBM, you cannot use cudf
\nI really want to know how to overcome inference problem in test data (2.5 million questions). Despite no feature engineering at all, submission takes at least more than 3 hours…

", "votes": null, "replies": [] }, { "id": 1072497, "author_name": "ant3ng", "author_url": "", "post_date": "11/08/2020 10:27:53", "content": "

Self-solving
\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, X_trn.to_pandas() because LGBM seems not support cudf.

\n

And LGBM with GPU doesn't accelerate training speed so much, as many people said in past discussion. While xgboost is so speedy.

", "votes": null, "replies": [] } ] }, { "id": 1104187, "author_name": "feriiiiiiiiii", "author_url": "", "post_date": "12/06/2020 17:30:42", "content": "

whats the speed improvement ratio? I heard lgbm doesnt gain alot from gpu, however I tested catboost gpu and it is much faster than cpu verion (almost 10x)

", "votes": null, "replies": [ { "id": 1109754, "author_name": "ant3ng", "author_url": "", "post_date": "12/12/2020 03:10:08", "content": "

@feriiiiiiiiii
\nI didn't compare speed ratio exactly so I cannot say how it is different. As far as I did, xgboost with GPU is significantly faster than one with CPU, but not so much about LGBM

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1065939": "## UPDATE\nSelf-solving\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, `X_trn.to_pandas()` because training in LGBM seems not support cudf.\n\nIf you don't take process of conversion,\n```\n---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n```\n\nAnd comparing xgboost with GPU, LGBM ones doesn't accelerate training speed so much in my quick observation. There are [similar discussion in past competitions](https://www.kaggle.com/c/champs-scalar-coupling/discussion/103037).\n\n------------------------------------------------------------\nSeeing at [Answer Correctness - RAPIDS crazy fast](https://www.kaggle.com/andradaolteanu/answer-correctness-rapids-crazy-fast), I'm considering use of LGBM with GPU (cudf and cupy).\n\nLooking over this issue, it occurs to me three questions. Maybe newbie question..\n\n1. Until now, there is no notebook about that and no discussion so far in this competition. It means LGBM with GPU isn't wise choice in the first place?\n\n2. If this choice is somewhat good, or at least not bad, do you know how to use it without internet connection? (This competition seems not to allow internet connection)\n([There is already a way to do with internet connection](https://www.kaggle.com/kirankunapuli/ieee-fraud-lightgbm-with-gpu))\n\n3. In [Answer Correctness - RAPIDS crazy fast](https://www.kaggle.com/andradaolteanu/answer-correctness-rapids-crazy-fast), it seems there is no process of installing or setting for GPU ver of xgboost. This means xgboost has GPU setting inherently, unlike LGBM?? (and all you need to do in xgboost is just add hyper-parameter for switching GPU/CPU?)\n\nThank you", "1066241": "Try these Parameters,It may help \n```\n lg= LGBMClassifier( metric= 'auc',\n # GPU PARAMETERS #\n device = \"gpu\",\n gpu_device_id =0,\n max_bin = 63,\n gpu_platform_id=1,\n # GPU PARAMETERS #\n)\n```\n**These Links may also be helpful**\n[Link1](https://lightgbm.readthedocs.io/en/latest/GPU-Tutorial.html)\n[Link2](https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm)", "1068046": "vpkprasanna \nThanks your reply\n\nI directly download lgbm from github and put into dataset in kaggle. (Because internet connection is not allowed in this competition) And Install in kaggle notebook was done.\nHowever, LGBM with cudf seems to cause problems, saying \n\n```\n---> model.fit(**params_fit)\n\nTypeError: Implicit conversion to a host NumPy array via __array__ is not allowed, To explicitly construct a GPU array, consider using cupy.asarray(...)\nTo explicitly construct a host array, consider using .to_array()\n```\n\nMy guess is when it comes to training with LGBM, you cannot use cudf\nI really want to know how to overcome inference problem in test data (2.5 million questions). Despite no feature engineering at all, submission takes at least more than 3 hours...", "1072497": "Self-solving\nRegardless of train-api or sklearn-api, cudf data should be converted into pandas like, `X_trn.to_pandas()` because LGBM seems not support cudf.\n\nAnd LGBM with GPU doesn't accelerate training speed so much, as many people said in past discussion. While xgboost is so speedy.", "1104187": "whats the speed improvement ratio? I heard lgbm doesnt gain alot from gpu, however I tested catboost gpu and it is much faster than cpu verion (almost 10x)", "1109754": "feriiiiiiiiii \nI didn't compare speed ratio exactly so I cannot say how it is different. As far as I did, xgboost with GPU is significantly faster than one with CPU, but not so much about LGBM" }, "source": "meta" }