![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2666506%2Fcf068fe379cfb39f940338d2900b8983%2FCapture%20decran%202023-01-27%20a%206.38.11%20PM.png?generation=1674841108941183&alt=media\")
Hello community, I wanted to share this experiments with you.
\nI struggled for a long time with the Ranker inference, especially when using multiples rankers through GroupKFold or any other k-fold techniques.
\nHere is my results:
\n| Number of models | \nNumber of estimators | \ntime per chunk | \noverall time | \n
|---|---|---|---|
| 3 | \n50 | \n1min12 | \n11 min | \n
| 3 | \n100 | \n2 min | \n20 min | \n
| 3 | \n300 | \n5min23 sec | \n51 min | \n
| 3 | \n1000 | \n22 min. | \n3h 43min | \n
As you can see, using the CPU kaggle kernel is not enough, as pointed out by @buumoo here.
\nHowever, it's probably ideal to use GPU but I got some limitations and memory errors in my pipeline with kaggle kernels, and my ranker is already trained in CPU.
\nUsing the TPU VM -3-8, the inference time is reduced by about 15 times :
\n| Number of models | \nNumber of estimators | \ntime per chunk | \noverall time | \n
|---|---|---|---|
| 3 | \n1000 | \n1 min36. | \n~15 min | \n
| \n\n | \n | \n |
The only constraint is that I couldn't use Polars, so I convert it to Pandas and you'll have to wait in order to use the TPU.
\nSummary:
\n| Case | \nDisadvantages | \nAdvantages | \n
|---|---|---|
| CPU | \nToo slow | \nimmediat use / you don't have to wait | \n
| TPU | \nFast / don't need to chunk | \nQueue / Be aware of the lib you're using | \n
Another essential point before executing your code in TPU, please be really careful, test your code before execution, because if you made a little typo, you'll have to wait again.
", "messages": [ { "id": "2118003", "postDate": "01/27/2023 17:43:57", "content": "Hello community, I wanted to share this experiments with you.
\nI struggled for a long time with the Ranker inference, especially when using multiples rankers through GroupKFold or any other k-fold techniques.
\nHere is my results:
\n| Number of models | \nNumber of estimators | \ntime per chunk | \noverall time | \n
|---|---|---|---|
| 3 | \n50 | \n1min12 | \n11 min | \n
| 3 | \n100 | \n2 min | \n20 min | \n
| 3 | \n300 | \n5min23 sec | \n51 min | \n
| 3 | \n1000 | \n22 min. | \n3h 43min | \n
As you can see, using the CPU kaggle kernel is not enough, as pointed out by @buumoo here.
\nHowever, it's probably ideal to use GPU but I got some limitations and memory errors in my pipeline with kaggle kernels, and my ranker is already trained in CPU.
\nUsing the TPU VM -3-8, the inference time is reduced by about 15 times :
\n| Number of models | \nNumber of estimators | \ntime per chunk | \noverall time | \n
|---|---|---|---|
| 3 | \n1000 | \n1 min36. | \n~15 min | \n
| \n\n | \n | \n |
The only constraint is that I couldn't use Polars, so I convert it to Pandas and you'll have to wait in order to use the TPU.
\nSummary:
\n| Case | \nDisadvantages | \nAdvantages | \n
|---|---|---|
| CPU | \nToo slow | \nimmediat use / you don't have to wait | \n
| TPU | \nFast / don't need to chunk | \nQueue / Be aware of the lib you're using | \n
Another essential point before executing your code in TPU, please be really careful, test your code before execution, because if you made a little typo, you'll have to wait again.
", "rawMarkdown": "Hello community, I wanted to share this experiments with you.\n\nI struggled for a long time with the Ranker inference, especially when using multiples rankers through GroupKFold or any other k-fold techniques.\n\nHere is my results:\n- I'm using **50 variables** ( it's important to note that ).\n- Those results concern 1/10 of the validation set, because I'm using 10 CHUNKS, just multiply the **time per chunk** by 10 to get the overall time elapsed .\n\n| Number of models | Number of estimators | time per chunk | overall time\n|---|---|---| ---|\n| 3 | 50 | 1min12 | 11 min\n| 3 | 100 | 2 min | 20 min\n| 3 | 300 | 5min23 sec | 51 min\n| 3 | 1000 | 22 min. | 3h 43min\n\n\nAs you can see, using the CPU kaggle kernel is not enough, as pointed out by @buumoo [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/381469). \n\nHowever, it's probably ideal to use GPU but I got some limitations and memory errors in my pipeline with kaggle kernels, and my ranker is already trained in CPU. \n\n\nUsing the TPU VM -3-8, the inference time is reduced by about 15 times :\n| Number of models | Number of estimators | time per chunk | overall time\n|---|---|---| ---|\n| 3 | 1000 | 1 min36. | ~15 min\n\n\nThe only constraint is that I couldn't use Polars, so I convert it to Pandas and you'll have to wait in order to use the TPU.\n\nSummary:\n| Case | Disadvantages | Advantages\n| --- | --- |\n| CPU | Too slow | immediat use / you don't have to wait\n| TPU | Fast / don't need to chunk | Queue / Be aware of the lib you're using \n\n*Another essential point before executing your code in TPU, please be really careful, test your code before execution, because if you made a little typo, you'll have to wait again.*", "votes": null }, { "id": "2118073", "postDate": "01/27/2023 18:39:34", "content": "You may want to use cuml.ForestInference to accelerate the inference speed for tree-based models, no matter it is trained by GPU or CPU.
", "rawMarkdown": "You may want to use cuml.ForestInference to accelerate the inference speed for tree-based models, no matter it is trained by GPU or CPU.", "votes": null }, { "id": "2119094", "postDate": "01/28/2023 13:49:17", "content": "is it faster than treelite ? If run on CPU ?
", "rawMarkdown": "is it faster than treelite ? If run on CPU ?", "votes": null }, { "id": "2119158", "postDate": "01/28/2023 14:21:35", "content": "If I am right, I think ForestInference is a GPU-only lib, I am not very sure.
\nIf runing it on GPU, it is supposed to be much faster than treelite.
\nI found a comparision figure for your reference.
\nanyway, the syntax is simple, just give it a shot, maybe it will surprise you.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4993592%2F8dd3e9d24a135ed92197153baeffe7b5%2F0_KAOl-8S4gIzFyhly.png?generation=1674915594600121&alt=media\")
You may want to use cuml.ForestInference to accelerate the inference speed for tree-based models, no matter it is trained by GPU or CPU.
", "votes": null, "replies": [ { "id": 2119094, "author_name": "nikhilmishradev", "author_url": "", "post_date": "01/28/2023 13:49:17", "content": "is it faster than treelite ? If run on CPU ?
", "votes": null, "replies": [ { "id": 2119158, "author_name": "buumoo", "author_url": "", "post_date": "01/28/2023 14:21:35", "content": "If I am right, I think ForestInference is a GPU-only lib, I am not very sure.
\nIf runing it on GPU, it is supposed to be much faster than treelite.
\nI found a comparision figure for your reference.
\nanyway, the syntax is simple, just give it a shot, maybe it will surprise you.
\n