![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F83267%2Fc143f9c3a84bb7767f957142dd3a5cfd%2Fcpu_with_annoy.png?generation=1670472787700069&alt=media\")
UPDATE: I seem to have a tendency for \"rediscovering\" solutions to problems that my colleagues already have implemented a much better solution to π For running NN search on the GPU please see: [Matrix Factorization with GPU: 6.5x faster!] by @cpmpm.
\nUsing cuml is a very elegant solution to this problem π
Plus you get many algorithms that do NN search on the GPU for free!
\nIf you would like to read a justification of why that is important, please read the post below.
\nThank you for stopping by π
\nHey,
\nANN (approximate nearest neighbor search) helps us deal with a situation where there are more items to retrieve than our compute can handle.
\nI initially jumped to using ANN because that is generally what people do with millions of items to recommend, but I didn't realize how far you can get when running NN (nearest neighbor search) on the GPU!
\nHere are the results (the first are runtime on the CPU and the scores with ANN, the second are on the GPU):
\nCPU ANN:
\n
GPU NN:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F83267%2F24bbd1d97f66938dcc577976877b0051%2FGPU_with_faiss.png?generation=1670472817907642&alt=media\")
In this relatively simple example, ANN gets worse results and takes longer to run than NN on the GPU!
\nBut for the benchmarking to be complete, I attempted to run NN both on the GPU and the CPU. The CPU is quite powerful (24-core AMD Ryzen 9 3900X)
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F83267%2F7fa1b89c3a12d39b8a6d68af91c5732d%2FScreenshot%202022-12-08%20135420.png?generation=1670471752759475&alt=media\")
NN search on the GPU is 72x faster!
\nI was very excited when I stumbled into this π This not only helps you get a better result in much shorter time, but also (which I really appreciate) takes away the overhead of having to find the number of trees for your ANN index to get a good trade-off between run time and performance, etc (in fact, once you move to more complex and powerful ANN implementations than annoy there are even more hyperparameters to tune!)
Of course, beyond a certain point even the sheer speed of the GPU will not save you and you will have to switch to ANN. But it was quite amazing to me that with 1.7 million aids and a 32-dimensional vector this executed so quickly πThat GPUs push the boundaries of what is possible in this domain by that much (NN search is a key component of many RecSys solutions).
\nWanted to share this with you as this can definitely streamline the solutions and lead to an improved result. Happy Kaggling! π
\nUPDATE: I seem to have a tendency for \"rediscovering\" solutions to problems that my colleagues already have implemented a much better solution to π For running NN search on the GPU please see: [Matrix Factorization with GPU: 6.5x faster!] by @cpmpm.
\nUsing cuml is a very elegant solution to this problem π
Plus you get many algorithms that do NN search on the GPU for free!
\nIf you would like to read a justification of why that is important, please read the post below.
\nThank you for stopping by π
\nHey,
\nANN (approximate nearest neighbor search) helps us deal with a situation where there are more items to retrieve than our compute can handle.
\nI initially jumped to using ANN because that is generally what people do with millions of items to recommend, but I didn't realize how far you can get when running NN (nearest neighbor search) on the GPU!
\nHere are the results (the first are runtime on the CPU and the scores with ANN, the second are on the GPU):
\nCPU ANN:
\n
GPU NN:
\n
In this relatively simple example, ANN gets worse results and takes longer to run than NN on the GPU!
\nBut for the benchmarking to be complete, I attempted to run NN both on the GPU and the CPU. The CPU is quite powerful (24-core AMD Ryzen 9 3900X)
\n
NN search on the GPU is 72x faster!
\nI was very excited when I stumbled into this π This not only helps you get a better result in much shorter time, but also (which I really appreciate) takes away the overhead of having to find the number of trees for your ANN index to get a good trade-off between run time and performance, etc (in fact, once you move to more complex and powerful ANN implementations than annoy there are even more hyperparameters to tune!)
Of course, beyond a certain point even the sheer speed of the GPU will not save you and you will have to switch to ANN. But it was quite amazing to me that with 1.7 million aids and a 32-dimensional vector this executed so quickly πThat GPUs push the boundaries of what is possible in this domain by that much (NN search is a key component of many RecSys solutions).
\nWanted to share this with you as this can definitely streamline the solutions and lead to an improved result. Happy Kaggling! π
\n