In this competition, we can use only 13GB CPU RAM and I would like to reduce memory usage. But I'm a beginner of pytorch and don't know how to free CPU RAM after transferring model to GPU RAM. Is there way to achieve this?
\nP.S. It is probably inappropriate to ask this question here, because this question is not related to this competition and doesn't need the knowledge of this competiton to answer.
In this competition, we can use only 13GB CPU RAM and I would like to reduce memory usage. But I'm a beginner of pytorch and don't know how to free CPU RAM after transferring model to GPU RAM. Is there way to achieve this?
\nP.S. It is probably inappropriate to ask this question here, because this question is not related to this competition and doesn't need the knowledge of this competiton to answer.
First world problems 😄
", "rawMarkdown": "First world problems 😄", "votes": 7 }, { "id": 1122714, "postDate": "2020-12-22T16:42:34.757Z", "content": "seems it is caused by pin memory buffers
\nhttps://pytorch.org/docs/stable/notes/cuda.html#use-pinned-memory-buffers
Hi @mamasinkgs thank you for the link. Did it help you reduce cpu ram usage? It is not working for me.
", "rawMarkdown": "Hi @mamasinkgs thank you for the link. Did it help you reduce cpu ram usage? It is not working for me. " } ] }, { "id": 1122404, "postDate": "2020-12-22T12:12:43.393Z", "content": "Frankly speaking, i am pretty sure that CPU RAM will be cleared for sure; Need to write some code to test that, that's all i have to do to verify.
", "rawMarkdown": "Frankly speaking, i am pretty sure that CPU RAM will be cleared for sure; Need to write some code to test that, that's all i have to do to verify." }, { "id": 1121932, "postDate": "2020-12-22T03:08:04.820Z", "content": "I found a discussion on PyTorch community, but it seems that this issue is not solved (at least in this case).
\nhttps://discuss.pytorch.org/t/how-to-free-cpu-ram-after-module-to-cuda-device/20381/5
Thank you!
", "rawMarkdown": "Thank you!" } ] }, { "id": 1121892, "postDate": "2020-12-22T02:17:01.573Z", "content": "Are you sure that it's the PyTorch that's causing you OOM's if any? This is a cool link to debug the tensors to track GPU mem.
", "rawMarkdown": "Are you sure that it's the PyTorch that's causing you OOM's if any? [This](https://forum.pyro.ai/t/a-clever-trick-to-debug-tensor-memory/556) is a cool link to debug the tensors to track GPU mem.", "replies": [ { "id": 1121911, "postDate": "2020-12-22T02:41:24.013Z", "content": "Thank you for a great link! I checked my model uses more than 1GB CPU RAM even after model.to(args.device), by using the great memory profiler https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/203020. For me, GPU RAM is sufficient, but CPU RAM is not sufficient.
", "rawMarkdown": "Thank you for a great link! I checked my model uses more than 1GB CPU RAM even after model.to(args.device), by using the great memory profiler https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/203020. For me, GPU RAM is sufficient, but CPU RAM is not sufficient." }, { "id": 1121913, "postDate": "2020-12-22T02:46:24.687Z", "content": "Are you using num_workers > 0? If so, it's spinning up additional processes, which will (almost) *n your memory usage in the various data loaders.
I think @authman Mamas is more worried about the model being copied to CPU first and then moved to GPU. But the copy is still left there in RAM even though the model is moved to GPU.
\nWe don't need num_workers here for sure! Rather we don't quite need the data-loader either as we can prepare the batch ourselves almost.. (wrt inference)
\nI am not quite sure what PyTorch does when it comes to this..
yes, i want pytorch expert that knows how to free memory. maybe I should ask this question in the issue of pytorch, because the content of the question is not related to this competition and does not need the knowledge of this competition.
", "rawMarkdown": "yes, i want pytorch expert that knows how to free memory. maybe I should ask this question in the issue of pytorch, because the content of the question is not related to this competition and does not need the knowledge of this competition." } ] } ], "comments": [ { "id": 1122281, "author_name": "Nikola Bacic", "author_url": "", "post_date": "2020-12-22T10:28:23.867000", "content": "First world problems 😄
", "votes": 7, "replies": [] }, { "id": 1122714, "author_name": "mamas", "author_url": "", "post_date": "2020-12-22T16:42:34.757000", "content": "seems it is caused by pin memory buffers
\nhttps://pytorch.org/docs/stable/notes/cuda.html#use-pinned-memory-buffers
Hi @mamasinkgs thank you for the link. Did it help you reduce cpu ram usage? It is not working for me.
", "votes": 0, "replies": [] } ] }, { "id": 1122404, "author_name": "Aditya Soni", "author_url": "", "post_date": "2020-12-22T12:12:43.393000", "content": "Frankly speaking, i am pretty sure that CPU RAM will be cleared for sure; Need to write some code to test that, that's all i have to do to verify.
", "votes": 0, "replies": [] }, { "id": 1121932, "author_name": "u++", "author_url": "", "post_date": "2020-12-22T03:08:04.820000", "content": "I found a discussion on PyTorch community, but it seems that this issue is not solved (at least in this case).
\nhttps://discuss.pytorch.org/t/how-to-free-cpu-ram-after-module-to-cuda-device/20381/5
Thank you!
", "votes": 0, "replies": [] } ] }, { "id": 1121892, "author_name": "Aditya Soni", "author_url": "", "post_date": "2020-12-22T02:17:01.573000", "content": "Are you sure that it's the PyTorch that's causing you OOM's if any? This is a cool link to debug the tensors to track GPU mem.
", "votes": 0, "replies": [ { "id": 1121911, "author_name": "mamas", "author_url": "", "post_date": "2020-12-22T02:41:24.013000", "content": "Thank you for a great link! I checked my model uses more than 1GB CPU RAM even after model.to(args.device), by using the great memory profiler https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/203020. For me, GPU RAM is sufficient, but CPU RAM is not sufficient.
", "votes": 0, "replies": [] }, { "id": 1121913, "author_name": "عثمان", "author_url": "", "post_date": "2020-12-22T02:46:24.687000", "content": "Are you using num_workers > 0? If so, it's spinning up additional processes, which will (almost) *n your memory usage in the various data loaders.
I think @authman Mamas is more worried about the model being copied to CPU first and then moved to GPU. But the copy is still left there in RAM even though the model is moved to GPU.
\nWe don't need num_workers here for sure! Rather we don't quite need the data-loader either as we can prepare the batch ourselves almost.. (wrt inference)
\nI am not quite sure what PyTorch does when it comes to this..
yes, i want pytorch expert that knows how to free memory. maybe I should ask this question in the issue of pytorch, because the content of the question is not related to this competition and does not need the knowledge of this competition.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1121862": "In this competition, we can use only 13GB CPU RAM and I would like to reduce memory usage. But I'm a beginner of pytorch and don't know how to free CPU RAM after transferring model to GPU RAM. Is there way to achieve this?\nP.S. It is probably inappropriate to ask this question here, because this question is not related to this competition and doesn't need the knowledge of this competiton to answer.", "1122281": "First world problems 😄", "1122714": "seems it is caused by pin memory buffers\nhttps://pytorch.org/docs/stable/notes/cuda.html#use-pinned-memory-buffers", "1122404": "Frankly speaking, i am pretty sure that CPU RAM will be cleared for sure; Need to write some code to test that, that's all i have to do to verify.", "1121932": "I found a discussion on PyTorch community, but it seems that this issue is not solved (at least in this case). \nhttps://discuss.pytorch.org/t/how-to-free-cpu-ram-after-module-to-cuda-device/20381/5", "1121892": "Are you sure that it's the PyTorch that's causing you OOM's if any? [This](https://forum.pyro.ai/t/a-clever-trick-to-debug-tensor-memory/556) is a cool link to debug the tensors to track GPU mem." } }