I'm new to fastai but I'm decent with pytorch. I noticed that when I'm training my classification network, Volatile GPU utilization jumps between 0% and 100% constantly (around 60% of the time it's 0%, with spikes of 100%). Also, all CPU cores were used 100%. This is different from pytorch behavior where CPU utilization per core were around 20%~30% and volatile GPU utilization was constantly high. Is it a sign that fastai is slower than pytorch?
", "messages": [ { "id": "410947", "postDate": "10/27/2018 01:41:03", "content": "I'm new to fastai but I'm decent with pytorch. I noticed that when I'm training my classification network, Volatile GPU utilization jumps between 0% and 100% constantly (around 60% of the time it's 0%, with spikes of 100%). Also, all CPU cores were used 100%. This is different from pytorch behavior where CPU utilization per core were around 20%~30% and volatile GPU utilization was constantly high. Is it a sign that fastai is slower than pytorch?
", "rawMarkdown": "I'm new to fastai but I'm decent with pytorch. I noticed that when I'm training my classification network, Volatile GPU utilization jumps between 0% and 100% constantly (around 60% of the time it's 0%, with spikes of 100%). Also, all CPU cores were used 100%. This is different from pytorch behavior where CPU utilization per core were around 20%~30% and volatile GPU utilization was constantly high. Is it a sign that fastai is slower than pytorch?", "votes": null }, { "id": "410957", "postDate": "10/27/2018 02:47:33", "content": "I think it has to do with how data is loaded.
", "rawMarkdown": "I think it has to do with how data is loaded.", "votes": null }, { "id": "410978", "postDate": "10/27/2018 03:42:48", "content": "fast.ai is just a wrapper around pytorch. So, it should be the same in terms of speed. But there are many tools and tricks built in fast.ai that it is very easy to write an efficient code. If one did something similar with plant pytorch, it would be much more work, and there would be many places where things could be done not effectively or screw up.
", "rawMarkdown": "fast.ai is just a wrapper around pytorch. So, it should be the same in terms of speed. But there are many tools and tricks built in fast.ai that it is very easy to write an efficient code. If one did something similar with plant pytorch, it would be much more work, and there would be many places where things could be done not effectively or screw up.", "votes": null }, { "id": "411000", "postDate": "10/27/2018 06:18:17", "content": "I got the same problem before. The reason is your dataloader. You take to much time for preprocessing, then your GPU is waiting for your data ready. So you can see your GPU jumps between 0% and 100%.
\nThe solutions can be:
\n 1. Improve your preprocessing (load image, augmentation,...)
\n 2. Using cache
Got it, thank you. I was trying to establish a baseline in fastai, and now that the LB seems to be good, I'm moving on with my own pytorch implementation.
", "rawMarkdown": "Got it, thank you. I was trying to establish a baseline in fastai, and now that the LB seems to be good, I'm moving on with my own pytorch implementation.", "votes": null } ], "comments": [ { "id": 410957, "author_name": "alexanderliao", "author_url": "", "post_date": "10/27/2018 02:47:33", "content": "I think it has to do with how data is loaded.
", "votes": null, "replies": [] }, { "id": 410978, "author_name": "iafoss", "author_url": "", "post_date": "10/27/2018 03:42:48", "content": "fast.ai is just a wrapper around pytorch. So, it should be the same in terms of speed. But there are many tools and tricks built in fast.ai that it is very easy to write an efficient code. If one did something similar with plant pytorch, it would be much more work, and there would be many places where things could be done not effectively or screw up.
", "votes": null, "replies": [] }, { "id": 411000, "author_name": "backaggle", "author_url": "", "post_date": "10/27/2018 06:18:17", "content": "I got the same problem before. The reason is your dataloader. You take to much time for preprocessing, then your GPU is waiting for your data ready. So you can see your GPU jumps between 0% and 100%.
\nThe solutions can be:
\n 1. Improve your preprocessing (load image, augmentation,...)
\n 2. Using cache
Got it, thank you. I was trying to establish a baseline in fastai, and now that the LB seems to be good, I'm moving on with my own pytorch implementation.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "410947": "I'm new to fastai but I'm decent with pytorch. I noticed that when I'm training my classification network, Volatile GPU utilization jumps between 0% and 100% constantly (around 60% of the time it's 0%, with spikes of 100%). Also, all CPU cores were used 100%. This is different from pytorch behavior where CPU utilization per core were around 20%~30% and volatile GPU utilization was constantly high. Is it a sign that fastai is slower than pytorch?", "410957": "I think it has to do with how data is loaded.", "410978": "fast.ai is just a wrapper around pytorch. So, it should be the same in terms of speed. But there are many tools and tricks built in fast.ai that it is very easy to write an efficient code. If one did something similar with plant pytorch, it would be much more work, and there would be many places where things could be done not effectively or screw up.", "411000": "I got the same problem before. The reason is your dataloader. You take to much time for preprocessing, then your GPU is waiting for your data ready. So you can see your GPU jumps between 0% and 100%. \nThe solutions can be: \n 1. Improve your preprocessing (load image, augmentation,...) \n 2. Using cache", "411147": "Got it, thank you. I was trying to establish a baseline in fastai, and now that the LB seems to be good, I'm moving on with my own pytorch implementation." }, "source": "meta" }