Just a note for those of you using PIL and/or default torchvision transforms. If you are going to use PIL, just don't. Uninstall it, and instead install PIL-SIMD. It's not just a little faster, it's way faster. Torchvision transforms uses PIL. PIL-SIMD is a drop in replacement for PIL, so just uninstall PIL and install PIL-SIMD instead.
\n\nInstructions here: https://docs.fast.ai/performance.html
\n\nAlso, you should note that OpenCV2 is way faster than PIL, and faster than PIL-SIMD as well. Here is one comparison of OpenCV to PIL:\nhttps://www.kaggle.com/vfdev5/pil-vs-opencv
\n\nHere is some data showing images/sec of various libraries (scroll to Benchmarking Results):\nhttps://github.com/albumentations-team/albumentations
\n\nAlso as noted in the fastai link above, you should make sure your libraries are linked with libjpeg-turbo.
\n\nThe basic differences of using say OpenCV vs PIL can be 10x, so this is definitely something to look into.
", "messages": [ { "id": "953861", "postDate": "08/01/2020 07:32:04", "content": "Just a note for those of you using PIL and/or default torchvision transforms. If you are going to use PIL, just don't. Uninstall it, and instead install PIL-SIMD. It's not just a little faster, it's way faster. Torchvision transforms uses PIL. PIL-SIMD is a drop in replacement for PIL, so just uninstall PIL and install PIL-SIMD instead.
\n\nInstructions here: https://docs.fast.ai/performance.html
\n\nAlso, you should note that OpenCV2 is way faster than PIL, and faster than PIL-SIMD as well. Here is one comparison of OpenCV to PIL:\nhttps://www.kaggle.com/vfdev5/pil-vs-opencv
\n\nHere is some data showing images/sec of various libraries (scroll to Benchmarking Results):\nhttps://github.com/albumentations-team/albumentations
\n\nAlso as noted in the fastai link above, you should make sure your libraries are linked with libjpeg-turbo.
\n\nThe basic differences of using say OpenCV vs PIL can be 10x, so this is definitely something to look into.
", "rawMarkdown": "Just a note for those of you using PIL and/or default torchvision transforms. If you are going to use PIL, just don't. Uninstall it, and instead install PIL-SIMD. It's not just a little faster, it's way faster. Torchvision transforms uses PIL. PIL-SIMD is a drop in replacement for PIL, so just uninstall PIL and install PIL-SIMD instead.\n\nInstructions here: https://docs.fast.ai/performance.html\n\nAlso, you should note that OpenCV2 is way faster than PIL, and faster than PIL-SIMD as well. Here is one comparison of OpenCV to PIL:\nhttps://www.kaggle.com/vfdev5/pil-vs-opencv\n\nHere is some data showing images/sec of various libraries (scroll to Benchmarking Results):\nhttps://github.com/albumentations-team/albumentations\n\nAlso as noted in the fastai link above, you should make sure your libraries are linked with libjpeg-turbo.\n\nThe basic differences of using say OpenCV vs PIL can be 10x, so this is definitely something to look into.", "votes": null }, { "id": "954669", "postDate": "08/02/2020 01:45:54", "content": "Thanks for the comparison. I switched from PIL to OpenCV2 a few months ago after noticing the speed increase and I've been pleased with the performance of CV2.
", "rawMarkdown": "Thanks for the comparison. I switched from PIL to OpenCV2 a few months ago after noticing the speed increase and I've been pleased with the performance of CV2.", "votes": null }, { "id": "955613", "postDate": "08/02/2020 18:29:21", "content": "If you want the fastest, try TurboJpeg: https://www.learnopencv.com/efficient-image-loading/
", "rawMarkdown": "If you want the fastest, try TurboJpeg: https://www.learnopencv.com/efficient-image-loading/", "votes": null }, { "id": "956216", "postDate": "08/03/2020 10:27:32", "content": "Okay😪. Now I understand why my training is taking an entire century to run. Thanks for the mention.
", "rawMarkdown": "Okay😪. Now I understand why my training is taking an entire century to run. Thanks for the mention.", "votes": null }, { "id": "956302", "postDate": "08/03/2020 11:54:16", "content": "PIL-SIMD is a drop in replacement to PIL, just make sure you uninstall PIL/pillow and verify you are running PIL-SIMD. Also make sure you compile with libjpeg-turbo. If you switch from PIL to CV2 then you have to redo a bit of stuff, including how you do transforms, but the library albumentations will do everything you are used to with torchvision and then some. Another option is torchtoolbox which uses CV2 and replicates everything in torchvision transforms. You can compile CV2 with libjpeg-turbo as well.
", "rawMarkdown": "PIL-SIMD is a drop in replacement to PIL, just make sure you uninstall PIL/pillow and verify you are running PIL-SIMD. Also make sure you compile with libjpeg-turbo. If you switch from PIL to CV2 then you have to redo a bit of stuff, including how you do transforms, but the library albumentations will do everything you are used to with torchvision and then some. Another option is torchtoolbox which uses CV2 and replicates everything in torchvision transforms. You can compile CV2 with libjpeg-turbo as well.", "votes": null } ], "comments": [ { "id": 954669, "author_name": "cdeotte", "author_url": "", "post_date": "08/02/2020 01:45:54", "content": "Thanks for the comparison. I switched from PIL to OpenCV2 a few months ago after noticing the speed increase and I've been pleased with the performance of CV2.
", "votes": null, "replies": [] }, { "id": 955613, "author_name": "anjum48", "author_url": "", "post_date": "08/02/2020 18:29:21", "content": "If you want the fastest, try TurboJpeg: https://www.learnopencv.com/efficient-image-loading/
", "votes": null, "replies": [] }, { "id": 956216, "author_name": "tawheedrony", "author_url": "", "post_date": "08/03/2020 10:27:32", "content": "Okay😪. Now I understand why my training is taking an entire century to run. Thanks for the mention.
", "votes": null, "replies": [ { "id": 956302, "author_name": "brianfeeny", "author_url": "", "post_date": "08/03/2020 11:54:16", "content": "PIL-SIMD is a drop in replacement to PIL, just make sure you uninstall PIL/pillow and verify you are running PIL-SIMD. Also make sure you compile with libjpeg-turbo. If you switch from PIL to CV2 then you have to redo a bit of stuff, including how you do transforms, but the library albumentations will do everything you are used to with torchvision and then some. Another option is torchtoolbox which uses CV2 and replicates everything in torchvision transforms. You can compile CV2 with libjpeg-turbo as well.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "953861": "Just a note for those of you using PIL and/or default torchvision transforms. If you are going to use PIL, just don't. Uninstall it, and instead install PIL-SIMD. It's not just a little faster, it's way faster. Torchvision transforms uses PIL. PIL-SIMD is a drop in replacement for PIL, so just uninstall PIL and install PIL-SIMD instead.\n\nInstructions here: https://docs.fast.ai/performance.html\n\nAlso, you should note that OpenCV2 is way faster than PIL, and faster than PIL-SIMD as well. Here is one comparison of OpenCV to PIL:\nhttps://www.kaggle.com/vfdev5/pil-vs-opencv\n\nHere is some data showing images/sec of various libraries (scroll to Benchmarking Results):\nhttps://github.com/albumentations-team/albumentations\n\nAlso as noted in the fastai link above, you should make sure your libraries are linked with libjpeg-turbo.\n\nThe basic differences of using say OpenCV vs PIL can be 10x, so this is definitely something to look into.", "954669": "Thanks for the comparison. I switched from PIL to OpenCV2 a few months ago after noticing the speed increase and I've been pleased with the performance of CV2.", "955613": "If you want the fastest, try TurboJpeg: https://www.learnopencv.com/efficient-image-loading/", "956216": "Okay😪. Now I understand why my training is taking an entire century to run. Thanks for the mention.", "956302": "PIL-SIMD is a drop in replacement to PIL, just make sure you uninstall PIL/pillow and verify you are running PIL-SIMD. Also make sure you compile with libjpeg-turbo. If you switch from PIL to CV2 then you have to redo a bit of stuff, including how you do transforms, but the library albumentations will do everything you are used to with torchvision and then some. Another option is torchtoolbox which uses CV2 and replicates everything in torchvision transforms. You can compile CV2 with libjpeg-turbo as well." }, "source": "meta" }