In this competition, for the first time, I'm trying to work only with PyTorch.
\nWith TF I know that loading of data from TFRecords is really really fast and, in addition, it is easy to work with TPU.
\nI'm trying to optimize my pipeline with PyTorch and with GPU and what I find frustrating is that the bottleneck is image loading.
The best thing I have found so far is :
\nI know that I could use NVIDIA DALI, but, as far I understand I should implement the entire pipeline in DALI with a code very different from the usual PyTorch and I would need to create a \"dataset\" for DALI, for the inference part.
\nHow could I speed image loading, what are the best approaches?
", "messages": [ { "id": "1094173", "postDate": "11/28/2020 11:30:59", "content": "In this competition, for the first time, I'm trying to work only with PyTorch.
\nWith TF I know that loading of data from TFRecords is really really fast and, in addition, it is easy to work with TPU.
\nI'm trying to optimize my pipeline with PyTorch and with GPU and what I find frustrating is that the bottleneck is image loading.
The best thing I have found so far is :
\nI know that I could use NVIDIA DALI, but, as far I understand I should implement the entire pipeline in DALI with a code very different from the usual PyTorch and I would need to create a \"dataset\" for DALI, for the inference part.
\nHow could I speed image loading, what are the best approaches?
", "rawMarkdown": "In this competition, for the first time, I'm trying to work only with PyTorch.\nWith TF I know that loading of data from TFRecords is really really fast and, in addition, it is easy to work with TPU.\nI'm trying to optimize my pipeline with PyTorch and with GPU and what I find frustrating is that the bottleneck is image loading.\n\nThe best thing I have found so far is :\n* use Albumentations\n* use jpeg4py (about 15% gain in speed)\n\nI know that I could use NVIDIA DALI, but, as far I understand I should implement the entire pipeline in DALI with a code very different from the usual PyTorch and I would need to create a \"dataset\" for DALI, for the inference part.\n\nHow could I speed image loading, what are the best approaches?", "votes": null }, { "id": "1095259", "postDate": "11/29/2020 12:12:01", "content": "I don't understand. What's wrong with loading images using OpenCV or Pillow? PyTorch offers Dataset and Dataloader which allow loading images in batches. For me, my inference code that generates predictions over ~15k images during submission takes around 4-5 mins. I've tested with EfficientNet B4 and image size 384. It would take less time if you use smaller models with smaller images.
It is not wrong. But you're loading a bunch of images one file a time. If you try with TF and TFRecords with TPU you'll see that with TFRecords you're loading faster. One operation that is slow on CPU is JPEG image decompression. With JPeg4Py it is faster than openCV and with DALI is more faster. I was asking if there is something we could do in PyTorch in addition to num_workers
", "rawMarkdown": "It is not wrong. But you're loading a bunch of images one file a time. If you try with TF and TFRecords with TPU you'll see that with TFRecords you're loading faster. One operation that is slow on CPU is JPEG image decompression. With JPeg4Py it is faster than openCV and with DALI is more faster. I was asking if there is something we could do in PyTorch in addition to num_workers", "votes": null } ], "comments": [ { "id": 1095259, "author_name": "tahsin", "author_url": "", "post_date": "11/29/2020 12:12:01", "content": "I don't understand. What's wrong with loading images using OpenCV or Pillow? PyTorch offers Dataset and Dataloader which allow loading images in batches. For me, my inference code that generates predictions over ~15k images during submission takes around 4-5 mins. I've tested with EfficientNet B4 and image size 384. It would take less time if you use smaller models with smaller images.
It is not wrong. But you're loading a bunch of images one file a time. If you try with TF and TFRecords with TPU you'll see that with TFRecords you're loading faster. One operation that is slow on CPU is JPEG image decompression. With JPeg4Py it is faster than openCV and with DALI is more faster. I was asking if there is something we could do in PyTorch in addition to num_workers
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1094173": "In this competition, for the first time, I'm trying to work only with PyTorch.\nWith TF I know that loading of data from TFRecords is really really fast and, in addition, it is easy to work with TPU.\nI'm trying to optimize my pipeline with PyTorch and with GPU and what I find frustrating is that the bottleneck is image loading.\n\nThe best thing I have found so far is :\n* use Albumentations\n* use jpeg4py (about 15% gain in speed)\n\nI know that I could use NVIDIA DALI, but, as far I understand I should implement the entire pipeline in DALI with a code very different from the usual PyTorch and I would need to create a \"dataset\" for DALI, for the inference part.\n\nHow could I speed image loading, what are the best approaches?", "1095259": "I don't understand. What's wrong with loading images using `OpenCV` or `Pillow`? PyTorch offers `Dataset` and `Dataloader` which allow loading images in batches. For me, my inference code that generates predictions over ~15k images during submission takes around 4-5 mins. I've tested with EfficientNet B4 and image size 384. It would take less time if you use smaller models with smaller images.", "1095698": "It is not wrong. But you're loading a bunch of images one file a time. If you try with TF and TFRecords with TPU you'll see that with TFRecords you're loading faster. One operation that is slow on CPU is JPEG image decompression. With JPeg4Py it is faster than openCV and with DALI is more faster. I was asking if there is something we could do in PyTorch in addition to num_workers" }, "source": "meta" }