{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://i.redd.it/nklty63uzav41.png)","metadata":{}},{"cell_type":"markdown","source":"# Introduction\n\nIn this notebook, let's get to know about the fastest way of reading and processing images. We will use iWildCam2021 dataset and compare various libraries available using a benchmarking task. A fast and efficient data loading pipeline will help us train our models faster and **save a lot of GPU time**.\n\nNotebook is distributed in following sections -\n1. Libraries for image processing\n2. Create a benchmark task\n3. Compare the libraries\n4. Conclusion\n5. References and further reading","metadata":{}},{"cell_type":"markdown","source":"
\n

1. Libraries for image processing


\n
\n\n### The most famous ones - \n1. OpenCV\n2. Scikit Image\n3. Pillow\n4. Pillow-SIMD\n\nThere is a lot of documentation available for all of them. I recommend reading basic things about all of them in case you are not aware.","metadata":{}},{"cell_type":"code","source":"# OpenCV\nimport cv2\n\n# Scikit-Image\nimport skimage\nfrom skimage import io\nfrom skimage.transform import resize\n\n# Pillow/PIL\nimport PIL\nfrom PIL import Image\n\n# To use Pillow-simd you just need to unistall PIL and install PIL-simd\n\n# Other helper libraries\nimport json\nfrom matplotlib import pyplot as plt","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Versions used\nprint(\"OpenCV: \", cv2.__version__)\nprint(\"Scikit Image: \", skimage.__version__)\nprint(\"PIL: \", PIL.__version__)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

2. Create a benchmark task


\n
\n\n
\n\nWe will ofcourse use [iWildCam2021](https://www.kaggle.com/c/iwildcam2021-fgvc8/overview) dataset. I recommend going through this wonderful [notebook](https://www.kaggle.com/nayuts/iwildcam-2021-starter-notebook) if you are not fully aware about the competition and dataset. \n\nThe task will be simple but a little tiresome.\n\n**The task is to read 500 images and for each image, crop the annotated detections (provided in json file) and then resize the same.**\n\n1. For each image in 500 images
\n i. Read the image
\n ii. Crop the detections
\n iii. Resize the cropped detections
\n\nThis task will test Reading, Cropping and Resizing operations of all libraries which is core to any image preprocessing pipeline","metadata":{}},{"cell_type":"code","source":"with open('../input/iwildcam2021-fgvc8/metadata/iwildcam2021_megadetector_results.json', encoding='utf-8') as json_file:\n detections = json.load(json_file)\n\n# detections['images'][0] looks like\n# {\n# 'detections': [{'category': '1', 'bbox': [0.6529, 0.5425, 0.3471, 0.4038], 'conf': 0.999}], \n# 'id': '905a3c8c-21bc-11ea-a13a-137349068a90', \n# 'max_detection_conf': 0.999\n# }\n\ndata = {} # {image_name:[detections]}\n\ntotal_im_reads = 500\ntotal_im_crops = 0\nfor detection in detections['images'][:total_im_reads]:\n data[detection['id']+ '.jpg'] = [x['bbox'] for x in detection['detections']]\n total_im_crops += len(detection['detections'])\n \nprint(f\"The benchmarking task involves reading images {total_im_reads} times, cropping and resizing images {total_im_crops} times.\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We have created a dictionary {} called data which has names and detections of our images.","metadata":{}},{"cell_type":"markdown","source":"
\n

3. Compare the libraries


\n
\n\n
\n\nWe will use %timeit module to measure the time. And every crop will be resized to 200 x 200 image size. ","metadata":{}},{"cell_type":"code","source":"SIZE = 200","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. OpenCV","metadata":{}},{"cell_type":"code","source":"def numpy_crop_image(img, bbox):\n h,w,c = img.shape\n x1, y1,w_box, h_box = bbox\n ymin,xmin,ymax,xmax = y1, x1, y1 + h_box, x1 + w_box\n ymin,xmin,ymax,xmax = ymin*h,xmin*w,ymax*h,xmax*w\n crop_img = img[int(ymin):int(ymax), int(xmin):int(xmax)]\n return crop_img\n\ndef opencv_task():\n for image, detections in data.items():\n img = cv2.imread(\"../input/iwildcam2021-fgvc8/train/\"+image)\n for detection in detections:\n crop = numpy_crop_image(img, detection)\n resized = cv2.resize(crop,(SIZE,SIZE))\n results.append(resized)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = []\n%timeit opencv_task()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# See some results\nfig = plt.figure(figsize=(25, 25))\nfor i,result in enumerate(results[:16]):\n ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n ax.title.set_text(f'OpenCV {i}')\n plt.imshow(result)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Scikit-Image\n\nOpenCV and Scikit-Image both work with numpy arrays. Thus, both use numpy array slicing for cropping images. And hence we will use extact same function for both of these.","metadata":{}},{"cell_type":"code","source":"def skimage_task():\n for image, detections in data.items():\n img = io.imread(\"../input/iwildcam2021-fgvc8/train/\"+image)\n for detection in detections:\n crop = numpy_crop_image(img, detection)\n resized = resize(crop,(SIZE,SIZE))\n results.append(resized)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = []\n%timeit skimage_task()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# See some results\nfig = plt.figure(figsize=(25, 25))\nfor i,result in enumerate(results[:16]):\n ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n ax.title.set_text(f'Scikit-Image {i}')\n plt.imshow(result)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. PIL","metadata":{}},{"cell_type":"code","source":"def pil_crop_image(bbox, image_size):\n x1, y1,w_box, h_box = bbox\n ymin,xmin,ymax, xmax = y1, x1, y1 + h_box, x1 + w_box\n area = (xmin * image_size[0], ymin * image_size[1], \n xmax * image_size[0], ymax * image_size[1])\n return area\n\ndef pil_task():\n for image, detections in data.items():\n img = Image.open(\"../input/iwildcam2021-fgvc8/train/\"+image)\n for detection in detections:\n area = pil_crop_image(detection, img.size)\n crop = img.crop(area)\n resized = crop.resize((SIZE,SIZE))\n results.append(resized)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = []\n%timeit pil_task()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# See some results\nfig = plt.figure(figsize=(25, 25))\nfor i,result in enumerate(results[:16]):\n ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n ax.title.set_text(f'PIL {i}')\n plt.imshow(result)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Pillow-simd\n\n
\n\nTo use pillow-simd you just need to unistall pillow and install pillow-simd. And it works exactly same as normal PIL.","metadata":{}},{"cell_type":"code","source":"!pip uninstall -y pillow\n!pip install pillow-simd","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = []\n%timeit pil_task()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# See some results\nfig = plt.figure(figsize=(25, 25))\nfor i,result in enumerate(results[:16]):\n ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n ax.title.set_text(f'PIL-SIMD {i}')\n plt.imshow(result)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

4. Conclusion


\n
\n\n
\n\n### Winner : PIL 🔥🔥🔥\n\n
\n\n### Comparison :\n
\n\nWe ran each method of the same task 7 times and following is the average time for one loop. Width of each bar = (2 x (mean_time x 10)) pixels\n
Opencv: 17.5 s ± 66 ms
\n
Skimage: 49.1 s ± 2.26 s
\n
PIL: 11.5 s ± 33.2 ms
\n
PIL-simd: 11.6 s ± 53.1 ms
\n\n
\n\nEven if PIL-SIMD is highly optimized library, we see that PIL performs slightly better. \n\n**Only a very detailed benchmarking would reveal strengths and weaknesses of each library. But as of now, to do the basic reading, cropping and resizing we see that PIL is the fastest.**","metadata":{}},{"cell_type":"markdown","source":"
\n

5. References and further reading


\n
\n\n
\n\nhttps://learnopencv.com/efficient-image-loading/\n\nhttps://python-pillow.org/pillow-perf/\n\nhttps://www.kaggle.com/vfdev5/pil-vs-opencv","metadata":{}},{"cell_type":"markdown","source":"
\n
\n

Please don't forget to upvote if you find this notebook useful :)

\n
\n
","metadata":{}}]}