{"cells":[{"metadata":{},"cell_type":"markdown","source":"# APTOS 2019 - [Image Super Resolution](https://github.com/idealo/image-super-resolution)\n\n---\n\nOne of my first thoughts in this competition was to undertand the quality of the given images.\n\nMy first aim with this kernel was to improve images quality to make better prediction, but after a few days, I realized that I couldn't take much advantage as predictive power from this process. That's because the images resolution is already pretty high and when we process them we usually reduce it around 256x256.\n\nHowever, I guess many laboratories can't have the same tools and, most probably, there are many hospitals around the world that can't afford taking those high quality images.\n\nSo, that's the goal of this kernel: \n> _Improving Diabetic Retinopathy images using free and open source tools!_\n\nThe process will be:\n\n**1. Start from a low resolution images**\n\n**2. Increase their resolution with \"Image Super Resolution\" - [ISR](https://github.com/idealo/image-super-resolution) project**\n\n**3. Compare the results with the original ones**\n\n---"},{"metadata":{},"cell_type":"markdown","source":"## What is exactly Image Super Resolution project?\n\n![ISR Image](https://idealo.github.io/image-super-resolution/figures/butterfly.png)\n\n**From their website:**\n\nThe goal of this project is to upscale and improve the quality of low resolution images.\n\nThis project contains Keras implementations of different Residual Dense Networks for Single Image Super-Resolution (ISR) as well as scripts to train these networks using content and adversarial loss components.\n\nThe implemented networks include:\n\n- The super-scaling Residual Dense Network described in [Residual Dense Network for Image Super-Resolution](https://arxiv.org/abs/1802.08797) (Zhang et al. 2018)\n- The super-scaling Residual in Residual Dense Network described in [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://arxiv.org/abs/1809.00219) (Wang et al. 2018)\n- A multi-output version of the Keras VGG19 network for deep features extraction used in the perceptual loss\n- A custom discriminator network based on the one described in [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://arxiv.org/abs/1609.04802) (SRGANS, Ledig et al. 2017)\n\nRead the full documentation at: https://idealo.github.io/image-super-resolution/.\n\nDocker scripts and Google Colab notebooks are available to carry training and prediction. Also, we provide scripts to facilitate training on the cloud with AWS and nvidia-docker with only a few commands.\n\nISR is compatible with Python 3.6 and is distributed under the Apache 2.0 license. We welcome any kind of contribution. If you wish to contribute, please see the Contribute section.\n\n![Arch](https://idealo.github.io/image-super-resolution/figures/RRDN.jpg)\n\n---\n\n_NOTE:_\n\nI Discovered this project following [Dat Tran](https://www.linkedin.com/in/dat-tran-a1602320/) on LinkedIn, you can always learn from his posts.\n\nHe and his team realized this amazing open source project that can be applied to many domains!\n\nHere is the full team:\n\n**Francesco Cardinale**, github: [cfrancesco](https://github.com/cfrancesco)\n\n**Zubin John**, github: [valiantone](https://github.com/valiantone)\n\n**Dat Tran**, github: [datitran](https://github.com/datitran)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# libraries\nimport numpy as np\nimport pandas as pd\n\nimport os\nprint(os.listdir(\"../input\"))\n\nimport cv2\nfrom PIL import Image\nimport matplotlib.pyplot as plt\n\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Firstly, We are going so resize the images to Low Resolution.. Let's say 200x200\nim_size = 200\n\n# Then, we apply ISR and see the results","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2019 Competition Image"},{"metadata":{"trusted":true},"cell_type":"code","source":"# I'm going to take just 2 images (new and old competition) and work on them\n\nnew_path = f\"../input/aptos2019-blindness-detection/train_images/d7bc00091cfc.png\"\nnew_image = cv2.imread(new_path)\nnew_image = cv2.cvtColor(new_image, cv2.COLOR_BGR2RGB)\nh, w, c = new_image.shape\nprint('Original Size: {}, {}'.format(w, h))\nnew_image = cv2.resize(new_image, (im_size,im_size))\n\nfig = plt.figure(figsize=(10,10))\nplt.imshow(new_image)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2015 Competition Image"},{"metadata":{"trusted":true},"cell_type":"code","source":"old_path = f\"../input/diabetic-retinopathy-resized/resized_train/resized_train/22_left.jpeg\"\nold_image = cv2.imread(old_path)\nold_image = cv2.cvtColor(old_image, cv2.COLOR_BGR2RGB)\nh, w, c = old_image.shape\nprint('Original Size: {}, {}'.format(w, h))\nold_image = cv2.resize(old_image, (im_size,im_size))\n\nfig = plt.figure(figsize=(10,10))\nplt.imshow(old_image)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Install ISR"},{"metadata":{"trusted":true,"_kg_hide-output":true,"collapsed":true},"cell_type":"code","source":"!pip install ISR","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true,"collapsed":true},"cell_type":"code","source":"!wget https://github.com/idealo/image-super-resolution/raw/master/weights/sample_weights/rdn-C6-D20-G64-G064-x2/ArtefactCancelling/rdn-C6-D20-G64-G064-x2_ArtefactCancelling_epoch219.hdf5\n!wget https://github.com/idealo/image-super-resolution/raw/master/weights/sample_weights/rdn-C6-D20-G64-G064-x2/PSNR-driven/rdn-C6-D20-G64-G064-x2_PSNR_epoch086.hdf5\n!wget https://github.com/idealo/image-super-resolution/raw/master/weights/sample_weights/rdn-C3-D10-G64-G064-x2/PSNR-driven/rdn-C3-D10-G64-G064-x2_PSNR_epoch134.hdf5\n!mkdir weights\n!mv *.hdf5 weights","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# import model\nfrom ISR.models import RDN","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nrdn = RDN(arch_params={'C': 6, 'D':20, 'G':64, 'G0':64, 'x':2})\nrdn.model.load_weights('weights/rdn-C6-D20-G64-G064-x2_ArtefactCancelling_epoch219.hdf5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nnew_image_isr = rdn.predict(np.asarray(new_image))\nold_image_isr = rdn.predict(np.asarray(old_image))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# This is to compare to the images created with ISR so we can better see differences\nold_image = cv2.resize(old_image,(im_size*2,im_size*2))\nnew_image = cv2.resize(new_image,(im_size*2,im_size*2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"f = plt.figure(figsize=(10,10))\nf.suptitle(\"Old Competition Image: Resized\", fontsize=16)\nplt.imshow(old_image)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"f = plt.figure(figsize=(10,10))\nf.suptitle(\"Old Competition Image: ImageSuperResolution\", fontsize=16)\nplt.imshow(old_image_isr)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"f = plt.figure(figsize=(10,10))\nf.suptitle(\"New Competition Image: Resized\", fontsize=16)\nplt.imshow(new_image)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"f = plt.figure(figsize=(10,10))\nf.suptitle(\"New Competition Image: ImageSuperResolution\", fontsize=16)\nplt.imshow(new_image_isr)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Conclusions\n\nWe can easily notice the differences between the processed images and the resized ones.\nFor instance, the outlines and veins details improved significantly!\n\nWhat do you think? Could it be useful for hospitals and labs to save in super expensive machinery?\n\nI would love to discuss further in comments\n\n---\n\n**If you enjoyed the kernel, please consider upvoting it.  xD **"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}