{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Filter Application and Data Visualization"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Goals\nThis notebook is about image preprocessing to optimize any model you use, whether it be a typical CNN, or a U-net, or anything else.\n1. Analyze and get a feel for the data\n2. Filtration and Image Normalization:\n    1. Application of image zero-padding using numpy.pad\n    2. Application of Contrast Limited Adaptive Histogram Equalization (referred to as CLAHE from now on)<sup>1</sup>\n3. Apply scientifically described and trained models to identify and separate medical devices from organs <sup>2</sup>\n4. Discuss image filtration techniques to provide before convolutions\n\n<sup>1</sup> http://www.cs.unc.edu/techreports/90-035.pdf\n\n<sup>2</sup> https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6113157/\n\nInterested in separate applications of CLAHE, in Python:\n\nThis one describes a model of CLAHE, and some of the limits of traditional models: https://towardsdatascience.com/increase-your-face-recognition-models-accuracy-by-improving-face-contrast-a3e71bb6b9fb\n\nThis one describes the opencv application of adaptive Histogram equalization on grayscale images: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_histograms/py_histogram_equalization/py_histogram_equalization.html\n\nSame as above, but in \"layman's terms\" (is anything AI ever in layman's terms): https://www.geeksforgeeks.org/clahe-histogram-eqalization-opencv/\n\nScikit Image application of CLAHE (not used below): https://scikit-image.org/docs/dev/api/skimage.exposure.html#skimage.exposure.equalize_adapthist"},{"metadata":{},"cell_type":"markdown","source":"## Imports"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# Basics\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os # File/directory scanning and editting\n\n# Image Processing\nfrom PIL import Image \nimport cv2 as cv\n\n# Image Displaying\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Directory Structure"},{"metadata":{"trusted":true},"cell_type":"code","source":"# File Folders:\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    print(dirname)\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Files\ndisplay_folders = \"n\" # y = display, anything else = no\nif (display_folders == 'y'):\n    for dirname, _, filenames in os.walk('/kaggle/input'):\n        print(\"-\"*30, \"\\n\"*10,\"-\"*30, sep = \"\\n\")\n        for filename in filenames:\n            print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Image Padding and Contrast Limited Adaptive Histogram Equalization (CLAHE)"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Image Padding:\n\n1. We have to figure out the maximum dimensions of the images\n2. We convert images to a numpy array\n3. We apply the numpy.pad function, and voilà, you have your padded image\n4. Optional: Convert numpy array to image type, or so that it is accessible by a library (such as PIL or OpenCV)"},{"metadata":{},"cell_type":"markdown","source":"### A \"quick\" discussion on image resizing/padding:\n\n\n* Should I resize my images, pad my images, or both? There are pros and cons to both approaches:\n1. Resizing:\n    * Shrinking:\n        * Pros:\n            * Speeds up training process\n            * Reduces RAM requirement\n            * Allows for larger batch sizes\n            * Can focus images on larger portions of images\n        * Cons:\n            * Reduces image quality\n            * Removes finer details of the image\n    * Enlarging:\n        * Pros:\n            * Keeps all details of image\n            * Allows for more pooling layers/convolutional layers, allowing for more sophisticated networks\n            * Allows network to train on finer details of image\n        * Cons:\n            * Increases training time\n            * Uses large amount or RAM\n            * Smaller batch sizes often required\n            * Images are often pixelated and/or stretched\n                * **Note:** Images are not necessarily stretched, in either scenario, as one can shrink/enlarge keeping the same aspect ratio, and then pad for the remaining pixels.\n                    * Example: Say I have a 100 by 200 pixel image, and I want it to be shrunk to a size of 50 by 50. I have two options: \n                        1. No Padding:\n                            * Reduce width by a factor of 2, and the height by a factor of 4.\n                        2. Padding:\n                            * Reduct width and height by a factor of 4\n                            * Pad an extra 25 by 50 region in the photo as desired\n2. Padding:\n    * Pros:\n        * Keeps image aspect ratios\n        * Retains all fine details\n    * Cons:\n        * Photos must be padded to the size of the largest photograph\n            * ***Common Mistake:*** This must be the largest photograph of both the training and test set\n            * This means that, depending on the sets, they can take up a lot of RAM, or not a lot, it varies\n3. Combination of Resizing and Padding:\n    * Pros and Cons:\n        * Depend on situation/circumstances"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"## Step 1:\n\npicture_height = 3567 # Manually set\npicture_width = 3827 # Manually set\n\n## Note: I coded, as below, for a program to find the largest dimensions. \n##   It takes about 5-10 minutes to run, so to save the valuable kernel time,\n##   I manually set the values above, according to the result of the program below.\n\n## Code to identify the largest photo\n# for dirname, _, filenames in os.walk(train_path):\n#     for filename in filenames:\n#         temp_img = mpimg.imread(os.path.join(dirname, filename))\n#         (temp_height, temp_width) = temp_img.shape\n#         if temp_height > picture_height:\n#             picture_height = temp_height\n#         if temp_width > picture_width:\n#             picture_width = temp_width\n\n# for dirname, _, filenames in os.walk(test_path):\n#     for filename in filenames:\n#         temp_img = mpimg.imread(os.path.join(dirname, filename))\n#         (temp_height, temp_width) = temp_img.shape\n#         if temp_height > picture_height:\n#             picture_height = temp_height\n#         if temp_width > picture_width:\n#             picture_width = temp_width","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Step 2:\n\n# We will use some example images. I'll label them, img_1 and img_2\n# We will read them, and then convert them to a numpy array.\n\n# img_1 = \"/kaggle/input/ranzcr-clip-catheter-line-classification/train/1.2.826.0.1.3680043.8.498.17952552645001544825751321016030941058.jpg\"\n# img_2 = '/kaggle/input/ranzcr-clip-catheter-line-classification/train/1.2.826.0.1.3680043.8.498.10370758874574386468962321364924311754.jpg'\n\n\n# For this purpose, I will use the PIL (Pillow), library\n\nimg_1 = Image.open(\"/kaggle/input/ranzcr-clip-catheter-line-classification/train/1.2.826.0.1.3680043.8.498.17952552645001544825751321016030941058.jpg\")\nimg_2 = Image.open('/kaggle/input/ranzcr-clip-catheter-line-classification/train/1.2.826.0.1.3680043.8.498.10370758874574386468962321364924311754.jpg')\n\n# print(img_1.mode) # Prints the mode of the images (RGB, HSV, L, P, ...)\n# print(img_2.mode)\n# # The mode of the photos is \"L\", which are grayscale images with 8-bit pixels\n\nimg_1_np = np.array(img_1)\nimg_2_np = np.array(img_2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Step 3\n\nimg_1_np = np.pad(img_1_np, ((0, picture_height - img_1_np.shape[0]),(0, picture_width - img_1_np.shape[1])))\nimg_2_np = np.pad(img_2_np, ((0, picture_height - img_2_np.shape[0]),(0, picture_width - img_2_np.shape[1])))\nprint(img_1_np, \"\\n\\n\\n\")\nprint(img_2_np)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Optional Step 4:\n# Now we can convert back to a PIL Image, and display it using matplot, along with the original image:\n\nnew_img_1 = Image.fromarray(img_1_np)\nnew_img_2 = Image.fromarray(img_2_np)\n\n# You can see the padding below, comparing the two images.\n# The white space, is space that is not part of the photo,\n# and the black is padding\n\nfig, ((ax1, ax2),(ax3, ax4)) = plt.subplots(nrows = 2, ncols = 2, sharex = True, sharey = True, figsize = (15, 15), dpi = 150, num = 1)\nax1.imshow(img_1, cmap = \"gray\") # Images are gray scale, ensuring that matplotlib displays them as such\nax2.imshow(new_img_1, cmap = \"gray\")\nax3.imshow(img_2, cmap = \"gray\")\nax4.imshow(new_img_2, cmap = \"gray\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Applying Contrast Limited Adaptive Histogram Equalization (CLAHE)"},{"metadata":{"trusted":true},"cell_type":"code","source":"clahe = cv.createCLAHE(clipLimit=15.0, tileGridSize=(8,8))\n\nclahe_img_1 = clahe.apply(img_1_np)\nclahe_img_2 = clahe.apply(img_2_np)\n\nfig, ((new_ax1,  new_ax2),(new_ax3, new_ax4)) = plt.subplots(nrows = 2, ncols = 2, sharex = True, sharey = True, figsize = (20, 20), dpi = 150, num = 1)\nnew_ax1.imshow(clahe_img_1, cmap = \"gray\") # Images are gray scale, ensuring that matplotlib displays them as such\nnew_ax2.imshow(new_img_1, cmap = \"gray\")\nnew_ax3.imshow(clahe_img_2, cmap = \"gray\")\nnew_ax4.imshow(new_img_2, cmap = \"gray\")\n\n# Images on the left are the images with contrast applied, on the right are the non-contrasted images\n# I recommend values from 2 - 20\n# Notice, the difference in how well it works. In the top image, \n# you get black spots, for seemingly no reason, yet in the bottom \n# image, the spine becomes clearer.\n# Most importantly, the lines in both images become clearer for the catheters/tubing.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Medically Valid Models\n\nYou can apply some of the following models for further image preprocessing\n* https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6113157/\n* https://arxiv.org/pdf/2011.07394.pdf\n\nIn addition, one can apply further image filtration:\n* https://humanhealth.iaea.org/HHW/MedicalPhysics/TheMedicalPhysicist/Studentscorner/HandbookforTeachersandStudents/Chapter_17.pdf"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}