{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Summary\nThis kernel does preliminary exploratory data analysis on this dataset and explores some of the pre-processing techniques, some of which are proven to be potentially useful. Some code are borrowed or adapted from other public kernels and discussion threads. Credits are listed below.\n\n#### Thanks to:\n@Neuron Engineer for his excellent EDA code https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping#data\n\n@Bharat Singh for his great starter kernel https://www.kaggle.com/bharatsingh213/keras-resnet-tta\n\nPeople in this discussion https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/102613#latest-614367"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport tensorflow as tf\nfrom tensorflow import keras\nimport cv2\nimport albumentations\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport random\n\nprint(tf.__version__)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames[:5]:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define some important global variables here"},{"metadata":{"trusted":true},"cell_type":"code","source":"CLASSS = {0: \"No DR\", 1: \"Mild\", 2: \"Moderate\", 3: \"Severe\", 4: \"Proliferative DR\"}\nSEED = 77\nrandom.seed(SEED)\nIMG_CHANNELS = 3\nIMG_WIDTH = 512\n\n# These are used for histogram equalization\nclipLimit=2.0 \ntileGridSize=(8, 8)  \n\nchannels = {\"R\":0, \"G\": 1, \"B\":2}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load data and check distribution"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"sample_submission = pd.read_csv(\"/kaggle/input/aptos2019-blindness-detection/sample_submission.csv\")\nprint(sample_submission.head())\ntest_file = pd.read_csv(\"/kaggle/input/aptos2019-blindness-detection/test.csv\")\ntrain_file = pd.read_csv(\"/kaggle/input/aptos2019-blindness-detection/train.csv\")\nprint(test_file.head())\nprint(train_file.head())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Now check the distribution of train images\nprint(len(train_file))\ntrain_file['diagnosis'].hist(figsize = (8,4))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that this dataset is quite imbalanced.\nNext we will visualize some of the training images"},{"metadata":{},"cell_type":"markdown","source":"## Display Original Images"},{"metadata":{"trusted":true},"cell_type":"code","source":"def display_samples(df, columns=4, rows=3):\n    fig=plt.figure(figsize=(4*columns, 3*rows))\n    \n    random_indices = random.sample(range(0, len(train_file)), columns*rows)\n    count = 0\n    for i in random_indices:\n        image_path = df.loc[i,'id_code']\n        image_rating = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        \n        fig.add_subplot(rows, columns, count+1)\n        count += 1\n        plt.title(image_rating)\n        plt.imshow(img)\n    \n    plt.tight_layout()\n\ndisplay_samples(train_file, 4, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Some meta-data features of these images\n* They are actually very large\n* They can be in very different lighting conditions ==> Consider this in augmentations\n* They vary largely in size\n* They have quite different proportion of dark fringes"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_img_path = random.choice(train_file[\"id_code\"])\nsample_img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{sample_img_path}.png')\nsample_img = cv2.cvtColor(sample_img, cv2.COLOR_BGR2RGB)\nprint(sample_img.shape)\nplt.imshow(sample_img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now try to gain some visual intuition into healthy and unhealthy patients' pictures.\n\nFollowing code borrowed from @Bharat Singh"},{"metadata":{"trusted":true},"cell_type":"code","source":"def draw_img(imgs, class_label='0'):\n    fig, axis = plt.subplots(2, 6, figsize=(15, 6))\n    for idnx, (idx, row) in enumerate(imgs.iterrows()):\n        imgPath = (f'../input/aptos2019-blindness-detection/train_images/{row[\"id_code\"]}.png')\n        img = cv2.imread(imgPath)\n        row = idnx // 6\n        col = idnx % 6\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        axis[row, col].imshow(img)\n    plt.suptitle(class_label)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CLASS_ID = 0\ndraw_img(train_file[train_file.diagnosis == CLASS_ID].head(12), CLASSS[CLASS_ID])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CLASS_ID = 4\ndraw_img(train_file[train_file.diagnosis == CLASS_ID].head(12), CLASSS[CLASS_ID])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> ## Basic Image Pre-Processing"},{"metadata":{},"cell_type":"markdown","source":"**Now we see the three different channels and see if one channel may contain most of the useful information** \n\nThe idea of single channel (green) images and the histogram equalization comes from this topic: https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/102613#latest-614367\n\nThe CLAHE (Contrast Limited Adaptive Histogram Equalization) used here\nhttps://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_equalization.html\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# HE --> Histogram Equalization: True to apply CLAHE to the color channel image\n\nprint(channels)\nprint(clipLimit)\nprint(tileGridSize)\n\ndef display_single_channel_samples(df, columns=4, rows=3, channel = \"G\", HE = False):\n    fig=plt.figure(figsize=(4*columns, 3*rows))\n    random.seed(SEED) # This lines make sure that all the following function calls will\n                    # show the same set of randomly selected images\n    random_indices = random.sample(range(0, len(train_file)), columns*rows)\n    \n    count = 0\n    for i in random_indices:\n        # Load images and convert to RGB\n        image_path = df.loc[i,'id_code']\n        image_rating = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        \n        # Apply some pre-processing\n        img = img[:,:,channels[channel]]\n        if HE: #If the histogram equalization is applied\n            clahe = cv2.createCLAHE(clipLimit=clipLimit, tileGridSize=tileGridSize)\n            img = clahe.apply(img) #This is for creating the image with a higher contrast\n        else:\n            pass\n        \n        # Actually drawing stuff \n        fig.add_subplot(rows, columns, count+1)\n#         fig.add_subplot()\n        count += 1\n        plt.title(image_rating)\n        plt.imshow(img)\n    \n    plt.tight_layout()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Green Channel WITHOUT histogram equalization**"},{"metadata":{"trusted":true},"cell_type":"code","source":"display_single_channel_samples(train_file, 4, 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"** Green Channel WITH Histogram Equalization**\n\nWe can see that this is **slightly better** than the un-equalized version of these images as the contrast has been increased. In the green channel, the vessels are much more clear.\n\nNext, we try to visualiza another two channels, each **WITH** HE applied.\n\nNow, we can see that "},{"metadata":{"trusted":true},"cell_type":"code","source":"display_single_channel_samples(train_file,4,3, \"G\", HE = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"** Red Channel WITH histogram equalization**"},{"metadata":{"trusted":true},"cell_type":"code","source":"display_single_channel_samples(train_file,4,3, \"R\", HE = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"** Blue Channel WITH histogram equalization**"},{"metadata":{"trusted":true},"cell_type":"code","source":"display_single_channel_samples(train_file,4,2, \"B\", HE = False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"display_single_channel_samples(train_file,4,2, \"B\", HE = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From the comparison above, we can see that histogram equalization is indeed helping out the blue channel **a little bit**. But it's still not providing much high-quality information."},{"metadata":{},"cell_type":"markdown","source":"#### Conclusion:\nThe green channel is indeed very helpful. The contrast can be effectively adjusted by a simple line of code. I will probably try to use pure green channel to fit a model, as an experiment."},{"metadata":{},"cell_type":"markdown","source":"### Resize & Crop & Combine\nAs described in @Neuron engineer's EDA kernel, another good way to pre-process the RGB images is Ben Grahem's method (https://github.com/btgraham/SparseConvNet/tree/kaggle_Diabetic_Retinopathy_competition), which is simple yet powerful. The following function implements this method."},{"metadata":{"trusted":true},"cell_type":"code","source":"def resize_bens(df, columns=4, rows=3, sigmaX = 20, img_width = IMG_WIDTH): # Assume image is square \n    fig=plt.figure(figsize=(4*columns, 3*rows))\n    \n    random.seed(SEED)\n    random_indices = random.sample(range(0, len(train_file)), columns*rows)\n    count = 0\n    for i in random_indices:\n        image_path = df.loc[i,'id_code']\n        image_rating = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        img = cv2.resize(img, (img_width, img_width))\n        img = cv2.addWeighted ( img,4, cv2.GaussianBlur(img , (0,0) , sigmaX) ,-4 ,128)\n        \n        fig.add_subplot(rows, columns, count+1)\n        count += 1\n        plt.title(image_rating)\n        plt.imshow(img)\n    \n    plt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"resize_bens(train_file, 4, 4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Try some different **sigmaX** values.\nWe can see below that maybe sigmaX = 50 is too high, causing some bright white fringes of picture at location (0,0).\n\nNext we tried sigmaX = 16. It looks very similar to sigmaX = 10"},{"metadata":{"trusted":true},"cell_type":"code","source":"resize_bens(train_file,4,4, sigmaX = 50)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"resize_bens(train_file,4,4, sigmaX = 16)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Crop them!\nFrom the examples above, it's intuitive to not set the sigmaX any where too high.\n\nNext, we want to be able to crop these images so that the dark fringes have approx. the same distribution among all images fed into the network. The code below are directly adapted from the kernel of @Neuron Engineer"},{"metadata":{"trusted":true},"cell_type":"code","source":"def crop_image_from_gray(img,tol=7):\n    if img.ndim ==2:\n        mask = img>tol\n        return img[np.ix_(mask.any(1),mask.any(0))]\n    elif img.ndim==3:\n        gray_img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)\n        mask = gray_img>tol\n        \n        check_shape = img[:,:,0][np.ix_(mask.any(1),mask.any(0))].shape[0]\n        if (check_shape == 0): # image is too dark so that we crop out everything,\n            return img # return original image\n        else:\n            img1=img[:,:,0][np.ix_(mask.any(1),mask.any(0))]\n            img2=img[:,:,1][np.ix_(mask.any(1),mask.any(0))]\n            img3=img[:,:,2][np.ix_(mask.any(1),mask.any(0))]\n    #         print(img1.shape,img2.shape,img3.shape)\n            img = np.stack([img1,img2,img3],axis=-1)\n    #         print(img.shape)\n        return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def resize_bens_and_crop(df, columns=4, rows=3, sigmaX = 20, img_width = IMG_WIDTH): # Assume image is square \n    fig=plt.figure(figsize=(4*columns, 3*rows))\n    \n    random.seed(SEED)\n    random_indices = random.sample(range(0, len(train_file)), columns*rows)\n    count = 0\n    for i in random_indices:\n        image_path = df.loc[i,'id_code']\n        image_rating = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        \n        # First crop, then resize.\n        img = crop_image_from_gray(img)\n        img = cv2.resize(img, (img_width, img_width))\n        \n        # Applying Ben's method\n        img = cv2.addWeighted ( img,4, cv2.GaussianBlur(img , (0,0) , sigmaX) ,-4 ,128)\n        \n        fig.add_subplot(rows, columns, count+1)\n        count += 1\n        plt.title(image_rating)\n        plt.imshow(img)\n    \n    plt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"resize_bens_and_crop(train_file, 4, 4, sigmaX = 10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Finally\nHere we provide an intuitive comparison between the effectiveness of using SINGLE BLUE CHANNEL WITH HISTOGRAM EQUALIZATION and using RGB IMAGE WITH BEN'S METHOD"},{"metadata":{},"cell_type":"markdown","source":"** Don't forget to add cropping and resizing functions to the SINGLE CHANNEL method we used before **"},{"metadata":{"trusted":true},"cell_type":"code","source":"# HE --> Histogram Equalization: Try to apply CLAHE to the color channel image\n\nprint(channels)\nprint(clipLimit)\nprint(tileGridSize)\n\ndef display_single_channel_crop_resize(df, columns=4, rows=3, channel = \"G\", HE = False):\n    fig=plt.figure(figsize=(4*columns, 3*rows))\n    random.seed(SEED) # This lines make sure that all the following function calls will\n                    # show the same set of randomly selected images\n    random_indices = random.sample(range(0, len(train_file)), columns*rows)\n    \n    count = 0\n    for i in random_indices:\n        # Load images and convert to RGB\n        image_path = df.loc[i,'id_code']\n        image_rating = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        \n        # Crop and then resize the image\n        img = crop_image_from_gray(img)\n        img = cv2.resize(img, (IMG_WIDTH, IMG_WIDTH))\n        \n        # Apply some pre-processing\n        img = img[:,:,channels[channel]]\n        if HE: #If the histogram equalization is applied\n            clahe = cv2.createCLAHE(clipLimit=clipLimit, tileGridSize=tileGridSize)\n            img = clahe.apply(img) #This is for creating the image with a higher contrast\n        else:\n            pass\n        \n        # Actually drawing stuff \n        fig.add_subplot(rows, columns, count+1)\n#         fig.add_subplot()\n        count += 1\n        plt.title(image_rating)\n        plt.imshow(img)\n    \n    plt.tight_layout()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"resize_bens_and_crop(train_file, 5, 7, sigmaX = 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"display_single_channel_crop_resize(train_file, 5, 7, HE = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}