{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os, sys\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport skimage.io\nfrom skimage.transform import resize\nfrom imgaug import augmenters as iaa\nfrom tqdm import tqdm\nimport PIL\nfrom PIL import Image, ImageOps\nimport cv2\nfrom sklearn.utils import class_weight, shuffle\nfrom keras.losses import binary_crossentropy\nfrom keras.applications.resnet50 import preprocess_input\nimport keras.backend as K\nimport tensorflow as tf\nfrom sklearn.metrics import f1_score, fbeta_score\nfrom keras.utils import Sequence\nfrom keras.utils import to_categorical\nfrom sklearn.model_selection import train_test_split\n\nWORKERS = 2\nCHANNEL = 3\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nIMG_SIZE = 512\nNUM_CLASSES = 5\nSEED = 77\nTRAIN_NUM = 1000 # use 1000 when you just want to explore new idea, use -1 for full train","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"1. Introduction. Explore first, train later:\n\n One intuitive way to improve the performance of our model is to simply improve the quality of input images. \n\nReducing lighting-condition effects: as we will see, images come with many different lighting conditions, some images are very dark and difficult to visualize. We can try to convert the image to gray scale, and visualize better. Alternatively, there is a better approach. We can try the method of [Ben Graham]\n\nCropping uninformative area: everyone know this :) Here, I just find the codes from internet and choose the best ."},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv('../input/aptos2019-blindness-detection/train.csv')\ndf_test = pd.read_csv('../input/aptos2019-blindness-detection/test.csv')\n#print(df_train)\nprint(df_train.head(10))\nprint(df_train.describe)\nprint(df_train.shape)\nx = df_train['id_code']\ny = df_train['diagnosis']\n\nx, y = shuffle(x, y, random_state=SEED)\nprint(\"x data after shuffling\")\nprint(x)\nprint(\"y data after shuffling\")\nprint(y)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_x, valid_x, train_y, valid_y = train_test_split(x, y, test_size=0.15,\n                                                      stratify=y, random_state=SEED)\nprint(train_x.shape, train_y.shape, valid_x.shape, valid_y.shape)\ntrain_y.hist()\nvalid_y.hist()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.2 Original Inputs|\n\nFirst, let have a glance of original inputs. Each row depicts each severity level. We can see two problems which make the severity difficult to spot on. First, some images are very dark [pic(0,2) and pic(4,4) ] and sometimes different color illumination is confusing [pic (3,3)]. Second, we can get the uninformative dark areas for some pictures [pic(0,1), pic(0,3)]. This is important when we reduce the picture size, as informative areas become too small. So it is intuitive to crop the uninformative areas out in the second case."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nfig = plt.figure(figsize=(25, 16))\n# display 10 images from each class\nfor class_id in sorted(train_y.unique()):\n    for i, (idx, row) in enumerate(df_train.loc[df_train['diagnosis'] == class_id].sample(5, random_state=SEED).iterrows()):\n        ax = fig.add_subplot(5, 5, class_id * 5 + i + 1, xticks=[], yticks=[])\n        path=f\"../input/aptos2019-blindness-detection/train_images/{row['id_code']}.png\"\n        image = cv2.imread(path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        image = cv2.resize(image, (IMG_SIZE, IMG_SIZE))\n\n        plt.imshow(image)\n        ax.set_title('Label: %d - %d - %s' % (class_id, idx, row['id_code']) )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can try gray scale and feel understand better for some pictures, as color distraction is gone. For example, we can see more blood clearer in the upper part of pic(4,4), which has severity of level 4."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nfig = plt.figure(figsize=(25, 16))\nfor class_id in sorted(train_y.unique()):\n    for i, (idx, row) in enumerate(df_train.loc[df_train['diagnosis'] == class_id].sample(5, random_state=SEED).iterrows()):\n        ax = fig.add_subplot(5, 5, class_id * 5 + i + 1, xticks=[], yticks=[])\n        path=f\"../input/aptos2019-blindness-detection/train_images/{row['id_code']}.png\"\n        image = cv2.imread(path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n#         image=cv2.addWeighted ( image, 0 , cv2.GaussianBlur( image , (0 ,0 ) , 10) ,-4 ,128)\n        image = cv2.resize(image, (IMG_SIZE, IMG_SIZE))\n\n        plt.imshow(image, cmap='gray')\n        ax.set_title('Label: %d-%d-%s' % (class_id, idx, row['id_code']) )\n        ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For severity level 4, I feel that two examples here are difficult to spot on, pic(4,1) and pic(4,4). As we try zooming to see the details (use real size image), we can see some abnormalities (*cotton wool spots* or *hard exudates* ?) in those eyes clearer (observe the lower-right part of the eye). Therefore, `IMG_SIZE` is definitely important for this problem. In the next section, we shall see better method than gray-scale conversion."},{"metadata":{"trusted":true},"cell_type":"code","source":"dpi = 80 #inch\n\n# path=f\"../input/aptos2019-blindness-detection/train_images/5c7ab966a3ee.png\" # notice upper part\npath=f\"../input/aptos2019-blindness-detection/train_images/cd54d022e37d.png\" # lower-right, this still looks not so severe, can be class3\nimage = cv2.imread(path)\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\nheight, width = image.shape\nprint(height, width)\n\nSCALE=2\nfigsize = (width / float(dpi))/SCALE, (height / float(dpi))/SCALE\n\nfig = plt.figure(figsize=figsize)\nplt.imshow(image, cmap='gray')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. Try Ben Graham's preprocessing method."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nfig = plt.figure(figsize=(25, 16))\nfor class_id in sorted(train_y.unique()):\n    for i, (idx, row) in enumerate(df_train.loc[df_train['diagnosis'] == class_id].sample(5, random_state=SEED).iterrows()):\n        ax = fig.add_subplot(5, 5, class_id * 5 + i + 1, xticks=[], yticks=[])\n        path=f\"../input/aptos2019-blindness-detection/train_images/{row['id_code']}.png\"\n        image = cv2.imread(path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n#         image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        image = cv2.resize(image, (IMG_SIZE, IMG_SIZE))\n        image=cv2.addWeighted ( image,4, cv2.GaussianBlur( image , (0,0) , IMG_SIZE/10) ,-4 ,128) # the trick is to add this line\n\n        plt.imshow(image, cmap='gray')\n        ax.set_title('Label: %d-%d-%s' % (class_id, idx, row['id_code']) )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def crop_image1(img,tol=7):\n    # img is image data\n    # tol  is tolerance\n        \n    mask = img>tol\n    return img[np.ix_(mask.any(1),mask.any(0))]\n\ndef crop_image_from_gray(img,tol=7):\n    if img.ndim ==2:\n        mask = img>tol\n        return img[np.ix_(mask.any(1),mask.any(0))]\n    elif img.ndim==3:\n        gray_img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)\n        mask = gray_img>tol\n        \n        check_shape = img[:,:,0][np.ix_(mask.any(1),mask.any(0))].shape[0]\n        if (check_shape == 0): # image is too dark so that we crop out everything,\n            return img # return original image\n        else:\n            img1=img[:,:,0][np.ix_(mask.any(1),mask.any(0))]\n            img2=img[:,:,1][np.ix_(mask.any(1),mask.any(0))]\n            img3=img[:,:,2][np.ix_(mask.any(1),mask.any(0))]\n    #         print(img1.shape,img2.shape,img3.shape)\n            img = np.stack([img1,img2,img3],axis=-1)\n    #         print(img.shape)\n        return img\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# OLD version of image color cropping, use crop_image_from_gray instead\n# The above code work only for 1-channel. Here is my simple extension for 3-channels image\ndef crop_image(img,tol=7):\n    if img.ndim ==2:\n        mask = img>tol\n        return img[np.ix_(mask.any(1),mask.any(0))]\n    elif img.ndim==3:\n        h,w,_=img.shape\n#         print(h,w)\n        img1=cv2.resize(crop_image1(img[:,:,0]),(w,h))\n        img2=cv2.resize(crop_image1(img[:,:,1]),(w,h))\n        img3=cv2.resize(crop_image1(img[:,:,2]),(w,h))\n        \n#         print(img1.shape,img2.shape,img3.shape)\n        img[:,:,0]=img1\n        img[:,:,1]=img2\n        img[:,:,2]=img3\n        return img\n\n'''all of these do not work'''\n\ndef crop_image2(image,threshold=5):\n    if len(image.shape) == 3:\n        flatImage = np.max(image, 2)\n    else:\n        flatImage = image\n    assert len(flatImage.shape) == 2\n\n    rows = np.where(np.max(flatImage, 0) > threshold)[0]\n    if rows.size:\n        cols = np.where(np.max(flatImage, 1) > threshold)[0]\n        image = image[cols[0]: cols[-1] + 1, rows[0]: rows[-1] + 1]\n    else:\n        image = image[:1, :1]\n\n    return image\n\ndef crop_image3(image):\n    mask = image > 0\n\n    # Coordinates of non-black pixels.\n    coords = np.argwhere(mask)\n\n    # Bounding box of non-black pixels.\n    x0, y0 = coords.min(axis=0)\n    x1, y1 = coords.max(axis=0) + 1   # slices are exclusive at the top\n    \n    # Get the contents of the bounding box.\n    cropped = image[x0:x1, y0:y1]\n    return cropped\n\ndef crop_image4(image):\n    _,thresh = cv2.threshold(image,1,255,cv2.THRESH_BINARY)\n    contours,hierarchy = cv2.findContours(thresh,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)\n    cnt = contours[0]\n    x,y,w,h = cv2.boundingRect(cnt)\n    crop = image[y:y+h,x:x+w]\n    return crop\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_ben_color(path, sigmaX=10):\n    image = cv2.imread(path)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    image = crop_image_from_gray(image)\n    image = cv2.resize(image, (IMG_SIZE, IMG_SIZE))\n    image=cv2.addWeighted ( image,4, cv2.GaussianBlur( image , (0,0) , sigmaX) ,-4 ,128)\n        \n    return image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\nNUM_SAMP=7\nfig = plt.figure(figsize=(25, 16))\nfor class_id in sorted(train_y.unique()):\n    for i, (idx, row) in enumerate(df_train.loc[df_train['diagnosis'] == class_id].sample(NUM_SAMP, random_state=SEED).iterrows()):\n        ax = fig.add_subplot(5, NUM_SAMP, class_id * NUM_SAMP + i + 1, xticks=[], yticks=[])\n        path=f\"../input/aptos2019-blindness-detection/train_images/{row['id_code']}.png\"\n        image = load_ben_color(path,sigmaX=30)\n\n        plt.imshow(image)\n        ax.set_title('%d-%d-%s' % (class_id, idx, row['id_code']) )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ## try circle crop\ndef circle_crop(img, sigmaX=10):   \n    \"\"\"\n    Create circular crop around image centre    \n    \"\"\"    \n    \n    img = cv2.imread(img)\n    img = crop_image_from_gray(img)    \n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    \n    height, width, depth = img.shape    \n    \n    x = int(width/2)\n    y = int(height/2)\n    r = np.amin((x,y))\n    \n    circle_img = np.zeros((height, width), np.uint8)\n    cv2.circle(circle_img, (x,y), int(r), 1, thickness=-1)\n    img = cv2.bitwise_and(img, img, mask=circle_img)\n    img = crop_image_from_gray(img)\n    img=cv2.addWeighted ( img,4, cv2.GaussianBlur( img , (0,0) , sigmaX) ,-4 ,128)\n    return img ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n## try circle crop\nNUM_SAMP=7\nfig = plt.figure(figsize=(25, 16))\nfor class_id in sorted(train_y.unique()):\n    for i, (idx, row) in enumerate(df_train.loc[df_train['diagnosis'] == class_id].sample(NUM_SAMP, random_state=SEED).iterrows()):\n        ax = fig.add_subplot(5, NUM_SAMP, class_id * NUM_SAMP + i + 1, xticks=[], yticks=[])\n        path=f\"../input/aptos2019-blindness-detection/train_images/{row['id_code']}.png\"\n        image = circle_crop(path,sigmaX=30)\n\n        plt.imshow(image)\n        ax.set_title('%d-%d-%s' % (class_id, idx, row['id_code']) )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dpi = 80 #inch\n\n# path=f\"../input/aptos2019-blindness-detection/train_images/5c7ab966a3ee.png\" # notice upper part\npath=f\"../input/aptos2019-blindness-detection/train_images/cd54d022e37d.png\" # lower-right, can be class3\nimage = load_ben_color(path,sigmaX=10)\n\nheight, width = IMG_SIZE, IMG_SIZE\nprint(height, width)\n\nSCALE=1\nfigsize = (width / float(dpi))/SCALE, (height / float(dpi))/SCALE\n\nfig = plt.figure(figsize=figsize)\nplt.imshow(image, cmap='gray')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}