{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\nWhen working on my kernel, I have spent some time on preprocessing and augmentation, and encountered some problems. Cropping, preprocessing and augmentation can be quite computationally expensive (even to the extent images are not loaded quickly enough to keep the GPU running at full speed). Furthermore, default ImageDataGenerators couldn't crop or preprocess the images in exactly the way and order I wanted.\n\nTo speed things up, I show a few tricks here. This hopefully allows you to train your neural nets for more epochs or pre-train on (much more!) data from the previous challenge. Note that most of the methods presented here, are copied from [this excellent kernel](https://www.kaggle.com/ratthachat/aptos-updated-preprocessing-ben-s-cropping) (recommended read), which in turn is based on the preprocessing method of [Ben Graham](https://github.com/btgraham/SparseConvNet/tree/kaggle_Diabetic_Retinopathy_competition) used in the previous challenge.\n\nI am new to Kaggle, and this is the first kernel I share, so please be a bit gentle in your feedback."},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport cv2\nimport matplotlib.pyplot as plt\nIMAGE_SIZE = 224","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Code"},{"metadata":{"trusted":true},"cell_type":"code","source":"def info_image(im):\n    # Compute the center (cx, cy) and radius of the eye\n    cy = im.shape[0]//2\n    midline = im[cy,:]\n    midline = np.where(midline>midline.mean()/3)[0]\n    if len(midline)>im.shape[1]//2:\n        x_start, x_end = np.min(midline), np.max(midline)\n    else: # This actually rarely happens p~1/10000\n        x_start, x_end = im.shape[1]//10, 9*im.shape[1]//10\n    cx = (x_start + x_end)/2\n    r = (x_end - x_start)/2\n    return cx, cy, r\n\ndef resize_image(im, augmentation=True):\n    # Crops, resizes and potentially augments the image to IMAGE_SIZE\n    cx, cy, r = info_image(im)\n    scaling = IMAGE_SIZE/(2*r)\n    rotation = 0\n    if augmentation:\n        scaling *= 1 + 0.3 * (np.random.rand()-0.5)\n        rotation = 360 * np.random.rand()\n    M = cv2.getRotationMatrix2D((cx,cy), rotation, scaling)\n    M[0,2] -= cx - IMAGE_SIZE/2\n    M[1,2] -= cy - IMAGE_SIZE/2\n    return cv2.warpAffine(im,M,(IMAGE_SIZE,IMAGE_SIZE)) # This is the most important line\n\ndef subtract_median_bg_image(im):\n    k = np.max(im.shape)//20*2+1\n    bg = cv2.medianBlur(im, k)\n    return cv2.addWeighted (im, 4, bg, -4, 128)\n\ndef subtract_gaussian_bg_image(im):\n    k = np.max(im.shape)/10\n    bg = cv2.GaussianBlur(im ,(0,0) ,k)\n    return cv2.addWeighted (im, 4, bg, -4, 128)\n\ndef id_to_image(id_code, resize=True, augmentation=False, subtract_gaussian=False, subtract_median=False):\n    path = '../input/train_images/{}.png'.format(id_code)\n    im = cv2.imread(path)\n    im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n    if resize_image:\n        im = resize_image(im, augmentation)\n    if subtract_gaussian:\n        im = subtract_gaussian_bg_image(im)\n    if subtract_median:\n        im = subtract_median_bg_image(im)\n    return im","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The main trick for speed-up the line *cv2.warpAffine(im,M,(IMAGE_SIZE,IMAGE_SIZE))*. This effectively does cropping, resizing, scaling and rotation in a single affine transformation. I got a speedup of at least a factor 2 (and probably more), compared to doing those operations one-by-one. Another advantage of combining resizing and augmentation into a single operation is that you do not have to interpolate the image multiple times, which typically degrades image quality."},{"metadata":{},"cell_type":"markdown","source":"# Sample images"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ndf_train = pd.read_csv('../input/train.csv')\nfig = plt.figure(figsize=(25, 16))\nSEED = np.random.randint(0,100)\n\ndef plot_col(col, id2im, n_cols=6):\n    for class_id in range(0,5):\n        for i, (idx, row) in enumerate(df_train.loc[df_train['diagnosis'] == class_id].sample(1, random_state=SEED).iterrows()):\n            ax = fig.add_subplot(5, n_cols, class_id * n_cols + i + col, xticks=[], yticks=[])\n            im = id2im(row['id_code'])\n            plt.imshow(im)\n            ax.set_title('Label: %d-%d-%s' % (class_id, idx, row['id_code']) )\n\n# display normal image of each class\nplot_col(1, lambda x: id_to_image(x))\n\n# display normal image of each class\nplot_col(4, lambda x: id_to_image(x, augmentation=True))\n\n# display normal image of each class\nplot_col(2, lambda x: id_to_image(x, subtract_gaussian=True))\n\n# display normal image of each class\nplot_col(5, lambda x: id_to_image(x, subtract_gaussian=True, augmentation=True))\n\n# display normal image of each class\nplot_col(3, lambda x: id_to_image(x, subtract_median=True))\n\n# display normal image of each class\nplot_col(6, lambda x: id_to_image(x, subtract_median=True, augmentation=True))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The first column shows the normal images, the second columns shows the images processed with an approach similar to Ben Graham's approach.\n\nThe third column shows the images when a Median filter is used (instead of a Gaussian filter). A median filter is very effective in seperating small or narrow objects from a relatively continous background. As a result, the blood vessels and small objects are much more apparant in the third column compared to the first two. That may be useful (or not!). It is also relatively succesful in removing color from the image, which may make images shot with different cameras more comparable (or removes potentially usefull information!). Put it in the comments if you get better/worse results with the Median filter.\n\nThe last three columns just show random rotations/scalings. You can use the code to implement your own ImageGenerator and generate augmented images on the fly.\n\nOk, thanks. that's it. Happy training!"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}