{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Polar unrolling"},{"metadata":{},"cell_type":"markdown","source":"Polar unrolling allows to better utilize pixel space, remove \"rotation\" from a list of augmentations and obtain uniformly scaled eye images (for the cases of no/partial cropping of the fundus image with preservation of radius)."},{"metadata":{},"cell_type":"markdown","source":"Initially we have images with noticeable black areas.\nIn order to remove them we at the beginning apply an autocrop."},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport cv2\ndf_train = pd.read_csv('../input/aptos2019-blindness-detection/train.csv')\ndf_test = pd.read_csv('../input/aptos2019-blindness-detection/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nfig=plt.figure(figsize=(10, 9))\n\nfor i in range(3):\n    image_path = df_train.loc[i,'id_code']\n    image_id = df_train.loc[i,'diagnosis']\n    img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n#     img = preprocess(img)\n\n    fig.add_subplot(3,1, i+1)\n    plt.title(image_id)\n    plt.imshow(img)\n\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def autocrop(image, threshold=0):\n    if len(image.shape) == 3:\n        flatImage = np.max(image, 2)\n    else:\n        flatImage = image\n    assert len(flatImage.shape) == 2\n\n    rows = np.where(np.max(flatImage, 0) > threshold)[0]\n    if rows.size:\n        cols = np.where(np.max(flatImage, 1) > threshold)[0]\n        image = image[cols[0]: cols[-1] + 1, rows[0]: rows[-1] + 1]\n    else:\n        image = image[:1, :1]\n\n    return image\n\nfig=plt.figure(figsize=(10, 9))\n\nfor i in range(3):\n    image_path = df_train.loc[i,'id_code']\n    image_id = df_train.loc[i,'diagnosis']\n    img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    img = autocrop(img,10)\n\n    fig.add_subplot(3,1, i+1)\n    plt.title(image_id)\n    plt.imshow(img)\n\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After autocrop, we have radius of a circle (represented by widest side of an image). We can extract circle using polar unrolling.\n\nThe method is applied as follows:\n- Radius of a circle determited by maximum width of an image.\n- Perform the unrolling (using cv2.linearPolar).\n- Remove very close and very far sides of unrolled image (because areas close to the center appear to be stretched and areas far from center appear highlighted).\n- Transpose an image (initially, height of unrolled image is much larger than width)\n- Resize an image. In fact, by unrolling we should get images with sizes R (radius) and C (C = 2 * pi * R). But initial size of C is too wide and can increase memory consumption. Thus, image resized not to ratio 1:3.14, but 1:2.56 (subjectively selected value, which maintains ballance between memory consumtion and perception of features)"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"def preprocess(img):\n    img = autocrop(img,10)\n    value = np.max([img.shape[1]/2.0, img.shape[0]/2.0])\n    value = value - value/20.0\n    polar_image = cv2.linearPolar(img,(img.shape[1]/2, img.shape[0]/2), value, cv2.INTER_CUBIC+cv2.WARP_FILL_OUTLIERS)\n    polar_image = polar_image.astype(np.uint8)\n    polar_image = cv2.resize(polar_image,(316,768),interpolation=cv2.INTER_CUBIC)\n    res = cv2.transpose(polar_image[:,16:,:])\n    res = cv2.addWeighted (res,4, cv2.GaussianBlur(res, (0,0) ,10.0), -4, 128)\n    return res\n\nfig=plt.figure(figsize=(10, 9))\n\nfor i in range(3):\n    image_path = df_train.loc[i,'id_code']\n    image_id = df_train.loc[i,'diagnosis']\n    img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    img = preprocess(img)\n\n    fig.add_subplot(3,1, i+1)\n    plt.title(image_id)\n    plt.imshow(img)\n\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Advantages:\n- No need for rotation augmentation. By unrolling we changed coordinate space. Now rotation becomes just plain shift by x axis (what doesn't matter for convolutional neural networks). It is more than absence of this type of augmentation. Actually now we have all the possible rotations considered in the model (except some borders, but this can easily be solved by the single 50% shift by x axis).\n- Uniform scale (because we are extracting an actual circle based on image sizes). There are cropped images in the 2019 dataset, but we still have the radius. By having radius we have the same scale for features, because we assume very low variation in sizes of eyes.\n- No black pixels from after the radius. And consequently - better image space utilization (if circle is not cropped then we have an image completely filled with an image of an eye).\n- Much higher resultion. Obtained 300 x 768 is related to an image with size 600x600 (radius=300). 600 x 600 = 360,000 pixels (and ~20% are just black pixels if it is a complete circle). 300 x 768 = 230,400 pixels.\n\n# Disadvantages:\n- High memory consumption (an actual use of such preprocessing technique produces wide (2 x pi xR) images).\n- Correct unrolling is not always possible. Some circles just highly cropped. In this case, by using widest side, this technique can't produce correct unrolling. Because some parts of an image are out of radius defined by widest side (in this case, unrolling should be done using diagonal, not side)."},{"metadata":{},"cell_type":"markdown","source":"# About Augmentations\n- Unrolling makes it hard to use zoom with coefficients less than 1.0, because of cropped areas (represented as elliptical areas). One can use zoom 1.0 .. 1.*.\n- Horizontal shift by x axis can by done only once (and probably it is unnecessary at all).\n- Horizontal/vertical flip is ok.\n- No need for rotation.\n"},{"metadata":{},"cell_type":"markdown","source":"Polar unrolling perfectly fits 2015 dataset (it contains mostly complete circles). But in 2019 dataset there are images, which are highly cropped, which reduces applicability of such method.\n\nEfficientNetB3 model pretrained on images preprocessed by this technique on 2015 dataset and then fine-tuned on 2019 dataset allowed to get score 0.794 for a single model."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}