{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Diabetic Retinopathy Classification Using MobileNetV2"},{"metadata":{},"cell_type":"markdown","source":"# ABSTRACT\n\nDeep learning has been proposed as one of the automated solutions for diabetic retinopathy (DR) severity classification problem. However, most of the successful deep learning models are based on large convolutional neural network (CNN) architectures, requiring a vast volume of training data as well as dedicated computational resources. In this study, we used MobileNetV2 architecture, which was considered a small-scale architecture (4.2 million trainable parameters), to perform DR classification task in APTOS 2019 dataset (3662 color retinal images). We used the generic MobileNetV2 pre-trained weights from ImageNet as initialization and implemented data augmentation."},{"metadata":{},"cell_type":"markdown","source":"# Import Library"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"# Import Library\n\nimport numpy as np\nimport pandas as pd\n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nimport cv2\nfrom PIL import Image\n\nfrom keras import layers\nfrom tensorflow.keras import applications \nfrom keras.applications import MobileNetV2\nfrom keras.callbacks import Callback, ModelCheckpoint\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.models import Sequential\nfrom keras.optimizers import Adam\n\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import cohen_kappa_score, accuracy_score, confusion_matrix\n\nfrom tqdm import tqdm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loading Data and Exploration"},{"metadata":{},"cell_type":"markdown","source":"## Loading data\n\nLoading the csv data which contains the images file name and its labels. In this project I am using data that already splitted for training and validation."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('../input/valid-and-test-ta/x_train_8.csv')\nvalid_df = pd.read_csv('../input/valid-and-test-ta/x_valid_8.csv')\nprint(train_df.shape)\nprint(valid_df.shape)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df['diagnosis'].value_counts()\ntrain_df['diagnosis'].hist()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Displaying some Sample Images\n\nDisplay the color retinal images with its label as the title"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def display_samples(df, columns=4, rows=3):\n    fig=plt.figure(figsize=(5*columns, 4*rows))\n\n    for i in range(columns*rows):\n        image_path = df.loc[i,'id_code']\n        image_id = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        \n        fig.add_subplot(rows, columns, i+1)\n        plt.title(image_id)\n        plt.imshow(img)\n    \n    plt.tight_layout()\n\ndisplay_samples(train_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Pre-processing"},{"metadata":{},"cell_type":"markdown","source":"## Sampling Data\n\nData is sampled into 700 data for each class"},{"metadata":{"trusted":true},"cell_type":"code","source":"#resample\nfrom sklearn.utils import resample\nX=train_df\nnormal=X[X.diagnosis==0]\nmild=X[X.diagnosis==1]\nmoderate=X[X.diagnosis==2]\nsevere=X[X.diagnosis==3]\npdr=X[X.diagnosis==4]\n\n#downsampled\nmild = resample(mild,\n                replace=True, # sample with replacement\n                n_samples=700, # match number in majority class\n                random_state=2020) # reproducible results\nmoderate = resample(moderate,\n                    replace=False, # sample with replacement\n                    n_samples=700, # match number in majority class\n                    random_state=2020) # reproducible results\nsevere = resample(severe,\n                  replace=True, # sample with replacement\n                  n_samples=700, # match number in majority class\n                  random_state=2020) # reproducible results\nnormal = resample(normal,\n                  replace=False, # sample with replacement\n                  n_samples=700, # match number in majority class\n                  random_state=2020) # reproducible results\npdr = resample(pdr,\n               replace=True, # sample with replacement\n               n_samples=700, # match number in majority class\n               random_state=2020) # reproducible results    \n\n# combine minority and downsampled majority\nsampled = pd.concat([normal, mild, moderate, severe, pdr])\n\ntrain_df = sampled\ntrain_df = train_df.sample(frac=1).reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Resize Images \nWe will resize the images to 224x224 pixel (MobileNetV2 default input resolution), then create a single numpy array to hold the data (because the data is not too much)."},{"metadata":{"trusted":true},"cell_type":"code","source":"def preprocess_image(image_path, desired_size=224):\n    im = Image.open(image_path)\n    im = im.resize((desired_size, )*2, resample=Image.BILINEAR)\n    \n    return im\n\nN = train_df.shape[0]\nx_train = np.empty((N, 224, 224, 3), dtype=np.float32)\n\nfor i, image_id in enumerate(tqdm(train_df['id_code'])):\n    x_train[i, :, :, :] = preprocess_image(\n        f'../input/aptos2019-blindness-detection/train_images/{image_id}.png'\n    )\n    \nN = valid_df.shape[0]\nx_val = np.empty((N, 224, 224, 3), dtype=np.float32)\n\nfor i, image_id in enumerate(tqdm(valid_df['id_code'])):\n    x_val[i, :, :, :] = preprocess_image(\n        f'../input/aptos2019-blindness-detection/train_images/{image_id}.png'\n    )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Image Generator\n\nUsing Keras ImageDataGenerator to generate the images (x_train/valid) and its labels (y_train/valid)"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train = train_df['diagnosis']\ny_val = valid_df['diagnosis']\nprint(x_train.shape)\nprint(y_train.shape)\nprint(x_val.shape)\nprint(y_val.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BATCH_SIZE = 32\n\ndef create_datagen():\n    return ImageDataGenerator(\n        zoom_range=0.1,  # set range for random zoom\n        rotation_range = 360,\n        fill_mode='constant',\n        cval=0.,\n        horizontal_flip=True,  # randomly flip images\n        vertical_flip=True,  # randomly flip images\n    )\n\n# Using generator\ndata_generator = create_datagen().flow(x_train, y_train, batch_size=BATCH_SIZE, seed=2019)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Quadratic Weighted Kappa\n\nQuadratic Weighted Kappa (QWK, the greek letter $\\kappa$), also known as Cohen's Kappa, is the official evaluation metric. For our kernel, we will use a custom callback to monitor the score, and plot it at the end.\n\n## What is Cohen Kappa?\n\nAccording to the [wikipedia article](https://en.wikipedia.org/wiki/Cohen%27s_kappa), we have\n> The definition of $\\kappa$ is:\n> $$\\kappa \\equiv \\frac{p_o - p_e}{1 - p_e}$$\n> where $p_o$ is the relative observed agreement among raters (identical to accuracy), and $p_e$ is the hypothetical probability of chance agreement, using the observed data to calculate the probabilities of each observer randomly seeing each category.\n\nThis metric is used because if we just using accuracy as the metric, it will give spurious results (because the data is imbalance). The QWK is more fit to the problem."},{"metadata":{},"cell_type":"markdown","source":"## Creating keras callback for QWK\n\nBecause our main metric in this problem is QWK, we have to make a function that calculate the QWK for every epoch and also saving the model that got the highest score. "},{"metadata":{"trusted":true},"cell_type":"code","source":"class Metrics(Callback):\n    def on_train_begin(self, logs={}):\n        self.val_kappas = []\n\n    def on_epoch_end(self, epoch, logs={}):\n        X_val, y_val = self.validation_data[:2]\n        \n        y_pred = self.model.predict(X_val)\n        y_pred = np.clip(y_pred,0,4)\n        y_pred = y_pred.astype(int)\n\n        _val_kappa = cohen_kappa_score(\n            y_val,\n            y_pred, \n            weights='quadratic'\n        )\n\n        self.val_kappas.append(_val_kappa)\n\n        print(f\"val_kappa: {_val_kappa:.4f}\")\n        \n        if _val_kappa == max(self.val_kappas):\n            print(\"Validation Kappa has improved. Saving model.\")\n            self.model.save('model.h5')\n\n        return\n    \nkappa_metrics = Metrics()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model: MobileNetV2\n\nI am using MobileNetV2 architecture in this project. The architecture is adopted from [1], where they choose 1.3 as the width multiplier/ alpha (MobileNetV2 hyperparameter). They customize the last layer of the model to become 2 dense layer with 256 nodes and 1 nodes output layer. They use regression approach for this problem because diabetic retinopathy severity is an ordinal variables. Linear activation function is used in the output layer."},{"metadata":{"trusted":true},"cell_type":"code","source":"mobilenet = MobileNetV2(\n    alpha = 1.3,\n    weights='../input/valid-and-test-ta/mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.3_224_no_top.h5',\n    include_top=False,\n    input_shape=(224,224,3)\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def build_model():\n    model = Sequential()\n    model.add(mobilenet)\n    model.add(layers.GlobalAveragePooling2D())\n    model.add(layers.Dense(256))\n    model.add(layers.Dense(256))\n    model.add(layers.Dense(1))\n    \n    model.compile(\n        loss='mse',\n        optimizer=Adam(lr=0.0001),\n        metrics=['accuracy']\n    )\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = build_model()\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Training & Evaluation"},{"metadata":{},"cell_type":"markdown","source":"## Training\n\nI trained the model with 100 epoch, mean squared error (mse) as the loss function and Adam as the optimizer"},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(\n    data_generator,\n    steps_per_epoch=x_train.shape[0] / BATCH_SIZE,\n    epochs=100,\n    validation_data=(x_val, y_val),\n    callbacks=[kappa_metrics]\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#plotting accuracies and loss\nhistory_df = pd.DataFrame(history.history)\nhistory_df[['loss', 'val_loss']].plot()\nhistory_df[['acc', 'val_acc']].plot()\nhistory_df.to_csv('history.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#plotting the QWK score\nplt.plot(kappa_metrics.val_kappas)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Evaluation\n\nAfter training phase and already have the model with highest QWK score, we have to evaluate the model performance using confusion matrix to see whether the model is working properly or not"},{"metadata":{"trusted":true},"cell_type":"code","source":"model.load_weights('model.h5')\ny_val_pred = model.predict(x_val)\n#clipping the value to range of 0-4, and round it to the nearest integer\ny_val_pred = np.clip(y_val_pred,0,4).astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\ncnf_matrix = confusion_matrix(valid_df['diagnosis'].astype('int'), y_val_pred)\ndf_cm = pd.DataFrame(cnf_matrix, index=labels, columns=labels)\nplt.figure(figsize=(16, 7))\nsns.heatmap(df_cm, annot=True, cmap=\"Blues\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"kappa_val = cohen_kappa_score(\n            valid_df['diagnosis'].astype('int'),\n            y_val_pred, \n            weights='quadratic'\n        )\nprint(kappa_val)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"References:\n\n[1] J. Gao, C. Leung and C. Miao, \"Diabetic Retinopathy Classification Using an Efficient Convolutional Neural Network,\" in 2019 IEEE International Conference on Agents (ICA), Jinan, China, 2019, pp. 80-85."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}