{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Diabetic Retinopathy Classification Using MobileNetV2-SVM"},{"metadata":{},"cell_type":"markdown","source":"# ABSTRACT\nDeep learning has been proposed as one of the automated solutions for diabetic retinopathy (DR) severity classification problem. However, most of the successful deep learning models are based on large convolutional neural network (CNN) architectures, requiring a vast volume of training data as well as dedicated computational resources. In [previous study](https://www.kaggle.com/shidqiet/diabetic-retinopathy-mobilenetv2), we used MobileNetV2 architecture, which was considered a small-scale architecture (4.2 million trainable parameters), to perform DR classification task in APTOS 2019 dataset (3662 color retinal images). **In this study, we further optimized our model by implemented data augmentation and resampling during training and combining it with an SVM classifier, resulting a hybrid and computationally efficient deep learning model, MobileNetV2-SVM.**"},{"metadata":{},"cell_type":"markdown","source":"# Import Library"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"# Import Library\n\nimport numpy as np\nimport pandas as pd\n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nimport cv2\nfrom PIL import Image\n\nfrom keras import layers\nfrom tensorflow.keras import applications \nfrom keras.applications import MobileNetV2\nfrom keras.callbacks import Callback, ModelCheckpoint\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.models import Sequential, load_model\nfrom keras.optimizers import Adam\n\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import cohen_kappa_score, accuracy_score, confusion_matrix\n\nfrom tqdm import tqdm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loading Data and Exploration"},{"metadata":{},"cell_type":"markdown","source":"## Loading data\n\nLoading the csv data which contains the images file name and its labels. In this project I am using data that already splitted for training and validation."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('../input/valid-and-test-ta/x_train_8.csv')\nvalid_df = pd.read_csv('../input/valid-and-test-ta/x_valid_8.csv')\nprint(train_df.shape)\nprint(valid_df.shape)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df['diagnosis'].value_counts()\ntrain_df['diagnosis'].hist()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Displaying some Sample Images\nDisplay the color retinal images with its label as the title"},{"metadata":{"trusted":true},"cell_type":"code","source":"def display_samples(df, columns=4, rows=3):\n    fig=plt.figure(figsize=(5*columns, 4*rows))\n\n    for i in range(columns*rows):\n        image_path = df.loc[i,'id_code']\n        image_id = df.loc[i,'diagnosis']\n        img = cv2.imread(f'../input/aptos2019-blindness-detection/train_images/{image_path}.png')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        \n        fig.add_subplot(rows, columns, i+1)\n        plt.title(image_id)\n        plt.imshow(img)\n    \n    plt.tight_layout()\n\ndisplay_samples(train_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Preprocessing"},{"metadata":{},"cell_type":"markdown","source":"## Sampling Data\n\nData is sampled into 700 data for each class"},{"metadata":{"trusted":true},"cell_type":"code","source":"#resample\nfrom sklearn.utils import resample\nX=train_df\nnormal=X[X.diagnosis==0]\nmild=X[X.diagnosis==1]\nmoderate=X[X.diagnosis==2]\nsevere=X[X.diagnosis==3]\npdr=X[X.diagnosis==4]\n\n#downsampled\nmild = resample(mild,\n                replace=True, # sample with replacement\n                n_samples=700, # match number in majority class\n                random_state=2020) # reproducible results\nmoderate = resample(moderate,\n                    replace=False, # sample with replacement\n                    n_samples=700, # match number in majority class\n                    random_state=2020) # reproducible results\nsevere = resample(severe,\n                  replace=True, # sample with replacement\n                  n_samples=700, # match number in majority class\n                  random_state=2020) # reproducible results\nnormal = resample(normal,\n                  replace=False, # sample with replacement\n                  n_samples=700, # match number in majority class\n                  random_state=2020) # reproducible results\npdr = resample(pdr,\n               replace=True, # sample with replacement\n               n_samples=700, # match number in majority class\n               random_state=2020) # reproducible results    \n\n# combine minority and downsampled majority\nsampled = pd.concat([normal, mild, moderate, severe, pdr])\n\n# checking counts\nsampled.diagnosis.value_counts()\n\ntrain_df = sampled\ntrain_df = train_df.sample(frac=1).reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Pre-processing : Resize Images\nWe will resize the images to 224x224 pixel (MobileNetV2 default input resolution), then create a single numpy array to hold the data (because the data is not too much)."},{"metadata":{"trusted":true},"cell_type":"code","source":"def preprocess_image(image_path, desired_size=224):\n    im = Image.open(image_path)\n    im = im.resize((desired_size, )*2, resample=Image.BILINEAR)\n    \n    return im\n\nN = train_df.shape[0]\nx_train = np.empty((N, 224, 224, 3), dtype=np.float32)\n\nfor i, image_id in enumerate(tqdm(train_df['id_code'])):\n    x_train[i, :, :, :] = preprocess_image(\n        f'../input/aptos2019-blindness-detection/train_images/{image_id}.png'\n    )\n    \nN = valid_df.shape[0]\nx_val = np.empty((N, 224, 224, 3), dtype=np.float32)\n\nfor i, image_id in enumerate(tqdm(valid_df['id_code'])):\n    x_val[i, :, :, :] = preprocess_image(\n        f'../input/aptos2019-blindness-detection/train_images/{image_id}.png'\n    )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train = train_df['diagnosis']\ny_val = valid_df['diagnosis']\nprint(x_train.shape)\nprint(y_train.shape)\nprint(x_val.shape)\nprint(y_val.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model: MobilenetV2\n\nI am using MobileNetV2 architecture in this project. The architecture is adopted from [1], where they choose 1.3 as the width multiplier/ alpha (MobileNetV2 hyperparameter). They customize the last layer of the model to become 2 dense layer with 256 nodes and 1 nodes output layer. They use regression approach for this problem because diabetic retinopathy severity is an ordinal variables. Linear activation function is used in the output layer."},{"metadata":{"trusted":true},"cell_type":"code","source":"model = load_model('../input/diabetic-retinopathy-mobilenetv2/model.h5')\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Check performance on validation data"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_val_pred = model.predict(x_val)\ny_val_pred = np.clip(y_val_pred,0,4).astype(int)\n\nlabels = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\ncnf_matrix = confusion_matrix(valid_df['diagnosis'].astype('int'), y_val_pred)\ndf_cm = pd.DataFrame(cnf_matrix, index=labels, columns=labels)\nplt.figure(figsize=(16, 7))\nsns.heatmap(df_cm, annot=True, cmap=\"Blues\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"kappa_val = cohen_kappa_score(\n            valid_df['diagnosis'].astype('int'),\n            y_val_pred, \n            weights='quadratic'\n        )\nkappa_val","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import classification_report\ntarget_names = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\nprint(classification_report(valid_df['diagnosis'].astype('int'), y_val_pred, target_names=target_names))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# MobileNetV2-SVM\n\nIn most transfer learning approach, the pre-trained weights of the convolutional layers are left intact while the fully-connected layers are re-trained with the specific dataset. The motivation for such approach is the assumption that the convolutional layers pre-trained in large-scale dataset are already an optimum generic feature extractor on their current state. Following such procedure, we investigated the utilization of the MobileNetV2 convolutional layers as a fixed feature extractor, while replacing the fully-connected layer (i.e. the feature classifier) with different types of classifiers that may offer more flexibility, for instance the Support Vector Machine (SVM). "},{"metadata":{},"cell_type":"markdown","source":"## Import Library"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.svm import SVC\n\nfrom sklearn.preprocessing import label_binarize\nfrom sklearn.preprocessing import StandardScaler\n\nfrom sklearn.pipeline import make_pipeline\n\nfrom sklearn.metrics import roc_curve, auc\n\nfrom sklearn.multiclass import OneVsOneClassifier\n\nfrom scipy import interp","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Removing The Output Layer\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import keras\nlayer_name = 'dense_2'\nintermediate_layer_model = keras.Model(inputs=model.input,\n                                       outputs=model.get_layer(layer_name).output)\nintermediate_layer_model.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Extract 256 Features from each images"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train_pred = intermediate_layer_model.predict(x_train)\ny_valid_pred = intermediate_layer_model.predict(x_val)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training SVM"},{"metadata":{"trusted":true},"cell_type":"code","source":"y_val = label_binarize(y_val, classes=[0,1,2,3,4])\n# classifier\nclf = make_pipeline(StandardScaler(),SVC(probability=True))\nclf = OneVsOneClassifier(clf)\ny_score = clf.fit(y_train_pred, y_train).decision_function(y_valid_pred)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Evaluate"},{"metadata":{},"cell_type":"markdown","source":"## Quadratic Weighted Kappa"},{"metadata":{"trusted":true},"cell_type":"code","source":"kappa_val = cohen_kappa_score(\n            np.argmax(y_val,axis=1),\n            np.argmax(y_score,axis=1), \n            weights='quadratic'\n        )\n\nprint(kappa_val)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## AUROC"},{"metadata":{"trusted":true},"cell_type":"code","source":"n_classes = 5\n\n# Compute ROC curve and ROC area for each class\nfpr = dict()\ntpr = dict()\nroc_auc = dict()\nfor i in range(n_classes):\n    fpr[i], tpr[i], _ = roc_curve(y_val[:, i], y_score[:, i])\n    roc_auc[i] = auc(fpr[i], tpr[i])\n\n# Compute micro-average ROC curve and ROC area\nfpr[\"micro\"], tpr[\"micro\"], _ = roc_curve(y_val.ravel(), y_score.ravel())\nroc_auc[\"micro\"] = auc(fpr[\"micro\"], tpr[\"micro\"])\n\nfrom itertools import cycle\nlw = 2\nall_fpr = np.unique(np.concatenate([fpr[i] for i in range(n_classes)]))\n\n# Then interpolate all ROC curves at this points\nmean_tpr = np.zeros_like(all_fpr)\nfor i in range(n_classes):\n    mean_tpr += interp(all_fpr, fpr[i], tpr[i])\n\n# Finally average it and compute AUC\nmean_tpr /= n_classes\n\nfpr[\"macro\"] = all_fpr\ntpr[\"macro\"] = mean_tpr\nroc_auc[\"macro\"] = auc(fpr[\"macro\"], tpr[\"macro\"])\n\n# Plot all ROC curves\nplt.figure()\nplt.plot(fpr[\"macro\"], tpr[\"macro\"],\n         label='macro-average ROC curve (area = {0:0.2f})'\n               ''.format(roc_auc[\"macro\"]),\n         color='navy', linestyle=':', linewidth=4)\n\ncolors = cycle(['aqua', 'darkorange', 'cornflowerblue'])\nfor i, color in zip(range(n_classes), colors):\n    plt.plot(fpr[i], tpr[i], color=color, lw=lw,\n             label='ROC curve of class {0} (area = {1:0.2f})'\n             ''.format(i, roc_auc[i]))\n\nplt.plot([0, 1], [0, 1], 'k--', lw=lw)\nplt.xlim([0.0, 1.0])\nplt.ylim([0.0, 1.05])\nplt.xlabel('False Positive Rate')\nplt.ylabel('True Positive Rate')\nplt.title('Some extension of Receiver operating characteristic to multi-class')\nplt.legend(loc=\"lower right\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Confusion Matrix"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\ncnf_matrix = confusion_matrix(np.argmax(y_val,axis=1), np.argmax(y_score,axis=1))\ndf_cm = pd.DataFrame(cnf_matrix, index=labels, columns=labels)\nplt.figure(figsize=(16, 7))\nsns.heatmap(df_cm, annot=True, cmap=\"Blues\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Accuracy etc."},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import classification_report\ntarget_names = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\nprint(classification_report(np.argmax(y_val,axis=1), np.argmax(y_score,axis=1), target_names=target_names))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}