{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Aptos 2019 Blindness Detection\n\nThe goal of this competition is to use a set of supplied retina images to predict the severity of any diabetic retinopathy (diabetic blindness) that may be present, according to the following scale:\n* 0 - No DR\n* 1 - Mild\n* 2 - Moderate\n* 3 - Severe\n* 4 - Proliferative DR\n\nI will be using Pytorch with a pre-trained ResNet-50 model to adjust its weights according to the training set images for predicting the final diagnoses on the test set.\n\nAlso, the training set supplied will be enhanced by adding cropped and resized versions of a previous DR competition held in 2015 (according to ilovescience's kernel https://www.kaggle.com/tanlikesmath/diabetic-retinopathy-resnet50-binary-cropped ). The current images will also be cropped and resized, similarly to the algorithm used in said kernel, to eliminate blackspace and reduce image size for performance. These modifications will also be replicated on the test set images. To load and modify the images, I will be using PIL and OpenCV."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport PIL\nimport cv2\nimport seaborn as sns\n\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim \nimport torchvision\nfrom torchvision import models\nfrom torch.utils.data import DataLoader, Dataset\nimport torch.utils.data as utils\nfrom torchvision import transforms\n\nimport matplotlib.pyplot as plt\n\nprint(os.listdir('../input/'))\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Loading in Data\n\nBefore beginning, we want to load in the supplied datasets into corresponding Pandas dataframes and confirm the structure of the data fields is as expected."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Directories to be accessed for images\ntrain_dir = '../input/aptos2019-blindness-detection/train_images/'\ntest_dir = '../input/aptos2019-blindness-detection/test_images/'\nsupp_dir = '../input/diabetic-retinopathy-resized/resized_train_cropped/resized_train_cropped/'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Checking that the data is formatted as expected:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data = pd.read_csv(\"../input/aptos2019-blindness-detection/train.csv\")\nprint(\"Training set has %d rows and %d columns\" % train_data.shape)\ntrain_data.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data = pd.read_csv(\"../input/aptos2019-blindness-detection/test.csv\")\nprint(\"Test set has %d rows and %d columns\" % test_data.shape)\ntest_data.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.read_csv(\"../input/aptos2019-blindness-detection/sample_submission.csv\")\nprint(\"Submission set has %d rows and %d columns\" % submission.shape)\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As a quick sanity check, we want to confirm that the id_code column is identical for the test_data and submission dataframes:"},{"metadata":{"trusted":true},"cell_type":"code","source":"if test_data[\"id_code\"].equals(submission[\"id_code\"]):\n    print(\"The id_code columns are identical\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With equality of each dataset ensured, we can now simply copy the submission dataframe onto the test dataframe (so that the diagnosis column is added to the test)."},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data = submission.copy()\ntest_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Exploring the makeup of the training set:"},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set(style='darkgrid')\nsns.countplot(y = 'diagnosis',\n              data = train_data)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From this distribution, it is clear that the dataset is imbalanced in favor of DR-absent cases, as well as moderate DR cases."},{"metadata":{},"cell_type":"markdown","source":"To enhance the training data, the supplemental data from the previous competition will be loaded in (specifically, the resized and cropped versions)."},{"metadata":{"trusted":true},"cell_type":"code","source":"supp_data = pd.read_csv(\"../input/diabetic-retinopathy-resized/trainLabels_cropped.csv\")\nprint(\"Test set has %d rows and %d columns\" % supp_data.shape)\nsupp_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The supplemental dataset is improperly labeled as is, this code fixes it:"},{"metadata":{"trusted":true},"cell_type":"code","source":"supp_data = supp_data.drop([\"Unnamed: 0\",\"Unnamed: 0.1\"], axis = 1)\nsupp_data = supp_data.rename(columns = {\"image\":\"id_code\", \"level\":\"diagnosis\"})\nsupp_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Image Retrieval\n\nNow the images in each set of data will be loaded in, and the current competition's train and test data will be cropped according to the strategy in ilovescience's kernel (https://www.kaggle.com/tanlikesmath/diabetic-retinopathy-resnet50-binary-cropped) used to created the supplemental data added to this kernel."},{"metadata":{"trusted":true},"cell_type":"code","source":"def cropImage(image):\n# Credits to ilovescience, https://www.kaggle.com/tanlikesmath/diabetic-retinopathy-resnet50-binary-cropped\n# Crops out as much of the surrounding blackspace as possible by forming a bounding\n# box around the lens view of the microscope\n    output = image.copy()\n    gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)\n    thresh = 20 # Grayscale cutoff, increasing speeds up function but may result in more cropping failures\n    ret,gray = cv2.threshold(gray,thresh,255,cv2.THRESH_BINARY)\n    contours,hierarchy = cv2.findContours(gray,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)\n    if not contours:\n        return image\n    cnt = max(contours, key=cv2.contourArea)\n    ((x, y), r) = cv2.minEnclosingCircle(cnt)\n    x = int(x); y = int(y); r = int(r)\n    #print(x,y,r)\n    if r > 100:\n        return output[max(y-r,0):-1 + max(y+r+1,0),max(x-r,0):-1 + max(x+r+1,0)]\n    else:\n        return image\n    \ndef resizeImage(image, size):\n    # Image resizing if needed, preserves aspect ratio\n    width = image.shape[1]\n    height = image.shape[0]\n    if width > size and height > size:\n        if width > height:\n            newWidth = size\n            newHeight = int(height * (newWidth/width))\n        else:\n            newHeight = size\n            newWidth = int(width * (newHeight/height))\n        newDims = newWidth, newHeight\n        image = cv2.resize(image, newDims, interpolation = cv2.INTER_AREA)\n    return image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Running Model Using Pytorch\n\nCredits go to Alexander Teplyunk for their kernel sections utilizing Pytorch (https://www.kaggle.com/ateplyuk/aptos-pytorch-starter-rnet50), which I adapted for my data setup.\n\nPytorch will be used to ingest the data and adjust weights on the pre-trained ResNet-50 model accordingly"},{"metadata":{"trusted":true},"cell_type":"code","source":"def getImages(type, dataset):\n    images = []\n    if type == \"train\":\n        extension = \".png\"\n        folder = train_dir\n        # update_freq - Number of images to go through before providing progress updates\n        update_freq = 250\n        cropped = False\n    elif type == \"test\":\n        extension = \".png\"\n        folder = test_dir\n        update_freq = 250\n        cropped = False\n    elif type == \"supplemental\":\n        extension = \".jpeg\"\n        folder = supp_dir\n        update_freq = 2500\n        cropped = True\n    else:\n        print(\"Invalid folder provided, specify either 'train', 'test', or 'supplemental'.\")\n        return dataset\n    ids = dataset['id_code']\n    length = len(ids)\n    size = 265 # Maximum dimension size to downscale to\n    for i in range(length):\n        if (i + 1) % update_freq == 0:\n            print(\"Processed image %d of %d\" % (i+1,length))\n        id = ids[i]\n        fileName = folder + id + extension\n        im = cv2.cvtColor(cv2.imread(fileName),cv2.COLOR_BGR2RGB)\n        im = resizeImage(im, size)\n        if not cropped:\n            im = cropImage(im)\n        images.append(im)\n    dataset[\"image\"] = images\n    return dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Loading in each dataset:"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Loading in training images...\")\ntrain_data = getImages(\"train\", train_data)\nprint(\"Done loading training images!\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Loading in test images...\")\ntest_data = getImages(\"test\", test_data)\nprint(\"Done loading test images!\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Loading in supplemental images...\")\nsupp_data = getImages(\"supplemental\", supp_data)\nprint(\"Done loading supplemental images!\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If the images were properly loaded, a sample image from each set should be displayed below:"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(train_data[\"image\"][0])\nplt.grid(b=None)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(test_data[\"image\"][0])\nplt.grid(b=None)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(supp_data[\"image\"][0])\nplt.grid(b=None)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Combining Old and New Data\n\nTo enhance the training set, the supplemental data from the previous blindness competition will be combined with the current training data, we will also see how adding this data affects the score distribution:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data = train_data.append(supp_data,ignore_index=True)\nsns.set(style='darkgrid')\nsns.countplot(y = 'diagnosis',\n              data = train_data)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is clear that adding the supplemental data made the dataset even more imbalanced in favor of DR-absent cases, this may reduce the accuracy of the model on new data."},{"metadata":{"trusted":true},"cell_type":"code","source":"class ImageData(Dataset):\n    def __init__(self, df, transform):\n        super().__init__()\n        self.df = df\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, index):       \n        label = self.df.diagnosis[index]           \n        \n        image = self.df.image[index]\n        image = self.transform(image)\n        return image, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_transf = transforms.Compose([transforms.ToPILImage(mode='RGB'), \n                                  transforms.Resize(265),\n                                  transforms.CenterCrop(224),\n                                  transforms.ToTensor()])\ntrain = ImageData(df = train_data, transform = data_transf)\ntrain_loader = DataLoader(dataset = train, batch_size=32, drop_last=True, shuffle=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = models.resnet50()\nmodel.load_state_dict(torch.load(\"../input/resnet50/resnet50.pth\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Freeze model weights\nfor param in model.parameters():\n    param.requires_grad = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Changing number of model's output classes to 5\nmodel.fc = nn.Linear(2048, 5)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"# Transfer execution to GPU\nmodel = model.to('cuda')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"optimizer = optim.Adam(model.parameters())\nloss_func = nn.CrossEntropyLoss()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n# Train model\nloss_log=[]\nfor epoch in range(15):    \n    model.train()\n    for ii, (data, target) in enumerate(train_loader):       \n        data, target = data.cuda(), target.cuda()              \n        optimizer.zero_grad()\n        output = model(data)                    \n        loss = loss_func(output, target)\n        loss.backward()\n        optimizer.step()          \n        if ii % 1000 == 0:\n            loss_log.append(loss.item())       \n    print('Epoch: {} - Loss: {:.6f}'.format(epoch + 1, loss.item()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test = ImageData(df = test_data, transform = data_transf)\ntest_loader = DataLoader(dataset = test, shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Creating and Outputting predictions"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n# Prediction\npredict = []\nmodel.eval()\nfor i, (data, _) in enumerate(test_loader):\n    data = data.cuda()\n    output = model(data)  \n    output = output.cpu().detach().numpy()    \n    predict.append(output[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission['diagnosis'] = np.argmax(predict, axis=1)\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}