{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Glaucoma Diagnosis and Segmentation\n## Introduction :\nDuring this challenge, we tried to diagnose glaucoma from images of the inside back surface of the eye. We have developed a deep learning system that can detect glaucoma disease. We have worked with multiple convolutional neural networks models UNet, VGG, and AlexNet and tested different kinds of data augmentation.\n\n## Plan :\n   >  __1. Data Pre-processing__\n   \n   >  __2. VGG and AlexNet Networks__\n   \n   >  __3. Training and Testing__ \n   \n   >  __4. Conclusion__ \n","metadata":{}},{"cell_type":"code","source":"import os\nimport json\nimport csv\nimport random\nimport pickle\nimport cv2\nimport numpy as np\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport torchvision.transforms as transforms\n\nfrom PIL import Image\nfrom torch.utils.data import Dataset, DataLoader\nfrom scipy.ndimage.measurements import label\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import roc_auc_score, roc_curve\n\n#dimension of the images:2124x2056","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## README\nthis notebook presents two models: VGG and AlexNet. It is implemented so that the AlexNet can be runned. If you want to test with the VGG here are the parameters to change:\n\nchange ___output_size = (224,224)___\nfor the best model we found put __lr = 1e-4__, __batch_size = 4__, __num_workers = 8__\nThen you can uncomment all the code related to VGG (and comment the AlexNet code) in the following parts:\n\n- Train The VGG Model (you'll see. a variable set to true in the training function that adapats the training for VGG)\n\n- Confusion Matrix on training set\n\n- Load Models Weights\n\n- Predictions on test set\n\nOther paramters that didn't work but available if you want to try:\n\n- Confusion Matrix on training set\n\n- Load Models Weights\n\n- Predictions on test set\n\nOther paramters that didn't work but available if you want to try:\n\nwe implemented the clahe functions for __lightning Normalization and brightness__. As it didn't improved our model we didn't use it but if you want to activate it please put the variable normalize_light_using_CLAHE = True\n\nin the section ___\"UPSAMPLING of the training set With Rotation Functions\"___ we impemented a data augmentation that adds rotated images from our dataset. it didn't improve our results. if you want to uncomment the code and add samples through the variable added_values for each lable.","metadata":{}},{"cell_type":"markdown","source":"# 1. Data Pre-processing\n","metadata":{}},{"cell_type":"code","source":"# For Alex Net the ouput size is (256,256)\n# For the VGG net the output size is (224,224)\n\noutput_size = (256,256)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### CLAHE FUNCTION - LIGHTNING CONTRAST NORMALIZATION \nThe different lighting conditions and intensity variations among images across various databases were circumvented by performing normalization of the histogram using Contrast Limited Adaptive HistogramEqualization (CLAHE). \nFor this, we have implemented a function that allows to improve the contrast of images, it takes as input an image, and generates an image with a better contrast. Instead of using Adaptive histogram equalization (AHE) which over amplifies noise in relatively homogeneous regions of an image, we have used a variant of adaptive histogram equalization called contrast limited adaptive histogram equalization (CLAHE) prevents this by limiting the amplification.\n","metadata":{}},{"cell_type":"code","source":"# CLAHE Functions are designed for lightning Normalization and brightness  \n\ndef clahe_single(ori_img,clipLimit,tileGridSize):\n\n    img = np.array(Image.open(ori_img).convert('RGB'))\n    \n    lab = cv2.cvtColor(img, cv2.COLOR_RGB2LAB)\n    \n    lab_planes = cv2.split(lab)\n    clahe = cv2.createCLAHE(clipLimit,tileGridSize)\n    lab_planes[0] = clahe.apply(lab_planes[0])\n    \n    lab = cv2.merge(lab_planes)\n    rgb = cv2.cvtColor(lab, cv2.COLOR_LAB2RGB)\n\n    return rgb\n    \ndef clahe_all1(ori_img):\n    rgb = clahe_single(ori_img, 0.1 , (8,8))\n    return Image.fromarray(rgb)\n\ndef clahe_all2(ori_img):\n    rgb = clahe_single(ori_img, 2.0, (300,300))\n    return Image.fromarray(rgb)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"im = Image.open(\"../input/eurecom-aml-2021-challenge-2/refuge_data/refuge_data/train/images/g0002.jpg\")\nimport matplotlib.pyplot as plt\nplt.imshow(im)\n\n\nclahe_all2(\"../input/eurecom-aml-2021-challenge-2/refuge_data/refuge_data/train/images/g0001.jpg\")\n\nwidth, height = im.size\n  \n# Setting the points for cropped image\nleft = 5\ntop = 620\nright = 700\nbottom = 1400\n  \n# Cropped image of above dimension\n# (It will not change orginal image)\nim = im.crop((left, top, right, bottom))\nplt.imshow(im)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load Data ","metadata":{}},{"cell_type":"code","source":"# For the lightning Nomalization, set normalize_light to True \n\nnormalize_light_using_CLAHE = False\n\nclass RefugeDataset(Dataset):\n\n    def __init__(self, root_dir, split='train', output_size=output_size):\n        # Define attributes\n        self.output_size = output_size\n        self.root_dir = root_dir\n        self.split = split\n        \n        # Load data index\n        with open(os.path.join(self.root_dir, self.split, 'index.json')) as f:\n            self.index = json.load(f)\n            \n        self.images = []\n        \n            \n        for k in range(len(self.index)):\n            print('Loading {} image {}/{}...'.format(split, k, len(self.index)), end='\\r')\n            img_name = os.path.join(self.root_dir, self.split, 'images', self.index[str(k)]['ImgName'])\n            img = Image.open(img_name).crop((left, top, right, bottom))\n            if (normalize_light_using_CLAHE) :\n                img = clahe_all1(img_name)\n            else :\n                img = np.array(Image.open(img_name).convert('RGB'))\n            #img = img.crop((left, top, right, bottom))\n            img = transforms.functional.to_tensor(img)\n            img = transforms.functional.resize(img, self.output_size, interpolation=Image.BILINEAR)\n            self.images.append(img)\n            \n        # Load ground truth for 'train' and 'val' sets\n        if split != 'test':\n            self.segs = []\n            for k in range(len(self.index)):\n                print('Loading {} segmentation {}/{}...'.format(split, k, len(self.index)), end='\\r')\n                seg_name = os.path.join(self.root_dir, self.split, 'gts', self.index[str(k)]['ImgName'].split('.')[0]+'.bmp')\n                seg = Image.open(seg_name).crop((left, top, right, bottom))\n                seg = np.array(seg).copy()\n                seg = 255. - seg\n                od = (seg>=127.).astype(np.float32)\n                oc = (seg>=250.).astype(np.float32)\n                od = torch.from_numpy(od[None,:,:])\n                oc = torch.from_numpy(oc[None,:,:])\n                od = transforms.functional.resize(od, self.output_size, interpolation=Image.NEAREST)\n                oc = transforms.functional.resize(oc, self.output_size, interpolation=Image.NEAREST)\n                seg = torch.cat([od, oc], dim=0)\n                self.segs.append(seg)\n                \n        print('Succesfully loaded {} dataset.'.format(split) + ' '*50)\n            \n            \n    def __len__(self):\n        return len(self.index)\n\n    def __getitem__(self, idx):\n        # Image\n        img = self.images[idx]\n    \n        # Return only images for 'test' set\n        if self.split == 'test':\n            return img\n        \n        # Else, images and ground truth\n        else:\n            # Label\n            lab = torch.tensor(self.index[str(idx)]['Label'], dtype=torch.float32)\n\n            # Segmentation masks\n            seg = self.segs[idx]\n\n            # Fovea localization\n            f_x = self.index[str(idx)]['Fovea_X']\n            f_y = self.index[str(idx)]['Fovea_Y']\n            fov = torch.FloatTensor([f_x, f_y])\n        \n            return img, lab, seg, fov, self.index[str(idx)]['ImgName']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Datasets\nroot_dir = '/kaggle/input/eurecom-aml-2021-challenge-2/refuge_data/refuge_data'\nbatch_size = 8 # Initial value: 8\nnum_workers = 32\ntotal_epoch = 100 # Initial value: 100\n\n\ntrain_set = RefugeDataset(root_dir, \n                          split='train')\nval_set = RefugeDataset(root_dir, \n                        split='val')\ntest_set = RefugeDataset(root_dir, \n                         split='test')\n\n# Dataloaders\ntrain_loader = DataLoader(train_set, \n                          batch_size=batch_size, \n                          shuffle=True, \n                          num_workers=num_workers,\n                          pin_memory=True,\n                          #sampler=sampler\n                         )\nval_loader = DataLoader(val_set, \n                        batch_size=batch_size, \n                        shuffle=False, \n                        num_workers=num_workers,\n                        pin_memory=True,\n                        )\ntest_loader = DataLoader(test_set, \n                        batch_size=batch_size, \n                        shuffle=False, \n                        num_workers=num_workers,\n                        pin_memory=True,\n                        )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Upsamling Using Vanilla Upsampling ","metadata":{}},{"cell_type":"code","source":"import seaborn as sns \nimport matplotlib.pyplot as plt\n\nadded_values = 0 # number of values added to the imbalanced part of the train set\n\n######## Upsampling the class 1 ########\n\nupsample_nb = int(added_values) # sample number of label 1 we want to add\nclass_1_dict = {k: v for k, v in train_set.index.items() if v[\"Label\"] == 1} # keep only samples with label 1\n\nfor k in range(1,upsample_nb+1):\n  element = dict([random.choice(list(class_1_dict.items()))]) # Choose randomly one sample of label 1\n  \n  L = [(k,v) for k,v in element.items()] # {\"key\": {value}}\n  key,value = L[0][0],L[0][1] \n  \n  # add img\n  img_name = os.path.join(train_set.root_dir, train_set.split, 'images', train_set.index[key]['ImgName'])\n  \n  seg_name = os.path.join(train_set.root_dir, train_set.split, 'gts', \n                          train_set.index[key]['ImgName'].split('.')[0]+'.bmp')\n\n  #img, seg = augmentation_data(img_name,seg_name)\n  img, seg = Image.open(img_name),Image.open(seg_name)\n\n  img = np.array(img.convert('RGB'))\n  img = transforms.functional.to_tensor(img)\n  img = transforms.functional.resize(img, train_set.output_size, interpolation=Image.BILINEAR)\n  train_set.images.append(img)\n\n  # add segs\n  seg = np.array(seg).copy()\n  seg = 255. - seg\n  od = (seg>=127.).astype(np.float32)\n  oc = (seg>=250.).astype(np.float32)\n  od = torch.from_numpy(od[None,:,:])\n  oc = torch.from_numpy(oc[None,:,:])\n  od = transforms.functional.resize(od, train_set.output_size, interpolation=Image.NEAREST)\n  oc = transforms.functional.resize(oc, train_set.output_size, interpolation=Image.NEAREST)\n  seg = torch.cat([od, oc], dim=0)\n  train_set.segs.append(seg)\n\n  element = dict({str(399+k): value})\n  train_set.index = dict(train_set.index, **element)\n\n\n\n\n######## Upsampling the class  0 ########\nindex_start = 399+added_values\n\nadded_values = 0\n\nupsample_nb = int(added_values) # sample number of label 1 we want to add\nclass_0_dict = {k: v for k, v in train_set.index.items() if v[\"Label\"] == 0} # keep only samples with label 1\nfor k in range(1,upsample_nb+1):\n  element = dict([random.choice(list(class_0_dict.items()))]) # Choose randomly one sample of label 1\n  L = [(k,v) for k,v in element.items()] # {\"key\": {value}}\n  key,value = L[0][0],L[0][1] \n\n  # add img\n  img_name = os.path.join(train_set.root_dir, train_set.split, 'images', train_set.index[key]['ImgName'])\n  \n  seg_name = os.path.join(train_set.root_dir, train_set.split, 'gts', \n                          train_set.index[key]['ImgName'].split('.')[0]+'.bmp')\n\n  #img, seg = augmentation_data(img_name,seg_name)\n  img, seg = Image.open(img_name),Image.open(seg_name)\n\n  img = np.array(img.convert('RGB'))\n  img = transforms.functional.to_tensor(img)\n  img = transforms.functional.resize(img, train_set.output_size, interpolation=Image.BILINEAR)\n  train_set.images.append(img)\n\n  # add segs\n  \n  seg = np.array(seg).copy()\n  seg = 255. - seg\n  od = (seg>=127.).astype(np.float32)\n  oc = (seg>=250.).astype(np.float32)\n  od = torch.from_numpy(od[None,:,:])\n  oc = torch.from_numpy(oc[None,:,:])\n  od = transforms.functional.resize(od, train_set.output_size, interpolation=Image.NEAREST)\n  oc = transforms.functional.resize(oc, train_set.output_size, interpolation=Image.NEAREST)\n  seg = torch.cat([od, oc], dim=0)\n  train_set.segs.append(seg)\n\n  element = dict({str(index_start+k): value})\n  train_set.index = dict(train_set.index, **element)\n    \ntrain_loader = DataLoader(train_set, \n                          batch_size=batch_size, \n                          shuffle=True, \n                          num_workers=num_workers,\n                          pin_memory=True,\n                         )\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Upsampling Using Rotation Functions to images ","metadata":{}},{"cell_type":"code","source":"# Function to rotate Images \n\n# Rotate Image with 45 degrees\ndef rotate1(ori_img):\n    colorImage  = Image.open(ori_img)\n    # Rotate it by 45 degrees\n    rotated     = colorImage.rotate(45)\n    return rotated\n\n# Rotate Image with 90 degrees\ndef rotate2(ori_img):\n    colorImage  = Image.open(ori_img)\n    # Rotate it by 90 degrees\n    rotated     = colorImage.rotate(90)\n    \n    return rotated\n\n# Rotate Image with 180 degrees\ndef rotate3(ori_img):\n    colorImage  = Image.open(ori_img)\n    # Rotate it by 180 degrees\n    rotated     = colorImage.rotate(180)\n    return rotated\n\n# Rotate Image with 270 degrees\ndef rotate4(ori_img):\n    colorImage  = Image.open(ori_img)\n    # Rotate it by 180 degrees\n    rotated     = colorImage.rotate(270)\n    return rotated\n\n# select randomly a rotation and applied it to an image and the image segmentation \ndef augmentation_data(ori_img,seg_img) :\n    \n    index = random.randint(0,3)\n    #if index == 0 :\n    #    return clahe_all1(ori_img),clahe_all1(seg_img)\n    #if index == 1 :\n    #    return clahe_all2(ori_img),clahe_all2(seg_img)\n    if index == 0 :\n        return rotate1(ori_img),rotate1(seg_img)\n    if index == 1 :\n        return rotate2(ori_img),rotate2(seg_img)\n    if index == 2 :\n        return rotate3(ori_img),rotate3(seg_img)\n    if index == 3 :\n        return rotate4(ori_img),rotate4(seg_img)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import seaborn as sns \nimport matplotlib.pyplot as plt\n\nadded_values = 0 # number of values added to the imbalanced part of the train set\n\n######## Upsampling the class 1 ########\n\nupsample_nb = int(added_values) # sample number of label 1 we want to add\nclass_1_dict = {k: v for k, v in train_set.index.items() if v[\"Label\"] == 1} # keep only samples with label 1\n\nfor k in range(1,upsample_nb+1):\n  element = dict([random.choice(list(class_1_dict.items()))]) # Choose randomly one sample of label 1\n  \n  L = [(k,v) for k,v in element.items()] # {\"key\": {value}}\n  key,value = L[0][0],L[0][1] \n  \n  # add img\n  img_name = os.path.join(train_set.root_dir, train_set.split, 'images', train_set.index[key]['ImgName'])\n  \n  seg_name = os.path.join(train_set.root_dir, train_set.split, 'gts', \n                          train_set.index[key]['ImgName'].split('.')[0]+'.bmp')\n\n  img, seg = augmentation_data(img_name,seg_name)\n  #img, seg = Image.open(img_name),Image.open(seg_name)\n\n  img = np.array(img.convert('RGB'))\n  img = transforms.functional.to_tensor(img)\n  img = transforms.functional.resize(img, train_set.output_size, interpolation=Image.BILINEAR)\n  train_set.images.append(img)\n\n  # add segs\n  seg = np.array(seg).copy()\n  seg = 255. - seg\n  od = (seg>=127.).astype(np.float32)\n  oc = (seg>=250.).astype(np.float32)\n  od = torch.from_numpy(od[None,:,:])\n  oc = torch.from_numpy(oc[None,:,:])\n  od = transforms.functional.resize(od, train_set.output_size, interpolation=Image.NEAREST)\n  oc = transforms.functional.resize(oc, train_set.output_size, interpolation=Image.NEAREST)\n  seg = torch.cat([od, oc], dim=0)\n  train_set.segs.append(seg)\n\n  element = dict({str(399+k): value})\n  train_set.index = dict(train_set.index, **element)\n\n\n\n\n######## Upsampling the class  0 ########\nindex_start = 399+added_values\n\nadded_values = 0\n\nupsample_nb = int(added_values) # sample number of label 1 we want to add\nclass_0_dict = {k: v for k, v in train_set.index.items() if v[\"Label\"] == 0} # keep only samples with label 1\nfor k in range(1,upsample_nb+1):\n  element = dict([random.choice(list(class_0_dict.items()))]) # Choose randomly one sample of label 1\n  L = [(k,v) for k,v in element.items()] # {\"key\": {value}}\n  key,value = L[0][0],L[0][1] \n\n  # add img\n  img_name = os.path.join(train_set.root_dir, train_set.split, 'images', train_set.index[key]['ImgName'])\n  \n  seg_name = os.path.join(train_set.root_dir, train_set.split, 'gts', \n                          train_set.index[key]['ImgName'].split('.')[0]+'.bmp')\n\n  img, seg = augmentation_data(img_name,seg_name)\n  img, seg = Image.open(img_name),Image.open(seg_name)\n\n  #img = np.array(img.convert('RGB'))\n  img = transforms.functional.to_tensor(img)\n  img = transforms.functional.resize(img, train_set.output_size, interpolation=Image.BILINEAR)\n  train_set.images.append(img)\n\n  # add segs\n  \n  seg = np.array(seg).copy()\n  seg = 255. - seg\n  od = (seg>=127.).astype(np.float32)\n  oc = (seg>=250.).astype(np.float32)\n  od = torch.from_numpy(od[None,:,:])\n  oc = torch.from_numpy(oc[None,:,:])\n  od = transforms.functional.resize(od, train_set.output_size, interpolation=Image.NEAREST)\n  oc = transforms.functional.resize(oc, train_set.output_size, interpolation=Image.NEAREST)\n  seg = torch.cat([od, oc], dim=0)\n  train_set.segs.append(seg)\n\n  element = dict({str(index_start+k): value})\n  train_set.index = dict(train_set.index, **element)\n    \ntrain_loader = DataLoader(train_set, \n                          batch_size=batch_size, \n                          shuffle=True, \n                          num_workers=num_workers,\n                          pin_memory=True,\n                         )\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Networks - VGG and ALEXNET","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\n\nimport tensorflow as tf\nfrom tensorflow.keras.applications import *\nfrom tensorflow.keras.optimizers import *\nfrom tensorflow.keras.losses import *\nfrom tensorflow.keras.layers import *\nfrom tensorflow.keras.models import *\nfrom tensorflow.keras.callbacks import *\nfrom tensorflow.keras.preprocessing.image import *\nfrom tensorflow.keras.utils import *\n# import pydot\nfrom sklearn.metrics import *\nfrom sklearn.model_selection import *\nimport tensorflow.keras.backend as K\n\n# from tqdm import tqdm, tqdm_notebook\n# from colorama import Fore\n# import json\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom glob import glob\nfrom skimage.io import *\n%config Completer.use_jedi = False\nimport warnings\nwarnings.filterwarnings('ignore')\nprint(\"All modules have been imported\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## VGG\nVGG takes as input images of size 224 x 224. Thus we had to resize the images. VGG is composed of a series of convolutional networks in which max pooling is used and it is also composed of three fully-connected: the first two have 4096 channels and the third one 2 channels corresponding to each class (0 and 1). We also used some dropout to avoid overfitting and regularize the network. In the rest of this part we will compare different tunings that we’ve made. \n","metadata":{}},{"cell_type":"code","source":"VGG_types = {\n    'VGG11' : [64,'M',128,'M',256,256,'M',512,512,'M',512,512,'M'],\n    'VGG13' : [64,64,'M',128,128,'M',256,256,'M',512,512,'M',512,512,'M'],\n    'VGG16' : [64,64,'M',128,128,'M',256,256,256,'M',512,512,512,'M',512,512,512,'M'],\n    'VGG19' : [64,64,'M',128,128,'M',256,256,256,256,'M',512,512,512,512,'M',512,512,512,512,'M']\n}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Initializing a Sequential model\n\nclass VGG_net(nn.Module):\n\n    def __init__(self, num_classes=2, in_channels=3 , init_weights=True):\n        super(VGG_net, self).__init__()\n        self.in_channels = in_channels\n        self.num_classes = num_classes\n        self.conv_layers = self.create_conv_layers(VGG_types['VGG19'])\n        self.epoch = 0\n        \n# inplace=True means that it will modify the input directly, without allocating any additional output. \n# It can sometimes slightly decrease the memory usage, but may not always be a valid operation \n# (because the original input is destroyed). \n# However, if you don't see an error, it means that your use case is valid.\n        self.classifier = nn.Sequential(\n            nn.Linear(512*7*7,4096),\n            nn.ReLU(inplace=True),\n            nn.Dropout(p=0.72),   # initially p=0.5\n            nn.Linear(4096,4096),\n            nn.ReLU(True),\n            nn.Dropout(p=0.72),   # initially p=0.5\n            nn.Linear(4096, num_classes),\n        )\n        \n        if init_weights:\n            self._initialize_weights()\n\n    def forward(self, x):\n        x= self.conv_layers(x)\n        x = x.reshape(x.shape[0],-1)\n        x = self.classifier(x)\n        return x\n    \n    \n    def create_conv_layers(self, architecture, batch_norm=True):\n        layers = []\n        in_channels = self.in_channels\n        for v in architecture:\n            if type(v) == int:\n                out_channels = v\n                conv2d = nn.Conv2d(in_channels=in_channels, out_channels=out_channels, \n                                   kernel_size=(3,3),stride=(1,1), padding=(1,1))\n                if batch_norm:\n                    layers += [conv2d, nn.BatchNorm2d(v), nn.ReLU(inplace=True)]\n                else:\n                    layers += [conv2d, nn.ReLU(inplace=True)]\n                in_channels = v\n            elif v == 'M':\n                layers += [nn.MaxPool2d(kernel_size=(2,2), stride=(2,2))]\n\n        return nn.Sequential(*layers)\n\n    def _initialize_weights(self):\n        for m in self.modules():\n            if isinstance(m, nn.Conv2d):\n                nn.init.kaiming_normal_(\n                    m.weight, mode='fan_out', nonlinearity='relu')\n                if m.bias is not None:\n                    nn.init.constant_(m.bias, 0)\n            elif isinstance(m, nn.BatchNorm2d):\n                nn.init.constant_(m.weight, 1)\n                nn.init.constant_(m.bias, 0)\n            elif isinstance(m, nn.Linear):\n                nn.init.normal_(m.weight, 0, 0.01)\n                nn.init.constant_(m.bias, 0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## AlexNet \nAlexNet contained eight layers; the first five were convolutional layers, some of them followed by max-pooling layers (the second, the fourth and the fifth layers), and the last three were fully connected layers. It used the non-saturating ReLU activation function. AlexNet takes input images of size 256 x 256. We used some dropouts to avoid overfitting and regularize the network, and we normalized contrast images using the CLAHE function as it gives good results as presented above. \n","metadata":{}},{"cell_type":"code","source":"# The idea of the model implementation was taken from : https://github.com/supersmm/230project \n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport sys\nsys.path.append('../')\n\nfrom torch.hub import load_state_dict_from_url\n\n__all__ = ['AlexNet', 'alexnet']\n\n\nmodel_urls = {\n    'alexnet': 'https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth',\n}\n\n\nclass AlexNet(nn.Module):\n\n    def __init__(self, num_classes=1000):\n        super(AlexNet, self).__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=3, stride=2),\n            nn.Conv2d(64, 192, kernel_size=5, padding=2),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=3, stride=2),\n            nn.Conv2d(192, 384, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(384, 256, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(256, 256, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=3, stride=2),            \n        )\n        self.epoch = 0 \n        self.avgpool = nn.AdaptiveAvgPool2d((6,6))\n        self.classifier = nn.Sequential(\n            nn.Dropout(0.72),\n            nn.Linear(256 * 6 * 6, 4096),\n            nn.ReLU(inplace=True),\n            nn.Dropout(),\n            nn.Linear(4096, 4096),\n            nn.ReLU(inplace=True),\n            nn.Linear(4096, num_classes),\n        )\n\n    def forward(self, x):\n        x = self.features(x)\n        x = self.avgpool(x)\n        x = x.view(x.size(0), 256 * 6 * 6)\n        x = self.classifier(x)\n        x = torch.sigmoid(x)\n        return x\n    \nclass Post_AlexNet(nn.Module):\n    def __init__(self, AlexnetClass = 1000, num_classes = 2):\n        super(Post_AlexNet, self).__init__()\n        self.classifier = nn.Sequential(\n            nn.Dropout(),\n            nn.Linear(AlexnetClass, 50),\n            nn.BatchNorm1d(50),\n            nn.ReLU(inplace=True),\n            nn.Dropout(),\n            nn.Linear(50, 50),\n            nn.BatchNorm1d(50),\n            nn.ReLU(inplace=True),\n            nn.Linear(50, num_classes),\n            \n        )\n    def forward(self, x):\n        x = self.classifier(x)\n        x = torch.sigmoid(x)\n        return x\n\n\ndef alexnet(pretrained=False, progress=True, num_classes = 2, **kwargs):\n   \n    model = AlexNet(**kwargs)\n    if pretrained:\n        state_dict = load_state_dict_from_url(model_urls['alexnet'],\n                                              progress=progress)\n        model.load_state_dict(state_dict)\n        \n    Post_model = Post_AlexNet( num_classes = num_classes)\n    model = nn.Sequential(model,Post_model)\n    return model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Training and Test ","metadata":{}},{"cell_type":"markdown","source":"### Settings","metadata":{}},{"cell_type":"code","source":"root_dir = '/kaggle/input/eurecom-aml-2021-challenge-2/refuge_data/refuge_data'\nlr = 1e-5","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Metrics","metadata":{}},{"cell_type":"code","source":"def classif_eval(classif_preds, classif_gts):\n    '''\n    Compute AUC classification score.\n    '''\n    auc = roc_auc_score(classif_gts, classif_preds)\n    return auc","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Loss Function \n# Exponential Weighted Cross Entropy Loss Function used for AlexNet \ndef ExpWCrossEntropy(outputs, labels, weights = (1,0.1)):\n    \n    labels = torch.tensor(labels)\n    labels = labels.to(device)\n    '''\n    Cross entropy loss with uneven weigth between positive and negative result, add exponential function to positive to manually adjust precision and recall\n    '''\n    loss = torch.sum(torch.add(weights[0]*torch.exp(-torch.mul(labels[:],torch.log(outputs[:])))-1, -weights[1]*torch.mul(1 - labels[:],torch.log(1 - outputs[:]))))\n    return loss","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train Functions \n# if you to train the VGG model, please set train_VGG to True, otherwise set it as False \n# We use Cross Entropy Loss Function for VGG \n\ndef Train_Model(train_VGG,model,name = \"VGG\") :\n    nb_train_batches = len(train_loader)\n    nb_val_batches = len(val_loader)\n    nb_iter = 0\n    best_val_auc = 0.\n\n    for epoch in range(total_epoch):\n        # Accumulators\n        train_loss, val_loss = 0., 0.\n        train_classif_gts, val_classif_gts = [], []\n        train_classif_pred, val_classif_pred = [], []\n        train_loss, val_loss = 0., 0.\n        train_error=0.\n        val_error=0.\n\n        ############\n        # TRAINING #\n        ############\n        model.train()\n        train_data = iter(train_loader)\n        for k in range(nb_train_batches):\n            # Loads data\n            imgs, classif_gts, seg_gts, fov_coords, names = train_data.next()\n            imgs, classif_gts, seg_gts = imgs.to(device), classif_gts.to(device), seg_gts.to(device)\n\n\n            # Forward pass\n            logits = model(imgs)\n            probas, preds = torch.max(logits, 1)\n            clz=classif_gts.long().to(device)\n\n            logits = model(imgs)\n            output_ = F.softmax(logits, dim = 1)\n            \n\n            if(train_VGG) :\n                loss = criterion(logits ,clz)\n            else : \n                loss = criterion(output_[:,1],clz)\n\n\n            # Backward pass\n            optimizer.zero_grad()\n            loss.backward()\n            optimizer.step()\n            train_loss += loss.item() / nb_train_batches\n\n            with torch.no_grad():\n\n                train_classif_pred += logits.tolist()\n                train_classif_gts += classif_gts.cpu().numpy().tolist()\n                # statistics\n                train_error += torch.sum(preds != clz) / nb_train_batches\n\n            # Increase iterations\n            nb_iter += 1\n\n            # Std out\n            print('Epoch {}, iter {}/{}, loss {:.6f}'.format(epoch+1, k+1, nb_train_batches, loss.item()) + ' '*20, \n                  end='\\r')\n\n        # Train a logistic regression on vCDRs\n        #print(train_classif_pred)\n        train_classif_pred = np.array(train_classif_pred)[:,1].reshape(-1,1)\n        train_classif_gts = np.array(train_classif_gts)\n        clf = LogisticRegression(random_state=0, solver='lbfgs').fit(train_classif_pred, train_classif_gts)\n        train_classif_preds = clf.predict_proba(train_classif_pred)[:,1]\n        train_auc = classif_eval(train_classif_preds, train_classif_gts)\n\n        ##############\n        # VALIDATION #\n        ##############\n        model.eval()\n        with torch.no_grad():\n            val_data = iter(val_loader)\n            for k in range(nb_val_batches):\n                # Loads data\n                imgs, classif_gts, seg_gts, fov_coords, names = val_data.next()\n                imgs, classif_gts, seg_gts = imgs.to(device), classif_gts.to(device), seg_gts.to(device)\n\n                # Forward pass\n                logits = model(imgs)\n                \n                probas, preds = torch.max(logits, 1)\n                clz=classif_gts.long().to(device)\n                \n                output_ = F.softmax(logits, dim = 1)\n            \n                if(train_VGG) :\n                    val_loss += criterion(logits, clz).item() / nb_val_batches\n                else : \n                    val_loss += criterion(output_[:,1],clz).item() / nb_val_batches\n\n                # Std out\n                print('Validation iter {}/{}'.format(k+1, nb_val_batches) + ' '*50, \n                      end='\\r')\n\n                val_classif_pred += logits.tolist()\n                val_classif_gts += classif_gts.cpu().numpy().tolist()\n                # statistics\n                val_error += torch.sum(preds != clz) / nb_train_batches\n\n        # Glaucoma predictions from vCDRs\n        \n        val_classif_pred = np.array(val_classif_pred)[:,1].reshape(-1,1)\n        val_classif_gts = np.array(val_classif_gts)\n        val_classif_preds = clf.predict_proba(val_classif_pred)[:,1]\n        val_auc = classif_eval(val_classif_preds, val_classif_gts)\n\n\n        # Validation results\n        print('VALIDATION epoch {}'.format(epoch+1)+' '*50)\n        print('LOSSES: {:.4f} (train), {:.4f} (val)'.format(train_loss, val_loss))\n        print('Classification (AUC): {:.4f} (train), {:.4f} (val)'.format(train_auc, val_auc))\n\n        # Save model if best validation AUC is reached\n        if val_auc > best_val_auc:\n            torch.save(model.state_dict(), '/kaggle/working/best_AUC_weights_'+name+'.pth')\n            with open('/kaggle/working/best_AUC_classifier_'+name+'.pkl', 'wb') as clf_file:\n                pickle.dump(clf, clf_file)\n            best_val_auc = val_auc\n            print('Best validation AUC reached. Saved model weights and classifier.')\n        print('_'*50)\n\n        # End of epoch\n        epoch += 1\n    return val_classif_preds,val_classif_gts","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train The VGG Model ","metadata":{}},{"cell_type":"code","source":"'''\n# Device\ndevice = torch.device(\"cuda:0\")\n# Network\nvgg_model = VGG_net(in_channels=3, num_classes=2).to(device)\n# Loss\ncriterion = torch.nn.CrossEntropyLoss()\n# Optimizer\noptimizer = optim.Adam(vgg_model.parameters(), lr=lr)\n'''","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"'''\nimport warnings\nwarnings.filterwarnings('ignore')\nval_classif_preds_vgg,val_classif_gts_vgg = Train_Model(True,vgg_model)\n'''","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train Alex Net ","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')\n\n#device = torch.device(\"cuda:0\")\nalex = alexnet(pretrained=True, progress=False, num_classes = 2)\nalex_model = alex.to(device)\n# Loss\ncriterion = ExpWCrossEntropy\n# Optimizer\noptimizer = optim.Adam(alex_model.parameters(), lr=lr, weight_decay=1e-05)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')\nval_classif_preds_alex,val_classif_gts_alex = Train_Model(False,alex_model,\"alex\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Confusion Matrix on Validation set","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt \nfrom sklearn.metrics import *\nimport itertools\ndef plot_confusion_matrix(cm, classes,\n                          normalize=False,\n                          title='Confusion matrix',\n                          cmap=plt.cm.Blues):\n    \"\"\"\n    This function prints and plots the confusion matrix.\n    Normalization can be applied by setting `normalize=True`.\n    \"\"\"\n    plt.figure(figsize = (6,6))\n    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n    plt.title(title)\n    plt.colorbar()\n    tick_marks = np.arange(len(classes))\n    plt.xticks(tick_marks, classes, rotation=90)\n    plt.yticks(tick_marks, classes)\n    if normalize:\n        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n\n    thresh = cm.max() / 2.\n    cm = np.round(cm,2)\n    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n        plt.text(j, i, cm[i, j],\n                 horizontalalignment=\"center\",\n                 color=\"white\" if cm[i, j] > thresh else \"black\")\n    plt.tight_layout()\n    plt.ylabel('True label')\n    plt.xlabel('Predicted label')\n    plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_confusion_Matrix(val_classif_preds,val_classif_gts) :\n    val_classif_preds_ = np.where(val_classif_preds > 0.5, 1, 0)\n    val_classif_gts_ = np.where(val_classif_gts > 0.5, 1, 0)\n    pred=np.argmax(val_classif_preds_,axis=-1)\n    cm=confusion_matrix( val_classif_gts_, val_classif_preds_)\n    cm_plot=plot_confusion_matrix(cm, classes=['0','1'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"'''\n#VGG_NET\nplot_confusion_Matrix(val_classif_preds_vgg,val_classif_gts_vgg)\n'''","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#ALEX_NET\nplot_confusion_Matrix(val_classif_preds_alex,val_classif_gts_alex)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load Models Weights ","metadata":{}},{"cell_type":"code","source":"'''\nvgg_model.load_state_dict(torch.load('/kaggle/working/best_AUC_weights_VGG.pth'))\nwith open('/kaggle/working/best_AUC_classifier_VGG.pkl', 'rb') as clf_file:\n    clf = pickle.load(clf_file)\n'''","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"alex_model.load_state_dict(torch.load('/kaggle/working/best_AUC_weights_alex.pth'))\nwith open('/kaggle/working/best_AUC_classifier_alex.pkl', 'rb') as clf_file:\n    clf = pickle.load(clf_file)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Predictions on test set","metadata":{}},{"cell_type":"code","source":"nb_test_batches = len(test_loader)\n\ndef test_probabilities(model) :\n    model.eval()\n    test_classif_pred = []\n    with torch.no_grad():\n        test_data = iter(test_loader)\n        for k in range(nb_test_batches):\n            # Loads data\n            imgs = test_data.next()\n            imgs = imgs.to(device)\n\n            # Forward pass\n            logits = model(imgs)\n            _, preds = torch.max(logits, 1)\n\n            output_ = F.softmax(logits, dim = 1)\n            # Std out\n            print('Test iter {}/{}'.format(k+1, nb_test_batches) + ' '*50, \n                  end='\\r')\n\n            # Compute and store probability\n            test_classif_pred += logits.tolist()\n            #running_loss += loss.item() * inputs.size(0)\n\n\n        # Glaucoma predictions\n        test_classif_pred = np.array(test_classif_pred)[:,1].reshape(-1,1)\n        test_classif_preds = clf.predict_proba(test_classif_pred)[:,1]\n    return test_classif_preds\n\n# Prepare and save .csv file\ndef create_submission_csv(prediction, submission_filename='/kaggle/working/submission.csv'):\n    \"\"\"Create a sumbission file in the appropriate format for evaluation.\n\n    :param\n    prediction: list of predictions (ex: [0.12720, 0.89289, ..., 0.29829])\n    \"\"\"\n    \n    with open(submission_filename, mode='w') as csv_file:\n        fieldnames = ['Id', 'Predicted']\n        writer = csv.DictWriter(csv_file, fieldnames=fieldnames)\n        writer.writeheader()\n\n        for i, p in enumerate(prediction):\n            writer.writerow({'Id': \"T{:04d}\".format(i+1), 'Predicted': '{:f}'.format(p)})\ntest_classif_preds = test_probabilities(alex_model)\ncreate_submission_csv(test_classif_preds)\n\n# The submission.csv file is under /kaggle/working/submission.csv.\n# If you want to submit it, you should download it before closing the current kernel.","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Conclusion \nThus through this project, we learnt how to create a model to classify image datasets and do its segmentation. For the scenario of classifying Glaucoma, we first preprocessed the dataset by upsampling and data augmentation methods and then fed the processed dataset through convolutional neural networks. By comparing VGG, Unet and Alexnet we found that the ALexnet model is performing better on the provided image dataset. The hyperparameters were tuned to avoid overfitting issues as well as enhanced the Test AUC of the model. If we would have more time to experiment, we would probably have experimented with the Dense Net model.\n","metadata":{}}]}