{"cells":[{"metadata":{},"cell_type":"markdown","source":"# APTOS 2019 Blindness Detection\n### Detect diabetic retinopathy to stop blindness before it's too late \n\n\n"},{"metadata":{},"cell_type":"markdown","source":"Importing Dependencies and defining file paths. "},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"from fastai.vision import *\nfrom fastai import *\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport os\nimport cv2\nimport glob\nimport torch\n\n\ndevice = 'cuda:0' if torch.cuda.is_available() else 'cpu'\nprint(f'Running on device: {device}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PATH = Path('../input/aptos2019-blindness-detection')\ntrain = PATH/'train_images'\ntest = PATH/'test_images'\ntrain_folder = 'train_images'\nmodel_dir = Path('/kaggle/working/')\n\ntrain_df = pd.read_csv(os.path.join(PATH, 'train.csv'))\ntrain_df['id_code'] = train_df['id_code'].apply(lambda x: f'{train_folder}/{x}.png')\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PATH.ls()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.countplot(train_df['diagnosis'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The plot above does indicate some imbalance between the classes. "},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"Size of Training set images: {len(list(train.glob('*.png')))}\")\nprint(f\"Size of Test set images: {len(list(test.glob('*.png')))}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### On to Fastai.\nThe Data block API makes it way easier to define a databunch. \nI use a couple of transformations for the first stage of training.\nThe values selected for these transformations are based on top performing solution summaries on the discussion tab. "},{"metadata":{"trusted":true},"cell_type":"code","source":"tfms = get_transforms(do_flip=True,flip_vert=True,max_rotate=180,\n                      max_warp=0,max_zoom=1.25,p_lighting=0.4,\n                      max_lighting=0.3,xtra_tfms=[flip_lr()] )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Defining the data to be used for training and validation. \n**Randomly setting aside 2% of the data for validation. **\nFor the first training stage, i used padding as the resize method. \nTo get an understanding of how resize methods work in fastai check out the data augmentations part from fastai docs [here](https://docs.fast.ai/vision.transform.html#Data-augmentation)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"img_size = 456\nbs = 64\nsrc = (ImageList.from_df(train_df, PATH)\n        .split_by_rand_pct()\n        .label_from_df())\n\ndata = (src.transform(tfms, size=img_size, resize_method=ResizeMethod.PAD,padding_mode='zeros')\n       .databunch(bs=32).normalize(imagenet_stats))\n\ndata\n\n# The more simpler ImageDataBunch shortcut method. \n# data = ImageDataBunch.from_df(PATH, train_csv, folder='train_images', \n#                               suffix='.png', no_check=True, \n#                               ds_tfms=get_transforms(), size=512, bs=32).normalize(imagenet_stats)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.train_ds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.valid_ds","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Having a look at our Data. "},{"metadata":{"trusted":true},"cell_type":"code","source":"data.show_batch(rows=3, figsize=(10,8))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Training. \n\nAs stated by the competition, they use the Quadratic Kappa score as the evaluation metric. \nI also print the error_rate metric to get an idea how it performs side by side with Kappa. \n\nThis is **Stage-1** Training. \n> Fine tuning only the newly added final layers of the model whilst freezing the earlier layrers and using a pretrained(ImageNet) Resnet50.  "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Training\nkappa = KappaScore()\nkappa.weights = \"quadratic\"\n\nlearner = cnn_learner(data, models.resnet50, metrics=[error_rate, kappa])\n\nlearner.fit_one_cycle(4)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Running the **LR finder** which would show us the optimal learning rate through a LR plot. "},{"metadata":{"trusted":true},"cell_type":"code","source":"learner.model_dir = '/kaggle/working'\nlearner.unfreeze()\nlearner.lr_find()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learner.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Based on the above plot and coupled with Jeremy Howards advices:\n- We select a value for the learning rate where the loss is minimum. \n- Use this small learning rate to train the earlier layers. \n\nThis time we fine tune the entire model, albiet with varying learning rates accross the layers. \n- Smaller learning rate for earlier layers (slow learning as not much weight updation needed). "},{"metadata":{"trusted":true},"cell_type":"code","source":"learner.fit_one_cycle(4, max_lr=slice(1e-5, 1e-3))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learner.save('stage-2', return_path = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is **Stage-2** training. \n> Here we create a new databunch with differnt sized pictures. We used 512x512 Images for stage-1 training. \n> Now we use 456x456 to fine tune our model further. \n> Following the same cycle, unfreezing --> lr_finder --> training with varying learning rates. "},{"metadata":{"trusted":true},"cell_type":"code","source":"tfms_456 = get_transforms(do_flip=True,flip_vert=True,max_rotate=360,\n                      max_warp=0,max_zoom=1.35,p_lighting=0.5,\n                      max_lighting=0.2)\n\ndata_456 = (src.transform(tfms_456, size=512)\n           ).databunch(bs=32).normalize(imagenet_stats)\n\ndata_456\n\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"learner_2 = cnn_learner(data_456, models.resnet50, metrics=[error_rate, kappa])\n\nlearner_2.load(model_dir/'stage-2')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"learner_2.model_dir = '/kaggle/working'\nlearner_2.unfreeze()\nlearner_2.lr_find()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learner_2.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learner_2.fit_one_cycle(4, max_lr=slice(1e-5, 1e-3))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}