{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Detect diabetic retinopathy to stop blindness before it's too late\n\nwe'll build a machine learning model to speed up disease detection. we’ll work with thousands of images collected in rural areas to help identify diabetic retinopathy automatically. If successful, it will not only help to prevent lifelong blindness, but these models may be used to detect other sorts of diseases in the future, like glaucoma and macular degeneration.\n\nThe dataset contains around 3.5ktraining images of retina which have been labelled by clinicians for the severity of diabetic retinopathy on a scale of 0 to 4:\n\n0 - No DR\n\n1 - Mild\n\n2 - Moderate\n\n3 - Severe\n\n4 - Proliferative DR\n\nWe will use Fastai to solve this problem. \n## What is Fastai\n\nThe fastai library simplifies training fast and accurate neural nets using modern best practices. It's based on research in to deep learning best practices undertaken at fast.ai, including \"out of the box\" support for vision, text, tabular, and collab (collaborative filtering) models.\n\n### Let's get started with the code."},{"metadata":{},"cell_type":"markdown","source":"The following 3 lines ensure that any edits to libraries you make are reloaded here automatically, and also that any charts or images displayed are shown in this notebook."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"%reload_ext autoreload\n%autoreload 2\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We import all the necessary packages. We are going to work with the fastai V1 library which sits on top of Pytorch 1.0. The fastai library provides many useful functions that enable us to quickly and easily build neural networks and train our models."},{"metadata":{"trusted":true},"cell_type":"code","source":"from fastai import *\nfrom fastai.vision import *\nimport pandas as pd\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If you're using a computer with an unusually small GPU, you may get an out of memory error when running this notebook. If this happens, click Kernel->Restart, uncomment the 2nd line below to use a smaller batch size and try again."},{"metadata":{"trusted":true},"cell_type":"code","source":"bs = 64\n# bs = 16   # uncomment this line if you run out of memory even after clicking Kernel->Restart","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Looking at the data\n\nBelow, We have defined a path variable to navigate to the data source for the problem.\n\nWe have created 2 different variables for train csv file (which contains image file names and their respective labels) and to the image data source folder itself which contains all the images. (train_img_path)."},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\npath = '../input/aptos2019-blindness-detection/'\ntrain_csv_path = path +'/train.csv'\ntrain_img_path = path + 'train_images/'\ntrain = pd.read_csv(train_csv_path)\n\n#test path strings\nprint(train_csv_path)\nprint(train_img_path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"There are total {} images in training dataset\".format(len(train)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"f_names = get_image_files(train_img_path)\nf_names[:5]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Fastai ImageList.from_folder() enables access to fetch image files from a folder (imagenet style) using single line of code.\n\nLet's take a look at one of the image files using Imagelist."},{"metadata":{"trusted":true},"cell_type":"code","source":"il = ImageList.from_folder(train_img_path)\nil","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"il.open(il.items[10])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"train['id_code'] = train['id_code'].map(lambda x: (train_img_path + x + '.png'))\n\n#test the paths\nprint(train['id_code'][1])\nprint(train['id_code'][2])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The below code transforms the images by flipping them horizontally and vertically along with rotating, zooming, lighting etc to make the model more robust by boosting the image samples available for training."},{"metadata":{"trusted":true},"cell_type":"code","source":"tfms = get_transforms(do_flip=True,flip_vert=True,\n                      max_rotate=360,max_warp=0,max_zoom=1.1,\n                      max_lighting=0.1,p_lighting=0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Before any work can be done a dataset needs to be converted into a DataBunch object, and in the case of the computer vision data - specifically into an ImageDataBunch subclass.\n\nThe from_df() method allows to conviniently fetch data from a dataframe and create a dataBunch object. The ImageDataBunch object contains both training set and validation set and a optional test set. \n\nThe ImageDataBunch helps setup the model training process very quickly.\n\nIn the below code, we have notified the ImageDataBunch object that our data is in the \"train\" dataframe and our label column is \"diagnosis\". \n\nAlso, the image sizes are currently non coherent (of different sizes) in our dataset. It is very important to regularize the images to the same size before training. \n\nAlso we have set the batchsize to 64. This simply means that we are showing 64 images to the model at once during training."},{"metadata":{"trusted":true},"cell_type":"code","source":"data = ImageDataBunch.from_df(path = '', df= train, label_col='diagnosis', ds_tfms=tfms,\n                              valid_pct=0.2, size=224, bs=64).normalize(imagenet_stats)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Show_batch() is a handy method of ImageDataBunch which displays sample images from our dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"data.show_batch(rows = 4, figsize= (12,10))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(data.classes)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The evaluation metric for the competition is quadratic kappa score. The good folk's at FASTAI have already implemented this metric.\n\nIn a nutshell, this metric determines the agreement between predicted labels and ground truth.\ne.g. If the prediction of a particular sample is mild but the ground truth is severe, the metric penalizes more compared to if the prediction is mild and ground truth is moderate."},{"metadata":{"trusted":true},"cell_type":"code","source":"kappa = KappaScore()\nkappa.weights = \"quadratic\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Once we have our data in the dataBunch object  specifically ImageDataBunch for Computer vision related problems), we are ready to kickstart our training process. We can choose from a number of model architectures available in Fastai model Zoo.\n\nHere, we use Resnet34 and Resnet50 which arguably work pretty well for the problem at hand."},{"metadata":{},"cell_type":"markdown","source":"# Training: resnet34\n\nNow we will start training our model. We will use a convolutional neural network backbone and a fully connected head with a single hidden layer as a classifier.\n\nWe will train for 4 epochs (4 cycles through all our data).\n\nResnet34, as the name indicates is a 34 layer deep neural network which has been pretrained on Imagenet."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(data, models.resnet34, metrics=[error_rate, kappa])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see below how the neural network's architecture looks like:"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.save('stage-1')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's take a look at some images where our model went wrong. We see what the model predicted, what was the ground truth and what was the loss."},{"metadata":{"trusted":true},"cell_type":"code","source":"interpret = ClassificationInterpretation.from_learner(learn)\nlosses,idx = interpret.top_losses()\n\ninterpret.plot_top_losses(9, figsize=(15,11))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"interpret.plot_confusion_matrix(figsize=(12,12), dpi=60)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"interpret.most_confused(min_val=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Unfreezing, finetuning and learning rates\n\n#### Since our model is working as we expect it to, we will unfreeze our model and train some more."},{"metadata":{},"cell_type":"markdown","source":"### Explaining the unfreezing step:\n\nIn the steps above, **we just trained the top few layers of the neural net**. Now, we will unfreeze the entire neural net and train it with the learning rate found with lr_find(). Explanations below:"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.unfreeze()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's load the previous state of the model."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.load('stage-1')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is known that the learning rate is the most important hyper-parameter to tune for training deep neural networks. A new method for setting the learning rate named cyclical learning rates, which practically eliminates the need to experimentally find the best values and schedule for the global learning rates. Instead of monotonically decreasing the learning rate, this method lets the learning rate cyclically vary between reasonable boundary values. Training with cyclical learning rates instead of fixed values achieves improved classification accuracy without a need to tune and often in fewer iterations. [link here](https://arxiv.org/abs/1506.01186)\n\nlr_find() does this for us. "},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above plot shows us the optimal learning rates to train the nn. So now Let's unfreeze the entire model and train it with the learning rates determined from the plot above. We train the model for 2 epochs."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.unfreeze()\nlearn.fit_one_cycle(2, max_lr=slice(1e-06,1e-03))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's a fairly accurate model. Now let's jump to resnet50 and see how it performs better than resnet34."},{"metadata":{},"cell_type":"markdown","source":"# Training: resnet50\nNow we will train in the same way as before but with one caveat: instead of using resnet34 as our backbone we will use resnet50 (resnet34 is a 34 layer residual network while resnet50 has 50 layers.\n\nBasically, resnet50 usually performs better because it is a deeper network with more parameters. Let's see if we can achieve a higher performance here. To help it along, let's us use larger images too, since that way the network can see more detail. We reduce the batch size a bit since otherwise this larger network will require more GPU memory."},{"metadata":{"trusted":true},"cell_type":"code","source":"data = ImageDataBunch.from_df(path = '', df= train, label_col='diagnosis', ds_tfms=tfms,\n                              valid_pct=0.2, size=299, bs=32).normalize(imagenet_stats)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(data, models.resnet50, metrics=[error_rate,kappa])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()\nlearn.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Feel free to fork the kernel and Uncomment the below lines to train the Resnet50 model and see high it performs better than the above model."},{"metadata":{},"cell_type":"markdown","source":"We train the nn with resnet50 model architecture for 8 epochs."},{"metadata":{"trusted":true},"cell_type":"code","source":"# learn.fit_one_cycle(8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# learn.save('stage-1-50')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Unfreeze all the layers and train again for 3 epochs and learning rates determined with lr_find()"},{"metadata":{"trusted":true},"cell_type":"code","source":"# learn.unfreeze()\n# learn.fit_one_cycle(3, max_lr=(slice(1e-6,1e-4)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#If it doesn't, you can always go back to your previous model.\n# learn.load('stage-1-50');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"interpret the model performance intuitively:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# interp = ClassificationInterpretation.from_learner(learn)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# interp.most_confused(min_val=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This tutorial is taken from the Fastai V3 MOOC taught by the legendary **Jeremy Howard** who is also one of the founder of Fastai."},{"metadata":{},"cell_type":"markdown","source":"Further reading : https://course.fast.ai/"},{"metadata":{},"cell_type":"markdown","source":"**Please drop an upvote if you find this helpful :)**"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}