{"cells":[{"metadata":{},"cell_type":"markdown","source":"This kernel uses fastai to predict the 5 classes of diabetic retinopathy"},{"metadata":{},"cell_type":"markdown","source":"## Loading libraries"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# loading the necessary libraries\nimport numpy as np\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom fastai.vision import *\nfrom fastai.metrics import error_rate\nimport os\nprint(os.listdir(\"../input\"))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"# Training\nbs = 64 //2\npath_anno = '../input/aptos2019-blindness-detection/train.csv'\npath_img = '../input/aptos2019-blindness-detection/train_images'\n\n# Test dataset\ntpath_anno = '../input/aptos2019-blindness-detection/test.csv'\ntpath_img = '../input/aptos2019-blindness-detection/test_images'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Later, I use resnet as the base architecture. However, since we can't use the internet for this kernel in this competition I will set these directories which will contain the models. This is because, `cnn_learner` will check those directories first before attempting to download. When internet for the kernel is turned off and these models don't exist, an error will be raised. To add the models, click on add dataset at the top right corner of this kernel and search for resnet. Make sure to choose resnet for `PyTorch`"},{"metadata":{"trusted":true},"cell_type":"code","source":"# creating directories and copying the models to those directories\n!mkdir -p /tmp/.cache/torch/checkpoints\n!cp ../input/resnet50/resnet50.pth /tmp/.cache/torch/checkpoints/resnet50-19c8e357.pth\n!cp ../input/resnet152/resnet152.pth /tmp/.cache/torch/checkpoints/resnet152-b121ed2d.pth","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# File names\nfnames = get_image_files(path_img)\nfnames[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Training images and their labels\ndf = pd.read_csv(path_anno)\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Test images \ntdf = pd.read_csv(tpath_anno)\ntdf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test = ImageList.from_df(df = tdf, path = tpath_img, suffix = '.png')\nlen(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Transformations"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Since these are microscopic images, I'll turn on random flipping of the images (since there's no really \n# up or down for the images) so the model generalizes well\n\ntfms = get_transforms(flip_vert=True, max_lighting=0.1, max_zoom=1, max_warp=0.)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(42)\nsrc = (ImageList.from_df(path = path_img, df = df, suffix = '.png')\n        .split_by_rand_pct(0.2)\n        .label_from_df(label_delim=None)\n      .add_test(test))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = (src.transform(tfms, size=126)\n        .databunch().normalize(imagenet_stats))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.classes","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualize data"},{"metadata":{"trusted":true},"cell_type":"code","source":"data.show_batch(rows=3, figsize=(7,6))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model Training"},{"metadata":{},"cell_type":"markdown","source":"### Resnet50"},{"metadata":{"trusted":true},"cell_type":"code","source":"kappa = KappaScore()\nkappa.weights = \"quadratic\"\n\nlearn = cnn_learner(data, models.resnet50, metrics=[error_rate, kappa], model_dir = Path('../kaggle/working'),\n                   path = Path(\".\"))\n\n# Let's fit a couple of cycles\nlearn.fit_one_cycle(2)\n\n# Now let's find a more accurate learning rate\nlearn.unfreeze()\nlearn.lr_find()\nlearn.recorder.plot(suggestion=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Let's fit some more with a more accurate learning rate\nlearn.fit_one_cycle(8, max_lr=slice(0.01))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# let's save the model in case, we want to return to this point\nlearn.save('train-1-rn50', return_path = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Fine tuning"},{"metadata":{"trusted":true},"cell_type":"code","source":"# let's see if we can get even a better learning rate from this point on\nlearn.load('train-1-rn50')\n\n# Let's freeze up to last layer group. That is to sets every layer group except the last to untrainable \nlearn.freeze()\n\n# I'll try a smaller learning rate\nlearn.fit_one_cycle(8, max_lr=slice(1e-03))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's replace the data with even a higher size and see "},{"metadata":{"trusted":true},"cell_type":"code","source":"data2 = (src.transform(tfms, size= 256)\n        .databunch().normalize(imagenet_stats))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Replacing data \nlearn.data = data2\n\n# train some more by building on the previous model\nlearn.fit_one_cycle(10)\n\n# Learning rate\nlearn.unfreeze()\nlearn.lr_find()\nlearn.recorder.plot(suggestion=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# choosing a learning rate based on the data \nlearn.fit_one_cycle(10, max_lr = slice(1e-4))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Prediction of test set and submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"preds, _ = learn.get_preds(ds_type=DatasetType.Test)\npred_prob, pred_class = preds.max(1)\nlen(pred_class)\n\nmy_submission = pd.DataFrame({'id_code': tdf['id_code'] , 'diagnosis': pred_class})\nmy_submission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}