{"cells":[{"metadata":{},"cell_type":"markdown","source":"# About this notebook\n* This kernel is intended for Fastai beginners. It is quite easy to get a good model setup due to the ease of access to state-of-the-art algorithms provided by Fastai."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\n\nimport numpy as np\nimport pandas as pd\n\nfrom fastai import *\nfrom fastai.imports import *\nfrom fastai.vision.all import *","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Making sure GPU is on, both needs to be True. If the first is False, make sure GPU is turned on in the notebook settings. For issues with second, please check google for help. :D"},{"metadata":{"trusted":true},"cell_type":"code","source":"torch.cuda.is_available(),torch.backends.cudnn.enabled","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def seed_everything(seed):\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    random.seed(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\nseed_everything(42)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Quick data exploration"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"path = Path('../input/cassava-leaf-disease-classification/')\nos.listdir(path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(path/'train.csv')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.value_counts('label')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"High volume of data with an imblance."},{"metadata":{"trusted":true},"cell_type":"code","source":"train['label'].hist(figsize = (8, 6));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Adding the path of each image makes life easier when running inference."},{"metadata":{"trusted":true},"cell_type":"code","source":"train['image_id'] = train['image_id'].map(lambda x:path/'train_images'/x)\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loading data into Fastai format"},{"metadata":{},"cell_type":"markdown","source":"* Fastai provides several options for loading images; csv, folders, path, df...\n* In this case; the images are loaded using the pandas dataframe which contains the path of each image. You can also specify the image folder's path to the ImageDataLoaders instead of adding the path to df explicitly.\n* item_tfm resizes each image in terms of pixels, large sizes greatly increase computation.\n* aug_transforms() applies image transformations. For a full detail of the transformations applied, check out their doc: https://docs.fast.ai/vision.augment.html#aug_transforms.\n* Normalize.from_stats(*imagenet_stats): Normalizes images based on mean and std of the imagenet pretrained model as we are using a pretrained resnet model.\n* batch_tfms applies the above two points to each batch of images.\n* Have a look at their doc if you want to learn more about the parameters and other methods to load data: https://docs.fast.ai/vision.data.html."},{"metadata":{"trusted":true},"cell_type":"code","source":"tfms = aug_transforms()\ndata = ImageDataLoaders.from_df(train, valid_pct=0.2, item_tfms=Resize(512), batch_tfms=[*tfms,Normalize.from_stats(*imagenet_stats)])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.show_batch()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Modelling"},{"metadata":{},"cell_type":"markdown","source":"* Fastai uses a single trainer class (cnn_learner) that takes in the data, model, metrics, loss functions, optimizer functions etc.\n* Here, the Resnet50 model is used. Its a convolutional neural network that has 50 layers and is pretrained on thousands of images with optimised weights.\n* You can also easily use techniques such as label smoothing with the Learner object if you wish. \n* to_native_fp16() changes floating point representation to 16-bit; GPU computation is faster for neural networks."},{"metadata":{},"cell_type":"markdown","source":"As we cannot use the internet for the competition, the pretrained resnet50 model is loaded (placed in the folder the online model would be placed in) in as a input dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"if not os.path.exists('/root/.cache/torch/hub/checkpoints/'):\n    os.makedirs('/root/.cache/torch/hub/checkpoints/')\n    !cp '../input/resnet50preload/resnet50-19c8e357.pth' '/root/.cache/torch/hub/checkpoints/resnet50-19c8e357.pth'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(data, models.resnet50, pretrained=True, metrics=accuracy, model_dir='../working/').to_native_fp16()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Training (Short version)**\n1. First we find the learning rate.\n2. Freeze the model and train only the last layer(has random weights).\n> As the pretrained model is used, the initial weights on the other layers have 'good' values. Hence we freeze the first layers and train only the last layer that has random weights.\n3. Unfreeze, find the learning rate again, and train all layers.\n> Model is unfreezed to train all layers."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* fit_one_cycle is widely used to train models, you can also try other fit methods.\n* We freeze and train the last layer.\n* The first parameter is the Epoch; Number of times the dataset is iterated by the CNN. Second is the learning rate.\n* You can increase the epochs to experiment. Higher values usually give better results however it increases computation drastically."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.freeze()\nlearn.fit_one_cycle(1,1e-1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Observing the loss function."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.recorder.plot_loss()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* We unfreeze to train all the layers.\n* The learning rate changes due to unlocking all layers."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.unfreeze()\nlearn.lr_find()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* An approximated value of the learning rate is used. You can try others, using small rates increases computation.\n* Use more epochs, at least 10 (experiment with values to get optimal validation loss values) for actual model."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(5, 1e-4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.recorder.plot_loss()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Change back to 32-bit."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = learn.to_native_fp32()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inference"},{"metadata":{},"cell_type":"markdown","source":"* It is easy to get the predictions on the test set as long as the test data is in the same format as train data.\n* Add in path of test images similar to train images."},{"metadata":{"trusted":true},"cell_type":"code","source":"test = pd.read_csv(path/'sample_submission.csv')\ntmp_test = test.copy()\ntmp_test['image_id'] = tmp_test['image_id'].map(lambda x:path/'test_images'/x)\ntmp_test.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Fastai provides a method (test_dl) that can parse the test data with the same parameters as the training data."},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data = data.test_dl(tmp_test)\ntest_data.show_batch()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* tta applies the same image transformations as on the training data.\n* In practise, the image is transformed n times and its predicted results are averaged."},{"metadata":{"trusted":true},"cell_type":"code","source":"predict, t = learn.tta(dl=test_data, n=8, beta=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test['label'] = predict.argmax(dim=-1).numpy()\ntest.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}