{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":13836,"databundleVersionId":1718836,"sourceType":"competition"},{"sourceId":7548366,"sourceType":"datasetVersion","datasetId":4396147}],"dockerImageVersionId":30646,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# [Kaggle competition: Cassava Leaf Disease Classification](https://www.kaggle.com/c/cassava-leaf-disease-classification/)\n#### *Identifing the type of disease present on a Cassava Leaf image*\n\nIn this competition, we explore a dataset consisting of $21,367$ labeled photographs of Cassava leaves collected during a regular survey in Uganda. Images were crowdsourced from farmers and annotated by experts at the National Crops Resources Research Institute (NaCRRI) in collaboration with the AI lab at Makerere University, Kampala.\n\nOur task is to classify each image into one of $5$ categories: one indicating a **healthy** leaf, and the remaining four categories indicating different diseases. These are **Cassava Bacteria Blight** (CBB), **Cassava Brown Streak Disease** (CBSD), **Cassava Green Mottle** (CGM), and **Cassava Mosaic Disease** (CMD). \n\nIn a previous Notebook, [finetuned-resnet152-model-eda-and-training-1-2](http://www.kaggle.com/owenpb/finetuned-resnet152-model-eda-and-training-1-2), we performed:\n\n$1.$ Exploratory Data Analysis (EDA) of the Cassava leaf image dataset, and demonstration of image augmentation techniques using the Albumentations library.\n\n$2.$ Training and finetuning a ResNet-152 model from torchvision (PyTorch) with GPU P100 accelerator. \n\nIn this Notebook, we will load our trained model, ensembling our out-of-fold (OOF) predictions while making use of test time augmentations (TTA). This Notebook contains:\n\n$3.$ Ensembling out-of-fold (OOF) predictions with test time augmentations (TTA). \n\n$4.$ Preparing our final submission for this competition.","metadata":{}},{"cell_type":"markdown","source":"## Import libraries:","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torchvision\nfrom torchvision import transforms\nfrom torch.utils.data import DataLoader\nfrom torch.utils.data import Dataset\nfrom torchinfo import summary\n\nfrom torchvision.models import resnet152, ResNet152_Weights\n\nimport albumentations \nfrom albumentations.pytorch.transforms import ToTensorV2\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\n\nimport os\nimport copy\nimport glob\nimport json\nimport random\nimport pathlib\nfrom PIL import Image\nimport pickle \n\n\nBASE_PATH = '/kaggle/input/cassava-leaf-disease-classification/'\nDEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nprint(f'Device: {DEVICE}')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-02-10T16:50:29.320688Z","iopub.execute_input":"2024-02-10T16:50:29.320934Z","iopub.status.idle":"2024-02-10T16:50:37.559253Z","shell.execute_reply.started":"2024-02-10T16:50:29.320911Z","shell.execute_reply":"2024-02-10T16:50:37.558250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Inference: Ensembling out-of-fold (OOF) predictions with test time augmentations (TTA)","metadata":{}},{"cell_type":"markdown","source":"Here we load models trained (in individual Notebooks) on our $K=5$ separate folds:","metadata":{}},{"cell_type":"code","source":"loaded_model_0 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_resnet152_10_epochs_fold_0.pt')\nloaded_weights_0 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_weights_resnet152_10_epochs_fold_0.pt')\n\nloaded_model_1 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_resnet152_10_epochs_fold_1.pt')\nloaded_weights_1 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_weights_resnet152_10_epochs_fold_1.pt')\n\nloaded_model_2 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_resnet152_10_epochs_fold_2.pt')\nloaded_weights_2 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_weights_resnet152_10_epochs_fold_2.pt')\n\nloaded_model_3 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_resnet152_10_epochs_fold_3.pt')\nloaded_weights_3 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_weights_resnet152_10_epochs_fold_3.pt')\n\nloaded_model_4 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_resnet152_10_epochs_fold_4.pt')\nloaded_weights_4 = torch.load('/kaggle/input/best-model-tuned-resnet152-10-epochs-all-folds/best_model_tuned_weights_resnet152_10_epochs_fold_4.pt')","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:51:15.320680Z","iopub.execute_input":"2024-02-10T16:51:15.321300Z","iopub.status.idle":"2024-02-10T16:51:49.448465Z","shell.execute_reply.started":"2024-02-10T16:51:15.321266Z","shell.execute_reply":"2024-02-10T16:51:49.447446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loaded_model_0.load_state_dict(loaded_weights_0)\nloaded_model_0.eval()\n\nloaded_model_1.load_state_dict(loaded_weights_1)\nloaded_model_1.eval()\n\nloaded_model_2.load_state_dict(loaded_weights_2)\nloaded_model_2.eval()\n\nloaded_model_3.load_state_dict(loaded_weights_3)\nloaded_model_3.eval()\n\nloaded_model_4.load_state_dict(loaded_weights_4)\nloaded_model_4.eval()","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:51:51.964066Z","iopub.execute_input":"2024-02-10T16:51:51.964907Z","iopub.status.idle":"2024-02-10T16:51:52.150987Z","shell.execute_reply.started":"2024-02-10T16:51:51.964874Z","shell.execute_reply":"2024-02-10T16:51:52.150055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"models = [loaded_model_0, loaded_model_1, loaded_model_2, loaded_model_3, loaded_model_4]","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:52:08.646135Z","iopub.execute_input":"2024-02-10T16:52:08.646780Z","iopub.status.idle":"2024-02-10T16:52:08.651290Z","shell.execute_reply.started":"2024-02-10T16:52:08.646746Z","shell.execute_reply":"2024-02-10T16:52:08.650216Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load test images into dataframe\n\nNote that only one test image is visible in the test_images directory. However, by collecting all images present in the test_images directory into df_test as shown below, we ensure all test images (hidden by Kaggle for the purposes of this competition) will be processed when we make our final submission. ","metadata":{}},{"cell_type":"code","source":"test_images = glob.glob('../input/cassava-leaf-disease-classification/test_images/*.jpg')\ndf_test = pd.DataFrame(test_images, columns = ['path'])\ndf_test['label'] = -1","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:52:26.857358Z","iopub.execute_input":"2024-02-10T16:52:26.857708Z","iopub.status.idle":"2024-02-10T16:52:26.874729Z","shell.execute_reply.started":"2024-02-10T16:52:26.857672Z","shell.execute_reply":"2024-02-10T16:52:26.873906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Test Time Augmentation (TTA):\n\nWe will augment each image in the test set $5$ times. Along with the original test image, we thus have $6$ versions of each image. After feeding an image to our model, we get a set of class probabilities for each of our num_classes=$5$ disease categories. For each image in the test set, we simply average the probabilities obtained from its $6$ versions, giving $5$ averaged class probabilities per image.\n\nNote that even for the case where we perform no image augmentation, we still need to appropriately resize and normalize the image before it is fed into our network.","metadata":{}},{"cell_type":"code","source":"width = 512\nheight = 512\n\ntest_transforms = albumentations.Compose([\n    albumentations.CenterCrop(width, height, p=1.0),\n    albumentations.Resize(width, height),\n    albumentations.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\ntta_transforms = albumentations.Compose([\n    \n    albumentations.RandomResizedCrop(width, height),\n    albumentations.HorizontalFlip(p=0.5),\n    albumentations.Transpose(p=0.5),\n    albumentations.VerticalFlip(p=0.5),\n    albumentations.ShiftScaleRotate(p=0.5),\n    albumentations.HueSaturationValue(\n                hue_shift_limit=0.2, \n                sat_shift_limit=0.2, \n                val_shift_limit=0.2, \n                p=0.5\n            ),\n    albumentations.RandomBrightnessContrast(\n                brightness_limit=(-0.1, 0.1), \n                contrast_limit=(-0.1, 0.1), \n                p=0.5),\n    albumentations.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2()\n    \n])","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:52:56.452646Z","iopub.execute_input":"2024-02-10T16:52:56.453644Z","iopub.status.idle":"2024-02-10T16:52:56.462700Z","shell.execute_reply.started":"2024-02-10T16:52:56.453587Z","shell.execute_reply":"2024-02-10T16:52:56.461602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class TestDataset(Dataset):\n    \n    def __init__(self, image_ids, labels, transform=None):\n        \n        self.transform = transform\n        self.image_ids = image_ids\n        self.labels = labels\n        \n    def __len__(self):\n        return len(self.image_ids)\n    \n    def __getitem__(self, index):\n        \n        img = Image.open(self.image_ids[index])\n        img = np.array(img)\n        label = torch.tensor(self.labels[index], dtype=torch.long)\n        \n        if self.transform:\n            return self.transform(image=img)['image'], label \n        else:\n            return img, label ","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:53:01.443410Z","iopub.execute_input":"2024-02-10T16:53:01.443801Z","iopub.status.idle":"2024-02-10T16:53:01.451145Z","shell.execute_reply.started":"2024-02-10T16:53:01.443769Z","shell.execute_reply":"2024-02-10T16:53:01.450012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_dataset = TestDataset(image_ids=df_test.path, labels=df_test.label, transform=test_transforms)\ntta_dataset = TestDataset(image_ids=df_test.path, labels=df_test.label, transform=tta_transforms)\n\ntest_dl = DataLoader(test_dataset, batch_size=1, shuffle=False)\ntta_dl = DataLoader(tta_dataset, batch_size=1, shuffle=False)\n\ntest_set_size = len(test_dataset)\ntta_num = 5","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:53:05.245329Z","iopub.execute_input":"2024-02-10T16:53:05.245945Z","iopub.status.idle":"2024-02-10T16:53:05.252256Z","shell.execute_reply.started":"2024-02-10T16:53:05.245912Z","shell.execute_reply":"2024-02-10T16:53:05.251265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_model_probabilities(model):\n\n    final_probabilities = np.zeros(shape=(test_set_size, num_classes))\n\n    probabilities = np.zeros(shape=(test_set_size, num_classes))\n\n    count = 0\n\n    with torch.no_grad():\n        for image, label in test_dl:\n\n            image = image.to(DEVICE)\n            label = label.to(DEVICE)\n\n            logits = model(image)\n\n            probs = torch.nn.functional.softmax(logits, dim=1).detach().cpu().numpy()\n\n            probabilities[count] = probs            \n\n            count += 1\n\n    final_probabilities += probabilities\n\n\n    for i in range(tta_num):\n\n        probabilities = np.zeros(shape=(test_set_size, num_classes))\n\n        count = 0\n\n        with torch.no_grad():\n            for image, label in tta_dl:\n\n                image = image.to(DEVICE)\n                label = label.to(DEVICE)\n\n                logits = model(image)\n\n                probs = torch.nn.functional.softmax(logits, dim=1).detach().cpu().numpy()\n\n                probabilities[count] = probs            \n\n                count += 1\n\n\n        final_probabilities += probabilities\n\n    final_probabilities /= (tta_num + 1)\n    \n    return final_probabilities\n","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:53:10.163492Z","iopub.execute_input":"2024-02-10T16:53:10.164175Z","iopub.status.idle":"2024-02-10T16:53:10.174347Z","shell.execute_reply.started":"2024-02-10T16:53:10.164140Z","shell.execute_reply":"2024-02-10T16:53:10.173243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ensemble OOF predictions:\n\nNow, each of our $K=5$ models will separately yield $5$ averaged class probabilities per image. These can be stored in array of dimensions (test_set_size, num_classes). However, we also want to ensemble our separately trained models to obtain our final predictions. To do this we simply average the $5$ arrays obtained from each model via get_model_probabilities(model).","metadata":{}},{"cell_type":"code","source":"num_models = len(models)\nnum_classes = 5\n\nensemble_probabilities = np.zeros(shape=(test_set_size, num_classes))\n\nfor model in models:\n    \n    model_probabilities = get_model_probabilities(model)\n    \n    ensemble_probabilities += model_probabilities\n    \nensemble_probabilities /= num_models\n\nensemble_predictions = ensemble_probabilities.argmax(axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:53:52.663929Z","iopub.execute_input":"2024-02-10T16:53:52.664323Z","iopub.status.idle":"2024-02-10T16:53:54.537204Z","shell.execute_reply.started":"2024-02-10T16:53:52.664292Z","shell.execute_reply":"2024-02-10T16:53:54.536407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Prepare final submission for competition\n\nWe now have a list of final predictions from our ensemble of models. Let us replace the \"label\" column of our df_test dataframe with these predictions.\n\n\nFor our final submission to the competition, we should also create a csv file \"submission.csv\" with columns \"image_id\" and \"label\".","metadata":{}},{"cell_type":"code","source":"df_test.label = ensemble_predictions\n\nfinal_test_submission = df_test\nfinal_test_submission['image_id'] = final_test_submission.path.str.split('/').str[-1]\nfinal_test_submission['label'] = ensemble_predictions\nfinal_test_csv = final_test_submission[['image_id', 'label']]","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:54:20.855978Z","iopub.execute_input":"2024-02-10T16:54:20.856332Z","iopub.status.idle":"2024-02-10T16:54:20.868516Z","shell.execute_reply.started":"2024-02-10T16:54:20.856304Z","shell.execute_reply":"2024-02-10T16:54:20.867631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can check our inference code is working for the one visible image in the test directory. Let's now display the df_test dataframe which now contains our predicted labels.  ","metadata":{}},{"cell_type":"code","source":"df_test","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:54:38.796499Z","iopub.execute_input":"2024-02-10T16:54:38.797195Z","iopub.status.idle":"2024-02-10T16:54:38.809322Z","shell.execute_reply.started":"2024-02-10T16:54:38.797165Z","shell.execute_reply":"2024-02-10T16:54:38.808355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_test_csv.to_csv('submission.csv', index=False)\nprint('Submission csv file created!')","metadata":{"execution":{"iopub.status.busy":"2024-02-10T16:54:44.810004Z","iopub.execute_input":"2024-02-10T16:54:44.810640Z","iopub.status.idle":"2024-02-10T16:54:44.818434Z","shell.execute_reply.started":"2024-02-10T16:54:44.810597Z","shell.execute_reply":"2024-02-10T16:54:44.817556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}