{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Cassava Leaf Disease Classification and Visualization\n\nCassava is the third largest source of carbohydrates for human food in the world but is vulnerable to virus diseases, which threaten to destabilize food security in sub-Saharan Africa. To prevent this crisis, we need to detect the disease on a cassava plant. Classifying the type of the disease helps to deliver the right pesticide to the affected plant. Let's explore each disease in the dataset.\n\n* [Introduction](#section-one)\n    - [Cassava Bacterial Blight (CBB)](#subsection-one)\n    - [Cassava Brown Streak Disease (CBSD)](#subsection-two)\n    - [Cassava Green Mottle (CGM)](#subsection-three)\n    - [Cassava Mosaic Disease (CMD)](#subsection-four)\n* [Transfer learning modelling](#section-two)\n    - [Loading the model](#subsection-oneintwo)\n    - [Training and Validation](#subsection-twointwo)\n* [Testing and Salience mapping visualization](#section-three)\n    - [Saliency map and Guided Backpropagation](#gbp)\n    - [Cassava Bacterial Blight (CBB)](#subsection-oneinthree)\n    - [Cassava Brown Streak Disease (CBSD)](#subsection-twointhree)\n    - [Cassava Green Mottle (CGM)](#subsection-threeinthree)\n    - [Cassava Mosaic Disease (CMD)](#subsection-fourinthree) \n    - [Cassava healthy](#subsection-fiveinthree) \n* [Conclusion](#section-four)\n\n"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-one\"></a>\n# Introduction\n\nGeneral analysis on the data labels and it's counts\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import time\nfrom PIL import Image\nimport os\nimport cv2\nimport numpy as np\nimport torch\nimport torch.optim as optim\nfrom torch import nn\nfrom torch.autograd import Variable\nfrom collections import OrderedDict\nimport torchvision\nimport torchvision.models as models\nimport torch.utils.model_zoo as model_zoo\nimport torchvision.transforms as transforms\nfrom torchvision import datasets\n\nfrom torch.utils.data import Dataset,DataLoader\nfrom itertools import accumulate\nfrom functools import reduce\n\n\nfrom glob import glob\nfrom sklearn.model_selection import GroupKFold, StratifiedKFold\nimport cv2\nfrom skimage import io\nimport torch\nfrom torch import nn\nimport os\nfrom datetime import datetime\nimport time\nimport random\nimport cv2\nimport torchvision\nfrom torchvision import transforms\nimport pandas as pd\nimport numpy as np\nfrom tqdm import tqdm\n\nimport matplotlib.pyplot as plt\nfrom torch.utils.data import Dataset,DataLoader\nfrom torch.utils.data.sampler import SequentialSampler, RandomSampler\nfrom torch.cuda.amp import autocast, GradScaler\nfrom torch.nn.modules.loss import _WeightedLoss\nimport torch.nn.functional as F\n\n# import timm\n\nimport sklearn\nimport warnings\nimport joblib\nfrom sklearn.metrics import roc_auc_score, log_loss\nfrom sklearn import metrics\nimport warnings\nimport cv2\nimport json","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"train_data = pd.read_csv('../input/cassava-leaf-disease-classification/train.csv')\nprint('Total nuber of images in the dataset', len(train_data))\ntrain_data.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"with open('../input/cassava-leaf-disease-classification/label_num_to_disease_map.json') as f:\n    labels = json.load(f)\nlabels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = train_data['label'].astype(str).map(labels)\np = plt.hist(data)\nplt.xticks(rotation='vertical')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Obviously, the data is imbalanced and we have 5 classes including healthy class. Lets see the science behind each class to understand the data "},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-one\"></a>\n## Cassava Bacterial Blight (CBB)\n\nAt first, angular, water-soaked spots occur on the leaves which are restricted by the veins; the spots are more clearly seen on the lower leaf surface. The spots expand rapidly, join together, especially along the margins of the leaves, and turn brown with yellow borders. The green part of the stem will also be affected. The tips of the stems blacken resulting in a \"candle\" appearance, and new shoots and leaves develop. Roots are rarely affected, although rots around dead vascular tissue occasionally occur on susceptible varieties.\n\n### Detection & Inspection\nLook to see if leaves are drying and dying early. Look for angular spots on the leaves, and cut out small pieces of the leaf from the edge of the spots and place them in a drop of water. Look for bacterial streaming - the streaming appears as white streaks in the water. Look for dark brown to black streaks on the green part of the stem, and for the presence of sticky liquid. Look for browning in the vascular tissues, i.e., the water conducting tubes, after peeling the bark and splitting the stem.\n\n### Possible types of images for this disease\n* Plant leaves\n* Plant stems\n\nLet's see few samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"class_0 = train_data[train_data['label']==0]\nsample = class_0.sample(4)\ncnt=1\nplt.figure(figsize=(50,50))\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    img=Image.open(image_path)\n    plt.subplot(1,5,cnt)\n    plt.imshow(img)\n    plt.axis('off')\n    plt.title(sample.loc[r,'image_id']+' (CBB)',fontsize=25)\n    cnt=cnt+1\n    \n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-two\"></a>\n## Cassava Brown Streak Disease (CBSD)\n\nWhen infected, cassava leaves show a mottled yellowing pattern typically beginning from the secondary veins and progressing to tertiary veins as the infection gets more severe. This yellowish chlorosis spreads along the veins until severely infected leaves are mostly yellow.Tolerant varieties and plants at a young age may be infected but asymptomatic. It may also cause brown streaks on stems of infected plants and brown necrotic rotting in tuberous roots which may render them inedible.\n\n### Detection & Inspection\n\nLook for yellow infection on leaves. As the name suggests, presence of brown streaks on the stems can be found. Also look for root infections where evident rotten brown appearance. \n\n### Possible types of images for this disease\n* Plant leaves\n* Plant stems\n* Plant roots\n\nLet's see few samples\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# class_0 = train_data[train_data['label']==1]\nsample = ['5912799.jpg','9224019.jpg','84787134.jpg','199112616.jpg']\ncnt=1\nplt.figure(figsize=(50,50))\nfor im in sample:\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+im\n    img=Image.open(image_path)\n    plt.subplot(1,5,cnt)\n    plt.imshow(img)\n    plt.axis('off')\n    plt.title(im+' (CBSD)',fontsize=25)\n    cnt=cnt+1\n    \n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-three\"></a>\n## Cassava Green Mottle (CGM)\n\nYoung leaves are puckered with faint to distinct yellow spots, green patterns, and twisted margins. Usually, the shoots recover from symptoms and appear healthy. Occasionally, plants become severely stunted, edible roots are absent. \n\n### Detection & Inspection\nLook for yellow patterns on the leaves, from small dots to irregular patches of yellow and green. Look for leaf margins that are distorted. The plants may be stunted (dwarf).\n\n### Possible types of images for this disease\n* Plant leaves\n* Whole plant (to show the dwarfness)"},{"metadata":{"trusted":true},"cell_type":"code","source":"class_0 = train_data[train_data['label']==2]\nsample = class_0.sample(4)\ncnt=1\nplt.figure(figsize=(50,50))\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    img=Image.open(image_path)\n    plt.subplot(1,5,cnt)\n    plt.imshow(img)\n    plt.axis('off')\n    plt.title(sample.loc[r,'image_id']+' (CGM)',fontsize=25)\n    cnt=cnt+1\n    \n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-four\"></a>\n## Cassava Mosaic Disease (CMD)\n\nNewly-infected plants begin to express symptoms from the top, while plants infected through the planted cutting often show symptoms in all leaves. Symptoms of CMD are a typical mosaic in which there is a mix of yellow/pale green chlorotic patches and green areas. Unlike CBSD, leaves are usually distorted in shape, and where symptoms are severe the size of leaves is greatly reduced and the plant is stunted.\n\n### Detection & Inspection\n\nLook for yellow mosaic pattern on the leaves and distorted size and shape of the leaves. The plants may be stunted (dwarf).\n\n### Possible types of images for this disease\n* Plant leaves\n* Whole plant (to show the dwarfness)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"class_0 = train_data[train_data['label']==3]\nsample = class_0.sample(4)\ncnt=1\nplt.figure(figsize=(50,50))\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    img=Image.open(image_path)\n    plt.subplot(1,5,cnt)\n    plt.imshow(img)\n    plt.axis('off')\n    plt.title(sample.loc[r,'image_id']+' (CMD)',fontsize=25)\n    cnt=cnt+1\n    \n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-two\"></a>\n# Transfer learning \n\nTransfer learning always helps as a pre-trained model had already learnt the high level features. Selecting the VGG-19 model for initial modelling and visualization\n"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-oneintwo\"></a>\n## Loading the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"model_urls = {\n    'vgg19': 'https://download.pytorch.org/models/vgg19-dcbb9e9d.pth',    \n}\n\nmodel_names = model_urls.keys()\n\ninput_sizes = {\n    'vgg' : (224,224)\n}\n\nbatch_size = 20\nuse_gpu = torch.cuda.is_available()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Sanity check that param names overlap\n#Note that params are not necessarily in the same order for every pretrained model\ndef diff_states(dict_canonical, dict_subset):\n    names1, names2 = (list(dict_canonical.keys()), list(dict_subset.keys()))\n    not_in_1 = [n for n in names1 if n not in names2]\n    not_in_2 = [n for n in names2 if n not in names1]\n    assert len(not_in_1) == 0\n    assert len(not_in_2) == 0\n\n    for name, v1 in dict_canonical.items():\n        v2 = dict_subset[name]\n        assert hasattr(v2, 'size')\n        if v1.size() != v2.size():\n            yield (name, v1)          ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#loading and finding the different states in the model and merging them\ndef load_defined_model(name, num_classes):\n    \n    model = models.__dict__[name](num_classes=num_classes)\n    \n    pretrained_state = model_zoo.load_url(model_urls[name])\n\n    #Diff\n    diff = [s for s in diff_states(model.state_dict(), pretrained_state)]\n    print(\"Replacing the following state from initialized\", name, \":\", \\\n          [d[0] for d in diff])\n    \n    for name, value in diff:\n        pretrained_state[name] = value\n    \n    assert len([s for s in diff_states(model.state_dict(), pretrained_state)]) == 0\n    \n    #Merge\n    model.load_state_dict(pretrained_state)\n    return model, diff","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def prepare_dataloader(df, trn_idx, val_idx, data_root='../input/cassava-leaf-disease-classification/train_images/'):\n    \n    from catalyst.data.sampler import BalanceClassSampler\n    \n    train_ = df.loc[trn_idx,:].reset_index(drop=True)\n    valid_ = df.loc[val_idx,:].reset_index(drop=True)\n        \n    train_ds = CassavaDataset(train_, data_root, transforms=get_train_transforms(), output_label=True, one_hot_label=False, do_fmix=False, do_cutmix=False)\n    valid_ds = CassavaDataset(valid_, data_root, transforms=get_valid_transforms(), output_label=True)\n    \n    train_loader = torch.utils.data.DataLoader(\n        train_ds,\n        batch_size=20,\n        pin_memory=False,\n        drop_last=False,\n        shuffle=True\n        #sampler=BalanceClassSampler(labels=train_['label'].values, mode=\"downsampling\")\n    )\n    val_loader = torch.utils.data.DataLoader(\n        valid_ds, \n        batch_size=20,\n        shuffle=False,\n        pin_memory=False,\n    )\n    return train_loader, val_loader","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def filtered_params(net, param_list=None):\n    def in_param_list(s):\n        for p in param_list:\n            if s.endswith(p):\n                return True\n        return False    \n    #Caution: DataParallel prefixes '.module' to every parameter name\n    params = net.named_parameters() if param_list is None \\\n    else (p for p in net.named_parameters() if in_param_list(p[0]))\n    return params","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndef rand_bbox(size, lam):\n    W = size[0]\n    H = size[1]\n    cut_rat = np.sqrt(1. - lam)\n    cut_w = np.int(W * cut_rat)\n    cut_h = np.int(H * cut_rat)\n\n    # uniform\n    cx = np.random.randint(W)\n    cy = np.random.randint(H)\n\n    bbx1 = np.clip(cx - cut_w // 2, 0, W)\n    bby1 = np.clip(cy - cut_h // 2, 0, H)\n    bbx2 = np.clip(cx + cut_w // 2, 0, W)\n    bby2 = np.clip(cy + cut_h // 2, 0, H)\n    return bbx1, bby1, bbx2, bby2\n\n\nclass CassavaDataset(Dataset):\n    def __init__(self, df, data_root, \n                 transforms=None, \n                 output_label=True, \n                 one_hot_label=False,\n                 do_fmix=False, \n                 fmix_params={\n                     'alpha': 1., \n                     'decay_power': 3., \n                     'shape': (224,224),\n                     'max_soft': True, \n                     'reformulate': False\n                 },\n                 do_cutmix=False,\n                 cutmix_params={\n                     'alpha': 1,\n                 }\n                ):\n        \n        super().__init__()\n        self.df = df.reset_index(drop=True).copy()\n        self.transforms = transforms\n        self.data_root = data_root\n        self.do_fmix = do_fmix\n        self.fmix_params = fmix_params\n        self.do_cutmix = do_cutmix\n        self.cutmix_params = cutmix_params\n        \n        self.output_label = output_label\n        self.one_hot_label = one_hot_label\n        \n        if output_label == True:\n            self.labels = self.df['label'].values\n            #print(self.labels)\n            \n            if one_hot_label is True:\n                self.labels = np.eye(self.df['label'].max()+1)[self.labels]\n                #print(self.labels)\n            \n    def __len__(self):\n        return self.df.shape[0]\n    \n    def __getitem__(self, index: int):\n        \n        # get labels\n        if self.output_label:\n            target = self.labels[index]\n          \n        img  = get_img(\"{}/{}\".format(self.data_root, self.df.loc[index]['image_id']))\n\n        if self.transforms:\n            img = self.transforms(image=img)['image']\n        \n        if self.do_fmix and np.random.uniform(0., 1., size=1)[0] > 0.5:\n            with torch.no_grad():\n                #lam, mask = sample_mask(**self.fmix_params)\n                \n                lam = np.clip(np.random.beta(self.fmix_params['alpha'], self.fmix_params['alpha']),0.6,0.7)\n                \n                # Make mask, get mean / std\n                mask = make_low_freq_image(self.fmix_params['decay_power'], self.fmix_params['shape'])\n                mask = binarise_mask(mask, lam, self.fmix_params['shape'], self.fmix_params['max_soft'])\n    \n                fmix_ix = np.random.choice(self.df.index, size=1)[0]\n                fmix_img  = get_img(\"{}/{}\".format(self.data_root, self.df.iloc[fmix_ix]['image_id']))\n\n                if self.transforms:\n                    fmix_img = self.transforms(image=fmix_img)['image']\n\n                mask_torch = torch.from_numpy(mask)\n                \n                # mix image\n                img = mask_torch*img+(1.-mask_torch)*fmix_img\n\n                #print(mask.shape)\n\n                #assert self.output_label==True and self.one_hot_label==True\n\n                # mix target\n                rate = mask.sum()/224/224\n                target = rate*target + (1.-rate)*self.labels[fmix_ix]\n                #print(target, mask, img)\n                #assert False\n        \n        if self.do_cutmix and np.random.uniform(0., 1., size=1)[0] > 0.5:\n            #print(img.sum(), img.shape)\n            with torch.no_grad():\n                cmix_ix = np.random.choice(self.df.index, size=1)[0]\n                cmix_img  = get_img(\"{}/{}\".format(self.data_root, self.df.iloc[cmix_ix]['image_id']))\n                if self.transforms:\n                    cmix_img = self.transforms(image=cmix_img)['image']\n                    \n                lam = np.clip(np.random.beta(self.cutmix_params['alpha'], self.cutmix_params['alpha']),0.3,0.4)\n                bbx1, bby1, bbx2, bby2 = rand_bbox((224, 224), lam)\n\n                img[:, bbx1:bbx2, bby1:bby2] = cmix_img[:, bbx1:bbx2, bby1:bby2]\n\n                rate = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (224* 224))\n                target = rate*target + (1.-rate)*self.labels[cmix_ix]\n                \n            #print('-', img.sum())\n            #print(target)\n            #assert False\n                            \n        # do label smoothing\n        #print(type(img), type(target))\n        if self.output_label == True:\n            return img, target\n        else:\n            return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-twointwo\"></a>\n## Training and Validation"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Data augmentation\nfrom albumentations.augmentations.transforms import CLAHE\nfrom albumentations import (\n    HorizontalFlip, VerticalFlip, IAAPerspective, ShiftScaleRotate, CLAHE, RandomRotate90,\n    Transpose, ShiftScaleRotate, Blur, OpticalDistortion, GridDistortion, HueSaturationValue,\n    IAAAdditiveGaussianNoise, GaussNoise, MotionBlur, MedianBlur, IAAPiecewiseAffine, RandomResizedCrop,\n    IAASharpen, IAAEmboss, RandomBrightnessContrast, Flip, OneOf, Compose, Normalize, Cutout, CoarseDropout, ShiftScaleRotate, CenterCrop, Resize\n)\n\nfrom albumentations.pytorch import ToTensorV2\n\ndef get_train_transforms():\n    return Compose([\n            RandomResizedCrop(224, 224),\n            Transpose(p=0.5),\n            HorizontalFlip(p=0.5),\n            VerticalFlip(p=0.5),\n            ShiftScaleRotate(p=0.5),\n            HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n            RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n            CLAHE(clip_limit=4.0, tile_grid_size=(8, 8), always_apply=True, p=1.0),\n            Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n            CoarseDropout(p=0.5),\n            Cutout(p=0.5),\n            ToTensorV2(p=1.0),\n        ], p=1.)\n  \n        \ndef get_valid_transforms():\n    return Compose([\n            CenterCrop(224, 224, p=1.),\n            Resize(224,224),\n            CLAHE(clip_limit=4.0, tile_grid_size=(8, 8), always_apply=True, p=1.0),\n            Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n            ToTensorV2(p=1.0),\n        ], p=1.)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_one_epoch(epoch, model, loss_fn, optimizer, train_loader, device, scheduler=None, schd_batch_update=False):\n    model.train()\n\n    t = time.time()\n    running_loss = None\n\n    pbar = tqdm(enumerate(train_loader), total=len(train_loader))\n    for step, (imgs, image_labels) in pbar:\n        imgs = imgs.to(device).float()\n        image_labels = image_labels.to(device).long()\n\n        #print(image_labels.shape, exam_label.shape)\n        \n        image_preds = model(imgs)   #output = model(input)\n        #print(image_preds.shape, exam_pred.shape)\n\n        loss = loss_fn(image_preds, image_labels)\n            \n        loss.backward()\n\n        if running_loss is None:\n            running_loss = loss.item()\n        else:\n            running_loss = running_loss * .99 + loss.item() * .01\n\n        if ((step + 1) %  2 == 0) or ((step + 1) == len(train_loader)):\n                # may unscale_ here if desired (e.g., to allow clipping unscaled gradients)\n\n            optimizer.step()\n            optimizer.zero_grad() \n                \n            if scheduler is not None and schd_batch_update:\n                scheduler.step()\n\n        if ((step + 1) % 1 == 0) or ((step + 1) == len(train_loader)):\n            description = f'epoch {epoch} loss: {running_loss:.4f}'\n                \n            pbar.set_description(description)\n                \n    if scheduler is not None and not schd_batch_update:\n        scheduler.step()\n        \ndef valid_one_epoch(epoch, model, loss_fn, val_loader, device, scheduler=None, schd_loss_update=False):\n    model.eval()\n\n    t = time.time()\n    loss_sum = 0\n    sample_num = 0\n    image_preds_all = []\n    image_targets_all = []\n    \n    pbar = tqdm(enumerate(val_loader), total=len(val_loader))\n    for step, (imgs, image_labels) in pbar:\n        imgs = imgs.to(device).float()\n        image_labels = image_labels.to(device).long()\n        \n        image_preds = model(imgs)   #output = model(input)\n        #print(image_preds.shape, exam_pred.shape)\n        image_preds_all += [torch.argmax(image_preds, 1).detach().cpu().numpy()]\n        image_targets_all += [image_labels.detach().cpu().numpy()]\n        \n        loss = loss_fn(image_preds, image_labels)\n        \n        loss_sum += loss.item()*image_labels.shape[0]\n        sample_num += image_labels.shape[0]  \n\n        if ((step + 1) % 1 == 0) or ((step + 1) == len(val_loader)):\n            description = f'epoch {epoch} loss: {loss_sum/sample_num:.4f}'\n            pbar.set_description(description)\n    \n    image_preds_all = np.concatenate(image_preds_all)\n    image_targets_all = np.concatenate(image_targets_all)\n    print('validation multi-class accuracy = {:.4f}'.format((image_preds_all==image_targets_all).mean()))\n    \n    if scheduler is not None:\n        if schd_loss_update:\n            scheduler.step(loss_sum/sample_num)\n        else:\n            scheduler.step()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_img(path):\n    im_bgr = cv2.imread(path)\n    im_rgb = im_bgr[:, :, ::-1]\n    #print(im_rgb)\n    return im_rgb","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"seed_everything(719)\n    \nfolds = StratifiedKFold(n_splits=3, shuffle=True, random_state=719).split(np.arange(train_data.shape[0]), train_data.label.values)\n\nfor fold, (trn_idx, val_idx) in enumerate(folds):\n# we'll train fold 0 first\n    if fold > 0:\n        break \n\n    print('Training with {} started'.format(fold))\n\n    print(len(trn_idx), len(val_idx))\n    train_loader, val_loader = prepare_dataloader(train_data, trn_idx, val_idx, data_root='../input/cassava-leaf-disease-classification/train_images/')\n\n    device = torch.device('cuda:0')\n\n    model, diff = load_defined_model('vgg19', 5)\n    model =model.cuda()\n    optimizer = torch.optim.Adam(model.parameters(), lr=1e-4, weight_decay=1e-6)\n    scheduler = torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=1, eta_min=1e-6, last_epoch=-1)\n                                            \n\n    loss_tr = nn.CrossEntropyLoss().to('cuda:0') \n    loss_fn = nn.CrossEntropyLoss().to('cuda:0')\n\n    for epoch in range(1):\n        train_one_epoch(epoch, model, loss_tr, optimizer, train_loader, device, scheduler=scheduler, schd_batch_update=False)\n\n        with torch.no_grad():\n            valid_one_epoch(epoch, model, loss_fn, val_loader, device, scheduler=None, schd_loss_update=False)\n\n        torch.save(model.state_dict(),'vgg19_epoch_20')\n\n    #torch.save(model.cnn_model.state_dict(),'{}/cnn_model_fold_{}_{}'.format(CFG['model_path'], fold, CFG['tag']))\n    del model, optimizer, train_loader, val_loader, scheduler\n    torch.cuda.empty_cache()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-three\"></a>\n# Testing and Salience mapping visualization\n\nLoading the model and visualizing the model performance based on the guided back propagation"},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_defined_model(path, num_classes,name):\n    model = models.__dict__[name](num_classes=num_classes)\n    pretrained_state = torch.load(path)\n    new_pretrained_state= OrderedDict()\n   \n    for k, v in pretrained_state.items():\n        layer_name = k.replace(\"module.\", \"\")\n        new_pretrained_state[layer_name] = v\n        \n    #Diff\n    diff = [s for s in diff_states(model.state_dict(), new_pretrained_state)]\n    if(len(diff)!=0):\n        print(\"Mismatch in these layers :\", name, \":\", [d[0] for d in diff])\n   \n    assert len(diff) == 0\n    \n    #Merge\n    model.load_state_dict(new_pretrained_state)\n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# #Load the model\nmodel_path='../input/pretrained-model-plant-disease/vgg19_epoch_20'\nmodel= load_defined_model(model_path,5,'vgg19')\nuse_gpu = torch.cuda.is_available()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"normalize = transforms.Normalize(\n   mean=[0.485, 0.456, 0.406],\n   std=[0.229, 0.224, 0.225]\n)\npreprocess = transforms.Compose([\n   transforms.Scale(256),\n  #  transforms.CenterCrop(224),\n   transforms.ToTensor(),\n   normalize\n])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import sys\nsys.path.append('../input/dlplantdiseasevis/visualization/')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from torchvis import util\nvis_param_dict, reset_state, remove_handles = util.augment_module(model)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"gbp\"></a>\n## Saliency map and Guided Backpropagation\nThe guided backpropagation method adds an additional rule during the backward pass. This rule is applied during the backpropagation through the nonlinear function called rectified linear (ReLU). In contrast with the standard backpropagation, only positive gradients are backward through ReLU. This rule prevents the backward flow of negative gradients on ReLU from the higher layer in the CNN architecture. This stops the gradients originated from the neurons that decrease the activation of the class node f (x)y and keeps the gradients from neurons that increase the activation of class node. Interestingly, unlike the standard backpropagation, this method produces more precise visualisations which help the user in detection of infected regions.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"def Saliency_map(image,model,preprocess,use_gpu=False,method=util.GradType.GUIDED):\n    vis_param_dict['method'] = method\n    img_tensor = preprocess(image)\n    img_tensor.unsqueeze_(0)\n    if use_gpu:\n        img_tensor=img_tensor.cuda()\n    input = Variable(img_tensor,requires_grad=True)\n    \n    if  input.grad is not None:\n        input.grad.data.zero_()\n    \n    model.zero_grad()\n    output = model(input)\n    ground_truth = np.argmax(output.detach().cpu().numpy())\n    ind=torch.LongTensor(1)\n    if(isinstance(ground_truth,np.int64)):\n        ground_truth=np.asscalar(ground_truth)\n    ind[0]=ground_truth\n    ind=Variable(ind)\n    energy=output[0,ground_truth]\n    energy.backward() \n    grad=input.grad\n    if use_gpu:\n        return np.abs(grad.data.cpu().numpy()[0]).max(axis=0), ground_truth\n    return np.abs(grad.data.numpy()[0]).max(axis=0), ground_truth\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"use_gpu = torch.cuda.is_available()\n\nif use_gpu:\n    print(\"Transfering models to GPU(s)\")\n    model= torch.nn.DataParallel(model).cuda()\n\nmodel.eval()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-oneinthree\"></a>\n## Cassava Bacterial Blight (CBB) visualizations"},{"metadata":{},"cell_type":"markdown","source":"In the corresponding guided backpropagated energy of each image, you can find the intensity mapping correlating with the actual detection and inspection of each diseases. Let's now visualize the images we saw initially while exploring the diseases using the salience mapping technique."},{"metadata":{"trusted":true},"cell_type":"code","source":"\nclass_0 = train_data[train_data['label']==0]\nsample = class_0.sample(5)\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    gt = sample.loc[r,'label']\n    img=Image.open(image_path)\n    method=util.GradType.GUIDED\n    map1, output1=Saliency_map(img,model,preprocess,use_gpu,method)\n    plt.figure(figsize=(15,15))\n    plt.subplot(1,2,1)\n    plt.subplots_adjust(top=0.85)\n    plt.imshow(img)\n    plt.axis('off') \n    plt.title('GT: '+labels[str(gt)]+'  Pred:'+labels[str(output1)])\n    plt.subplot(1,2,2)\n    plt.imshow(map1,cmap='hot', interpolation='nearest')\n    plt.axis('off')\n    plt.title('Guided BP')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With respect to CBB, the brown angular spot formation on the leaves are evidently mapped in the salience visualization."},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-twointhree\"></a>\n## Cassava Brown Streak Disease (CBSD) visualizations"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample = ['5912799.jpg','9224019.jpg','84787134.jpg','199112616.jpg']\nfor im in sample:\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+im\n    gt = train_data[train_data['image_id']==im]['label'].values[0]\n    img=Image.open(image_path)\n    method=util.GradType.GUIDED\n    map1, output1=Saliency_map(img,model,preprocess,use_gpu,method)\n    plt.figure(figsize=(15,15))\n    plt.subplot(1,2,1)\n    plt.subplots_adjust(top=0.85)\n    plt.imshow(img)\n    plt.axis('off') \n    plt.title('GT: '+labels[str(gt)]+'  Pred:'+labels[str(output1)])\n    plt.subplot(1,2,2)\n    plt.imshow(map1,cmap='hot', interpolation='nearest')\n    plt.axis('off')\n    plt.title('Guided BP')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With respect to CBSD, the brown straks on the stem is visible and the yellow patches on the leaves"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-threeinthree\"></a>\n## Cassava Green Mottle (CGM) visualizations"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nclass_0 = train_data[train_data['label']==2]\nsample = class_0.sample(5)\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    gt = sample.loc[r,'label']\n    img=Image.open(image_path)\n    method=util.GradType.GUIDED\n    map1, output1=Saliency_map(img,model,preprocess,use_gpu,method)\n    plt.figure(figsize=(15,15))\n    plt.subplot(1,2,1)\n    plt.subplots_adjust(top=0.85)\n    plt.imshow(img)\n    plt.axis('off') \n    plt.title('GT: '+labels[str(gt)]+'  Pred:'+labels[str(output1)])\n    plt.subplot(1,2,2)\n    plt.imshow(map1,cmap='hot', interpolation='nearest')\n    plt.axis('off')\n    plt.title('Guided BP')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With respect to CGM, the yellow and green dots are seen with high intensity in the maps. "},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-fourinthree\"></a>\n## Cassava Mossaic Disease (CMD) visualizations"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nclass_0 = train_data[train_data['label']==3]\nsample = class_0.sample(5)\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    gt = sample.loc[r,'label']\n    img=Image.open(image_path)\n    method=util.GradType.GUIDED\n    map1, output1=Saliency_map(img,model,preprocess,use_gpu,method)\n    plt.figure(figsize=(15,15))\n    plt.subplot(1,2,1)\n    plt.subplots_adjust(top=0.85)\n    plt.imshow(img)\n    plt.axis('off') \n    plt.title('GT: '+labels[str(gt)]+'  Pred:'+labels[str(output1)])\n    plt.subplot(1,2,2)\n    plt.imshow(map1,cmap='hot', interpolation='nearest')\n    plt.axis('off')\n    plt.title('Guided BP')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With respect to CMD, the mosaic pattern/yellow patches are mapped with high intensities."},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-fiveinthree\"></a>\n## Cassava Healthy plant visualizations"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nclass_0 = train_data[train_data['label']==4]\nsample = class_0.sample(5)\nfor r,c in sample.iterrows():\n    image_path ='../input/cassava-leaf-disease-classification/train_images/'+sample.loc[r,'image_id']\n    gt = sample.loc[r,'label']\n    img=Image.open(image_path)\n    method=util.GradType.GUIDED\n    map1, output1=Saliency_map(img,model,preprocess,use_gpu,method)\n    plt.figure(figsize=(15,15))\n    plt.subplot(1,2,1)\n    plt.subplots_adjust(top=0.85)\n    plt.imshow(img)\n    plt.axis('off') \n    plt.title('GT: '+labels[str(gt)]+'  Pred:'+labels[str(output1)])\n    plt.subplot(1,2,2)\n    plt.imshow(map1,cmap='hot', interpolation='nearest')\n    plt.axis('off')\n    plt.title('Guided BP')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"What is an healthy cassava plant, if you could extract the veins and leaf margins from the images evidently, it is healthy."},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-four\"></a>\n# Conclusion\n\n\n1. We understood the symptoms of each diseases with respect to the labels in the dataset\n2. Through Transfer learning, and training we dealth with developing a deep learning model for Cassava Plant Disease Classification\n3. We also visualized the model performance and checked whether it matches the data analysis.\n4. While visualizing you might have come across the label noise (In healthy, you can see the CBB and CMD diseases plants)\n5. Dealing with the label noise is our next step."},{"metadata":{},"cell_type":"markdown","source":"#### Please follow for the next version- for solving the limitations we see in this version.\n\n#### Please do not forget to Upvote! Comments are welcomed. Thank you!!!"},{"metadata":{},"cell_type":"markdown","source":"-----"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}