{"cells":[{"metadata":{},"cell_type":"markdown","source":"# What is Cassava ? What are the types of disases?"},{"metadata":{},"cell_type":"markdown","source":"As the second-largest provider of carbohydrates in Africa, cassava is a key food security crop grown by smallholder farmers because it can withstand harsh conditions. At least 80% of household farms in Sub-Saharan Africa grow this starchy root, but viral diseases are major sources of poor yields.\n\nExisting methods of disease detection require farmers to solicit the help of government-funded agricultural experts to visually inspect and diagnose the plants. This suffers from being labor-intensive, low-supply and costly. As an added challenge, effective solutions for farmers must perform well under significant constraints, since African farmers may only have access to mobile-quality cameras with low-bandwidth.\n\nSo, in this competition through the training set we have, we will try to classify which disease type cassava caught with the help of image processing techniques and AI.\n"},{"metadata":{},"cell_type":"markdown","source":"<img alt=\"Profit-making idea: Industrialisation of cassava one of Africa's biggest  opportunities\" class=\"n3VNCb\" src=\"https://www.howwemadeitinafrica.com/wp-content/uploads/2020/07/PMI-Philafrica-cassava-1200x630-1.jpg\" data-noaft=\"1\" jsname=\"HiaYvf\" jsaction=\"load:XAeZkd;\" style=\"width: 1024px; height: 2040; margin: 0px;\">"},{"metadata":{},"cell_type":"markdown","source":"# Let's look at the types of diseases:"},{"metadata":{},"cell_type":"markdown","source":"**1 - Cassava Bacterial Blight (CBB)**\n\nXanthomonas axonopodis pv. manihotis is the pathogen that causes bacterial blight of cassava. Originally discovered in Brazil in 1912, the disease has followed cultivation of cassava across the world.[1] Among diseases which afflict cassava worldwide, bacterial blight causes the largest losses in terms of yield.\n\n**Symptoms:**\n\n* Symptoms include leaf spotting, wilting, dying, gum oozing on young shoots, and vascular coloration of mature stems and roots of susceptible varieties."},{"metadata":{},"cell_type":"markdown","source":"**2 - Cassava Brown Streak Disease (CBSD)**\n\nCassava brown streak virus disease (CBSD) is a damaging disease of cassava plants, and is especially troublesome in East Africa. It was first identified in 1936 in Tanzania, and has spread to other coastal areas of East Africa, from Kenya to Mozambique. Recently, it was found that two distinct viruses are responsible for the disease: cassava brown streak virus (CBSV) and Ugandan cassava brown streak virus (UCBSV).\n\n**Symptoms:**\n\n* CBSD is characterized by severe chlorosis and necrosis on infected leaves, giving them a yellowish, mottled appearance.\n* Chlorosis may be associated with the veins, spanning from the mid vein, secondary and tertiary veins, or rather in blotches unconnected to veins.\n* Leaf symptoms vary greatly depending on a variety of factors. \n* The growing conditions (i.e. altitude, rainfall quantity), plant age, and the virus species account for these differences. \n* Brown streaks may appear on the stems of the cassava plant. Also, a dry brown-black necrotic rot of the cassava tuber exists, which may progress from a small lesion to the whole root. \n* Finally, the roots can become constricted due to the tuber rot, stunting growth"},{"metadata":{},"cell_type":"markdown","source":"**3 - Cassava Green Mottle (CGM)**\n\nIt has not been confirmed to be a nepovirus; these are viruses that are transmitted by nematodes - hence the name. Narrow. Only known from Solomon Islands. It was first found on Choiseul in the 1970s; more recently (2010), similar symptoms were seen on Malaita.\n\n**Symptoms:**\n\n* Look for yellow patterns on the leaves, from small dots to irregular patches of yellow and green. \n* Look for leaf margins that are distorted. \n* The plants may be stunted."},{"metadata":{},"cell_type":"markdown","source":"**4 - Cassava Mosaic Disease**\n\nCassava mosaic virus is the common name used to refer to any of eleven different species of plant pathogenic virus in the genus Begomovirus. African cassava mosaic virus (ACMV), East African cassava mosaic virus (EACMV), and South African cassava mosaic virus (SACMV) are distinct species of circular single-stranded DNA viruses which are transmitted by whiteflies and primarily infect cassava plants; these have thus far only been reported from Africa.\n\n**Symptoms:**\n\n* Initially following infection of a cassava geminivirus in cassava, systemic symptoms develop. \n* These symptoms include chlorotic mosaic of the leaves, leaf distortion, and stunted growth. \n* Leaf stalks have a characteristic S-shape.\n* Infection can be overcome by the plant especially when a rapid onset of symptoms occurs. A slow onset of disease development usually correlates with death of the plant.\n* affected by whiteflies\n* affected by environmental factors such as temperature, wind, precipitation and plant density"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"collapsed":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport pathlib\nimport imageio\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Import Necessary Libraries"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt \nimport plotly.express as px\nimport os\nimport cv2\nfrom PIL import Image\nimport keras\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score\nimport tensorflow as tf\nfrom tensorflow.keras import models, layers\nfrom tensorflow.keras.preprocessing import image\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\nfrom tensorflow.keras.applications import EfficientNetB0, Xception\nfrom tensorflow.keras.optimizers import Adam","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"input_dir = \"../input/cassava-leaf-disease-classification\"\n\ntrain_images_path = os.path.join(input_dir,\"train_images\")\ntest_images_path = os.path.join(input_dir,'test_images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/cassava-leaf-disease-classification/train.csv')\ntrain.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Total Number of Images in Training Data : \",train.shape[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"image_list = train['image_id'].to_list()\nlabel_list = train['label'].to_list()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import json\n\nwith open(\"../input/cassava-leaf-disease-classification/label_num_to_disease_map.json\") as f:\n    class_mapping = json.load(f)\n\nclass_mapping2 ={int(k):v for k,v in class_mapping.items()}\n\nclass_mapping2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Distribution of Diseases:"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(8,5))\n\nsns.set_style('whitegrid')\n\nax=sns.countplot(data=train, x='label', palette=\"Pastel1\")\n\n\n#  '0': 'Cassava Bacterial Blight (CBB)\n#  '1': 'Cassava Brown Streak Disease (CBSD)\n#  '2': 'Cassava Green Mottle (CGM)\n#  '3': 'Cassava Mosaic Disease (CMD)\n#  '4': 'Healthy","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train2 = train.copy()\ntrain2.replace({\"label\": class_mapping2}, inplace=True)\n\npie_df = train2['label'].value_counts().reset_index()\npie_df.columns = ['label', 'count']\nfig = px.pie(pie_df, values = 'count', names = 'label', hole=.3, color_discrete_sequence = px.colors.qualitative.Pastel1)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualization"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_samples(class_):\n    \n    print(f'Some Sample Images belonging to Class {class_mapping[f\"{class_}\"]}')\n    \n    sample_images = train[train.label == class_].sample(8)\n    \n    plt.rcParams[\"axes.grid\"] = False\n\n    fig,ax = plt.subplots(nrows=2,ncols=4,figsize=(20,12))\n\n    for e,img in enumerate(sample_images.image_id):\n        image_path = os.path.join(input_dir,f'train_images/{img}')\n        image = cv2.imread(image_path)\n        ax[e//4][e%4].imshow(image)\n    \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(0) #Cassava Bacterial Blight","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(1) #Cassava Brown Streak Disease","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(2) #Cassava Green Mottle","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(3) #Cassava Mosaic Disease","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Converting from BGR to RGB"},{"metadata":{},"cell_type":"markdown","source":"### Cassava Bacterial Blight (CBB) Samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_images = train[train.label == 0].sample(5)\nplt.figure(figsize=(35, 20))\nfor e,img in enumerate(sample_images.image_id):\n    plt.subplot(1, 5, e + 1)\n    img = cv2.imread(os.path.join(input_dir,f'train_images/{img}'))\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Cassava Brown Streak Disease (CBSD) Samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_images = train[train.label == 1].sample(5)\nplt.figure(figsize=(35, 20))\nfor e,img in enumerate(sample_images.image_id):\n    plt.subplot(1, 5, e + 1)\n    img = cv2.imread(os.path.join(input_dir,f'train_images/{img}'))\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Cassava Green Mottle (CGM) Samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_images = train[train.label == 2].sample(5)\nplt.figure(figsize=(35, 20))\nfor e,img in enumerate(sample_images.image_id):\n    plt.subplot(1, 5, e + 1)\n    img = cv2.imread(os.path.join(input_dir,f'train_images/{img}'))\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Cassava Mosaic Disease (CMD) Samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_images = train[train.label == 3].sample(5)\nplt.figure(figsize=(35, 20))\nfor e,img in enumerate(sample_images.image_id):\n    plt.subplot(1, 5, e + 1)\n    img = cv2.imread(os.path.join(input_dir,f'train_images/{img}'))\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Healty Samples "},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_images = train[train.label == 4].sample(5)\nplt.figure(figsize=(35, 20))\nfor e,img in enumerate(sample_images.image_id):\n    plt.subplot(1, 5, e + 1)\n    img = cv2.imread(os.path.join(input_dir,f'train_images/{img}'))\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model Implemetation and Augmentation"},{"metadata":{"trusted":true},"cell_type":"code","source":"BATCH_SIZE =8 #Mini-Batch Gradient Descent\nSTEPS_PER_EPOCH = len(train)*0.8 / BATCH_SIZE\nVALIDATION_STEPS = len(train)*0.2 / BATCH_SIZE\nEPOCHS = 20\nTARGET_SIZE = 350","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.label = train.label.astype('str')\n\ntrain_datagen = ImageDataGenerator(validation_split = 0.2,\n                                     rotation_range = 45,\n                                     zoom_range = 0.3,\n                                     horizontal_flip = True,\n                                     vertical_flip = True,\n                                     fill_mode = 'nearest',\n                                     shear_range = 0.1,\n                                     height_shift_range = 0.1,\n                                     width_shift_range = 0.1,\n                                     featurewise_center = True,\n                                     featurewise_std_normalization = True)\n\ntrain_generator = train_datagen.flow_from_dataframe(train,\n                         directory = os.path.join('../input/cassava-leaf-disease-classification/train_images'),\n                         subset = \"training\",\n                         x_col = \"image_id\",\n                         y_col = \"label\",\n                         target_size = (TARGET_SIZE, TARGET_SIZE),\n                         batch_size = BATCH_SIZE,\n                         class_mode = \"sparse\",\n                         shuffle= True)\n\n\nvalidation_datagen = ImageDataGenerator(validation_split = 0.2)\n\nvalidation_generator = validation_datagen.flow_from_dataframe(train,\n                         directory = os.path.join('../input/cassava-leaf-disease-classification/train_images'),\n                         subset = \"validation\",\n                         x_col = \"image_id\",\n                         y_col = \"label\",\n                         target_size = (TARGET_SIZE, TARGET_SIZE),\n                         batch_size = BATCH_SIZE,\n                         class_mode = \"sparse\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_path = os.path.join('../input/cassava-leaf-disease-classification/train_images/1003442061.jpg')\nimg = image.load_img(img_path, target_size = (TARGET_SIZE, TARGET_SIZE))\nimg_tensor = image.img_to_array(img)\nimg_tensor = np.expand_dims(img_tensor, axis = 0)\nimg_tensor /= 255.\n\nplt.imshow(img_tensor[0])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"generator = train_datagen.flow_from_dataframe(train.iloc[17:18],\n                         directory = os.path.join('../input/cassava-leaf-disease-classification/train_images'),\n                         x_col = \"image_id\",\n                         y_col = \"label\",\n                         target_size = (TARGET_SIZE, TARGET_SIZE),\n                         batch_size = BATCH_SIZE,\n                         class_mode = \"sparse\")\n\naug_images = [generator[0][0][0]/255 for i in range(10)]\nfig, axes = plt.subplots(2, 5, figsize = (20, 10))\naxes = axes.flatten()\nfor img, ax in zip(aug_images, axes):\n    ax.imshow(img)\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_model():\n    conv_base = Xception(include_top=False, input_tensor=None,\n    pooling=None, input_shape=(TARGET_SIZE, TARGET_SIZE, 3), classifier_activation='softmax')\n                               \n    model = conv_base.output\n    model = layers.GlobalAveragePooling2D()(model)\n    model = layers.Dense(5, activation = \"softmax\")(model)\n    model = models.Model(conv_base.input, model)\n\n    model.compile(optimizer = Adam(lr = 0.001),\n                  loss = \"sparse_categorical_crossentropy\",\n                  metrics = [\"acc\"])\n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = create_model()\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# model_save = ModelCheckpoint('./Xception_best_weights2.h5', \n#                              save_best_only = True, \n#                              save_weights_only = True,\n#                              monitor = 'val_loss', \n#                              mode = 'min', verbose = 1)\n# early_stop = EarlyStopping(monitor = 'val_loss', min_delta = 0.001, \n#                            patience = 5, mode = 'min', verbose = 1,\n#                            restore_best_weights = True)\n# reduce_lr = ReduceLROnPlateau(monitor = 'val_loss', factor = 0.3, \n#                               patience = 2, min_delta = 0.001, \n#                               mode = 'min', verbose = 1) #reduced learning rate\n\n\n# history = model.fit(\n#     train_generator,\n#     steps_per_epoch = STEPS_PER_EPOCH,\n#     epochs = EPOCHS,\n#     validation_data = validation_generator,\n#     validation_steps = VALIDATION_STEPS,\n#     callbacks = [model_save, early_stop, reduce_lr])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# model.save('./Xception_best_weights.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.models.load_model('../input/xception-best-weights/Xception_best_weights.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_file = pd.read_csv(os.path.join('../input/cassava-leaf-disease-classification/sample_submission.csv'))\nsubmission_file","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds = []\n\nfor image_id in submission_file.image_id:\n    image = Image.open(os.path.join(f'../input/cassava-leaf-disease-classification/test_images/{image_id}'))\n    image = image.resize((TARGET_SIZE, TARGET_SIZE))\n    image = np.expand_dims(image, axis = 0)\n    preds.append(np.argmax(model.predict(image)))\n\nsubmission_file['label'] = preds\nsubmission_file","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_file.to_csv('submission.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### PS: \n\nWhile creating this notebook, I was inspired by the notebook of a Kaggle member who name is Maksym Shkliarevskyi. This was my first attempt at computer vision, so his work was a good resource for me. \n\nThank you to him.\n\n\nResouce: https://www.kaggle.com/maksymshkliarevskyi/cassava-leaf-disease-best-keras-cnn\n\nMy Base Model: https://www.kaggle.com/eceifter/xception-cassava-leaf-disease-classification?scriptVersionId=48693427"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}