{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\nAs part of the Kaggle Competition **[Cassava Leaf Disease Classification competition](https://www.kaggle.com/c/cassava-leaf-disease-classification)** I will try to develop an effective model, because it would have a huge impact on farmers in Africa. The model should be able to classify 4 different diseases based on the pictures of the leaves. The fifth category is intended to classify healthy leaves.   \n\nFarmers may be able to quickly identify diseased plants, potentially saving their crops before they inflict irreparable damage. As an added challenge, effective solutions for farmers must perform well under significant constraints, since African farmers may only have access to mobile-quality cameras with low-bandwidth.\n\nSubmissions will be evaluated based on their categorization accuracy."},{"metadata":{},"cell_type":"markdown","source":"### Categories  \n\n\"0\": \"Cassava Bacterial Blight (CBB)\",   \n\"1\": \"Cassava Brown Streak Disease (CBSD)\",   \n\"2\": \"Cassava Green Mottle (CGM)\",   \n\"3\": \"Cassava Mosaic Disease (CMD)\",   \n\"4\": \"Healthy\"  "},{"metadata":{},"cell_type":"markdown","source":"# Set up environment"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"scrolled":true},"cell_type":"code","source":"import os\nimport tensorflow as tf\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport json\nimport cv2\nfrom PIL import Image\nfrom tensorflow import keras\nfrom tensorflow.keras import models, layers\nfrom tensorflow.keras.optimizers import Adam, SGD, RMSprop\nfrom tensorflow.keras.models import Sequential, Model\nfrom tensorflow.keras.layers import Dense, Flatten, Conv2D, BatchNormalization, MaxPool2D, Dropout, Activation, GlobalMaxPooling2D, GlobalAveragePooling2D\nfrom tensorflow.keras.applications.efficientnet import EfficientNetB0, EfficientNetB7, preprocess_input\nfrom tensorflow.keras.metrics import sparse_categorical_accuracy, sparse_categorical_crossentropy\nfrom tensorflow.keras.losses import sparse_categorical_crossentropy\nfrom tensorflow.keras.models import model_from_json\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.preprocessing.image import load_img, img_to_array\nfrom tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\nfrom functools import partial","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load the data"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"path = '/kaggle/input/cassava-leaf-disease-classification'\n\ntrain_images = os.listdir(os.path.join(path, \"train_images\"))\nprint(\"Total images for Train: \", len(train_images))\n\nwith open ('/kaggle/input/cassava-leaf-disease-classification/label_num_to_disease_map.json') as file:\n    classes = json.loads(file.read())\n    \nprint(json.dumps(classes,indent=4))\n\ntrain_df = pd.read_csv(os.path.join(path, \"train.csv\"))\ntrain_df.head()\n\ntrain_df['class'] = train_df['label'].map({int(i) : c for i, c in classes.items()}) \n\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# plot the categories/classes to visualize the distribution\n\nplt.subplots(figsize=(12,8))\nax  = sns.countplot(x='class', data=train_df)\n\nfor a in ax.patches:\n        ax.annotate('{:1}'.format(a.get_height()),\n                    (a.get_x()+0.3, a.get_height()))\nplt.xticks(rotation=90)\nax.set_title(\"classes\", fontdict={'fontsize':15})\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The plot shows us, that we have a unbalanced data set. "},{"metadata":{"trusted":true,"scrolled":false},"cell_type":"code","source":"def plot_images(class_id, label, images_number,verbose=0):\n\n    plot_list = train_df[train_df[\"label\"] == class_id].sample(images_number)['image_id'].tolist()\n    \n    # Printing list of images\n    if verbose:\n        print(plot_list)\n        \n    labels = [label for i in range(len(plot_list))]\n    size = np.sqrt(images_number)\n    if int(size)*int(size) < images_number:\n        size = int(size) + 1\n        \n    plt.figure(figsize=(20, 20))\n    \n    for ind, (image_id, label) in enumerate(zip(plot_list, labels)):\n        plt.subplot(size, size, ind + 1)\n        image = cv2.imread(os.path.join(path, \"train_images\", image_id))\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n        plt.imshow(image)\n        plt.title(label, fontsize=12)\n        plt.axis(\"off\")\n    \n    plt.show()\n\n\nplot_images(class_id=4, \n    label='Healthy',\n    images_number=6,\n    verbose=1)\n\nplot_images(class_id=3, \n    label='Cassava Mosaic Disease (CMD)',\n    images_number=6,\n    verbose=1)\n\nplot_images(class_id=2, \n    label='Cassava Green Mottle (CGM)',\n    images_number=6,\n    verbose=1)\n\nplot_images(class_id=1, \n    label='Cassava Brown Streak Disease (CBSD)',\n    images_number=6,\n    verbose=1)\n\nplot_images(class_id=0, \n    label='Cassava Bacterial Blight (CBB)',\n    images_number=6,\n    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The pictures are not all correctly labeled. I saw some \"Healthy\" pictures which look like \"CBB\", and there are also some fruits.  \nThese components will cause problems for the ML model; garbage in - garbage out  \nNormally I would try to identify and remove the wrongly labeled images, but I take part in the competition and if the test data for determining the accuracy have the same noise, my model will be worse. To find the right labels, I would prefer a k-means clustering. --> remove the \"fruit\" cluster, and have a look on the clusters. Maybe there will be some other diseases, or false labeled pictures. --> **Update: I tried to train the model without fruit pictures --> but it had a negative impact on the submission score** (0.106)"},{"metadata":{},"cell_type":"markdown","source":"# Set up variables"},{"metadata":{},"cell_type":"markdown","source":"In the next chunk are some variables defined. Changes would impact the accurracy.  \nI tried different target sizes. eg 380 **-> no impact on loss & accurracy**"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"scrolled":true},"cell_type":"code","source":"TARGET_SIZE = (240,240) #380\nBATCH_SIZE = 16\nSTEPS_PER_EPOCH = len(train_df)*0.8 // BATCH_SIZE\nVALIDATION_STEPS = len(train_df)*0.2 // BATCH_SIZE\nEPOCHS = 10","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The test data consists only one picture. So I will do a train-test-split on the training data set.  I split the data to 80%/20%\nThe function \"ImageDataGenerator\" is used to increase the amount of data by adding slightly modified copies of already existing data.\n\nThe there is no Cross Validtion parameter in the ImageDataGeneration function. (I have to write a function to implement it -> not done for now)"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"train_df.label = train_df.label.astype(str)\n\n\ntrain_datagen = ImageDataGenerator(validation_split = 0.2,\n                                    rotation_range = 45, \n                                    zoom_range = 0.2,\n                                    horizontal_flip = True,\n                                    vertical_flip = True,\n                                    fill_mode = 'nearest',\n                                    height_shift_range = 0.2,\n                                    width_shift_range = 0.2,\n                                  )\n\ntrain_generator = train_datagen.flow_from_dataframe(train_df,\n                         directory = '/kaggle/input/cassava-leaf-disease-classification/train_images/',\n                         subset = \"training\",\n                         x_col = \"image_id\",\n                         y_col = \"label\",\n                         target_size = TARGET_SIZE,\n                         batch_size = BATCH_SIZE,\n                         class_mode = \"sparse\",\n                         seed = 2021,\n                         shuffle= True)\n\n\nvalidation_datagen = ImageDataGenerator(validation_split = 0.2) # no data augmentation on validation set\n\nvalidation_generator = validation_datagen.flow_from_dataframe(train_df,\n                         directory = '/kaggle/input/cassava-leaf-disease-classification/train_images/',\n                         subset = \"validation\",\n                         x_col = \"image_id\",\n                         y_col = \"label\",\n                         target_size = TARGET_SIZE,\n                         batch_size = BATCH_SIZE,\n                         class_mode = \"sparse\",\n                         seed = 2021,\n                         shuffle= True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creating CNN"},{"metadata":{},"cell_type":"markdown","source":"- load the pretrained weights --> EfficientNetB0 \n- build the top Layers "},{"metadata":{"trusted":true},"cell_type":"code","source":"# I use this chunk, to load my pretrained weights. To build the model for training I used the next chunk.\n\"\"\"\nweights_path = '/kaggle/input/efficientnetb0/efficientnetb0_notop.h5'\n\nbasemodel = EfficientNetB0(\n    weights=weights_path, \n    include_top=False,\n    input_shape=TARGET_SIZE+(3,))\n\nheadmodel = layers.GlobalAveragePooling2D()(basemodel.output)\nheadmodel = layers.Dense(5, activation=\"softmax\")(headmodel) # 5 -> for five classes\nmodel = keras.Model(inputs=basemodel.input, outputs=headmodel)\n\nmodel.load_weights(\"../input/best-weights-efficient/best.h5\")\n\nmodel.compile(\n    optimizer=keras.optimizers.Adam(1e-3),\n    loss='categorical_cross_entropy', # becuse not all pictures have the right label\n    metrics=[\"accuracy\"],\n)\n\"\"\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"\nweights_path = '/kaggle/input/efficientnetb0/efficientnetb0_notop.h5' \n# the pretrained weights must be uploaded, because we don´t can use Internet for the competition - if there is a Internet connection use weights = 'imagenet'\n\nbasemodel = EfficientNetB0(\n    weights=weights_path, #'imagenet'\n    include_top=False, # we don´t need the top layers, because we built this layers for our dataset\n    input_shape=TARGET_SIZE+(3,))\n\nheadmodel = layers.GlobalAveragePooling2D()(basemodel.output)\nheadmodel = layers.Dense(5, activation=\"softmax\")(headmodel)\nmodel = keras.Model(inputs=basemodel.input, outputs=headmodel)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train the Modell  "},{"metadata":{},"cell_type":"markdown","source":"To train the model is time intensive. The process tooks hours with the GPU usage. \nThe biggest challenge is the notebook timeout, because it interrupts training."},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# I used this code to train the model... \n\n\nmodel_save = ModelCheckpoint('./best_weights.h5', \n                             save_best_only = True, \n                             monitor = 'val_loss', \n                             mode = 'min', verbose = 1)\nreduce_lr = ReduceLROnPlateau(monitor = 'val_loss', factor = 0.3, \n                              patience = 2, min_lr = 1e-6, \n                              mode = 'min', verbose = 1)\nearly_stop = EarlyStopping(monitor = 'val_loss', \n                           patience = 3, mode = 'min', verbose = 1,\n                           restore_best_weights = True)\n\n\nmodel.compile(\n    optimizer=keras.optimizers.Adam(1e-3),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"],\n)\n\nhistory = model.fit(\n    train_generator,\n    steps_per_epoch = STEPS_PER_EPOCH,\n    epochs = EPOCHS, \n    validation_data = validation_generator,\n    validation_steps = VALIDATION_STEPS,\n    callbacks = [model_save, early_stop, reduce_lr],\n)\n\nmodel.save(\"model.h5\")\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# to plot the model fit history \n\ndef plot_history(history):\n    loss_list = [s for s in history.history.keys() if 'loss' in s and 'val' not in s]\n    val_loss_list = [s for s in history.history.keys() if 'loss' in s and 'val' in s]\n    acc_list = [s for s in history.history.keys() if 'acc' in s and 'val' not in s]\n    val_acc_list = [s for s in history.history.keys() if 'acc' in s and 'val' in s]\n     \n    if len(loss_list) == 0: \n        print('Loss is missing in history') \n        return \n     \n    ## As loss always exists\n    epochs = range(1,len(history.history[loss_list[0]]) + 1)\n    \n    ## Loss\n    plt.figure(1)\n    for l in loss_list: \n        plt.plot(epochs, history.history[l], 'b', label='Training loss (' + str(str(format(history.history[l][-1],'.5f'))+')'))\n    for l in val_loss_list:\n        plt.plot(epochs, history.history[l], 'g', label='Validation loss (' + str(str(format(history.history[l][-1],'.5f'))+')'))\n    \n    plt.title('Loss')\n    plt.xlabel('Epochs')\n    plt.ylabel('Loss')\n    plt.legend()\n    \n    ## Accuracy\n    plt.figure(2)\n    for l in acc_list:\n        plt.plot(epochs, history.history[l], 'b', label='Training accuracy (' + str(format(history.history[l][-1],'.5f'))+')')\n    for l in val_acc_list:    \n        plt.plot(epochs, history.history[l], 'g', label='Validation accuracy (' + str(format(history.history[l][-1],'.5f'))+')')\n\n    plt.title('Accuracy')\n    plt.xlabel('Epochs')\n    plt.ylabel('Accuracy')\n    plt.legend()\n    plt.show()\n\n\nplot_history(history)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creating a submission file\nready to submit to the competition!"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"ss = pd.read_csv(os.path.join('/kaggle/input/cassava-leaf-disease-classification', \"sample_submission.csv\"))\npreds = []\nresults = []\n\nfor image_id in ss.image_id:\n    image = Image.open(os.path.join('/kaggle/input/cassava-leaf-disease-classification', \"test_images\", image_id))\n    image = image.resize(TARGET_SIZE)\n    image = np.expand_dims(image, axis = 0)\n    preds.append(np.argmax(model.predict(image)))\n    res = max(set(preds), key = preds.count)\n    results.append(res)\n\nss['label'] = results\nss.to_csv('submission.csv', index = False)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Path of research\n\nI read a lot of papers and articles about image classification. Transfer learning is a popular approach.\nThere are a lot of nets eg. VGG16, ResNet50, EfficientNet, etc.\n\n#### Some links:\nhttps://www.nature.com/articles/s41598-020-59108-x  \nhttps://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a  \nhttps://www.frontiersin.org/articles/10.3389/fpls.2016.01419/full  \nhttps://www.mdpi.com/2223-7747/9/10/1319/htm  \nhttps://thebinarynotes.com/transfer-learning-keras-vgg16/  \n\nThere are also a lot of puplic notebooks in this competition which inspired me."},{"metadata":{},"cell_type":"markdown","source":"## Challenges\n\n- notebook runtime timeout  \n- GPU hours (30h per week are a lot, but if there is a notebook timeout and training is interrupted, then the GPU time is wasted)  \n- unbalanced dataset\n- submission error (maybe because there is no free GPU left for now)"},{"metadata":{},"cell_type":"markdown","source":"## Success  \n- implement Transfer Learning  \n- reached Validation Accurracy ~85%"},{"metadata":{},"cell_type":"markdown","source":"## Next Steps\n\n- implement Cross Validation (for now the submission score is ~60, the validation accurracy ~85)\n- fine tuning "}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}