{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **WHAT IS THE PROBLEM ABOUT?**","metadata":{}},{"cell_type":"markdown","source":"Apples are one of the most important temperate fruit crops in the world. Foliar (leaf) diseases pose a major threat to the overall productivity and quality of apple orchards. The current process for disease diagnosis in apple orchards is based on manual scouting by humans, which is time-consuming and expensive.\n\nAlthough computer vision-based models have shown promise for plant disease identification, there are some limitations that need to be addressed. Large variations in visual symptoms of a single disease across different apple cultivars, or new varieties that originated under cultivation, are major challenges for computer vision-based disease identification. These variations arise from differences in natural and image capturing environments, for example, leaf color and leaf morphology, the age of infected tissues, non-uniform image background, and different light illumination during imaging etc.\n\nPlant Pathology 2020-FGVC7 challenge competition had a pilot dataset of 3,651 RGB images of foliar disease of apples. For Plant Pathology 2021-FGVC8, we have significantly increased the number of foliar disease images and added additional disease categories. This year’s dataset contains approximately 23,000 high-quality RGB images of apple foliar diseases, including a large expert-annotated disease dataset. This dataset reflects real field scenarios by representing non-homogeneous backgrounds of leaf images taken at different maturity stages and at different times of day under different focal camera settings.","metadata":{}},{"cell_type":"markdown","source":"# Specific Objectives","metadata":{}},{"cell_type":"markdown","source":"The main objective of the competition is to develop machine learning-based models to accurately classify a given leaf image from the test dataset to a particular disease category, and to identify an individual disease from multiple disease symptoms on a single leaf image.","metadata":{}},{"cell_type":"markdown","source":"# Resources","metadata":{}},{"cell_type":"markdown","source":"I thank Kaggle for providing the dataset and Data without whom this wouldn't have been possible. Also I would like to thank Ankur Singh for this amazing dataset as without it , it would have taken hours and hours to train the below mentioned model [Ankur Singh](http://https://www.kaggle.com/ankursingh12/resized-plant2021).","metadata":{}},{"cell_type":"markdown","source":"# Importing All the Necessary Libraries","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport tensorflow as tf\nimport tensorflow.keras as keras\nimport PIL\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport os\nimport random\nfrom tqdm import tqdm\nimport tensorflow_addons as tfa\nimport random\nfrom sklearn.preprocessing import MultiLabelBinarizer\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport keras\nfrom keras.preprocessing import image\nfrom keras.models import Sequential\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array, smart_resize\nfrom keras.layers import Dense, Dropout, Flatten, BatchNormalization, Activation\nfrom keras.constraints import maxnorm\nfrom keras.layers.convolutional import Conv2D, MaxPooling2D\nfrom tensorflow.keras.optimizers import Adam\nimport cv2\nfrom PIL import Image\nfrom keras.preprocessing.image import load_img, img_to_array\nfrom keras.models import load_model\nfrom keras.metrics import AUC\nimport matplotlib.pyplot as plt\nimport tensorflow as tf \nimport random as rn\nimport numpy as np\nimport os\n\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.applications import ResNet50V2\nfrom tensorflow.keras.applications import ResNet50\n\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.layers import Input\nfrom tensorflow.keras.layers import Dense,Dropout,Flatten\nfrom tensorflow.keras.layers import MaxPooling2D,GlobalAveragePooling2D,BatchNormalization,Activation\nfrom tensorflow import keras\nfrom keras import backend as K\n# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport tensorflow as tf\nimport logging\nlogging.basicConfig()\nimport struct\nfrom tensorflow.keras.layers import Dense, Activation, Conv2D, Flatten, Dropout, Input, BatchNormalization\n%matplotlib inline\nimport IPython.core.display         \n# setup output image format (Chrome works best)\nIPython.core.display.set_matplotlib_formats(\"svg\")\nimport matplotlib.pyplot as plt\nimport matplotlib\nimport numpy as np\nimport sklearn\nimport pandas as pd\nimport seaborn as sns\nfrom sklearn.preprocessing import MultiLabelBinarizer,OneHotEncoder\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras import backend as K\nfrom tensorflow.keras.applications import ResNet50V2\n\n%matplotlib inline","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Performing EDA And Understanding Our Data Better","metadata":{}},{"cell_type":"code","source":"train_dir= '../input/plant-pathology-2021-fgvc8/train_images'\ntest_dir =  '../input/plant-pathology-2021-fgvc8/test_images'\ntrain = pd.read_csv('../input/plant-pathology-2021-fgvc8/train.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We consider the two columns only which are images and the labels because that's all  we need to perform EDA and train the model.** ","metadata":{}},{"cell_type":"code","source":"#dup = pd.DataFrame(dup,columns = ['dup1','dup2'])\ntrain = pd.DataFrame(train,columns = ['image','labels'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let Us Find Out the Different Types of Classes(Diseases)","metadata":{}},{"cell_type":"code","source":"train['labels'].value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Frequency of Each Class","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(20,12))\nlabels = sns.barplot(train.labels.value_counts().index,train.labels.value_counts())\nfor item in labels.get_xticklabels():\n    item.set_rotation(45)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Important Observation","metadata":{}},{"cell_type":"markdown","source":"**Look at the labels, does anything strike you ??\nSome of the labels are mixture of one or more types !!! And thus the problem becomes Multilabel Problem**","metadata":{}},{"cell_type":"markdown","source":"So there are not 12 labels, its actually just 6 labels. 5 diseases:\n**1. Rust 2.Scab 3.Complex 4.Frog eye leaf spot 5.Powdery Mildew  and Last Label is \"Healthy\"**\nNow the most important thing is, as one image can have multiple diseases, that means this problem is Multi label classification problem. Many get confused betweeen multilabel and multiclass classification. if you are new to multilabel classification I would suggest going over this.\n[Multilabel Classification](http://https://www.geeksforgeeks.org/an-introduction-to-multilabel-classification/)\nSo now we gotta process the labels. And then lets find out the actual frequencies of the labels.\nWe divide it based on \" \" or space character , in order to get the labels for each of the image","metadata":{}},{"cell_type":"code","source":"train['labels'] = train['labels'].apply(lambda s: s.split(' '))\ntrain[:10]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Preparing Our Model**","metadata":{}},{"cell_type":"markdown","source":"# Image Preprocessing Using Keras Image Data Generator\nThose who want to know about Image Data Generator can look at this Documentation in order to get idea on what are the things we can perform [Keras Image Data Generator](http://https://keras.io/api/preprocessing/image/)","metadata":{}},{"cell_type":"code","source":"def add_gauss_noise(x,sigma2=0.05):\n    return x+np.random.normal(0, sigma2, x.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"datagen = ImageDataGenerator(\n    rotation_range = 10,#Performing Rotation\n    width_shift_range = 0.1,\n    height_shift_range = 0.1,\n    brightness_range = None,\n    shear_range = 0.1,\n    zoom_range = 0.1,\n    rescale = 1./255,\n    horizontal_flip=True,\n    vertical_flip=True,\n    \n    validation_split= 0.1# We will split the training data into training and validation set in the ration 90:10\n)\nbsize  = 16 # Giving a batch size of 16","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preprocessing the Training and Validation Data","metadata":{}},{"cell_type":"code","source":"train_data = datagen.flow_from_dataframe(\n    train,\n    directory = '../input/resized-plant2021/img_sz_512',# We are using the resized images otherwise it will take a lot of time to train \n    x_col = 'image',\n    y_col = 'labels',\n    subset=\"training\",\n    color_mode=\"rgb\",\n    target_size = (224,224),\n    class_mode=\"categorical\",\n    batch_size=bsize,\n    shuffle=False,\n    seed=40,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preparing the Validation Data","metadata":{}},{"cell_type":"code","source":"valid_data = datagen.flow_from_dataframe(\n    train,\n    directory = '../input/resized-plant2021/img_sz_512',\n    x_col = 'image',\n    y_col = 'labels',\n    subset=\"validation\",\n    color_mode=\"rgb\",\n    target_size = (224,224),\n    class_mode=\"categorical\",\n    batch_size=bsize,\n    shuffle=False,\n    seed=40,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Now we are Using Transfer Learning Model\n**Here I used ResNet 50V2 , which was present in the Keras Library. It is a modified version of the famous ResNet 50 architecture. After Using ResNet 50v2 as a base we fine tune our model.**","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.preprocessing import image\nimport random\n\nK.clear_session()\nrandom.seed(4487); tf.random.set_seed(4487)\n\ninput_shape= (224,224,3)#Using the shape of (224,224)\n# \nbase_model = ResNet50V2(input_shape=input_shape, include_top=False,weights= \"imagenet\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.layers import MaxPooling2D,GlobalAveragePooling2D,BatchNormalization,Activation\nx = base_model.output\nx = GlobalAveragePooling2D()(x)\n#fully connected layer\nx = Dense(64, activation='relu')(x)\nx = Dense(16, activation='relu')(x)\n# finally, the softmax for the classifier \npredictions = Dense(6, activation='softmax')(x)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let Us Look at out Model ","metadata":{}},{"cell_type":"code","source":"model = tf.keras.Model(inputs=base_model.input ,outputs = predictions)\nmodel.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Training Our Model And Let's Find out the Results**","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.models import Sequential, Model\n\nmodel = tf.keras.Model(inputs=base_model.input ,outputs = predictions)\nimport tensorflow_addons as tfa\nimport keras \nf1 = tfa.metrics.F1Score(num_classes=6, average='macro')\nmodel.compile(optimizer=keras.optimizers.SGD(lr=0.03, decay=1e-4, momentum=0.8, nesterov=True), \n              loss='binary_crossentropy', metrics=[f1])\n\naccearlystop = keras.callbacks.EarlyStopping(\n    monitor=f1,     # look at the validation loss tf2.0 accuracy\n    min_delta=0.02,       # threshold to consider as no change\n    patience=5,             # stop if  epochs with no change\n    verbose=1, mode='max', restore_best_weights= True\n)\nlossearlystop = keras.callbacks.EarlyStopping(\n    monitor='val_loss',     # look at the validation loss tf2.0 accuracy\n    min_delta=0.02,       # threshold to consider as no change\n    patience=5,             # stop if  epochs with no change\n    verbose=1, mode='min', restore_best_weights= True\n)\n# callbacks_list = [earlystop]\nlrschedule = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', \n                                 factor=0.05, patience=5, verbose=1)\ncallbacks_list = [lrschedule]\n# callbacks_list = [accearlystop,lossearlystop]\n#callbacks_list = []\n\nhistory = model.fit_generator(\n            train_data,  # data from generator\n             #steps_per_epoch=1,    # should be number of batches per epoch\n            epochs=15,\n            callbacks=callbacks_list, \n            validation_data=valid_data, \n            #validation_steps = 1,\n            verbose=True)\n\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting Our Performance for the model ","metadata":{}},{"cell_type":"code","source":"accname = 'f1_score'\n\ndef plot_history(history): \n    fig, ax1 = plt.subplots()\n    \n    ax1.plot(history.history['loss'], 'r', label=\"training loss ({:.6f})\".format(history.history['loss'][-1]))\n    ax1.plot(history.history['val_loss'], 'r--', label=\"validation loss ({:.6f})\".format(history.history['val_loss'][-1]))\n    ax1.grid(True)\n    ax1.set_xlabel('iteration')\n    ax1.legend(loc=\"best\", fontsize=9)    \n    ax1.set_ylabel('loss', color='r')\n    ax1.tick_params('y', colors='r')\n\n    if accname in history.history:\n        ax2 = ax1.twinx()\n\n        ax2.plot(history.history[accname], 'b', label=\"training f1_score ({:.4f})\".format(history.history[accname][-1]))\n        ax2.plot(history.history['val_'+accname], 'b--', label=\"validation f1_score ({:.4f})\".format(history.history['val_'+accname][-1]))\n\n        ax2.legend(loc=\"lower right\", fontsize=9)\n        ax2.set_ylabel('acc', color='b')        \n        ax2.tick_params('y', colors='b')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_history(history)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Finding Out our Overall Performance","metadata":{}},{"cell_type":"code","source":"loss, f1score = model.evaluate_generator(valid_data,verbose=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save('ResNet 50 v2 Modified.h5')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Preparing Our Testing Data and Finally the Submission File**","metadata":{}},{"cell_type":"code","source":"import tensorflow_addons as tfa\nf1 = tfa.metrics.F1Score(num_classes=6, average='macro')\naccearlystop = keras.callbacks.EarlyStopping(\n    monitor=f1,     # look at the validation loss tf2.0 accuracy\n    min_delta=0.02,       # threshold to consider as no change\n    patience=5,             # stop if  epochs with no change\n    verbose=1, mode='max', restore_best_weights= True\n)\nlossearlystop = keras.callbacks.EarlyStopping(\n    monitor='val_loss',     # look at the validation loss tf2.0 accuracy\n    min_delta=0.02,       # threshold to consider as no change\n    patience=5,             # stop if  epochs with no change\n    verbose=1, mode='min', restore_best_weights= True\n)\n# callbacks_list = [earlystop]\nlrschedule = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', \n                                 factor=0.05, patience=5, verbose=1)\ncallbacks_list = [lrschedule]","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\nmodel = load_model(\"../input/new-model/NewFinal.h5\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tqdm import tqdm\nimport PIL\n\ntest = pd.read_csv('../input/plant-pathology-2021-fgvc8/sample_submission.csv')\n\nfor img_name in tqdm(test['image']):\n    path = '../input/plant-pathology-2021-fgvc8/test_images/'+str(img_name)\n    with PIL.Image.open(path) as img:\n        img = img.resize((256,256))\n        img.save(f'./{img_name}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preprocessing Our Testing Data Like We did for the Training Dataset\nWe first pre process the data , then find an ideal threshold for the Multilabel classification. And finally get the predictions using it. Notice that the threshold has been kept low as an image may have more than 2 diseases for which 50% threshold won't work \n","metadata":{}},{"cell_type":"code","source":"test_data = datagen.flow_from_dataframe(\n    test,\n    directory = './',\n    x_col=\"image\",\n    y_col= None,\n    color_mode=\"rgb\",\n    target_size = (224,224),\n    classes=None,\n    class_mode=None,\n    batch_size=bsize,\n    shuffle=False,\n    seed=40,\n)\nbest_threshold = 0.23 #This threshold can be changed according to the people's wish\npreds = model.predict(test_data)\nprint(preds)\npreds = preds.tolist()\n\nindices = []\nfor pred in preds:\n    temp = []\n    for category in pred:\n        if category>=best_threshold:\n            temp.append(pred.index(category))\n    if temp!=[]:\n        indices.append(temp)\n    else:\n        temp.append(np.argmax(pred))\n        indices.append(temp)\n    \nprint(indices)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preparing the Submission File","metadata":{}},{"cell_type":"code","source":"labels = (train_data.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\nprint(labels)\n\ntestlabels = []\n\n\nfor image in indices:\n    temp = []\n    for i in image:\n        temp.append(str(labels[i]))\n    testlabels.append(' '.join(temp))\n\nprint(testlabels)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\ndelfiles = tf.io.gfile.glob('./*.jpg')\n\nfor file in delfiles:\n    os.remove(file)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv('../input/plant-pathology-2021-fgvc8/sample_submission.csv')\nsub['labels'] = testlabels\nsub.to_csv('submission.csv', index=False)\nsub","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Please Do Upvote the Notebook. Thank You**","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}