{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**What is this about?**\n\nApples are one of the most important temperate fruit crops in the world. Foliar (leaf) diseases pose a major threat to the overall productivity and quality of apple orchards. The current process for disease diagnosis in apple orchards is based on manual scouting by humans, which is time-consuming and expensive.\n\nAlthough computer vision-based models have shown promise for plant disease identification, there are some limitations that need to be addressed. Large variations in visual symptoms of a single disease across different apple cultivars, or new varieties that originated under cultivation, are major challenges for computer vision-based disease identification. These variations arise from differences in natural and image capturing environments, for example, leaf color and leaf morphology, the age of infected tissues, non-uniform image background, and different light illumination during imaging etc.\n\nPlant Pathology 2020-FGVC7 challenge competition had a pilot dataset of 3,651 RGB images of foliar disease of apples. For Plant Pathology 2021-FGVC8, we have significantly increased the number of foliar disease images and added additional disease categories. This year’s dataset contains approximately 23,000 high-quality RGB images of apple foliar diseases, including a large expert-annotated disease dataset. This dataset reflects real field scenarios by representing non-homogeneous backgrounds of leaf images taken at different maturity stages and at different times of day under different focal camera settings.\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"# **Specific Objectives**\n\nThe main objective of the competition is to develop machine learning-based models to accurately classify a given leaf image from the test dataset to a particular disease category, and to identify an individual disease from multiple disease symptoms on a single leaf image.","metadata":{}},{"cell_type":"markdown","source":"# **Resources**\nI thank Kaggle for providing the dataset and [Data](http://https://bsapubs.onlinelibrary.wiley.com/doi/10.1002/aps3.11390)\nwithout whom this wouldn't have been Possible. \nAlso I would like to thank [Ankur Singh](http://https://www.kaggle.com/ankursingh12/resized-plant2021) for this amazing dataset as without it , it would have taken hours and hours to train the below mentioned model. ","metadata":{}},{"cell_type":"markdown","source":"# Import the necessary libraries","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport tensorflow as tf\nimport tensorflow.keras as keras\nimport PIL\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport os\nimport random\nfrom tqdm import tqdm\nimport tensorflow_addons as tfa\nimport random\nfrom sklearn.preprocessing import MultiLabelBinarizer\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport keras\nfrom keras.preprocessing import image\nfrom keras.models import Sequential\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array, smart_resize\nfrom keras.layers import Dense, Dropout, Flatten, BatchNormalization, Activation\nfrom keras.constraints import maxnorm\nfrom keras.layers.convolutional import Conv2D, MaxPooling2D\nfrom tensorflow.keras.optimizers import Adam\nimport cv2\nfrom PIL import Image\nfrom keras.preprocessing.image import load_img, img_to_array\nfrom keras.models import load_model\nfrom keras.metrics import AUC\n\npd.set_option(\"display.max_columns\", None)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Let's Now Have a look at the Dataset and Study it better**\n\nI would like to thank [Praveen](http://https://www.kaggle.com/praveengovi/plant-pathology-detail-eda-pytorch) for this amazing EDA and analysis and also [Arnab](http://https://www.kaggle.com/arnabs007/apple-leaf-diseases-with-inceptionresnetv2-keras) from whom I have taken reference from . I have implemented various EDA and studied from their models and approached it with Transfer Learning Model of ResNet 50v2 base.\n ","metadata":{}},{"cell_type":"code","source":"train_dir= '../input/plant-pathology-2021-fgvc8/train_images'\ntest_dir =  '../input/plant-pathology-2021-fgvc8/test_images'\ntrain = pd.read_csv('../input/plant-pathology-2021-fgvc8/train.csv')\n#print(len(train))\n#print(train.columns)\n# print(train['labels'].value_counts())\n#print(train['labels'].value_counts().plot.bar())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let's Study the dataset in a better way and try to find some interesting stuff!!! ","metadata":{}},{"cell_type":"code","source":"train.head","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We get to know that we have \"many\" images with mostly 12 types of labels (but there is a twist) which we will comeback to later.**","metadata":{}},{"cell_type":"markdown","source":"Let's look at the number of images for various of 12 categories present","metadata":{}},{"cell_type":"code","source":"train['labels'].value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,12))\nlabels = sns.barplot(train.labels.value_counts().index,train.labels.value_counts())\nfor item in labels.get_xticklabels():\n    item.set_rotation(45)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note**\n\nNotice that there is a huge imbalance in dataset with \"scab\" having the highest number of frequency and \"powdery_mildew complex\" , the least","metadata":{}},{"cell_type":"markdown","source":"# Important Observation\n\n**Look at the labels, doesn't it strike you ??** \n\n**Some of the labels are mixture of one or more types !!! And thus the problem becomes Multilabel Problem**\n","metadata":{}},{"cell_type":"markdown","source":"So there are not 12 labels, its actually just 6 labels.\n5 diseases: \n1. rust\n2. scab \n3. complex \n4. frog eye leaf spot\n5. powdery mildew \n\nand another label is \n\n6. healthy (healthy leaves) \n\nNow the most important thing is, as one image can have multiple diseases, that means this problem is **Multi label classification** problem. Many get confused betweeen multilabel and multiclass classification. if you are new to multilabel classification I would suggest going over this [An introduction to MultiLabel classification](https://www.geeksforgeeks.org/an-introduction-to-multilabel-classification/) . \n\nSo now we gotta process the labels. And then lets find out the actual frequencies of the labels. \n","metadata":{}},{"cell_type":"markdown","source":"We divide it based on \" \" or space character , in order to get the labels for each of the image","metadata":{}},{"cell_type":"code","source":"train['labels'] = train['labels'].apply(lambda string: string.split(' '))\ntrain","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Converting the labels representation into **one hot encoded format** using MultilabelBinarizer from Scikit learn. Now we can see and plot the frequencies of each label. ","metadata":{}},{"cell_type":"code","source":"s = list(train['labels'])\nmlb = MultiLabelBinarizer()\ntrainx = pd.DataFrame(mlb.fit_transform(s), columns=mlb.classes_, index=train.index)\nprint(trainx.columns)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"These are the 6 different labels ","metadata":{}},{"cell_type":"code","source":"print(trainx.sum())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels = list(trainx.sum().keys())\n#print(labels)\nlabel_counts = trainx.sum().values.tolist()\n\nfig, ax = plt.subplots(1,1, figsize=(20,6))\n\nsns.barplot(x= labels, y= label_counts, ax=ax)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**NOW WE CAN SEE THE DATASET BECOMES MORE OR LESS BALANCED , AT LEAST BETTER THAN WHAT IT WAS PREVIOUSLY!**","metadata":{}},{"cell_type":"markdown","source":"# Let's See the Plant Pathology Images","metadata":{}},{"cell_type":"code","source":"labels = pd.concat([train['image'], trainx], axis=1)\nlabels.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig1 = plt.figure(figsize=(20,10))\n\nfor i in range(1, 10):\n    \n    rand =  random.randrange(1, 18000)\n    sample = os.path.join('../input/plant-pathology-2021-fgvc8/train_images/', train['image'][rand])\n    \n    img = PIL.Image.open(sample)\n    \n    ax = fig1.add_subplot(4,3,i)\n    ax.imshow(img)\n    \n    title = f\"{train['labels'][rand]}{img.size}\"\n    plt.title(title)\n    \n    fig1.tight_layout()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Imaze Size & Processing\nfrom the titles we can see some random image sizes - (4000, 2672). Larger images are harder to process hence takes much longer to train the CNN. Downsampling all these 18632 images is also a time consuming task. This is I am going to use the resized imaged for this dataset [resized-plant2021](https://www.kaggle.com/ankursingh12/resized-plant2021) by Ankur Singh. He has already downsampled the images into size of 256, 384, 512 & 640px.\nNow for Pre Processing I take help of the [Keras Image Data Generator](http://https://keras.io/api/preprocessing/image/). We transform it to size of (256,256,3) .","metadata":{}},{"cell_type":"code","source":"%%time\ndatagen = keras.preprocessing.image.ImageDataGenerator(rescale=1/255.0,\n                                                        preprocessing_function=None,\n                                                        data_format=None,\n                                                    )\n\ntrain_data = datagen.flow_from_dataframe(\n    train,\n    directory= '../input/resized-plant2021/img_sz_512',\n    x_col=\"image\",\n    y_col= 'labels',\n    color_mode=\"rgb\",\n    target_size = (256,256),\n    class_mode=\"categorical\",\n    batch_size=32,\n    shuffle=False,\n    seed=40,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Transfer Learning Using ResNet 50V2 Base\n\n[Transfer learning](http://builtin.com/data-science/transfer-learning) In transfer learning, the knowledge of an already trained machine learning model is applied to a different but related problem. For example, if you trained a simple classifier to predict whether an image contains a backpack, you could use the knowledge that the model gained during its training to recognize other objects like sunglasses.\nWith transfer learning, we basically try to exploit what has been learned in one task to improve generalization in another. We transfer the weights that a network has learned at \"task A\" to a new \"task B\". \nHere I will use ResNet 50V2 model, and load the previously saved weights from the \"imagenet\" competition . [ResNet 50 ](http://https://keras.io/api/applications/resnet/) is a famous model, which won the ImageNet Competition in 2016, it is based on the idea of residual networks. ResNet 50V2 is a modified version of ResNet 50 which I am going to use here now.\n","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport tensorflow as tf \nimport random as rn\nimport numpy as np\nimport os\n\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.applications import ResNet50V2\nfrom tensorflow.keras.applications import ResNet50\n\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.layers import Input\nfrom tensorflow.keras.layers import Dense,Dropout,Flatten\nfrom tensorflow.keras.layers import MaxPooling2D,GlobalAveragePooling2D,BatchNormalization,Activation\nfrom tensorflow import keras\nfrom keras import backend as K\n\n%matplotlib inline","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"seed = 1200\ntf.random.set_seed(seed)\ninput_shape= (256,256,3)\n#weights_path = '../input/keras-pretrained-models/inception_resnet_v2_weights_tf_dim_ordering_tf_kernels_notop.h5'\n#model = keras.applications.InceptionResNetV2(weights=weights_path, include_top=False, input_shape=(256, 256, 3))\nbase_model = ResNet50V2(input_shape=input_shape, include_top=False,weights= \"imagenet\")\nbase_model.trainable = False\nprint(base_model.input)\nprint(base_model.output)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Architecture and Information\n\nAs mentioned above here I used a ResNet 50V2 based model. So,first we need to keep in mind that we **should not** train the layers of the ResNet part of the model. Because it has already been trained for 1000 classes of image. So what we need to do , is to remove the last Fully Connected Layer and then build our model based upon it . After various changes to the model, I noticed that the model overfits whenever I add more than one layer in between the base model i.e ResNet 50V2 and the final layer . So I only added a single layer with BatchNormalisation and a Dropout layer , before giving out the outputs. \n","metadata":{}},{"cell_type":"markdown","source":"# Dealing with Multi Label Classification\n\nOut general thught when we come across more than one classes , is to use \"softmax\" activation function in the last layer . But remember as this is a multilabel classification problem, we can't use softmax here, hence the sigmoid activation.\nAnother thing to keep in mind is that Binary crossentropy is used instead of categorical crossentropy. We use categorical cross-entropy in multi-class problems, but for multi-label problems, we use binary cross-entropy. Think of it this way, an image may have multiple labels, and we need the probabilities that each of these labels corresponds to the given image - this can be considered as n independent binary classifiers for the n labels.\n\nFor evaluation I have used F1 accuracy metrics instead of binary accuracy. F1 and its variants are better for evaluation when it comes to multiclass and multilabel problems. if you want to know F1 score works for Multilabel classification go through this https://medium.com/synthesio-engineering/precision-accuracy-and-f1-score-for-multi-label-classification-34ac6bdfb404https://medium.com/synthesio-engineering/precision-accuracy-and-f1-score-for-multi-label-classification-34ac6bdfb404. ","metadata":{}},{"cell_type":"code","source":"\n# Freezing the weights\nfor layer in base_model.layers[:-1]:\n    layer.trainable=False\n    \n#new_model.summary()\n\nclasses = 6\nx = base_model.output\nx = GlobalAveragePooling2D()(x)\n#x = Flatten()(x)\nx = Dense(1024,activation='relu')(x)\n\nx = BatchNormalization()(x)\n\n#x = Dropout(0.10)(x)\n\n#x = Dense(1024,activation= 'relu')(x)\n#x = BatchNormalization()(x)\n#x = Dropout(0.50)(x)#jn\n\noutput = Dense(classes,activation='sigmoid')(x)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\nf1 = tfa.metrics.F1Score(num_classes=6, average='macro')\n\ncallbacks = keras.callbacks.EarlyStopping(monitor=f1, patience=3, mode='max', restore_best_weights=True)\nthreshold_x=0.25\noptimizer = Adam(learning_rate = 0.0001)\nf1 = tfa.metrics.F1Score(num_classes=6, average='macro',threshold = threshold_x)\ncallbacks = keras.callbacks.EarlyStopping(monitor=f1, patience=3, mode='max', restore_best_weights=True)\nmodel = tf.keras.Model(inputs = base_model.input, outputs = output)\n#model = load_model(\"../input/resnet50-v2\")\nmodel.compile(loss=tf.keras.losses.BinaryCrossentropy(), optimizer=optimizer,metrics=[f1])\nmodel.fit(train_data, epochs=15, callbacks=callbacks)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save('ResNet50Base12.h5')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# But How to Submit the Results? \n\nFor submission I will resize the test images as I did for the train images , and doing the same operations as the train set and then predict the labels for them.\n\n**Remember, since we cannot use the internet while submitting the solution , we should download the model as a \".h5\" file , then upload it into Input Folder and finally load the model , for predicting**","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv('../input/plant-pathology-2021-fgvc8/sample_submission.csv')\n\nfor img_name in tqdm(test['image']):\n    path = '../input/plant-pathology-2021-fgvc8/test_images/'+str(img_name)\n    with PIL.Image.open(path) as img:\n        img = img.resize((256,256))\n        img.save(f'./{img_name}')\n        \n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\nmodel = load_model(\"../input/resnet-50v2final/ResNet50Base12.h5\")\n ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data = datagen.flow_from_dataframe(\n    test,\n    directory = './',\n    x_col=\"image\",\n    y_col= None,\n    color_mode=\"rgb\",\n    target_size = (256,256),\n    classes=None,\n    class_mode=None,\n    batch_size=32,\n    shuffle=False,\n    seed=40,\n)\n\npreds = model.predict(test_data)\nprint(preds)\npreds = preds.tolist()\n\nindices = []\nfor pred in preds:\n    temp = []\n    for category in pred:\n        if category>=0.25:\n            temp.append(pred.index(category))\n    if temp!=[]:\n        indices.append(temp)\n    else:\n        temp.append(np.argmax(pred))\n        indices.append(temp)\n    \nprint(indices)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nlabels = (train_data.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\nprint(labels)\n\ntestlabels = []\n\n\nfor image in indices:\n    temp = []\n    for i in image:\n        temp.append(str(labels[i]))\n    testlabels.append(' '.join(temp))\n\nprint(testlabels)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Remove the resized images from output before submission. if there are any other files present except 'submission.csv' it will throw an error when submitting.","metadata":{}},{"cell_type":"code","source":"\ndelfiles = tf.io.gfile.glob('./*.jpg')\n\nfor file in delfiles:\n    os.remove(file)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nsub = pd.read_csv('../input/plant-pathology-2021-fgvc8/sample_submission.csv')\nsub['labels'] = testlabels\nsub.to_csv('submission.csv', index=False)\nsub\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# If you like the notebook please upvote the notebook\n\n**This is my First notebook , so please forgive me if there was some errors and please do suggest me on how to improve my writing in Kaggle Notebooks**\n\n","metadata":{}},{"cell_type":"markdown","source":"# Thank You","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}