{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Plant Pathology 2021 - FGVC8\n\n### What is this competition about?\n\nApples are one of the most important temperate fruit crops in the world. Foliar (leaf) diseases pose a major threat to the overall productivity and quality of apple orchards. The current process for disease diagnosis in apple orchards is based on manual scouting by humans, which is time-consuming and expensive.\n\nAlthough computer vision-based models have shown promise for plant disease identification, there are some limitations that need to be addressed. Large variations in visual symptoms of a single disease across different apple cultivars, or new varieties that originated under cultivation, are major challenges for computer vision-based disease identification. These variations arise from differences in natural and image capturing environments, for example, leaf color and leaf morphology, the age of infected tissues, non-uniform image background, and different light illumination during imaging etc.","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Let's import some libraries!","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport tensorflow as tf\nimport PIL\nimport cv2\nfrom keras_preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, GlobalAveragePooling2D\nfrom tensorflow.keras.callbacks import EarlyStopping\nfrom tensorflow.keras.optimizers import Adam\nimport tensorflow_addons as tfadd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom tensorflow.keras.applications import InceptionResNetV2","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, we will set the data directories for images and datasets","metadata":{}},{"cell_type":"code","source":"train_data = pd.read_csv('/kaggle/input/plant-pathology-2021-fgvc8/train.csv')\ntest_data = pd.read_csv('/kaggle/input/plant-pathology-2021-fgvc8/sample_submission.csv')\nTRAIN_IMG_DIR = '../input/plant-pathology-2021-fgvc8/train_images/'\nTEST_IMG_DIR = '../input/plant-pathology-2021-fgvc8/test_images/'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels = train_data.labels.unique()\nvalue_counts = train_data['labels'].value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 6))\nplt.xticks(rotation=90)\nplt.title(\"Comparison of Different Labels\")\nsns.barplot(x=labels, y=value_counts)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Take a look at the labels!\n\nAn image may belong to one class or multiple classes. So, in short we have 6 classes of labels. Out of these 6 classes, there are 5 diseases, namely:\n\n* scab\n* complex\n* rust\n* frog eye leaf spot\n* powdery mildew\n\nThe remaining one label is \"healthy\" which is pretty much self explanatory\n\nThis is a multi-label classification problem as one image can represent more than one class of diseases.\n\nLet's have a look at some of the images","metadata":{}},{"cell_type":"code","source":"IMG_SIZE = 250\n\nfor i in range(0, 100, 10):\n    img_array = cv2.imread(TRAIN_IMG_DIR + train_data['image'][i])\n    new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))\n    plt.imshow(new_array)\n    plt.title(train_data['labels'][i])\n    plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We might as well look at the individual label comparison with each other. I will create a copy of my training data as I don't want to make changes to original training data","metadata":{}},{"cell_type":"code","source":"dummy_train_data = train_data\ndummy_train_data = dummy_train_data['labels'].str.split(\" \", expand=True).stack()\nlabel_dummies = pd.get_dummies(dummy_train_data).groupby(level=0).sum()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We have converted the categorical data via dummy variable from pandas","metadata":{}},{"cell_type":"code","source":"label_dummies.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So far so good. Let's plot the labels and their occurrence in the training images","metadata":{}},{"cell_type":"code","source":"cols = label_dummies.columns\nlabel_counts = label_dummies[cols].sum()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 6))\nplt.title(\"Comparison of all unique Labels\")\nsns.barplot(x=cols, y=label_counts)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Problem with the original image size\n\nThe image sizes of all the training images is very high. I encountered two issues due to that:\n\n1. While the model was getting trained, most of the CPU time was used to load the images. Due to that, a single epoch took somewhere around 45 mins. GPU was getting used but the main lag was due to the CPU which was busy loading the images from the dataset\n\n2. When I tried downsizing the images to a lower resolution, my RAM got used fully and I was not able to continue ahead. Plus, it also took a lot of time to downsize 18632 images.\n\nTherefore, I will be using a resized images dataset ([link to the dataset](https://www.kaggle.com/ankursingh12/resized-plant2021))\n\nAlso, we need the labels of training data in a comma separated fashion","metadata":{}},{"cell_type":"code","source":"train_data['labels'] = train_data['labels'].str.split(\" \")\n\ndatagen = ImageDataGenerator(rescale=1./255, validation_split=0.1)\nfinal_train_data = datagen.flow_from_dataframe(train_data,\n    directory='/kaggle/input/resized-plant2021/img_sz_512',\n    x_col=\"image\",\n    y_col=\"labels\",\n    target_size=(256, 256),\n    color_mode=\"rgb\",\n    class_mode=\"categorical\",\n    subset=\"training\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"validation_data = datagen.flow_from_dataframe(train_data,\n    directory='/kaggle/input/resized-plant2021/img_sz_512',\n    x_col=\"image\",\n    y_col=\"labels\",\n    target_size=(256, 256),\n    color_mode=\"rgb\",\n    class_mode=\"categorical\",\n    subset=\"validation\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model Creation and Fitting\n\nI will be using InceptionResNetV2 pre-trained model. In addition to this, I will be adding a GlobalAveragePooling2D layer and one last Dense layer with 6 nodes, one for each class with 'sigmoid' as activation, one node for each label(this is a multilabel classification problem)","metadata":{}},{"cell_type":"code","source":"weights = '../input/keras-pretrained-models/inception_resnet_v2_weights_tf_dim_ordering_tf_kernels_notop.h5'\n\npretrained_weight_model = InceptionResNetV2(\n    include_top=False,\n    weights=weights,\n    input_shape=(256, 256, 3)\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pretrained_weight_model.input\npretrained_weight_model.output","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##  F1 score as metrics\n\nSince it is a multilabel image classification, I will be going for F1 accuracy instead of binary accuracy in macro mode. A macro-average will compute the metric independently for each class and then take the average (hence treating all classes equally)","metadata":{}},{"cell_type":"code","source":"final_model = Sequential([\n    pretrained_weight_model,\n    GlobalAveragePooling2D(),\n    Dense(units=6, activation = 'sigmoid')\n])\n\nfor layer in final_model.layers[:-1]:\n    layer.trainable=False\n\nfinal_model.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's create a callback to prevent overfitting/underfitting","metadata":{}},{"cell_type":"code","source":"f1_score = tfadd.metrics.F1Score(num_classes=6, average='macro')\n\nearly_stopping = EarlyStopping(monitor=f1_score, patience=3, mode='max', restore_best_weights=True)\n\n\nfinal_model.compile(loss='binary_crossentropy', optimizer=Adam(epsilon=0.01), \n              metrics= [f1_score])\n\nhistory = final_model.fit(final_train_data, epochs=60, \n        callbacks=early_stopping, validation_data=validation_data)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Analysis of scores\n\n1. loss vs f1 score\n2. validation loss vs validation f1 score","metadata":{}},{"cell_type":"code","source":"history_frame = pd.DataFrame(history.history)\nhistory_frame.loc[:, ['loss', 'val_loss']].plot()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_frame.loc[:, ['f1_score', 'val_f1_score']].plot();","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's now predict on the test images that we have. Firstly, I will resize the images to 256X256 and then predict the values on it.","metadata":{}},{"cell_type":"code","source":"for i in range(test_data.shape[0]):\n    image_path = TEST_IMG_DIR+'/'+test_data.image[i]\n    with PIL.Image.open(image_path) as image_data:\n        image_data = image_data.resize((256, 256))\n        image_data.save(f'./{test_data.image[i]}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_test_data = datagen.flow_from_dataframe(test_data,\n    directory='./',\n    x_col=\"image\",\n    y_col=None,\n    target_size=(256, 256),\n    color_mode=\"rgb\",\n    class_mode=None,\n    classes=None,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_data = final_model.predict(final_test_data)\npred_data = pred_data.tolist()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_data","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Converting the output values to the indices of labels","metadata":{}},{"cell_type":"code","source":"index_list = []\n\nfor pred in pred_data:\n    index = []\n    for value in pred:\n        if value >= 0.3:\n            index.append(pred.index(value))\n    if index != []:\n        index_list.append(index)\n    else:\n        index.append(np.argmax(pred))\n        index_list.append(index)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here are the predicted labels:","metadata":{}},{"cell_type":"code","source":"index_list","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Mapping the labels back to the disease names","metadata":{}},{"cell_type":"code","source":"pred_labels = final_train_data.class_indices\npred_labels = dict((value, key) for key, value in pred_labels.items())\n\npred_label_names = []\n\nfor indices in index_list:\n    index = []\n    for i in indices:\n        index.append(str(pred_labels[i]))\n    pred_label_names.append(' '.join(index))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_label_names","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We have our predicted labels. Now the final step is the submission of our predicted labels","metadata":{}},{"cell_type":"code","source":"resized_test_images = tf.io.gfile.glob('./*.jpg')\n\nfor image in resized_test_images:\n    os.remove(image)\n\ntest_data['labels'] = pred_label_names\n# test_data.to_csv('submission.csv', index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The below code is just for model saving purpose as the submission has some time constraints attached to it","metadata":{}},{"cell_type":"code","source":"final_model.save(\"plant_pathology_2021.h5\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}