{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# What is this about?\nApples are one of the most important temperate fruit crops in the world. Foliar (leaf) diseases pose a major threat to the overall productivity and quality of apple orchards. The current process for disease diagnosis in apple orchards is based on manual scouting by humans, which is time-consuming and expensive. \n\nThe main objective of the competition is to develop machine learning-based models to accurately classify a given leaf image from the test dataset to a particular disease category, and to identify an individual disease from multiple disease symptoms on a single leaf image. ","metadata":{}},{"cell_type":"markdown","source":"# Import the necessary libraries","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport tensorflow as tf\nimport tensorflow.keras as keras\nimport PIL\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport os\nimport random\nfrom tqdm import tqdm\nimport tensorflow_addons as tfa\nimport random\nfrom sklearn.preprocessing import MultiLabelBinarizer\n\npd.set_option(\"display.max_columns\", None)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's explore the data.\nHow many images are in the datset, the labels and their frequencies.","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('../input/plant-pathology-2021-fgvc8/train.csv')\nprint(len(train))\nprint(train.columns)\n# print(train['labels'].value_counts())\nprint(train['labels'].value_counts().plot.bar())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Wait\nNotice something. Unlike the previous year's competition [Plant Pathology 2020 - FGVC7](https://www.kaggle.com/c/plant-pathology-2020-fgvc7) one image can belong to more than one disease. Notice how in case of multiple diseases the diseases are seperated by an space. So there are not 12 labels, its actually just 6 labels.\n5 dieseases: \n\n1. rust\n2. scab \n3. complex \n4. frog eye leaf spot\n5. powdery mildew \n\nand another label is \n\n6. healthy (healthy leaves) \n\nNow the most important thing is, as one image can have multiple diseases, that means this problem is **Multi label classification** problem. Many get confused betweeen multilabel and multiclass classification. if you are new to multilabel classification I would suggest going over this [An introduction to MultiLabel classification](https://www.geeksforgeeks.org/an-introduction-to-multilabel-classification/) . \n\nSo now we gotta process the labels. And then lets find out the actual frequencies of the labels. \n","metadata":{}},{"cell_type":"code","source":"train['labels'] = train['labels'].apply(lambda string: string.split(' '))\ntrain","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First I convert the labels representation into **one hot encoded format** using MultilabelBinarizer from sklearn. Now we can see and plot the frequencies of each label. ","metadata":{}},{"cell_type":"code","source":"s = list(train['labels'])\nmlb = MultiLabelBinarizer()\ntrainx = pd.DataFrame(mlb.fit_transform(s), columns=mlb.classes_, index=train.index)\nprint(trainx.columns)\nprint(trainx.sum())\n\nlabels = list(trainx.sum().keys())\nprint(labels)\nlabel_counts = trainx.sum().values.tolist()\n\nfig, ax = plt.subplots(1,1, figsize=(20,6))\n\nsns.barplot(x= labels, y= label_counts, ax=ax)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let's view some of the images","metadata":{}},{"cell_type":"code","source":"fig1 = plt.figure(figsize=(26,10))\n\nfor i in range(1, 13):\n    \n    rand =  random.randrange(1, 18000)\n    sample = os.path.join('../input/plant-pathology-2021-fgvc8/train_images/', train['image'][rand])\n    \n    img = PIL.Image.open(sample)\n    \n    ax = fig1.add_subplot(4,3,i)\n    ax.imshow(img)\n    \n    title = f\"{train['labels'][rand]}{img.size}\"\n    plt.title(title)\n    \n    fig1.tight_layout()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Imaze Size & Processing\nfrom the titles we can see some random image sizes - (4000, 2672). Larger images are harder to process hence takes much longer to train the CNN. Downsampling all these 18632 images is also a time consuming task. This is I am going to use the resized imaged for this dataset [resized-plant2021](https://www.kaggle.com/ankursingh12/resized-plant2021) by Ankur Singh. He has already downsampled the images into size of 256, 384, 512 & 640px.\n\nThere are 18632 images in the training set. Even after using the downsampled images we cant fit all of the images into memory at once. So I have used the flow_from_dataframe method from keras. This method reads images in batch size from the storage without loading all the images at once and saving us from **GPU Out of Memory (OOM)** issue. ","metadata":{}},{"cell_type":"code","source":"%%time\ndatagen = keras.preprocessing.image.ImageDataGenerator(rescale=1/255.0,\n                                                        preprocessing_function=None,\n                                                        data_format=None,\n                                                    )\n\ntrain_data = datagen.flow_from_dataframe(\n    train,\n    directory='../input/resized-plant2021/img_sz_512',\n    x_col=\"image\",\n    y_col= 'labels',\n    color_mode=\"rgb\",\n    target_size = (256,256),\n    class_mode=\"categorical\",\n    batch_size=32,\n    shuffle=False,\n    seed=40,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Transfer Learning\nTransfer learning is the process of using frozen weights from a large pre-trained model for a downstream task which is in our case classifying leaf diseases. As we can't use internet in this notebook, I will use the dataset of keras's pretrained models containing the weights of 'imagenet'. The output/top layer of a pretrained layer is a dense layer containing number of nodes = number of output classes. All the models here are pre-trained on 'imagenet' hence they have a output/top layer of 1000 nodes. We will have to replace the output/top layer with our own dense layer with 6 nodes (for 6 classes). \n\nI am going to be using **Inception ResNet v2**.\n","metadata":{}},{"cell_type":"code","source":"seed = 1200\ntf.random.set_seed(seed)\n\nweights_path = '../input/keras-pretrained-models/inception_resnet_v2_weights_tf_dim_ordering_tf_kernels_notop.h5'\nmodel = keras.applications.InceptionResNetV2(weights=weights_path, include_top=False, input_shape=(256, 256, 3))\n\nprint(model.input)\nprint(model.output)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Activation, Losses & Metrices\n\nAs this is a multilabel classification problem, we can't use softmax here, hence the sigmoid activation.\n\nBinary crossentropy is used instead of categorical crossentropy. We use categorical cross-entropy in multi-class problems, but for multi-label problems, we use binary cross-entropy. Think of it this way, an image may have multiple labels, and we need the probabilities that each of these labels corresponds to the given image - this can be considered as n independent binary classifiers for the n labels.\n\nFor evaluation I have used F1 accuracy metrics instead of binary accuracy. F1 and its variants are better for evaluation when it comes to multiclass and multilabel problems. if you want to know F1 score works for Multilabel classification go through this https://medium.com/synthesio-engineering/precision-accuracy-and-f1-score-for-multi-label-classification-34ac6bdfb404https://medium.com/synthesio-engineering/precision-accuracy-and-f1-score-for-multi-label-classification-34ac6bdfb404. ","metadata":{}},{"cell_type":"code","source":"new_model = tf.keras.Sequential([\n    model,\n    keras.layers.GlobalAveragePooling2D(),\n    keras.layers.Dense(6, \n        kernel_initializer=keras.initializers.RandomUniform(seed=seed),\n        bias_initializer=keras.initializers.Zeros(), name='dense_top', activation='sigmoid')\n])\n\n# Freezing the weights\nfor layer in new_model.layers[:-1]:\n    layer.trainable=False\n    \nnew_model.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f1 = tfa.metrics.F1Score(num_classes=6, average='macro')\n\ncallbacks = keras.callbacks.EarlyStopping(monitor=f1, patience=3, mode='max', restore_best_weights=True)\n\n\nnew_model.compile(loss=tf.keras.losses.BinaryCrossentropy(), optimizer=keras.optimizers.Adam(lr=1e-4), \n              metrics= [f1])\n\nnew_model.fit(train_data, epochs=40, callbacks=callbacks)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission\n\nFor submission I will resize the test images and then predict the labels for them.","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv('../input/plant-pathology-2021-fgvc8/sample_submission.csv')\n\nfor img_name in tqdm(test['image']):\n    path = '../input/plant-pathology-2021-fgvc8/test_images/'+str(img_name)\n    with PIL.Image.open(path) as img:\n        img = img.resize((256,256))\n        img.save(f'./{img_name}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data = datagen.flow_from_dataframe(\n    test,\n    directory = './',\n    x_col=\"image\",\n    y_col= None,\n    color_mode=\"rgb\",\n    target_size = (256,256),\n    classes=None,\n    class_mode=None,\n    batch_size=32,\n    shuffle=False,\n    seed=40,\n)\n\npreds = new_model.predict(test_data)\nprint(preds)\npreds = preds.tolist()\n\nindices = []\nfor pred in preds:\n    temp = []\n    for category in pred:\n        if category>=0.3:\n            temp.append(pred.index(category))\n    if temp!=[]:\n        indices.append(temp)\n    else:\n        temp.append(np.argmax(pred))\n        indices.append(temp)\n    \nprint(indices)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels = (train_data.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\nprint(labels)\n\ntestlabels = []\n\n\nfor image in indices:\n    temp = []\n    for i in image:\n        temp.append(str(labels[i]))\n    testlabels.append(' '.join(temp))\n\nprint(testlabels)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Remove the resized images from output before submission. if there are any other files present except 'submission.csv' it will throw an error when submitting.","metadata":{}},{"cell_type":"code","source":"delfiles = tf.io.gfile.glob('./*.jpg')\n\nfor file in delfiles:\n    os.remove(file)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv('../input/plant-pathology-2021-fgvc8/sample_submission.csv')\nsub['labels'] = testlabels\nsub.to_csv('submission.csv', index=False)\nsub","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}