{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction","metadata":{"id":"WbY308Alxq_W"}},{"cell_type":"markdown","source":"<center><img src=\"https://i.imgur.com/vSUSbDf.jpg\" width=\"500px\"></center>","metadata":{}},{"cell_type":"markdown","source":"Welcome to the \"Plant Pathology 2020 - FGVC7\" competition! In this competition, contestants are challenged to diagnose plant diseases solely based on leaf images. The categories include \"healthy\", \"scab\", \"rust\", and \"multiple diseases\". Solving this problem is important because diagnosing plant diseases early can save tonnes of agricultural produce every year. This will benefit not only the general population by reducing hunger, but also the farmers by ensuring they get the harvest they deserve.\n\nIn this kernel, I will visualize the data with Matplotlib and Plotly and then demonstrate some important image processing and augmentation techniques using OpenCV. Finally, I will show how different pretrained Keras models, such as DenseNet and EfficientNet, can be used to solve the problem.\n\n<font color=\"red\" size=3>Please upvote this kernel if you like it. It motivates me to produce more quality content :)</font>","metadata":{}},{"cell_type":"markdown","source":"To get started, here is an excellent video about how data scientists use TensorFlow to detect diseases in Cassava plants in Africa:","metadata":{"id":"1f3iYUa60vaG"}},{"cell_type":"code","source":"from IPython.display import HTML\nHTML('<center><iframe width=\"700\" height=\"400\" src=\"https://www.youtube.com/embed/NlpS-DhayQA?rel=0&amp;controls=0&amp;showinfo=0\" frameborder=\"0\" allowfullscreen></iframe></center>')","metadata":{"_kg_hide-input":true,"id":"1d4uT6551Il-","execution":{"iopub.status.busy":"2022-08-09T18:15:31.229395Z","iopub.execute_input":"2022-08-09T18:15:31.229980Z","iopub.status.idle":"2022-08-09T18:15:31.240317Z","shell.execute_reply.started":"2022-08-09T18:15:31.229935Z","shell.execute_reply":"2022-08-09T18:15:31.239335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Acknowledgements\n\n1. [OpenCV Docs ~ by OpenCV](https://docs.opencv.org/master/)\n2. [OpenCV Python Tutorials ~ by OpenCV](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html)\n3. [Plant Pathology: Very Concise TPU EfficientNet ~ by xhulu](https://www.kaggle.com/xhlulu/plant-pathology-very-concise-tpu-efficientnet)\n4. [Plotly Express in Python ~ by Plotly](https://plot.ly/python/plotly-express/)\n5. [EDA - Plant Pathology 2020 ~ by Peter](https://www.kaggle.com/pestipeti/eda-plant-pathology-2020)\n6. [Fork of Plant 2020 TPU 915e9c ~ by Alexander](https://www.kaggle.com/ateplyuk/fork-of-plant-2020-tpu-915e9c)","metadata":{"id":"unq0JDLC1d06"}},{"cell_type":"markdown","source":"# Contents\n\n* [<font size=4>EDA</font>](#1)\n    * [Preparing the ground](#1.1)\n    * [Visualize one leaf](#1.2)\n    * [Channel distributions](#1.3)\n    * [Visualize sample leaves](#1.4)\n    * [Visualize targets](#1.5)\n\n\n* [<font size=4>Image processing and augmentation</font>](#2)\n    * [Canny edge detection](#2.1)\n    * [Flipping](#2.2)\n    * [Convolution](#2.3)\n    * [Blurring](#2.4)\n  \n\n* [<font size=4>Modeling</font>](#3)\n    * [Preparing the ground](#3.1)\n    * [DenseNet](#3.2)\n    * [EfficientNet](#3.3)\n    * [EfficientNet NoisyStudent](#3.4)\n    * [Ensembling](#3.5)\n\n\n* [<font size=4>Takeaways</font>](#4)\n\n\n* [<font size=4>Ending note</font>](#5)","metadata":{"id":"UKO5BDl72a3B"}},{"cell_type":"markdown","source":"# EDA <a id=\"1\"></a>","metadata":{"id":"wRNLR-MQ2i90"}},{"cell_type":"markdown","source":"## Preparing the ground <a id=\"1.1\"></a>","metadata":{"id":"ufcI1cah2o_T"}},{"cell_type":"markdown","source":"### Install and import necessary libraries","metadata":{"id":"1XvcaAzr2rjY"}},{"cell_type":"code","source":"!pip install -q efficientnet\n!pip install keras\n!pip install -U pip\n!pip install -U setuptools\n!pip install --upgrade tensorflow","metadata":{"id":"V8TgyWRwpnKH","execution":{"iopub.status.busy":"2022-08-09T18:15:31.261156Z","iopub.execute_input":"2022-08-09T18:15:31.261454Z","iopub.status.idle":"2022-08-09T18:16:18.128348Z","shell.execute_reply.started":"2022-08-09T18:15:31.261426Z","shell.execute_reply":"2022-08-09T18:16:18.127399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport gc\nimport re\n\nimport cv2\nimport math\nimport numpy as np\nimport scipy as sp\nimport pandas as pd\n\nimport tensorflow as tf\nfrom IPython.display import SVG\nimport efficientnet.tfkeras as efn\nfrom tensorflow.keras.utils import plot_model\nimport tensorflow.keras.layers as L\nfrom tensorflow.keras.utils import model_to_dot\nimport tensorflow.keras.backend as K\nfrom tensorflow.keras.models import Model\nfrom kaggle_datasets import KaggleDatasets\nfrom tensorflow.keras.applications import DenseNet121\n\nimport seaborn as sns\nfrom tqdm import tqdm\nimport matplotlib.cm as cm\nfrom sklearn import metrics\nimport matplotlib.pyplot as plt\nfrom sklearn.utils import shuffle\nfrom sklearn.model_selection import train_test_split\n\ntqdm.pandas()\nimport plotly.express as px\nimport plotly.graph_objects as go\nimport plotly.figure_factory as ff\nfrom plotly.subplots import make_subplots\n\nnp.random.seed(0)\ntf.random.set_seed(0)\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","id":"y4ElXcLopnKO","outputId":"72756d56-48f2-46c7-d8d0-56ec9fc395ce","execution":{"iopub.status.busy":"2022-08-09T18:16:18.130737Z","iopub.execute_input":"2022-08-09T18:16:18.131050Z","iopub.status.idle":"2022-08-09T18:16:18.143230Z","shell.execute_reply.started":"2022-08-09T18:16:18.131001Z","shell.execute_reply":"2022-08-09T18:16:18.142259Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load the data and define hyperparameters","metadata":{"id":"DyZKLcDg2yRi"}},{"cell_type":"code","source":"EPOCHS = 20\nSAMPLE_LEN = 100\nIMAGE_PATH = \"../input/plant-pathology-2020-fgvc7/images/\"\nTEST_PATH = \"../input/plant-pathology-2020-fgvc7/test.csv\"\nTRAIN_PATH = \"../input/plant-pathology-2020-fgvc7/train.csv\"\nSUB_PATH = \"../input/plant-pathology-2020-fgvc7/sample_submission.csv\"\n\nsub = pd.read_csv(SUB_PATH)\ntest_data = pd.read_csv(TEST_PATH)\ntrain_data = pd.read_csv(TRAIN_PATH)","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","id":"mneU8D9bpnKS","execution":{"iopub.status.busy":"2022-08-09T18:16:18.144584Z","iopub.execute_input":"2022-08-09T18:16:18.145156Z","iopub.status.idle":"2022-08-09T18:16:18.198700Z","shell.execute_reply.started":"2022-08-09T18:16:18.145116Z","shell.execute_reply":"2022-08-09T18:16:18.197667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data.head()","metadata":{"id":"lSXP0xubpnKW","outputId":"80d177b3-cf7d-4088-90ff-149edfb86295","execution":{"iopub.status.busy":"2022-08-09T18:16:18.201963Z","iopub.execute_input":"2022-08-09T18:16:18.202575Z","iopub.status.idle":"2022-08-09T18:16:18.221808Z","shell.execute_reply.started":"2022-08-09T18:16:18.202528Z","shell.execute_reply":"2022-08-09T18:16:18.220713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data.head()","metadata":{"id":"XISwCeBjpnKa","outputId":"61f70add-a4df-452e-feee-f0ffe0dfac14","execution":{"iopub.status.busy":"2022-08-09T18:16:18.222929Z","iopub.execute_input":"2022-08-09T18:16:18.223201Z","iopub.status.idle":"2022-08-09T18:16:18.234106Z","shell.execute_reply.started":"2022-08-09T18:16:18.223144Z","shell.execute_reply":"2022-08-09T18:16:18.233085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load sample images","metadata":{"id":"O6DgMHJz293K"}},{"cell_type":"code","source":"def load_image(image_id):\n    file_path = image_id + \".jpg\"\n    image = cv2.imread(IMAGE_PATH + file_path)\n    return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\ntrain_images = train_data[\"image_id\"][:SAMPLE_LEN].progress_apply(load_image)","metadata":{"id":"uJD4JbtHpnKd","outputId":"afb14520-4a05-4f26-e7bc-96a86b3f478c","execution":{"iopub.status.busy":"2022-08-09T18:16:18.236157Z","iopub.execute_input":"2022-08-09T18:16:18.236843Z","iopub.status.idle":"2022-08-09T18:16:22.911377Z","shell.execute_reply.started":"2022-08-09T18:16:18.236799Z","shell.execute_reply":"2022-08-09T18:16:22.910270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize one leaf <a id=\"1.2\"></a>","metadata":{"id":"aI1QC_K-3HjL"}},{"cell_type":"markdown","source":"### Sample image","metadata":{"id":"UY_1XspN3Afg"}},{"cell_type":"code","source":"fig = px.imshow(cv2.resize(train_images[0], (205, 136)))\nfig.show()","metadata":{"_kg_hide-input":true,"id":"LNXpowwwpnKg","outputId":"7a677746-3d36-4b54-d201-63474cefb493","execution":{"iopub.status.busy":"2022-08-09T18:16:22.914685Z","iopub.execute_input":"2022-08-09T18:16:22.915014Z","iopub.status.idle":"2022-08-09T18:16:23.658138Z","shell.execute_reply.started":"2022-08-09T18:16:22.914980Z","shell.execute_reply":"2022-08-09T18:16:23.657020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I have plotted the first image in the training data above (the RGB values can be seen by hovering over the image). The green parts of the image have very low blue values, but by contrast, the brown parts have high blue values. This suggests that green (healthy) parts of the image have low blue values, whereas unhealthy parts are more likely to have high blue values. **This might suggest that the blue channel may be the key to detecting diseases in plants.**","metadata":{"id":"6iwEoQ90WnnA"}},{"cell_type":"markdown","source":"## Channel distributions <a id=\"1.3\"></a>","metadata":{"id":"JGNq3QwJ3vs9"}},{"cell_type":"code","source":"red_values = [np.mean(train_images[idx][:, :, 0]) for idx in range(len(train_images))]\ngreen_values = [np.mean(train_images[idx][:, :, 1]) for idx in range(len(train_images))]\nblue_values = [np.mean(train_images[idx][:, :, 2]) for idx in range(len(train_images))]\nvalues = [np.mean(train_images[idx]) for idx in range(len(train_images))]","metadata":{"_kg_hide-input":true,"id":"SKlfmsiR3yVn","execution":{"iopub.status.busy":"2022-08-09T18:16:23.659323Z","iopub.execute_input":"2022-08-09T18:16:23.659542Z","iopub.status.idle":"2022-08-09T18:16:25.387355Z","shell.execute_reply.started":"2022-08-09T18:16:23.659518Z","shell.execute_reply":"2022-08-09T18:16:25.386350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### All channel values","metadata":{"id":"cluIike23zBw"}},{"cell_type":"code","source":"fig = ff.create_distplot([values], group_labels=[\"Channels\"], colors=[\"purple\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribution of channel values\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"Eb32WiwY34At","execution":{"iopub.status.busy":"2022-08-09T18:16:25.388889Z","iopub.execute_input":"2022-08-09T18:16:25.389243Z","iopub.status.idle":"2022-08-09T18:16:25.538733Z","shell.execute_reply.started":"2022-08-09T18:16:25.389194Z","shell.execute_reply":"2022-08-09T18:16:25.537808Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The channel values seem to have a roughly normal distribution centered around 105. The maximum channel activation is 255. This means that the average channel value is less than half the maximum value, which indicates that channels are minimally activated most of the time.","metadata":{"id":"Xr1kMIFtVAB_"}},{"cell_type":"markdown","source":"### Red channel values","metadata":{"id":"utld_a6W37SB"}},{"cell_type":"code","source":"fig = ff.create_distplot([red_values], group_labels=[\"R\"], colors=[\"red\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribution of red channel values\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"jDIqxHgj3_zj","execution":{"iopub.status.busy":"2022-08-09T18:16:25.541308Z","iopub.execute_input":"2022-08-09T18:16:25.541554Z","iopub.status.idle":"2022-08-09T18:16:25.592968Z","shell.execute_reply.started":"2022-08-09T18:16:25.541527Z","shell.execute_reply":"2022-08-09T18:16:25.592100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The red channel values seem to roughly normal distribution, but with a slight rightward (positive skew). This indicates that the red channel tends to be more concentrated at lower values, at around 100. There is large variation in average red values across images.","metadata":{"id":"r6HLx5fYVDIi"}},{"cell_type":"markdown","source":"### Green channel values","metadata":{"id":"7Gr6JXgg4QAo"}},{"cell_type":"code","source":"fig = ff.create_distplot([green_values], group_labels=[\"G\"], colors=[\"green\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribution of green channel values\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"75DRD2Jy4Sp-","execution":{"iopub.status.busy":"2022-08-09T18:16:25.594127Z","iopub.execute_input":"2022-08-09T18:16:25.594459Z","iopub.status.idle":"2022-08-09T18:16:25.646762Z","shell.execute_reply.started":"2022-08-09T18:16:25.594420Z","shell.execute_reply":"2022-08-09T18:16:25.645781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The green channel values have a more uniform distribution than the red channel values, with a smaller peak. The distribution also has a leftward skew (in contrast to red) and a larger mode of around 140. This indicates that green is more pronounced in these images than red, which makes sense, because these are images of leaves!","metadata":{"id":"nkpWr3JTVTQe"}},{"cell_type":"markdown","source":"### Blue channel values","metadata":{"id":"Dqs2zNPC4ZkG"}},{"cell_type":"code","source":"fig = ff.create_distplot([blue_values], group_labels=[\"B\"], colors=[\"blue\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribution of blue channel values\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"03Sp06S54bpy","execution":{"iopub.status.busy":"2022-08-09T18:16:25.648025Z","iopub.execute_input":"2022-08-09T18:16:25.648270Z","iopub.status.idle":"2022-08-09T18:16:25.699819Z","shell.execute_reply.started":"2022-08-09T18:16:25.648242Z","shell.execute_reply":"2022-08-09T18:16:25.698972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The blue channel has the most uniform distribution out of the three color channels, with minimal skew (slight leftward skew). The blue channel shows great variation across images in the dataset.","metadata":{"id":"L7dmewbgV4Cg"}},{"cell_type":"markdown","source":"### All channel values (together)","metadata":{"id":"jjFXtVFM4l0w"}},{"cell_type":"code","source":"fig = go.Figure()\n\nfor idx, values in enumerate([red_values, green_values, blue_values]):\n    if idx == 0:\n        color = \"Red\"\n    if idx == 1:\n        color = \"Green\"\n    if idx == 2:\n        color = \"Blue\"\n    fig.add_trace(go.Box(x=[color]*len(values), y=values, name=color, marker=dict(color=color.lower())))\n    \nfig.update_layout(yaxis_title=\"Mean value\", xaxis_title=\"Color channel\",\n                  title=\"Mean value vs. Color channel\", template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"id":"iUJeF2Ae4oUl","execution":{"iopub.status.busy":"2022-08-09T18:16:25.701309Z","iopub.execute_input":"2022-08-09T18:16:25.701626Z","iopub.status.idle":"2022-08-09T18:16:25.758863Z","shell.execute_reply.started":"2022-08-09T18:16:25.701584Z","shell.execute_reply":"2022-08-09T18:16:25.758258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = ff.create_distplot([red_values, green_values, blue_values],\n                         group_labels=[\"R\", \"G\", \"B\"],\n                         colors=[\"red\", \"green\", \"blue\"])\nfig.update_layout(title_text=\"Distribution of red channel values\", template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig.data[2].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[2].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"2x8aqxw04o8u","execution":{"iopub.status.busy":"2022-08-09T18:16:25.759819Z","iopub.execute_input":"2022-08-09T18:16:25.760051Z","iopub.status.idle":"2022-08-09T18:16:25.833217Z","shell.execute_reply.started":"2022-08-09T18:16:25.760025Z","shell.execute_reply":"2022-08-09T18:16:25.832431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the above plots, we can clearly see which colors are more common and which ones less common in the leaf images. Green is the most pronounced color, followed by red and blue respectively. The distributions, when plotted together, appear to have a similar shape, but shifted horizontally. ","metadata":{"id":"7tUmWvzYWMSs"}},{"cell_type":"markdown","source":"## Visualize sample leaves <a id=\"1.4\"></a>\n\nNow, I will visualize sample leaves beloning to different categories in the dataset.","metadata":{"id":"Ee2qutMw3FTP"}},{"cell_type":"code","source":"def visualize_leaves(cond=[0, 0, 0, 0], cond_cols=[\"healthy\"], is_cond=True):\n    if not is_cond:\n        cols, rows = 3, min([3, len(train_images)//3])\n        fig, ax = plt.subplots(nrows=rows, ncols=cols, figsize=(30, rows*20/3))\n        for col in range(cols):\n            for row in range(rows):\n                ax[row, col].imshow(train_images.loc[train_images.index[-row*3-col-1]])\n        return None\n        \n    cond_0 = \"healthy == {}\".format(cond[0])\n    cond_1 = \"scab == {}\".format(cond[1])\n    cond_2 = \"rust == {}\".format(cond[2])\n    cond_3 = \"multiple_diseases == {}\".format(cond[3])\n    \n    cond_list = []\n    for col in cond_cols:\n        if col == \"healthy\":\n            cond_list.append(cond_0)\n        if col == \"scab\":\n            cond_list.append(cond_1)\n        if col == \"rust\":\n            cond_list.append(cond_2)\n        if col == \"multiple_diseases\":\n            cond_list.append(cond_3)\n    \n    data = train_data.loc[:99]\n    for cond in cond_list:\n        data = data.query(cond)\n        \n    images = train_images.loc[list(data.index)]\n    cols, rows = 3, min([3, len(images)//3])\n    \n    fig, ax = plt.subplots(nrows=rows, ncols=cols, figsize=(30, rows*20/3))\n    for col in range(cols):\n        for row in range(rows):\n            ax[row, col].imshow(images.loc[images.index[row*3+col]])\n    plt.show()","metadata":{"_kg_hide-input":true,"id":"mQagns98pnKt","execution":{"iopub.status.busy":"2022-08-09T18:27:01.217597Z","iopub.execute_input":"2022-08-09T18:27:01.218282Z","iopub.status.idle":"2022-08-09T18:27:01.230726Z","shell.execute_reply.started":"2022-08-09T18:27:01.218232Z","shell.execute_reply":"2022-08-09T18:27:01.229887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Healthy leaves","metadata":{"id":"6YBTzqs53VHW"}},{"cell_type":"code","source":"visualize_leaves(cond=[1, 0, 0, 0], cond_cols=[\"healthy\"])","metadata":{"_kg_hide-input":true,"id":"_Gw1CH7bpnLc","outputId":"81b8d1ed-231f-4008-9508-23df527bbc80","execution":{"iopub.status.busy":"2022-08-09T18:27:02.266695Z","iopub.execute_input":"2022-08-09T18:27:02.267154Z","iopub.status.idle":"2022-08-09T18:27:07.222458Z","shell.execute_reply.started":"2022-08-09T18:27:02.267120Z","shell.execute_reply":"2022-08-09T18:27:07.221597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the above images, we can see that the healthy leaves are completely green, do not have any brown/yellow spots or scars. Healthy leaves do not have scab or rust. ","metadata":{"id":"v7T9CAFoXt9A"}},{"cell_type":"markdown","source":"### Leaves with scab","metadata":{"id":"lfWbbGfD5ACD"}},{"cell_type":"code","source":"visualize_leaves(cond=[0, 1, 0, 0], cond_cols=[\"scab\"])","metadata":{"_kg_hide-input":true,"id":"cWTEw_JZpnLj","outputId":"d34b44a9-f00b-4942-bd19-c055347fb186","execution":{"iopub.status.busy":"2022-08-09T18:24:38.283504Z","iopub.execute_input":"2022-08-09T18:24:38.283870Z","iopub.status.idle":"2022-08-09T18:24:43.307857Z","shell.execute_reply.started":"2022-08-09T18:24:38.283834Z","shell.execute_reply":"2022-08-09T18:24:43.306372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the above images, we can see that leaves with \"scab\" have large brown marks and stains across the leaf. Scab is defined as \"any of various plant diseases caused by fungi or bacteria and resulting in crustlike spots on fruit, leaves, or roots. The spots caused by such a disease\". The brown marks across the leaf are a sign of these bacterial/fungal infections. Once diagnosed, scab can be treated using chemical or non-chemical methods.","metadata":{"id":"qHR_5GsKYQWz"}},{"cell_type":"markdown","source":"### Leaves with rust","metadata":{"id":"1doTwr2S5Dxt"}},{"cell_type":"code","source":"visualize_leaves(cond=[0, 0, 1, 0], cond_cols=[\"rust\"])","metadata":{"_kg_hide-input":true,"id":"Xwn6GVaupnLl","outputId":"e109052f-3742-4212-c3bd-ec8fa82d2f0c","execution":{"iopub.status.busy":"2022-08-09T18:27:34.269776Z","iopub.execute_input":"2022-08-09T18:27:34.270368Z","iopub.status.idle":"2022-08-09T18:27:39.207914Z","shell.execute_reply.started":"2022-08-09T18:27:34.270323Z","shell.execute_reply":"2022-08-09T18:27:39.207104Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the above images, we can see that leaves with \"rust\" have several brownish-yellow spots across the leaf. Rust is defined as \"a disease, especially of cereals and other grasses, characterized by rust-colored pustules of spores on the affected leaf blades and sheaths and caused by any of several rust fungi\". The yellow spots are a sign of infection by a special type of fungi called \"rust fungi\". Rust can also be treated with several chemical and non-chemical methods once diagnosed.","metadata":{"id":"l_0g20pmZk17"}},{"cell_type":"markdown","source":"### Leaves with multiple diseases","metadata":{"id":"mKALD2vt5G78"}},{"cell_type":"code","source":"visualize_leaves(cond=[0, 0, 0, 1], cond_cols=[\"multiple_diseases\"])","metadata":{"_kg_hide-input":true,"id":"YQm_lj9TpnLo","outputId":"a0568b08-2df5-4275-ce41-1a461d8e0ebd","execution":{"iopub.status.busy":"2022-08-09T18:27:43.782459Z","iopub.execute_input":"2022-08-09T18:27:43.782906Z","iopub.status.idle":"2022-08-09T18:27:47.200620Z","shell.execute_reply.started":"2022-08-09T18:27:43.782873Z","shell.execute_reply":"2022-08-09T18:27:47.199565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the above images, we can see that the leaves show symptoms for several diseases, including brown marks and yellow spots. These plants have more than one of the above-described diseases.","metadata":{"id":"qyG-NpVDahDN"}},{"cell_type":"markdown","source":"## Visualize targets <a id=\"1.5\"></a>\n\nNow, I will visualize the labels and target data. **In all the below plots, blue represents the \"desired\" or \"healthy\" condition, and red represents the \"undesired\" or \"unhealthy\" condition.**","metadata":{"id":"yTFIf7dZ5MNe"}},{"cell_type":"markdown","source":"### All labels together (parallel plot)","metadata":{"id":"rTU7Plrb5R2v"}},{"cell_type":"code","source":"fig = px.parallel_categories(train_data[[\"healthy\", \"scab\", \"rust\", \"multiple_diseases\"]], color=\"healthy\", color_continuous_scale=\"sunset\",\\\n                             title=\"Parallel categories plot of targets\")\nfig","metadata":{"_kg_hide-input":true,"id":"vRDHR4G_pnLs","outputId":"7f935b25-2686-447a-bd57-8cd2429b15d2","execution":{"iopub.status.busy":"2022-08-09T18:28:17.045953Z","iopub.execute_input":"2022-08-09T18:28:17.046308Z","iopub.status.idle":"2022-08-09T18:28:17.147763Z","shell.execute_reply.started":"2022-08-09T18:28:17.046262Z","shell.execute_reply":"2022-08-09T18:28:17.147151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the above plot, we can see the relationship between all four categories. As expected, it is impossible for a healthy leaf (<code>healthy == 1</code>) to have scab, rust, or multiple diseases. Also, every unhealthy leaf has one of either scab, rust, or multiple diseases. The frequency of each combination can be seen by hovering over the plot.","metadata":{"id":"POYx63MXcfcQ"}},{"cell_type":"markdown","source":"### Pie chart","metadata":{}},{"cell_type":"code","source":"fig = go.Figure([go.Pie(labels=train_data.columns[1:],\n           values=train_data.iloc[:, 1:].sum().values)])\nfig.update_layout(title_text=\"Pie chart of targets\", template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:28:18.093793Z","iopub.execute_input":"2022-08-09T18:28:18.094833Z","iopub.status.idle":"2022-08-09T18:28:18.148910Z","shell.execute_reply.started":"2022-08-09T18:28:18.094786Z","shell.execute_reply":"2022-08-09T18:28:18.148057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the pie chart above, we can see that most leaves in the dataset are unhealthy (71.7%). Only 5% of plants have multiple diseases, and \"rust\" and \"scab\" occupy approximately one-third of the pie each.","metadata":{}},{"cell_type":"markdown","source":"### Healthy distribution","metadata":{"id":"kJNU54Yy5WST"}},{"cell_type":"code","source":"train_data[\"Healthy\"] = train_data[\"healthy\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Healthy\", title=\"Healthy distribution\", color=\"Healthy\",\\\n            color_discrete_map={\n                \"True\": px.colors.qualitative.Plotly[0],\n                \"False\": px.colors.qualitative.Plotly[1]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"lriYl3yXpnLv","outputId":"08bbe74b-5853-41e2-824e-e58adecfcb6b","execution":{"iopub.status.busy":"2022-08-09T18:28:18.751800Z","iopub.execute_input":"2022-08-09T18:28:18.752474Z","iopub.status.idle":"2022-08-09T18:28:18.892455Z","shell.execute_reply.started":"2022-08-09T18:28:18.752413Z","shell.execute_reply":"2022-08-09T18:28:18.891557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that there are more unhealthy (<code>healthy == 0</code>) plants than healthy (<code>healthy == 1</code>) ones. There are 1305 (72%) unhealthy plants and 516 (28%) healthy plants.","metadata":{"id":"qewm9cFSes3y"}},{"cell_type":"markdown","source":"### Scab distribution","metadata":{"id":"wDb-wmeA5ZMa"}},{"cell_type":"code","source":"train_data[\"Scab\"] = train_data[\"scab\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Scab\", color=\"Scab\", title=\"Scab distribution\",\\\n            color_discrete_map={\n                \"True\": px.colors.qualitative.Plotly[1],\n                \"False\": px.colors.qualitative.Plotly[0]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"AL3UmfaZpnLy","outputId":"61a61499-c8d0-41f5-af0c-8b0dd4bd9b93","execution":{"iopub.status.busy":"2022-08-09T18:28:19.536821Z","iopub.execute_input":"2022-08-09T18:28:19.537137Z","iopub.status.idle":"2022-08-09T18:28:19.647984Z","shell.execute_reply.started":"2022-08-09T18:28:19.537099Z","shell.execute_reply":"2022-08-09T18:28:19.646890Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that there are more plants without scab (<code>scab == 0</code>) than those with scab (<code>scab == 1</code>). There are 592 (33%) unhealthy plants and 1229 (67%) healthy plants. ","metadata":{"id":"Q3s0XzhwfOc-"}},{"cell_type":"markdown","source":"### Rust distribution","metadata":{"id":"fJ0g_MRy5bQb"}},{"cell_type":"code","source":"train_data[\"Rust\"] = train_data[\"rust\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Rust\", color=\"Rust\", title=\"Rust distribution\",\\\n            color_discrete_map={\n                \"True\": px.colors.qualitative.Plotly[1],\n                \"False\": px.colors.qualitative.Plotly[0]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"EamjMT0JpnL0","outputId":"4d6633ec-c3ff-409f-aa0f-90cb7a690fc0","execution":{"iopub.status.busy":"2022-08-09T18:28:20.219127Z","iopub.execute_input":"2022-08-09T18:28:20.219478Z","iopub.status.idle":"2022-08-09T18:28:20.330085Z","shell.execute_reply.started":"2022-08-09T18:28:20.219444Z","shell.execute_reply":"2022-08-09T18:28:20.329258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that there are more plants without rust (<code>rust == 0</code>) than those with rust (<code>rust == 1</code>). There are 622 (34%) unhealthy plants and 1199 (66%) healthy plants. We can see that the \"unhealthy\" percentage is very similar for both rust and scab.","metadata":{"id":"7EWabO690nSX"}},{"cell_type":"markdown","source":"### Multiple diseases distribution","metadata":{"id":"S0t3SEpB5dcS"}},{"cell_type":"code","source":"train_data[\"Multiple diseases\"] = train_data[\"multiple_diseases\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Multiple diseases\", color=\"Multiple diseases\", title=\"Multiple diseases distribution\",\\\n            color_discrete_map={\n                \"True\": px.colors.qualitative.Plotly[1],\n                \"False\": px.colors.qualitative.Plotly[0]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"9gYSWOl0pnL3","outputId":"29b0bdaf-d509-4436-a98a-628f1818341f","execution":{"iopub.status.busy":"2022-08-09T18:28:20.977849Z","iopub.execute_input":"2022-08-09T18:28:20.978216Z","iopub.status.idle":"2022-08-09T18:28:21.090527Z","shell.execute_reply.started":"2022-08-09T18:28:20.978151Z","shell.execute_reply":"2022-08-09T18:28:21.089598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that very few leaves have multiple diseases (it is a very occurance). There are 91 (5%) unhealthy plants and 1730 (95%) healthy plants.","metadata":{"id":"GDgVIT2p2YrZ"}},{"cell_type":"markdown","source":"# Image processing and augmentation <a id=\"2\"></a>","metadata":{"id":"aiPKQDkh5hF5"}},{"cell_type":"markdown","source":"## Canny edge detection <a id=\"2.1\"></a>\n\nCanny is a popular edge detection algorithm, and as the name suggests, it detects the edges of objects present in an image. It was developed by John F. Canny in 1986. The algorithm involves several steps.\n\n1. **Noise reduction:** Since edge detection is susceptible to noise in an image, we remove the noise in the image using a 5x5 Gaussian filter.\n\n\n2. **Finding Intensity Gradient of the Image**: The smoothened image is then filtered with a Sobel kernel in both horizontal and vertical directions to get the first derivative in the horizontal (*G<sub>x</sub>*) and vertical (*G<sub>y</sub>*) directions. From these two images, one can find the edge gradient and direction for each pixel:\n\n<center><img src=\"https://i.imgur.com/ntyjTep.png\" width=\"300px\"></center>\n<center><img src=\"https://i.imgur.com/75qDjv6.png\" width=\"260px\"></center>\n\n<br>\n\n3. **Rounding:** The gradient is always perpendicular to edges. So, it is rounded to one of the four angles representing vertical, horizontal and two diagonal directions.\n\n4. **Non-maximum suppression:** After getting the gradient magnitude and direction, a full scan of the image is done to remove any unwanted pixels which may not constitute the edge. For this, we check every pixel for being a local maximum in its neighborhood in the direction of the gradient.\n\n5. **Hysteresis Thresholding:** This stage decides which parts are edges and which are not. For this, we need two threshold values, *minVal* and *maxVal*. Any edges with intensity gradient greater than *maxVal* are considered edges and those lesser than *minVal* are considered non-edges, and discarded. Those who lie between these two thresholds are classified edges or non-edges based on their neighborhood. If they are near “sure-edge” pixels, they are considered edges, and otherwise, they are discarded.\n\nThe result of these five steps is a two-dimensional binary map (0 or 255) indicating the location of edges on the image. Canny edge is demonstrated below with a few leaf images:","metadata":{"id":"jC9wp-8N5kU4"}},{"cell_type":"markdown","source":"\n","metadata":{}},{"cell_type":"code","source":"def edge_and_cut(img):\n    emb_img = img.copy()\n    edges = cv2.Canny(img, 100, 200)\n    edge_coors = []\n    for i in range(edges.shape[0]):\n        for j in range(edges.shape[1]):\n            if edges[i][j] != 0:\n                edge_coors.append((i, j))\n    \n    row_min = edge_coors[np.argsort([coor[0] for coor in edge_coors])[0]][0]\n    row_max = edge_coors[np.argsort([coor[0] for coor in edge_coors])[-1]][0]\n    col_min = edge_coors[np.argsort([coor[1] for coor in edge_coors])[0]][1]\n    col_max = edge_coors[np.argsort([coor[1] for coor in edge_coors])[-1]][1]\n    new_img = img[row_min:row_max, col_min:col_max]\n    \n    emb_img[row_min-10:row_min+10, col_min:col_max] = [255, 0, 0]\n    emb_img[row_max-10:row_max+10, col_min:col_max] = [255, 0, 0]\n    emb_img[row_min:row_max, col_min-10:col_min+10] = [255, 0, 0]\n    emb_img[row_min:row_max, col_max-10:col_max+10] = [255, 0, 0]\n    \n    fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(30, 20))\n    ax[0].imshow(img, cmap='gray')\n    ax[0].set_title('Original Image', fontsize=24)\n    ax[1].imshow(edges, cmap='gray')\n    ax[1].set_title('Canny Edges', fontsize=24)\n    ax[2].imshow(emb_img, cmap='gray')\n    ax[2].set_title('Bounding Box', fontsize=24)\n    plt.show()","metadata":{"_kg_hide-input":true,"id":"uU_iqYaCpnL7","execution":{"iopub.status.busy":"2022-08-09T18:28:22.288749Z","iopub.execute_input":"2022-08-09T18:28:22.289269Z","iopub.status.idle":"2022-08-09T18:28:22.302626Z","shell.execute_reply.started":"2022-08-09T18:28:22.289210Z","shell.execute_reply":"2022-08-09T18:28:22.301894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"edge_and_cut(train_images[3])\nedge_and_cut(train_images[4])\nedge_and_cut(train_images[5])","metadata":{"_kg_hide-input":true,"id":"bYJ0t9kppnL9","outputId":"a2785c32-9151-4d8d-f837-ec9de9de49bf","execution":{"iopub.status.busy":"2022-08-09T18:28:22.694032Z","iopub.execute_input":"2022-08-09T18:28:22.694365Z","iopub.status.idle":"2022-08-09T18:28:43.042504Z","shell.execute_reply.started":"2022-08-09T18:28:22.694330Z","shell.execute_reply":"2022-08-09T18:28:43.041576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The second column of images above contains the Canny edges and the third column contains cropped images. I have taken the Canny edges and used it to predict a bounding box in which the actual leaf is contained. The most extreme edges at the four corners of the image are the vertices of the bounding box. This red box is likely to contain most of if not all of the leaf. These edges and bounding boxes can be used to build more accurate models.","metadata":{}},{"cell_type":"markdown","source":"## Flipping <a id=\"2.2\"></a>\n\nFlipping is a simple transformation that involves index-switching on the image channels. In vertical flipping, the order of rows is exchanged, whereas in vertical flipping, the order of rows is exchanged. Let us assume that *A<sub>ijk</sub>* (of size *(m, n, 3)*) is the image we want to flip. Horizontal and vertical flipping can be represented by the transformations below:\n\n<center><img src=\"https://i.imgur.com/B9y5apl.png\" width=\"135px\"></center>\n<center><img src=\"https://i.imgur.com/eQ1dyvN.png\" width=\"305px\"></center>\n<center><img src=\"https://i.imgur.com/i30LQgq.png\" width=\"305px\"></center>\n<br>\n\nWe can see that the order of columns is exchanged in horizontal flipping. While the *i* and *k* indices remain the same, the *j* index reverses. Whereas, in vertical flipping, the order of rows is exchanged in horizontal flipping. While the *j* and *k* indices remain the same, the *i* index reverses.\n\n","metadata":{"id":"Wh0WHTCC5sL_"}},{"cell_type":"code","source":"def invert(img):\n    fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(30, 20))\n    ax[0].imshow(img)\n    ax[0].set_title('Original Image', fontsize=24)\n    ax[1].imshow(cv2.flip(img, 0))\n    ax[1].set_title('Vertical Flip', fontsize=24)\n    ax[2].imshow(cv2.flip(img, 1))\n    ax[2].set_title('Horizontal Flip', fontsize=24)\n    plt.show()","metadata":{"_kg_hide-input":true,"id":"XPZwEZAepnMA","execution":{"iopub.status.busy":"2022-08-09T18:28:43.044461Z","iopub.execute_input":"2022-08-09T18:28:43.045331Z","iopub.status.idle":"2022-08-09T18:28:43.051833Z","shell.execute_reply.started":"2022-08-09T18:28:43.045293Z","shell.execute_reply":"2022-08-09T18:28:43.051141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"invert(train_images[3])\ninvert(train_images[4])\ninvert(train_images[5])","metadata":{"_kg_hide-input":true,"id":"gaiLRUzopnMD","outputId":"dc8f2538-abb7-4449-f0c9-dedd8a393843","execution":{"iopub.status.busy":"2022-08-09T18:28:43.053058Z","iopub.execute_input":"2022-08-09T18:28:43.053501Z","iopub.status.idle":"2022-08-09T18:28:48.487644Z","shell.execute_reply.started":"2022-08-09T18:28:43.053470Z","shell.execute_reply":"2022-08-09T18:28:48.486893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that the images are simply flipped. All major features in the image remain the same, but to a computer algorithm, the flipped images look completely different. These transformations can be used for data augmentation, making models more robust and accurate.","metadata":{}},{"cell_type":"markdown","source":"## Convolution <a id=\"2.3\"></a>\n\nConvolution is a rather simple algorithm which involves a kernel (a 2D matrix) which moves over the entire image, calculating dot products with each window along the way. The GIF below demonstrates convolution in action.\n\n<center><img src=\"https://i.imgur.com/wYUaqR3.gif\" width=\"450px\"></center>\n\nThe above process can be summarized with an equation, where *f* is the image and *h* is the kernel. The dimensions of *f* are *(m, n)* and the kernel is a square matrix with dimensions smaller than *f*:\n\n<center><img src=\"https://i.imgur.com/9scTOGv.png\" width=\"350px\"></center>\n<br>\n\nIn the above equation, the kernel *h* is moving across the length and breadth of the image. The dot product of *h* with a sub-matrix or window of matrix *f* is taken at each step, hence the double summation (rows and columns). Below I demonstrate the effect of convolution on leaf images.","metadata":{"id":"PqS2I93A5u_R"}},{"cell_type":"code","source":"def conv(img):\n    fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(20, 20))\n    kernel = np.ones((7, 7), np.float32)/25\n    conv = cv2.filter2D(img, -1, kernel)\n    ax[0].imshow(img)\n    ax[0].set_title('Original Image', fontsize=24)\n    ax[1].imshow(conv)\n    ax[1].set_title('Convolved Image', fontsize=24)\n    plt.show()","metadata":{"_kg_hide-input":true,"id":"aa81abmWpnMG","execution":{"iopub.status.busy":"2022-08-09T18:28:48.489426Z","iopub.execute_input":"2022-08-09T18:28:48.490350Z","iopub.status.idle":"2022-08-09T18:28:48.497002Z","shell.execute_reply.started":"2022-08-09T18:28:48.490307Z","shell.execute_reply":"2022-08-09T18:28:48.495979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv(train_images[3])\nconv(train_images[4])\nconv(train_images[5])","metadata":{"_kg_hide-input":true,"id":"2rqxzygspnMJ","outputId":"98092ab1-48d2-4ce6-e3a9-0342e31cf91c","execution":{"iopub.status.busy":"2022-08-09T18:28:48.498243Z","iopub.execute_input":"2022-08-09T18:28:48.498483Z","iopub.status.idle":"2022-08-09T18:28:52.181395Z","shell.execute_reply.started":"2022-08-09T18:28:48.498454Z","shell.execute_reply":"2022-08-09T18:28:52.180555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The convolution operator seems to have an apparent \"sunshine\" effect of the images. This may also serve the purpose of augmenting the data, thus helping to build more robust and accurate models. ","metadata":{}},{"cell_type":"markdown","source":"## Blurring <a id=\"2.4\"></a>\n\nBlurring is simply the addition of noise to the image, resulting in a less-clear image. The noise can be sampled from any distribution of choice, as long as the main content in the image does not become invisible. Only the minor details get obfuscated due to blurring. The blurring transformation can be represented using the equation below. \n\n<center><img src=\"https://i.imgur.com/zVM8HCU.png\" width=\"220px\"></center>\n<br>\n\nThe example uses a Gaussian distribution with mean 0 and variance 0.1. Below I demonstrate the effect of blurring on a few leaf images:","metadata":{"id":"xmmHZzUq5xu8"}},{"cell_type":"code","source":"def blur(img):\n    fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(20, 20))\n    ax[0].imshow(img)\n    ax[0].set_title('Original Image', fontsize=24)\n    ax[1].imshow(cv2.blur(img, (100, 100)))\n    ax[1].set_title('Blurred Image', fontsize=24)\n    plt.show()","metadata":{"_kg_hide-input":true,"id":"OcXTa-cxpnMM","execution":{"iopub.status.busy":"2022-08-09T18:28:52.182440Z","iopub.execute_input":"2022-08-09T18:28:52.183009Z","iopub.status.idle":"2022-08-09T18:28:52.189471Z","shell.execute_reply.started":"2022-08-09T18:28:52.182978Z","shell.execute_reply":"2022-08-09T18:28:52.188535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"blur(train_images[3])\nblur(train_images[4])\nblur(train_images[5])","metadata":{"_kg_hide-input":true,"id":"aH5qRDdupnMP","outputId":"5b649688-b7e6-419e-d060-ddb09a291e38","execution":{"iopub.status.busy":"2022-08-09T18:28:52.190965Z","iopub.execute_input":"2022-08-09T18:28:52.191205Z","iopub.status.idle":"2022-08-09T18:28:55.731766Z","shell.execute_reply.started":"2022-08-09T18:28:52.191178Z","shell.execute_reply":"2022-08-09T18:28:55.728305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The transformation clearly blurs the image by removing detailed, low-level features, while retaining the major, high-level features. This is once again a great way to augment images and train more robust models.","metadata":{}},{"cell_type":"markdown","source":"# Modeling <a id=\"3\"></a>","metadata":{"id":"wyfDeAJo6lGL"}},{"cell_type":"markdown","source":"## Preparing the ground <a id=\"3.1\"></a>\n\nBefore we move on to building the models, I will explain the major building blocks in pretrained CV models. Every major ImageNet model has a different architecture, but each one has the common building blocks: **Conv2D, MaxPool, ReLU**. I have already explained the mechanism behind convolution in the previous section, so I will now explain MaxPool and ReLU.\n\n### MaxPool\n\nMax pooling is very similar to convolution, except it involves finding the maximum value in a window instead of finding the dot product of the window with a kernel. Max pooling does not require a kernel and it is very useful in reducing the dimensionality of convolutional feature maps in CNNs. The image below demonstrates the working of MaxPool:\n\n\n<center><img src=\"https://i.imgur.com/rBNMsfi.png\" width=\"400px\"></center>\n<br></br>\n\nThe above example demonstrates max pooling with a window size of *(2, 2)*. This process can be represented with the equation below:\n<br></br>\n.\n\n<center><img src=\"https://i.imgur.com/FRyMNhI.png\" width=\"650px\"></center>\n<br></br>\n\nIn the above equation, the window moves across the image and the maximum value in each winow is calculated. Once again, this process is very important in reducing the complexity of CNNs while retaining features.","metadata":{"id":"J5f4-lkS7lKs"}},{"cell_type":"markdown","source":"### ReLU\n\nReLU is an activation function commonly used in neural network architectures. *ReLU(x)* returns 0 for *x < 0* and *x* otherwise. This function helps introducenon-linearity in the neural network, thus increasing its capacity ot model the image data. The graph and equation of *ReLU* are:\n\n<center><img src=\"https://i.imgur.com/eiRVQBh.png\" width=\"400px\"></center>\n\n<center><img src=\"https://i.imgur.com/0mBFAH0.png\" width=\"400px\"></center>\n<br></br>\n\nAs mentioned earlier, this function is non-linear and helps increase the modeling capacity of the CNN models. Now since we understand the basic building blocks of pretrained images models, let us finetune some pretained ImageNet models on TPU and visualize the results!","metadata":{}},{"cell_type":"markdown","source":"### Setup TPU Config","metadata":{"id":"zOfbl73V6t3p"}},{"cell_type":"code","source":"AUTO = tf.data.experimental.AUTOTUNE\ntpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n\ntf.config.experimental_connect_to_cluster(tpu)\ntf.tpu.experimental.initialize_tpu_system(tpu)\nstrategy = tf.distribute.experimental.TPUStrategy(tpu)\n\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync\nGCS_DS_PATH = KaggleDatasets().get_gcs_path()","metadata":{"id":"2ZC6VPQHpnMR","execution":{"iopub.status.busy":"2022-08-09T18:28:55.733227Z","iopub.execute_input":"2022-08-09T18:28:55.733759Z","iopub.status.idle":"2022-08-09T18:29:35.485044Z","shell.execute_reply.started":"2022-08-09T18:28:55.733727Z","shell.execute_reply":"2022-08-09T18:29:35.484063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load labels and paths","metadata":{"id":"SuAHc2hu6-Nu"}},{"cell_type":"code","source":"def format_path(st):\n    return GCS_DS_PATH + '/images/' + st + '.jpg'\n\ntest_paths = test_data.image_id.apply(format_path).values\ntrain_paths = train_data.image_id.apply(format_path).values\n\ntrain_labels = np.float32(train_data.loc[:, 'healthy':'scab'].values)\ntrain_paths, valid_paths, train_labels, valid_labels =\\\ntrain_test_split(train_paths, train_labels, test_size=0.15, random_state=2020)","metadata":{"id":"9BALmDtRpnMU","execution":{"iopub.status.busy":"2022-08-09T18:29:35.486478Z","iopub.execute_input":"2022-08-09T18:29:35.486879Z","iopub.status.idle":"2022-08-09T18:29:35.505816Z","shell.execute_reply.started":"2022-08-09T18:29:35.486839Z","shell.execute_reply":"2022-08-09T18:29:35.504837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def decode_image(filename, label=None, image_size=(512, 512)):\n    bits = tf.io.read_file(filename)\n    image = tf.image.decode_jpeg(bits, channels=3)\n    image = tf.cast(image, tf.float32) / 255.0\n    image = tf.image.resize(image, image_size)\n    \n    if label is None:\n        return image\n    else:\n        return image, label\n\ndef data_augment(image, label=None):\n    image = tf.image.random_flip_left_right(image)\n    image = tf.image.random_flip_up_down(image)\n    \n    if label is None:\n        return image\n    else:\n        return image, label","metadata":{"id":"T84Nnc1jpnMW","execution":{"iopub.status.busy":"2022-08-09T18:29:35.509268Z","iopub.execute_input":"2022-08-09T18:29:35.509565Z","iopub.status.idle":"2022-08-09T18:29:35.523282Z","shell.execute_reply.started":"2022-08-09T18:29:35.509535Z","shell.execute_reply":"2022-08-09T18:29:35.522305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Create Dataset objects","metadata":{"id":"tonEhhQ77Knh"}},{"cell_type":"code","source":"train_dataset = (\n    tf.data.Dataset\n    .from_tensor_slices((train_paths, train_labels))\n    .map(decode_image, num_parallel_calls=AUTO)\n    .map(data_augment, num_parallel_calls=AUTO)\n    .repeat()\n    .shuffle(512)\n    .batch(BATCH_SIZE)\n    .prefetch(AUTO)\n)\n\nvalid_dataset = (\n    tf.data.Dataset\n    .from_tensor_slices((valid_paths, valid_labels))\n    .map(decode_image, num_parallel_calls=AUTO)\n    .batch(BATCH_SIZE)\n    .cache()\n    .prefetch(AUTO)\n)\n\ntest_dataset = (\n    tf.data.Dataset\n    .from_tensor_slices(test_paths)\n    .map(decode_image, num_parallel_calls=AUTO)\n    .batch(BATCH_SIZE)\n)","metadata":{"id":"5rkIRCnupnMZ","execution":{"iopub.status.busy":"2022-08-09T18:29:35.524653Z","iopub.execute_input":"2022-08-09T18:29:35.524984Z","iopub.status.idle":"2022-08-09T18:29:35.769403Z","shell.execute_reply.started":"2022-08-09T18:29:35.524944Z","shell.execute_reply":"2022-08-09T18:29:35.768415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Helper functions","metadata":{"id":"xmirtR2L7TDC"}},{"cell_type":"code","source":"def build_lrfn(lr_start=0.00001, lr_max=0.00005, \n               lr_min=0.00001, lr_rampup_epochs=5, \n               lr_sustain_epochs=0, lr_exp_decay=.8):\n    lr_max = lr_max * strategy.num_replicas_in_sync\n\n    def lrfn(epoch):\n        if epoch < lr_rampup_epochs:\n            lr = (lr_max - lr_start) / lr_rampup_epochs * epoch + lr_start\n        elif epoch < lr_rampup_epochs + lr_sustain_epochs:\n            lr = lr_max\n        else:\n            lr = (lr_max - lr_min) *\\\n                 lr_exp_decay**(epoch - lr_rampup_epochs\\\n                                - lr_sustain_epochs) + lr_min\n        return lr\n    return lrfn","metadata":{"id":"uiiCB9SdpnMc","execution":{"iopub.status.busy":"2022-08-09T18:29:35.770790Z","iopub.execute_input":"2022-08-09T18:29:35.771048Z","iopub.status.idle":"2022-08-09T18:29:35.777863Z","shell.execute_reply.started":"2022-08-09T18:29:35.771019Z","shell.execute_reply":"2022-08-09T18:29:35.776817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define hyperparameters and callbacks","metadata":{}},{"cell_type":"code","source":"lrfn = build_lrfn()\nSTEPS_PER_EPOCH = train_labels.shape[0] // BATCH_SIZE\nlr_schedule = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2022-08-09T18:29:35.779197Z","iopub.execute_input":"2022-08-09T18:29:35.779799Z","iopub.status.idle":"2022-08-09T18:29:35.792150Z","shell.execute_reply.started":"2022-08-09T18:29:35.779764Z","shell.execute_reply":"2022-08-09T18:29:35.791042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## DenseNet <a id=\"3.2\"></a>\n\nDensely Connected Convolutional Networks (DenseNets), are a popular CNN-based ImageNet used for a variety of applications, inclusing classification, segmentation, localization, etc. Most models before DenseNet relied solely on network depth for representational power. **Instead of drawing representational power from extremely deep or wide architectures, DenseNets exploit the potential of the network through feature reuse.** This was the main motivation behind the DenseNet architecture. Now let us train DenseNet on leaf images and evaluate its performance.","metadata":{"id":"PRtfn6bJ7qI1"}},{"cell_type":"code","source":"with strategy.scope():\n    model = tf.keras.Sequential([DenseNet121(input_shape=(512, 512, 3),\n                                             weights='imagenet',\n                                             include_top=False),\n                                 L.GlobalAveragePooling2D(),\n                                 L.Dense(train_labels.shape[1],\n                                         activation='softmax')])\n        \n    model.compile(optimizer='adam',\n                  loss = 'categorical_crossentropy',\n                  metrics=['categorical_accuracy'])\n    model.summary()","metadata":{"id":"7fllHhN9pnMe","outputId":"c030160f-118a-43db-f431-0fd6b4379292","execution":{"iopub.status.busy":"2022-08-09T18:29:35.793563Z","iopub.execute_input":"2022-08-09T18:29:35.793798Z","iopub.status.idle":"2022-08-09T18:29:59.131524Z","shell.execute_reply.started":"2022-08-09T18:29:35.793771Z","shell.execute_reply":"2022-08-09T18:29:59.130589Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DenseNet fundamental block","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(Model(model.layers[0].input, model.layers[0].layers[13].output), dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:29:59.134710Z","iopub.execute_input":"2022-08-09T18:29:59.134960Z","iopub.status.idle":"2022-08-09T18:30:00.421773Z","shell.execute_reply.started":"2022-08-09T18:29:59.134932Z","shell.execute_reply":"2022-08-09T18:30:00.420581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above image shows the fundamental block in the DenseNet architecture. The architecture mainly involves Convolution, Maxpooling, ReLU, and concatenation.","metadata":{}},{"cell_type":"markdown","source":"### Visualize model architecture\n\nThe model consists of the DenseNet head (without the top), followed by global average pooling and a dense layer (with softmax) to generate probabilities.","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(model, dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:30:00.424536Z","iopub.execute_input":"2022-08-09T18:30:00.424829Z","iopub.status.idle":"2022-08-09T18:30:00.559016Z","shell.execute_reply.started":"2022-08-09T18:30:00.424795Z","shell.execute_reply":"2022-08-09T18:30:00.558203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train model","metadata":{"id":"V4yafObR7wIo"}},{"cell_type":"code","source":"history = model.fit(train_dataset,\n                    epochs=EPOCHS,\n                    callbacks=[lr_schedule],\n                    steps_per_epoch=STEPS_PER_EPOCH,\n                    validation_data=valid_dataset)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"id":"V56dwx6opnMh","outputId":"4fc94f9f-ac22-4386-a9d2-6e3192493b33","execution":{"iopub.status.busy":"2022-08-09T18:30:00.560824Z","iopub.execute_input":"2022-08-09T18:30:00.561123Z","iopub.status.idle":"2022-08-09T18:44:33.316250Z","shell.execute_reply.started":"2022-08-09T18:30:00.561086Z","shell.execute_reply":"2022-08-09T18:44:33.315117Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualize results","metadata":{"id":"H32GrB0L7ulh"}},{"cell_type":"code","source":"def display_training_curves(training, validation, yaxis):\n    if yaxis == \"loss\":\n        ylabel = \"Loss\"\n        title = \"Loss vs. Epochs\"\n    else:\n        ylabel = \"Accuracy\"\n        title = \"Accuracy vs. Epochs\"\n        \n    fig = go.Figure()\n        \n    fig.add_trace(\n        go.Scatter(x=np.arange(1, EPOCHS+1), mode='lines+markers', y=training, marker=dict(color=\"dodgerblue\"),\n               name=\"Train\"))\n    \n    fig.add_trace(\n        go.Scatter(x=np.arange(1, EPOCHS+1), mode='lines+markers', y=validation, marker=dict(color=\"darkorange\"),\n               name=\"Val\"))\n    \n    fig.update_layout(title_text=title, yaxis_title=ylabel, xaxis_title=\"Epochs\", template=\"plotly_white\")\n    fig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:44:33.317939Z","iopub.execute_input":"2022-08-09T18:44:33.318298Z","iopub.status.idle":"2022-08-09T18:44:33.326495Z","shell.execute_reply.started":"2022-08-09T18:44:33.318256Z","shell.execute_reply":"2022-08-09T18:44:33.325301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Scatter plots","metadata":{"id":"U2Rmet4p-ot3"}},{"cell_type":"code","source":"display_training_curves(\n    history.history['categorical_accuracy'], \n    history.history['val_categorical_accuracy'], \n    'accuracy')","metadata":{"_kg_hide-input":true,"id":"dKUl8NckpnMn","outputId":"eb37495e-52fc-4e0f-dc0b-e82e708dcf94","execution":{"iopub.status.busy":"2022-08-09T18:44:33.328146Z","iopub.execute_input":"2022-08-09T18:44:33.328710Z","iopub.status.idle":"2022-08-09T18:44:33.373077Z","shell.execute_reply.started":"2022-08-09T18:44:33.328667Z","shell.execute_reply":"2022-08-09T18:44:33.372293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the above plots, we can see that the losses decrease and accuracies increase quite consistently. The training metrics settle down very fast (after 1 or 2 epochs), whereas the validation metrics much greater volatility and start to settle down only after 7-8 epochs. This is expected because validation data is unseen and more diffcult to make predictions on than training data. ","metadata":{}},{"cell_type":"markdown","source":"### Animation (click ▶️)","metadata":{}},{"cell_type":"code","source":"acc_df = pd.DataFrame(np.transpose([[*np.arange(1, EPOCHS+1).tolist()*3], [\"Train\"]*EPOCHS + [\"Val\"]*EPOCHS + [\"Benchmark\"]*EPOCHS,\n                                     history.history['categorical_accuracy'] + history.history['val_categorical_accuracy'] + [1.0]*EPOCHS]))\nacc_df.columns = [\"Epochs\", \"Stage\", \"Accuracy\"]\nfig = px.bar(acc_df, x=\"Accuracy\", y=\"Stage\", animation_frame=\"Epochs\", title=\"Accuracy vs. Epochs\", color='Stage',\n       color_discrete_map={\"Train\":\"dodgerblue\", \"Val\":\"darkorange\", \"Benchmark\":\"seagreen\"}, orientation=\"h\")\n\nfig.update_layout(\n    xaxis = dict(\n        autorange=False,\n        range=[0, 1]\n    )\n)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:44:33.374687Z","iopub.execute_input":"2022-08-09T18:44:33.375006Z","iopub.status.idle":"2022-08-09T18:44:33.799719Z","shell.execute_reply.started":"2022-08-09T18:44:33.374969Z","shell.execute_reply":"2022-08-09T18:44:33.798759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the animations above, we can see the volatility in validation metrics a lot more clearly. The validation metrics oscillate in an erratic fashion until it reaches the 7th epoch and starts to generalize properly.","metadata":{}},{"cell_type":"markdown","source":"### Sample predictions\n\nNow, I will visualize some sample predictions made by the DenseNet model. The <font color=\"red\">red</font> bars represent the model's prediction (maximum probability), the <font color=\"green\">green</font> represent the ground truth (label), and the rest of the bars are <font color=\"blue\">blue</font>. When the model predicts correctly, the prediction bar is <font color=\"green\">green</font>.","metadata":{}},{"cell_type":"code","source":"def process(img):\n    return cv2.resize(img/255.0, (512, 512)).reshape(-1, 512, 512, 3)\ndef predict(img):\n    return model.layers[2](model.layers[1](model.layers[0](process(img)))).numpy()[0]\n\nfig = make_subplots(rows=4, cols=2)\npreds = predict(train_images[2])\n\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Scab\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Multiple diseases\"\n\ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Healthy\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[2], (205, 136))), row=1, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=1, col=2)\nfig.update_layout(height=1200, width=800, title_text=\"DenseNet Predictions\", showlegend=False)\n\npreds = predict(train_images[0])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Multiple diseases\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[0], (205, 136))), row=2, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=2, col=2)\n\npreds = predict(train_images[3])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Rust\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[3], (205, 136))), row=3, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=3, col=2)\n\npreds = predict(train_images[1])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Scab\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[1], (205, 136))), row=4, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=4, col=2)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:44:33.801702Z","iopub.execute_input":"2022-08-09T18:44:33.801962Z","iopub.status.idle":"2022-08-09T18:44:55.154268Z","shell.execute_reply.started":"2022-08-09T18:44:33.801931Z","shell.execute_reply":"2022-08-09T18:44:55.153325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that DenseNet predicts leaf diseases with great accuracy. No red or blue bars are seen. The probabilities are very polarized (one very high and the rest very low), indicating that the model is making these predictions with great confidence.","metadata":{}},{"cell_type":"markdown","source":"### Generate submission","metadata":{}},{"cell_type":"code","source":"probs_dnn = model.predict(test_dataset, verbose=1)\nsub.loc[:, 'healthy':] = probs_dnn\nsub.to_csv('submission_dnn.csv', index=False)\nsub.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T18:44:55.155452Z","iopub.execute_input":"2022-08-09T18:44:55.155682Z","iopub.status.idle":"2022-08-09T18:46:12.348620Z","shell.execute_reply.started":"2022-08-09T18:44:55.155657Z","shell.execute_reply":"2022-08-09T18:46:12.347650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EfficientNet <a id=\"3.3\"></a>\n\nEfficientNet is another popular (more recent) CNN-based ImageNet model which achieved the SOTA on several image-based tasks in 2019. EfficientNet performs model scaling in an innovative way to achieve excellent accuracy with significantly fewer parameters. It achieves the same if not greater accuracy than ResNet and DenseNet with a mcuh shallower architecture. Now let us train EfficientNet on leaf images and evaluate its performance.","metadata":{"id":"isy53hSJ72O5"}},{"cell_type":"code","source":"with strategy.scope():\n    model = tf.keras.Sequential([efn.EfficientNetB7(input_shape=(512, 512, 3),\n                                                    weights='imagenet',\n                                                    include_top=False),\n                                 L.GlobalAveragePooling2D(),\n                                 L.Dense(train_labels.shape[1],\n                                         activation='softmax')])\n    \n    \n        \n    model.compile(optimizer='adam',\n                  loss = 'categorical_crossentropy',\n                  metrics=['categorical_accuracy'])\n    model.summary()","metadata":{"id":"x8ELPOLJpnMp","outputId":"03e23ca7-1a98-4a3d-d61b-3edcb1ae0ced","execution":{"iopub.status.busy":"2022-08-09T18:46:12.349915Z","iopub.execute_input":"2022-08-09T18:46:12.350265Z","iopub.status.idle":"2022-08-09T18:47:11.490286Z","shell.execute_reply.started":"2022-08-09T18:46:12.350229Z","shell.execute_reply":"2022-08-09T18:47:11.489302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EfficientNet fundamental block","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(Model(model.layers[0].input, model.layers[0].layers[11].output), dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:47:11.492545Z","iopub.execute_input":"2022-08-09T18:47:11.492906Z","iopub.status.idle":"2022-08-09T18:47:11.711490Z","shell.execute_reply.started":"2022-08-09T18:47:11.492861Z","shell.execute_reply":"2022-08-09T18:47:11.710592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above image shows the fundamental block in the EfficientNet architecture. This architecture involves more addition and multiplication-based operators than DenseNet. These operations are less parameter-intensive than concatenation, which is much more common in DenseNet. Such transformations help EfficientNet achieve great efficiency (in terms of performance per parameter).","metadata":{}},{"cell_type":"markdown","source":"### Visualize model architecture\n\nThe model consists of the EfficientNet head (without the top), followed by global average pooling and a dense layer (with softmax) to generate probabilities.","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(model, dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T18:47:11.713117Z","iopub.execute_input":"2022-08-09T18:47:11.714206Z","iopub.status.idle":"2022-08-09T18:47:11.898879Z","shell.execute_reply.started":"2022-08-09T18:47:11.714126Z","shell.execute_reply":"2022-08-09T18:47:11.897604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train model","metadata":{"id":"DKydIBmq76tf"}},{"cell_type":"code","source":"history = model.fit(train_dataset,\n                    epochs=EPOCHS,\n                    callbacks=[lr_schedule],\n                    steps_per_epoch=STEPS_PER_EPOCH,\n                    validation_data=valid_dataset)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"id":"MHuypFwdpnMr","outputId":"3fde00ef-c8ac-46b4-bd0f-81a1294bf66f","execution":{"iopub.status.busy":"2022-08-09T18:47:11.900760Z","iopub.execute_input":"2022-08-09T18:47:11.901040Z","iopub.status.idle":"2022-08-09T19:02:18.735476Z","shell.execute_reply.started":"2022-08-09T18:47:11.901005Z","shell.execute_reply":"2022-08-09T19:02:18.734287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualize results","metadata":{"id":"hus0ZMYP8Ehy"}},{"cell_type":"markdown","source":"### Scatter plots","metadata":{"id":"gTrARyKO-aQP"}},{"cell_type":"code","source":"display_training_curves(\n    history.history['categorical_accuracy'], \n    history.history['val_categorical_accuracy'], \n    'accuracy')","metadata":{"_kg_hide-input":true,"id":"94JJ7UNDpnMu","outputId":"f4331b89-9fbc-47aa-884a-0b5d26df1479","execution":{"iopub.status.busy":"2022-08-09T19:02:18.736663Z","iopub.execute_input":"2022-08-09T19:02:18.737786Z","iopub.status.idle":"2022-08-09T19:02:18.776629Z","shell.execute_reply.started":"2022-08-09T19:02:18.737737Z","shell.execute_reply":"2022-08-09T19:02:18.775596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Animation (click ▶️)","metadata":{}},{"cell_type":"markdown","source":"From the above plots, we can once again see that the losses decrease and accuracies increase quite consistently. The training metrics settle down very fast (after 1 or 2 epochs). In this case, the validation metrics do not show high volatility as compared to the DenseNet model.","metadata":{}},{"cell_type":"code","source":"acc_df = pd.DataFrame(np.transpose([[*np.arange(1, EPOCHS+1).tolist()*3], [\"Train\"]*EPOCHS + [\"Val\"]*EPOCHS + [\"Benchmark\"]*EPOCHS,\n                                     history.history['categorical_accuracy'] + history.history['val_categorical_accuracy'] + [1.0]*EPOCHS]))\nacc_df.columns = [\"Epochs\", \"Stage\", \"Accuracy\"]\nfig = px.bar(acc_df, x=\"Accuracy\", y=\"Stage\", animation_frame=\"Epochs\", title=\"Accuracy vs. Epochs\", color='Stage',\n       color_discrete_map={\"Train\":\"dodgerblue\", \"Val\":\"darkorange\", \"Benchmark\":\"seagreen\"}, orientation=\"h\")\n\nfig.update_layout(\n    xaxis = dict(\n        autorange=False,\n        range=[0, 1]\n    )\n)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:02:18.780968Z","iopub.execute_input":"2022-08-09T19:02:18.781902Z","iopub.status.idle":"2022-08-09T19:02:19.170877Z","shell.execute_reply.started":"2022-08-09T19:02:18.781848Z","shell.execute_reply":"2022-08-09T19:02:19.170009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the animations above, we can see that the validation and training metrics do not show great volatility. They steadily rise towards 1.0.","metadata":{}},{"cell_type":"markdown","source":"### Sample predictions\n\nNow, I will visualize some sample predictions made by the EfficientNet model. The <font color=\"red\">red</font> bars represent the model's prediction (maximum probability), the <font color=\"green\">green</font> represent the ground truth (label), and the rest of the bars are <font color=\"blue\">blue</font>. When the model predicts correctly, the prediction bar is <font color=\"green\">green</font>.","metadata":{}},{"cell_type":"code","source":"def process(img):\n    return cv2.resize(img/255.0, (512, 512)).reshape(-1, 512, 512, 3)\ndef predict(img):\n    return model.layers[2](model.layers[1](model.layers[0](process(img)))).numpy()[0]\n\nfig = make_subplots(rows=4, cols=2)\npreds = predict(train_images[2])\n\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Scab\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Multiple diseases\"\n\ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Healthy\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[2], (205, 136))), row=1, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=1, col=2)\nfig.update_layout(height=1200, width=800, title_text=\"EfficientNet Predictions\", showlegend=False)\n\npreds = predict(train_images[0])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Multiple diseases\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[0], (205, 136))), row=2, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=2, col=2)\n\npreds = predict(train_images[3])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Rust\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[3], (205, 136))), row=3, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=3, col=2)\n\npreds = predict(train_images[1])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Scab\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[1], (205, 136))), row=4, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=4, col=2)\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:02:19.172547Z","iopub.execute_input":"2022-08-09T19:02:19.173151Z","iopub.status.idle":"2022-08-09T19:02:43.775684Z","shell.execute_reply.started":"2022-08-09T19:02:19.173108Z","shell.execute_reply":"2022-08-09T19:02:43.774577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The model predicts the leaf diseases with great accuracy. The level of performance is similar to that of DenseNet, as the green bars are very common. The red and blue bars are more prominent in the last (fourth) leaf labeled \"multiple diseases\". This is probably because leaves with multiple diseases may show symptoms of rust and scab as well, thus slightly confusing the model.","metadata":{}},{"cell_type":"markdown","source":"### Generate submission","metadata":{}},{"cell_type":"code","source":"probs_efn = model.predict(test_dataset, verbose=1)\nsub.loc[:, 'healthy':] = probs_efn\nsub.to_csv('submission_efn.csv', index=False)\nsub.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T19:02:43.777195Z","iopub.execute_input":"2022-08-09T19:02:43.777577Z","iopub.status.idle":"2022-08-09T19:03:44.955555Z","shell.execute_reply.started":"2022-08-09T19:02:43.777539Z","shell.execute_reply":"2022-08-09T19:03:44.954645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EfficientNet NoisyStudent <a id=\"3.4\"></a>\n\nEfficientNet NoisyStudent, released in 2020, is based on EfficientNet and uses semi-supervised learning on noisy images to learn rich visual representation. It outperformed EfficientNet on several tasks and is the SOTA at the time of writing (March 2020). Now let us train EfficientNet NoisyStudent on leaf images and evaluate its performance.","metadata":{}},{"cell_type":"code","source":"with strategy.scope():\n    model = tf.keras.Sequential([efn.EfficientNetB7(input_shape=(512, 512, 3),\n                                                    weights='noisy-student',\n                                                    include_top=False),\n                                 L.GlobalAveragePooling2D(),\n                                 L.Dense(train_labels.shape[1],\n                                         activation='softmax')])\n    \n    \n        \n    model.compile(optimizer='adam',\n                  loss = 'categorical_crossentropy',\n                  metrics=['categorical_accuracy'])\n    model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T19:03:44.956898Z","iopub.execute_input":"2022-08-09T19:03:44.957240Z","iopub.status.idle":"2022-08-09T19:04:49.218509Z","shell.execute_reply.started":"2022-08-09T19:03:44.957209Z","shell.execute_reply":"2022-08-09T19:04:49.217550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EfficientNet NoisyStudent","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(Model(model.layers[0].input, model.layers[0].layers[11].output), dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:04:49.219723Z","iopub.execute_input":"2022-08-09T19:04:49.219955Z","iopub.status.idle":"2022-08-09T19:04:49.497345Z","shell.execute_reply.started":"2022-08-09T19:04:49.219928Z","shell.execute_reply":"2022-08-09T19:04:49.496222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above image shows the fundamental block in the EfficientNet NoisyStudent architecture. This model has the same architecture as EfficientNet. Only the weights are different, as they are obtained through semi-supervision.","metadata":{}},{"cell_type":"markdown","source":"### Visualize model architecture\n\nThe model consists of the EfficientNet NoisyStudent head (without the top), followed by global average pooling and a dense layer (with softmax) to generate probabilities.","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(model, dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:04:49.498998Z","iopub.execute_input":"2022-08-09T19:04:49.499296Z","iopub.status.idle":"2022-08-09T19:04:49.762291Z","shell.execute_reply.started":"2022-08-09T19:04:49.499261Z","shell.execute_reply":"2022-08-09T19:04:49.761199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train model","metadata":{}},{"cell_type":"code","source":"history = model.fit(train_dataset,\n                    epochs=EPOCHS,\n                    callbacks=[lr_schedule],\n                    steps_per_epoch=STEPS_PER_EPOCH,\n                    validation_data=valid_dataset)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-08-09T19:04:49.764335Z","iopub.execute_input":"2022-08-09T19:04:49.764703Z","iopub.status.idle":"2022-08-09T19:19:41.692160Z","shell.execute_reply.started":"2022-08-09T19:04:49.764656Z","shell.execute_reply":"2022-08-09T19:19:41.691195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualize results","metadata":{}},{"cell_type":"markdown","source":"### Scatter plots","metadata":{}},{"cell_type":"code","source":"display_training_curves(\n    history.history['categorical_accuracy'], \n    history.history['val_categorical_accuracy'], \n    'accuracy')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:19:41.693673Z","iopub.execute_input":"2022-08-09T19:19:41.694028Z","iopub.status.idle":"2022-08-09T19:19:41.734450Z","shell.execute_reply.started":"2022-08-09T19:19:41.693984Z","shell.execute_reply":"2022-08-09T19:19:41.733482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the above plots, we can see that the losses decrease and accuracies increase quite consistently. The training metrics settle down very fast (after 1 or 2 epochs), whereas the validation metrics much greater volatility and start to settle down only after 12-13 epochs (similar to DenseNet). This is expected because validation data is unseen and more diffcult to make predictions on than training data. ","metadata":{}},{"cell_type":"markdown","source":"### Animation (click ▶️)","metadata":{}},{"cell_type":"code","source":"acc_df = pd.DataFrame(np.transpose([[*np.arange(1, EPOCHS+1).tolist()*3], [\"Train\"]*EPOCHS + [\"Val\"]*EPOCHS + [\"Benchmark\"]*EPOCHS,\n                                     history.history['categorical_accuracy'] + history.history['val_categorical_accuracy'] + [1.0]*EPOCHS]))\nacc_df.columns = [\"Epochs\", \"Stage\", \"Accuracy\"]\nfig = px.bar(acc_df, x=\"Accuracy\", y=\"Stage\", animation_frame=\"Epochs\", title=\"Accuracy vs. Epochs\", color='Stage',\n       color_discrete_map={\"Train\":\"dodgerblue\", \"Val\":\"darkorange\", \"Benchmark\":\"seagreen\"}, orientation=\"h\")\n\nfig.update_layout(\n    xaxis = dict(\n        autorange=False,\n        range=[0, 1]\n    )\n)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:19:41.736308Z","iopub.execute_input":"2022-08-09T19:19:41.736813Z","iopub.status.idle":"2022-08-09T19:19:42.096763Z","shell.execute_reply.started":"2022-08-09T19:19:41.736770Z","shell.execute_reply":"2022-08-09T19:19:42.095874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the animations above, we can see the volatility in validation metrics a lot more clearly. The validation metrics oscillate in an erratic fashion until it reaches the 12th epoch and starts to generalize properly.","metadata":{}},{"cell_type":"markdown","source":"### Sample predictions\n\nNow, I will visualize some sample predictions made by the EfficientNet NoisyStudent model. The <font color=\"red\">red</font> bars represent the model's prediction (maximum probability), the <font color=\"green\">green</font> represent the ground truth (label), and the rest of the bars are <font color=\"blue\">blue</font>. When the model predicts correctly, the prediction bar is <font color=\"green\">green</font>.","metadata":{}},{"cell_type":"code","source":"def process(img):\n    return cv2.resize(img/255.0, (512, 512)).reshape(-1, 512, 512, 3)\ndef predict(img):\n    return model.layers[2](model.layers[1](model.layers[0](process(img)))).numpy()[0]\n\nfig = make_subplots(rows=4, cols=2)\npreds = predict(train_images[2])\n\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Scab\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Multiple diseases\"\n\ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Healthy\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[2], (205, 136))), row=1, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=1, col=2)\nfig.update_layout(height=1200, width=800, title_text=\"EfficientNet NoisyStudent Predictions\", showlegend=False)\n\npreds = predict(train_images[0])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Multiple diseases\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[0], (205, 136))), row=2, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=2, col=2)\n\npreds = predict(train_images[3])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Rust\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[3], (205, 136))), row=3, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=3, col=2)\n\npreds = predict(train_images[1])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n    pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n    pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n    pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n    pred = \"Scab\"\n    \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Scab\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[1], (205, 136))), row=4, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=4, col=2)\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-09T19:19:42.098582Z","iopub.execute_input":"2022-08-09T19:19:42.099032Z","iopub.status.idle":"2022-08-09T19:20:02.633062Z","shell.execute_reply.started":"2022-08-09T19:19:42.098991Z","shell.execute_reply":"2022-08-09T19:20:02.632360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save(\"modelplant.hdf5\")","metadata":{"execution":{"iopub.status.busy":"2022-08-09T19:48:40.097057Z","iopub.execute_input":"2022-08-09T19:48:40.097382Z","iopub.status.idle":"2022-08-09T19:48:51.093780Z","shell.execute_reply.started":"2022-08-09T19:48:40.097350Z","shell.execute_reply":"2022-08-09T19:48:51.092840Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Similar to the DenseNet model, EfficientNet NoisyStudent predicts leaf diseases with great accuracy. No red bars are seen. The probabilities are very polarized (one very high and the rest very low), indicating that the model is making these predictions with great confidence. The semi-supervised weights seem to set this model apart from EfficientNet. The red and blue bars are, once again, more prominent in the last (fourth) leaf labeled \"multiple_diseases\". This is probably because leaves with multiple diseases may show symptoms of rust and scab as well, thus slightly confusing the model.","metadata":{}},{"cell_type":"markdown","source":"### Generate submission","metadata":{}},{"cell_type":"code","source":"probs_efnns = model.predict(test_dataset, verbose=1)\nsub.loc[:, 'healthy':] = probs_efnns\nsub.to_csv('submission_efnns.csv', index=False)\nsub.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T19:20:02.634247Z","iopub.execute_input":"2022-08-09T19:20:02.634977Z","iopub.status.idle":"2022-08-09T19:21:02.603388Z","shell.execute_reply.started":"2022-08-09T19:20:02.634940Z","shell.execute_reply":"2022-08-09T19:21:02.602399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ensembling <a id=\"3.5\"></a>\n\nEnsembling involves the averaging of multiple prediction vectos to reduce errors and improve accuracy. Now, I will ensemble predictions from DenseNet and EfficientNet to (hopefully) produce better results.","metadata":{}},{"cell_type":"code","source":"ensemble_1, ensemble_2, ensemble_3 = [sub]*3\n\nensemble_1.loc[:, 'healthy':] = 0.50*probs_dnn + 0.50*probs_efn\nensemble_2.loc[:, 'healthy':] = 0.25*probs_dnn + 0.75*probs_efn\nensemble_3.loc[:, 'healthy':] = 0.75*probs_dnn + 0.25*probs_efn\n\nensemble_1.to_csv('submission_ensemble_1.csv', index=False)\nensemble_2.to_csv('submission_ensemble_2.csv', index=False)\nensemble_3.to_csv('submission_ensemble_3.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-09T19:21:02.605233Z","iopub.execute_input":"2022-08-09T19:21:02.605781Z","iopub.status.idle":"2022-08-09T19:21:02.666395Z","shell.execute_reply.started":"2022-08-09T19:21:02.605724Z","shell.execute_reply":"2022-08-09T19:21:02.665263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2022-08-09T19:36:50.607130Z","iopub.execute_input":"2022-08-09T19:36:50.607492Z","iopub.status.idle":"2022-08-09T19:36:50.614349Z","shell.execute_reply.started":"2022-08-09T19:36:50.607459Z","shell.execute_reply":"2022-08-09T19:36:50.613059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Takeaways <a id=\"4\"></a>\n\n1. Image processing and augmentation methods such as edge detection, depth estimation, flipping, etc can be used to build  models.\n\n2. Several pretrained models like DenseNet and EfficientNet can be used to classify leaf diseases with high accuracy.\n\n3. Ensembling, stacking, and strong validation techniques may lead to more accurate and robust models.","metadata":{}},{"cell_type":"markdown","source":"# Ending note <a id=\"5\"></a>\n\n<font color=\"red\" size=4>This concludes my kernel. Please upvote if you like it. It motivates me to produce more quality content :)</font>","metadata":{}}]}