{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introdução","metadata":{"id":"WbY308Alxq_W"}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"Objetivo: diagnosticar doenças de plantas apenas com base em imagens de folhas. As categorias incluem \"saudável\", \"sarna\", \"ferrugem\" e \"múltiplas doenças\".","metadata":{}},{"cell_type":"markdown","source":"### Instale e importe as bibliotecas necessárias","metadata":{"id":"1XvcaAzr2rjY"}},{"cell_type":"code","source":"!pip install -q efficientnet","metadata":{"id":"V8TgyWRwpnKH","execution":{"iopub.status.busy":"2022-07-26T22:03:46.800153Z","iopub.execute_input":"2022-07-26T22:03:46.800497Z","iopub.status.idle":"2022-07-26T22:03:54.787043Z","shell.execute_reply.started":"2022-07-26T22:03:46.800461Z","shell.execute_reply":"2022-07-26T22:03:54.786083Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport gc\nimport re\n\nimport cv2\nimport math\nimport numpy as np\nimport scipy as sp\nimport pandas as pd\n\nimport tensorflow as tf\nfrom IPython.display import SVG\nimport efficientnet.tfkeras as efn\nfrom tensorflow.keras.utils import plot_model\nimport tensorflow.keras.layers as L\nfrom tensorflow.keras.utils import model_to_dot\nimport tensorflow.keras.backend as K\nfrom tensorflow.keras.models import Model\nfrom kaggle_datasets import KaggleDatasets\nfrom tensorflow.keras.applications import DenseNet121\n\nimport seaborn as sns\nfrom tqdm import tqdm\nimport matplotlib.cm as cm\nfrom sklearn import metrics\nimport matplotlib.pyplot as plt\nfrom sklearn.utils import shuffle\nfrom sklearn.model_selection import train_test_split\n\ntqdm.pandas()\nimport plotly.express as px\nimport plotly.graph_objects as go\nimport plotly.figure_factory as ff\nfrom plotly.subplots import make_subplots\n\nnp.random.seed(0)\ntf.random.set_seed(0)\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","id":"y4ElXcLopnKO","outputId":"72756d56-48f2-46c7-d8d0-56ec9fc395ce","execution":{"iopub.status.busy":"2022-07-26T22:03:54.789571Z","iopub.execute_input":"2022-07-26T22:03:54.789948Z","iopub.status.idle":"2022-07-26T22:03:54.802829Z","shell.execute_reply.started":"2022-07-26T22:03:54.789902Z","shell.execute_reply":"2022-07-26T22:03:54.801827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Carregar os dados e definir hiperparâmetros","metadata":{"id":"DyZKLcDg2yRi"}},{"cell_type":"code","source":"EPOCHS = 20\nSAMPLE_LEN = 100\nIMAGE_PATH = \"../input/plant-pathology-2020-fgvc7/images/\"\nTEST_PATH = \"../input/plant-pathology-2020-fgvc7/test.csv\"\nTRAIN_PATH = \"../input/plant-pathology-2020-fgvc7/train.csv\"\nSUB_PATH = \"../input/plant-pathology-2020-fgvc7/sample_submission.csv\"\n\nsub = pd.read_csv(SUB_PATH)\ntest_data = pd.read_csv(TEST_PATH)\ntrain_data = pd.read_csv(TRAIN_PATH)","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","id":"mneU8D9bpnKS","execution":{"iopub.status.busy":"2022-07-26T22:03:54.804085Z","iopub.execute_input":"2022-07-26T22:03:54.804416Z","iopub.status.idle":"2022-07-26T22:03:54.834030Z","shell.execute_reply.started":"2022-07-26T22:03:54.804377Z","shell.execute_reply":"2022-07-26T22:03:54.833156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data.head()","metadata":{"id":"lSXP0xubpnKW","outputId":"80d177b3-cf7d-4088-90ff-149edfb86295","execution":{"iopub.status.busy":"2022-07-26T22:03:54.836532Z","iopub.execute_input":"2022-07-26T22:03:54.836869Z","iopub.status.idle":"2022-07-26T22:03:54.848985Z","shell.execute_reply.started":"2022-07-26T22:03:54.836828Z","shell.execute_reply":"2022-07-26T22:03:54.848123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data.head()","metadata":{"id":"XISwCeBjpnKa","outputId":"61f70add-a4df-452e-feee-f0ffe0dfac14","execution":{"iopub.status.busy":"2022-07-26T22:03:54.850103Z","iopub.execute_input":"2022-07-26T22:03:54.850455Z","iopub.status.idle":"2022-07-26T22:03:54.863065Z","shell.execute_reply.started":"2022-07-26T22:03:54.850401Z","shell.execute_reply":"2022-07-26T22:03:54.862339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Carregar imagens de amostra","metadata":{"id":"O6DgMHJz293K"}},{"cell_type":"code","source":"def load_image(image_id):\n file_path = image_id + \".jpg\"\n image = cv2.imread(IMAGE_PATH + file_path)\n return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\ntrain_images = train_data[\"image_id\"][:SAMPLE_LEN].progress_apply(load_image)","metadata":{"id":"uJD4JbtHpnKd","outputId":"afb14520-4a05-4f26-e7bc-96a86b3f478c","execution":{"iopub.status.busy":"2022-07-26T22:03:54.864674Z","iopub.execute_input":"2022-07-26T22:03:54.864931Z","iopub.status.idle":"2022-07-26T22:03:58.695561Z","shell.execute_reply.started":"2022-07-26T22:03:54.864904Z","shell.execute_reply":"2022-07-26T22:03:58.694716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize uma folha ","metadata":{"id":"aI1QC_K-3HjL"}},{"cell_type":"markdown","source":"### Imagem de amostra","metadata":{"id":"UY_1XspN3Afg"}},{"cell_type":"code","source":"fig = px.imshow(cv2.resize(train_images[0], (205, 136)))\nfig.show()","metadata":{"_kg_hide-input":true,"id":"LNXpowwwpnKg","outputId":"7a677746-3d36-4b54-d201-63474cefb493","execution":{"iopub.status.busy":"2022-07-26T22:03:58.696938Z","iopub.execute_input":"2022-07-26T22:03:58.697314Z","iopub.status.idle":"2022-07-26T22:03:58.757903Z","shell.execute_reply.started":"2022-07-26T22:03:58.697273Z","shell.execute_reply":"2022-07-26T22:03:58.757047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Distribuições de canais ","metadata":{"id":"JGNq3QwJ3vs9"}},{"cell_type":"code","source":"red_values = [np.mean(train_images[idx][:, :, 0]) for idx in range(len(train_images))]\ngreen_values = [np.mean(train_images[idx][:, :, 1]) for idx in range(len(train_images))]\nblue_values = [np.mean(train_images[idx][:, :, 2]) for idx in range(len(train_images))]\nvalues = [np.mean(train_images[idx]) for idx in range(len(train_images))]","metadata":{"_kg_hide-input":true,"id":"SKlfmsiR3yVn","execution":{"iopub.status.busy":"2022-07-26T22:03:58.759196Z","iopub.execute_input":"2022-07-26T22:03:58.759417Z","iopub.status.idle":"2022-07-26T22:04:00.638440Z","shell.execute_reply.started":"2022-07-26T22:03:58.759392Z","shell.execute_reply":"2022-07-26T22:04:00.637697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Todos os valores do canal","metadata":{"id":"cluIike23zBw"}},{"cell_type":"code","source":"fig = ff.create_distplot([values], group_labels=[\"Channels\"], colors=[\"purple\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribuição de valores de canal\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"Eb32WiwY34At","execution":{"iopub.status.busy":"2022-07-26T22:04:00.639712Z","iopub.execute_input":"2022-07-26T22:04:00.639926Z","iopub.status.idle":"2022-07-26T22:04:00.693869Z","shell.execute_reply.started":"2022-07-26T22:04:00.639901Z","shell.execute_reply":"2022-07-26T22:04:00.693247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Valores do canal vermelho","metadata":{"id":"utld_a6W37SB"}},{"cell_type":"code","source":"fig = ff.create_distplot([red_values], group_labels=[\"R\"], colors=[\"red\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribuição dos valores do canal vermelho\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"jDIqxHgj3_zj","execution":{"iopub.status.busy":"2022-07-26T22:04:00.696325Z","iopub.execute_input":"2022-07-26T22:04:00.696646Z","iopub.status.idle":"2022-07-26T22:04:00.749570Z","shell.execute_reply.started":"2022-07-26T22:04:00.696612Z","shell.execute_reply":"2022-07-26T22:04:00.748770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Valores do canal verde","metadata":{"id":"7Gr6JXgg4QAo"}},{"cell_type":"code","source":"fig = ff.create_distplot([green_values], group_labels=[\"G\"], colors=[\"green\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribuição dos valores do canal verde\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"75DRD2Jy4Sp-","execution":{"iopub.status.busy":"2022-07-26T22:04:00.750777Z","iopub.execute_input":"2022-07-26T22:04:00.750986Z","iopub.status.idle":"2022-07-26T22:04:00.807641Z","shell.execute_reply.started":"2022-07-26T22:04:00.750962Z","shell.execute_reply":"2022-07-26T22:04:00.806811Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Valores do canal azul","metadata":{"id":"Dqs2zNPC4ZkG"}},{"cell_type":"code","source":"fig = ff.create_distplot([blue_values], group_labels=[\"B\"], colors=[\"blue\"])\nfig.update_layout(showlegend=False, template=\"simple_white\")\nfig.update_layout(title_text=\"Distribuição dos valores do canal azul\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"03Sp06S54bpy","execution":{"iopub.status.busy":"2022-07-26T22:04:00.808917Z","iopub.execute_input":"2022-07-26T22:04:00.809238Z","iopub.status.idle":"2022-07-26T22:04:00.866385Z","shell.execute_reply.started":"2022-07-26T22:04:00.809199Z","shell.execute_reply":"2022-07-26T22:04:00.865506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Todos os valores do canal (juntos)","metadata":{"id":"jjFXtVFM4l0w"}},{"cell_type":"code","source":"fig = go.Figure()\n\nfor idx, values in enumerate([red_values, green_values, blue_values]):\n if idx == 0:\n color = \"Red\"\n if idx == 1:\n color = \"Green\"\n if idx == 2:\n color = \"Blue\"\n fig.add_trace(go.Box(x=[color]*len(values), y=values, name=color, marker=dict(color=color.lower())))\n \nfig.update_layout(yaxis_title=\"Valor médio\", xaxis_title=\"Canal de cores\",\n title=\"Valor médio vs. Canal de cores\", template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"id":"iUJeF2Ae4oUl","execution":{"iopub.status.busy":"2022-07-26T22:04:00.867454Z","iopub.execute_input":"2022-07-26T22:04:00.867656Z","iopub.status.idle":"2022-07-26T22:04:00.914951Z","shell.execute_reply.started":"2022-07-26T22:04:00.867632Z","shell.execute_reply":"2022-07-26T22:04:00.914394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = ff.create_distplot([red_values, green_values, blue_values],\n group_labels=[\"R\", \"G\", \"B\"],\n colors=[\"red\", \"green\", \"blue\"])\nfig.update_layout(title_text=\"Distribuição dos valores do canal vermelho\", template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig.data[2].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[2].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"2x8aqxw04o8u","execution":{"iopub.status.busy":"2022-07-26T22:04:00.915850Z","iopub.execute_input":"2022-07-26T22:04:00.916445Z","iopub.status.idle":"2022-07-26T22:04:00.998432Z","shell.execute_reply.started":"2022-07-26T22:04:00.916412Z","shell.execute_reply":"2022-07-26T22:04:00.997814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize folhas de amostra ","metadata":{"id":"Ee2qutMw3FTP"}},{"cell_type":"code","source":"def visualize_leaves(cond=[0, 0, 0, 0], cond_cols=[\"healthy\"], is_cond=True):\n if not is_cond:\n cols, rows = 3, min([3, len(train_images)//3])\n fig, ax = plt.subplots(nrows=rows, ncols=cols, figsize=(30, rows*20/3))\n for col in range(cols):\n for row in range(rows):\n ax[row, col].imshow(train_images.loc[train_images.index[-row*3-col-1]])\n return None\n \n cond_0 = \"healthy == {}\".format(cond[0])\n cond_1 = \"scab == {}\".format(cond[1])\n cond_2 = \"rust == {}\".format(cond[2])\n cond_3 = \"multiple_diseases == {}\".format(cond[3])\n \n cond_list = []\n for col in cond_cols:\n if col == \"healthy\":\n cond_list.append(cond_0)\n if col == \"scab\":\n cond_list.append(cond_1)\n if col == \"rust\":\n cond_list.append(cond_2)\n if col == \"multiple_diseases\":\n cond_list.append(cond_3)\n \n data = train_data.loc[:100]\n for cond in cond_list:\n data = data.query(cond)\n \n images = train_images.loc[list(data.index)]\n cols, rows = 3, min([3, len(images)//3])\n \n fig, ax = plt.subplots(nrows=rows, ncols=cols, figsize=(30, rows*20/3))\n for col in range(cols):\n for row in range(rows):\n ax[row, col].imshow(images.loc[images.index[row*3+col]])\n plt.show()","metadata":{"_kg_hide-input":true,"id":"mQagns98pnKt","execution":{"iopub.status.busy":"2022-07-26T22:04:00.999326Z","iopub.execute_input":"2022-07-26T22:04:00.999964Z","iopub.status.idle":"2022-07-26T22:04:01.012905Z","shell.execute_reply.started":"2022-07-26T22:04:00.999932Z","shell.execute_reply":"2022-07-26T22:04:01.012011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Folhas saudáveis","metadata":{"id":"6YBTzqs53VHW"}},{"cell_type":"code","source":"#visualize_leaves(cond=[1, 0, 0, 0], cond_cols=[\"healthy\"])","metadata":{"_kg_hide-input":true,"id":"_Gw1CH7bpnLc","outputId":"81b8d1ed-231f-4008-9508-23df527bbc80","execution":{"iopub.status.busy":"2022-07-26T22:04:01.014423Z","iopub.execute_input":"2022-07-26T22:04:01.014879Z","iopub.status.idle":"2022-07-26T22:04:01.026279Z","shell.execute_reply.started":"2022-07-26T22:04:01.014841Z","shell.execute_reply":"2022-07-26T22:04:01.025492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Folhas com crosta","metadata":{"id":"lfWbbGfD5ACD"}},{"cell_type":"code","source":"visualize_leaves(cond=[0, 1, 0, 0], cond_cols=[\"scab\"])","metadata":{"_kg_hide-input":true,"id":"cWTEw_JZpnLj","outputId":"d34b44a9-f00b-4942-bd19-c055347fb186","execution":{"iopub.status.busy":"2022-07-26T22:04:01.027161Z","iopub.execute_input":"2022-07-26T22:04:01.027862Z","iopub.status.idle":"2022-07-26T22:04:06.475477Z","shell.execute_reply.started":"2022-07-26T22:04:01.027830Z","shell.execute_reply":"2022-07-26T22:04:06.472704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Folhas com ferrugem","metadata":{"id":"1doTwr2S5Dxt"}},{"cell_type":"code","source":"visualize_leaves(cond=[0, 0, 1, 0], cond_cols=[\"rust\"])","metadata":{"_kg_hide-input":true,"id":"Xwn6GVaupnLl","outputId":"e109052f-3742-4212-c3bd-ec8fa82d2f0c","execution":{"iopub.status.busy":"2022-07-26T22:04:06.476726Z","iopub.execute_input":"2022-07-26T22:04:06.476965Z","iopub.status.idle":"2022-07-26T22:04:11.366938Z","shell.execute_reply.started":"2022-07-26T22:04:06.476936Z","shell.execute_reply":"2022-07-26T22:04:11.363749Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Folhas com várias doenças","metadata":{"id":"mKALD2vt5G78"}},{"cell_type":"code","source":"visualize_leaves(cond=[0, 0, 0, 1], cond_cols=[\"multiple_diseases\"])","metadata":{"_kg_hide-input":true,"id":"YQm_lj9TpnLo","outputId":"a0568b08-2df5-4275-ce41-1a461d8e0ebd","execution":{"iopub.status.busy":"2022-07-26T22:04:11.368123Z","iopub.execute_input":"2022-07-26T22:04:11.368404Z","iopub.status.idle":"2022-07-26T22:04:14.790350Z","shell.execute_reply.started":"2022-07-26T22:04:11.368374Z","shell.execute_reply":"2022-07-26T22:04:14.789374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizar alvos \n\nAgora, vou visualizar os rótulos e os dados de destino. **Em todos os gráficos abaixo, o azul representa a condição \"desejada\" ou \"saudável\" e o vermelho representa a condição \"indesejada\" ou \"não saudável\".**","metadata":{"id":"yTFIf7dZ5MNe"}},{"cell_type":"markdown","source":"### Todos os rótulos juntos (plotagem paralela)","metadata":{"id":"rTU7Plrb5R2v"}},{"cell_type":"code","source":"fig = px.parallel_categories(train_data[[\"healthy\", \"scab\", \"rust\", \"multiple_diseases\"]], color=\"healthy\", color_continuous_scale=\"sunset\",\\\n title=\"Gráfico de categorias paralelas de alvos\")\nfig","metadata":{"_kg_hide-input":true,"id":"vRDHR4G_pnLs","outputId":"7f935b25-2686-447a-bd57-8cd2429b15d2","execution":{"iopub.status.busy":"2022-07-26T22:04:14.791661Z","iopub.execute_input":"2022-07-26T22:04:14.791954Z","iopub.status.idle":"2022-07-26T22:04:14.907370Z","shell.execute_reply.started":"2022-07-26T22:04:14.791919Z","shell.execute_reply":"2022-07-26T22:04:14.906301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Gráfico de pizza","metadata":{}},{"cell_type":"code","source":"fig = go.Figure([go.Pie(labels=train_data.columns[1:],\n values=train_data.iloc[:, 1:].sum().values)])\nfig.update_layout(title_text=\"Gráfico de pizza de alvos\", template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-26T22:04:14.908778Z","iopub.execute_input":"2022-07-26T22:04:14.909094Z","iopub.status.idle":"2022-07-26T22:04:14.966818Z","shell.execute_reply.started":"2022-07-26T22:04:14.909055Z","shell.execute_reply":"2022-07-26T22:04:14.965714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Distribuição saudável","metadata":{"id":"kJNU54Yy5WST"}},{"cell_type":"code","source":"train_data[\"Healthy\"] = train_data[\"healthy\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Healthy\", title=\"Distribuição saudável\", color=\"Healthy\",\\\n color_discrete_map={\n \"True\": px.colors.qualitative.Plotly[0],\n \"False\": px.colors.qualitative.Plotly[1]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"lriYl3yXpnLv","outputId":"08bbe74b-5853-41e2-824e-e58adecfcb6b","execution":{"iopub.status.busy":"2022-07-26T22:04:14.968331Z","iopub.execute_input":"2022-07-26T22:04:14.968811Z","iopub.status.idle":"2022-07-26T22:04:15.118706Z","shell.execute_reply.started":"2022-07-26T22:04:14.968768Z","shell.execute_reply":"2022-07-26T22:04:15.117809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Distribuição de sarna (Scab)","metadata":{"id":"wDb-wmeA5ZMa"}},{"cell_type":"code","source":"train_data[\"Scab\"] = train_data[\"scab\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Scab\", color=\"Scab\", title=\"Scab distribution\",\\\n color_discrete_map={\n \"True\": px.colors.qualitative.Plotly[1],\n \"False\": px.colors.qualitative.Plotly[0]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"AL3UmfaZpnLy","outputId":"61a61499-c8d0-41f5-af0c-8b0dd4bd9b93","execution":{"iopub.status.busy":"2022-07-26T22:04:15.120121Z","iopub.execute_input":"2022-07-26T22:04:15.120786Z","iopub.status.idle":"2022-07-26T22:04:15.236029Z","shell.execute_reply.started":"2022-07-26T22:04:15.120738Z","shell.execute_reply":"2022-07-26T22:04:15.235202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Distribuição de ferrugem (rust)","metadata":{"id":"fJ0g_MRy5bQb"}},{"cell_type":"code","source":"train_data[\"Rust\"] = train_data[\"rust\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Rust\", color=\"Rust\", title=\"Rust distribution\",\\\n color_discrete_map={\n \"True\": px.colors.qualitative.Plotly[1],\n \"False\": px.colors.qualitative.Plotly[0]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"EamjMT0JpnL0","outputId":"4d6633ec-c3ff-409f-aa0f-90cb7a690fc0","execution":{"iopub.status.busy":"2022-07-26T22:04:15.237405Z","iopub.execute_input":"2022-07-26T22:04:15.237859Z","iopub.status.idle":"2022-07-26T22:04:15.356048Z","shell.execute_reply.started":"2022-07-26T22:04:15.237818Z","shell.execute_reply":"2022-07-26T22:04:15.355223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Distribuição de várias doenças","metadata":{"id":"S0t3SEpB5dcS"}},{"cell_type":"code","source":"train_data[\"Multiple diseases\"] = train_data[\"multiple_diseases\"].apply(bool).apply(str)\nfig = px.histogram(train_data, x=\"Multiple diseases\", color=\"Multiple diseases\", title=\"Multiple diseases distribution\",\\\n color_discrete_map={\n \"True\": px.colors.qualitative.Plotly[1],\n \"False\": px.colors.qualitative.Plotly[0]})\nfig.update_layout(template=\"simple_white\")\nfig.data[0].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[0].marker.line.width = 0.5\nfig.data[1].marker.line.color = 'rgb(0, 0, 0)'\nfig.data[1].marker.line.width = 0.5\nfig","metadata":{"_kg_hide-input":true,"id":"9gYSWOl0pnL3","outputId":"29b0bdaf-d509-4436-a98a-628f1818341f","execution":{"iopub.status.busy":"2022-07-26T22:04:15.357637Z","iopub.execute_input":"2022-07-26T22:04:15.357958Z","iopub.status.idle":"2022-07-26T22:04:15.475806Z","shell.execute_reply.started":"2022-07-26T22:04:15.357921Z","shell.execute_reply":"2022-07-26T22:04:15.474881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Rede Neural Convolucional (CNN) pode ser considerada como uma\nvariante da rede neural Perceptron de Múltiplas Camadas","metadata":{}},{"cell_type":"markdown","source":"# Processamento e aumento de imagem (Data Augmentation)","metadata":{"id":"aiPKQDkh5hF5"}},{"cell_type":"markdown","source":"## Detecção de borda inteligente \n\nCanny é um algoritmo popular de detecção de bordas e, como o nome sugere, detecta as bordas dos objetos presentes em uma imagem. Foi desenvolvido por John F. Canny em 1986. O algoritmo envolve várias etapas.\n\n1. **Redução de ruído:** Como a detecção de bordas é suscetível a ruídos em uma imagem, removemos o ruído na imagem usando um filtro Gaussiano 5x5.\n\n\n2. **Encontrando o gradiente de intensidade da imagem**: A imagem suavizada é então filtrada com um kernel Sobel nas direções horizontal e vertical para obter a primeira derivada nas direções horizontal e vertical. A partir dessas duas imagens, pode-se encontrar o gradiente e a direção da borda para cada pixel:\n\n
\n
\n\n
\n\n3. **Arredondamento:** O gradiente é sempre perpendicular às arestas. Assim, é arredondado para um dos quatro ângulos que representam as direções vertical, horizontal e duas diagonais.\n\n4. **Supressão não máxima:** Depois de obter a magnitude e a direção do gradiente, é feita uma varredura completa da imagem para remover quaisquer pixels indesejados que possam não constituir a borda. Para isso, verificamos cada pixel por ser um máximo local em sua vizinhança na direção do gradiente.\n\n5. **Limite de histerese:** Este estágio decide quais partes são arestas e quais não são. Para isso, precisamos de dois valores de limite, *minVal* e *maxVal*. Quaisquer arestas com gradiente de intensidade maior que *maxVal* são consideradas arestas e aquelas menores que *minVal* são consideradas não arestas e descartadas. Aqueles que estão entre esses dois limites são classificados como arestas ou não arestas com base em sua vizinhança. Se estiverem próximos de pixels de “borda segura”, são considerados arestas e, caso contrário, são descartados.\n\nO resultado dessas cinco etapas é um mapa binário bidimensional (0 ou 255) indicando a localização das bordas na imagem. Canny edge é demonstrado abaixo com algumas imagens de folhas:","metadata":{"id":"jC9wp-8N5kU4"}},{"cell_type":"markdown","source":"\n","metadata":{}},{"cell_type":"code","source":"def edge_and_cut(img):\n emb_img = img.copy()\n edges = cv2.Canny(img, 100, 200)\n edge_coors = []\n for i in range(edges.shape[0]):\n for j in range(edges.shape[1]):\n if edges[i][j] != 0:\n edge_coors.append((i, j))\n \n row_min = edge_coors[np.argsort([coor[0] for coor in edge_coors])[0]][0]\n row_max = edge_coors[np.argsort([coor[0] for coor in edge_coors])[-1]][0]\n col_min = edge_coors[np.argsort([coor[1] for coor in edge_coors])[0]][1]\n col_max = edge_coors[np.argsort([coor[1] for coor in edge_coors])[-1]][1]\n new_img = img[row_min:row_max, col_min:col_max]\n \n emb_img[row_min-10:row_min+10, col_min:col_max] = [255, 0, 0]\n emb_img[row_max-10:row_max+10, col_min:col_max] = [255, 0, 0]\n emb_img[row_min:row_max, col_min-10:col_min+10] = [255, 0, 0]\n emb_img[row_min:row_max, col_max-10:col_max+10] = [255, 0, 0]\n \n fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(30, 20))\n ax[0].imshow(img, cmap='gray')\n ax[0].set_title('Original Image', fontsize=24)\n ax[1].imshow(edges, cmap='gray')\n ax[1].set_title('Canny Edges', fontsize=24)\n ax[2].imshow(emb_img, cmap='gray')\n ax[2].set_title('Bounding Box', fontsize=24)\n plt.show()","metadata":{"_kg_hide-input":true,"id":"uU_iqYaCpnL7","execution":{"iopub.status.busy":"2022-07-26T22:04:15.478001Z","iopub.execute_input":"2022-07-26T22:04:15.478665Z","iopub.status.idle":"2022-07-26T22:04:15.492660Z","shell.execute_reply.started":"2022-07-26T22:04:15.478627Z","shell.execute_reply":"2022-07-26T22:04:15.491956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"edge_and_cut(train_images[3])\nedge_and_cut(train_images[4])\nedge_and_cut(train_images[5])","metadata":{"_kg_hide-input":true,"id":"bYJ0t9kppnL9","outputId":"a2785c32-9151-4d8d-f837-ec9de9de49bf","execution":{"iopub.status.busy":"2022-07-26T22:04:15.493737Z","iopub.execute_input":"2022-07-26T22:04:15.494212Z","iopub.status.idle":"2022-07-26T22:04:48.756447Z","shell.execute_reply.started":"2022-07-26T22:04:15.494162Z","shell.execute_reply":"2022-07-26T22:04:48.755413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A segunda coluna de imagens acima contém as bordas do Canny e a terceira coluna contém imagens recortadas. Eu peguei as arestas do Canny e as usei para prever uma caixa delimitadora na qual a folha real está contida. As arestas mais extremas nos quatro cantos da imagem são os vértices da caixa delimitadora. Esta caixa vermelha provavelmente conterá a maior parte, senão toda a folha. Essas arestas e caixas delimitadoras podem ser usadas para construir modelos mais precisos.","metadata":{}},{"cell_type":"markdown","source":"## Flipping (Inversão)\n\nA inversão é uma transformação simples que envolve a troca de índice nos canais de imagem. No lançamento vertical, a ordem das linhas é trocada, enquanto no lançamento vertical, a ordem das linhas é trocada. Vamos supor que *Aijk* (de tamanho *(m, n, 3)*) é a imagem que queremos inverter. A inversão horizontal e vertical pode ser representada pelas transformações abaixo:\n\n
\n
\n
\n
\n\nPodemos ver que a ordem das colunas é trocada no flip horizontal. Enquanto os índices *i* e *k* permanecem os mesmos, o índice *j* se inverte. Considerando que, no lançamento vertical, a ordem das linhas é trocada no lançamento horizontal. Enquanto os índices *j* e *k* permanecem os mesmos, o índice *i* se inverte.\n\n","metadata":{"id":"Wh0WHTCC5sL_"}},{"cell_type":"code","source":"def invert(img):\n fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(30, 20))\n ax[0].imshow(img)\n ax[0].set_title('Original Image', fontsize=24)\n ax[1].imshow(cv2.flip(img, 0))\n ax[1].set_title('Vertical Flip', fontsize=24)\n ax[2].imshow(cv2.flip(img, 1))\n ax[2].set_title('Horizontal Flip', fontsize=24)\n plt.show()","metadata":{"_kg_hide-input":true,"id":"XPZwEZAepnMA","execution":{"iopub.status.busy":"2022-07-26T22:04:48.763353Z","iopub.execute_input":"2022-07-26T22:04:48.763679Z","iopub.status.idle":"2022-07-26T22:04:48.772785Z","shell.execute_reply.started":"2022-07-26T22:04:48.763643Z","shell.execute_reply":"2022-07-26T22:04:48.771317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Convolução \n\nA convolução é um algoritmo bastante simples que envolve um kernel (uma matriz 2D) que se move sobre toda a imagem, calculando produtos de ponto com cada janela ao longo do caminho. O GIF abaixo demonstra a convolução em ação.\n\n
\n\nO processo acima pode ser resumido com uma equação, onde *f* é a imagem e *h* é o kernel. As dimensões de *f* são *(m, n)* e o kernel é uma matriz quadrada com dimensões menores que *f*:\n\n
\n
","metadata":{"id":"PqS2I93A5u_R"}},{"cell_type":"code","source":"invert(train_images[3])\ninvert(train_images[4])\ninvert(train_images[5])","metadata":{"_kg_hide-input":true,"id":"gaiLRUzopnMD","outputId":"dc8f2538-abb7-4449-f0c9-dedd8a393843","execution":{"iopub.status.busy":"2022-07-26T22:04:48.774055Z","iopub.execute_input":"2022-07-26T22:04:48.774529Z","iopub.status.idle":"2022-07-26T22:04:54.170036Z","shell.execute_reply.started":"2022-07-26T22:04:48.774492Z","shell.execute_reply":"2022-07-26T22:04:54.168940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Podemos ver que as imagens são simplesmente invertidas. Todos os principais recursos da imagem permanecem os mesmos, mas para um algoritmo de computador, as imagens invertidas parecem completamente diferentes. Essas transformações podem ser usadas para aumento de dados, tornando os modelos mais robustos e precisos.","metadata":{}},{"cell_type":"code","source":"def conv(img):\n fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(20, 20))\n kernel = np.ones((7, 7), np.float32)/25\n conv = cv2.filter2D(img, -1, kernel)\n ax[0].imshow(img)\n ax[0].set_title('Original Image', fontsize=24)\n ax[1].imshow(conv)\n ax[1].set_title('Convolved Image', fontsize=24)\n plt.show()","metadata":{"_kg_hide-input":true,"id":"aa81abmWpnMG","execution":{"iopub.status.busy":"2022-07-26T22:04:54.171474Z","iopub.execute_input":"2022-07-26T22:04:54.171771Z","iopub.status.idle":"2022-07-26T22:04:54.180723Z","shell.execute_reply.started":"2022-07-26T22:04:54.171738Z","shell.execute_reply":"2022-07-26T22:04:54.179660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conv(train_images[3])\nconv(train_images[4])\nconv(train_images[5])","metadata":{"_kg_hide-input":true,"id":"2rqxzygspnMJ","outputId":"98092ab1-48d2-4ce6-e3a9-0342e31cf91c","execution":{"iopub.status.busy":"2022-07-26T22:04:54.182111Z","iopub.execute_input":"2022-07-26T22:04:54.182415Z","iopub.status.idle":"2022-07-26T22:04:57.811003Z","shell.execute_reply.started":"2022-07-26T22:04:54.182381Z","shell.execute_reply":"2022-07-26T22:04:57.810054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The convolution operator seems to have an apparent \"sunshine\" effect of the images. This may also serve the purpose of augmenting the data, thus helping to build more robust and accurate models. ","metadata":{}},{"cell_type":"markdown","source":"## Blurring (Desfoque) \n\nO desfoque é simplesmente a adição de ruído à imagem, resultando em uma imagem menos nítida. O ruído pode ser amostrado de qualquer distribuição de escolha, desde que o conteúdo principal da imagem não se torne invisível. Apenas os detalhes menores ficam ofuscados devido ao desfoque. A transformação de desfoque pode ser representada usando a equação abaixo.\n
\n
\n","metadata":{"id":"xmmHZzUq5xu8"}},{"cell_type":"code","source":"def blur(img):\n fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(20, 20))\n ax[0].imshow(img)\n ax[0].set_title('Original Image', fontsize=24)\n ax[1].imshow(cv2.blur(img, (100, 100)))\n ax[1].set_title('Blurred Image', fontsize=24)\n plt.show()","metadata":{"_kg_hide-input":true,"id":"OcXTa-cxpnMM","execution":{"iopub.status.busy":"2022-07-26T22:04:57.812239Z","iopub.execute_input":"2022-07-26T22:04:57.812483Z","iopub.status.idle":"2022-07-26T22:04:57.819121Z","shell.execute_reply.started":"2022-07-26T22:04:57.812455Z","shell.execute_reply":"2022-07-26T22:04:57.818122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"blur(train_images[3])\nblur(train_images[4])\nblur(train_images[5])","metadata":{"_kg_hide-input":true,"id":"aH5qRDdupnMP","outputId":"5b649688-b7e6-419e-d060-ddb09a291e38","execution":{"iopub.status.busy":"2022-07-26T22:04:57.820486Z","iopub.execute_input":"2022-07-26T22:04:57.820715Z","iopub.status.idle":"2022-07-26T22:05:01.262941Z","shell.execute_reply.started":"2022-07-26T22:04:57.820688Z","shell.execute_reply":"2022-07-26T22:05:01.262281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A transformação desfoca claramente a imagem removendo recursos detalhados de baixo nível, enquanto mantém os principais recursos de alto nível. Esta é mais uma vez uma ótima maneira de aumentar as imagens e treinar modelos mais robustos.","metadata":{}},{"cell_type":"markdown","source":"# Modeling ","metadata":{"id":"wyfDeAJo6lGL"}},{"cell_type":"markdown","source":"## Preparando o terreno \n\n\n### MaxPool (Camada de Pooling)\n\nTambém conhecida como sub amostragem, essa camada reduz a dimensionalidade de um mapa de característica fornecido como entrada e produz outro mapa de característica, uma espécie de resumo do primeiro (Hijazi, 2015).\n\n\n
\n

\n\nFiltros de tamanho igual a *(2, 2)*.\n

\nCalcula o válor máximo de cada janela. Isso reduz a complexidade das CNNs","metadata":{"id":"J5f4-lkS7lKs"}},{"cell_type":"markdown","source":"### ReLU - Função de Ativação\n\nAs camadas de convolução e de normalização em lote geralmente são seguidas por uma função de ativação não linear, como uma ReLU. \n\nReLU é uma função de ativação comumente usada em arquiteturas de redes neurais. *ReLU(x)* retorna 0 para *x < 0* e *x* caso contrário. Esta função ajuda a introduzir não linearidade na rede neural, aumentando assim sua capacidade de modelar os dados da imagem. O gráfico e a equação de *ReLU* são:\n\n
\n\n
\n

\n\nComo mencionado anteriormente, esta função ajuda a aumentar a capacidade de modelagem dos modelos CNN.\n\nReLU é mais eficiente computacionalmente sem grandes diferenças de acurácia.","metadata":{}},{"cell_type":"markdown","source":"### Setup TPU Config","metadata":{"id":"zOfbl73V6t3p"}},{"cell_type":"code","source":"AUTO = tf.data.experimental.AUTOTUNE\ntpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n\ntf.config.experimental_connect_to_cluster(tpu)\ntf.tpu.experimental.initialize_tpu_system(tpu)\nstrategy = tf.distribute.experimental.TPUStrategy(tpu)\n\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync\nGCS_DS_PATH = KaggleDatasets().get_gcs_path()","metadata":{"id":"2ZC6VPQHpnMR","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load labels and paths","metadata":{"id":"SuAHc2hu6-Nu"}},{"cell_type":"code","source":"def format_path(st):\n return GCS_DS_PATH + '/images/' + st + '.jpg'\n\ntest_paths = test_data.image_id.apply(format_path).values\ntrain_paths = train_data.image_id.apply(format_path).values\n\ntrain_labels = np.float32(train_data.loc[:, 'healthy':'scab'].values)\ntrain_paths, valid_paths, train_labels, valid_labels =\\\ntrain_test_split(train_paths, train_labels, test_size=0.15, random_state=2020)","metadata":{"id":"9BALmDtRpnMU","execution":{"iopub.status.busy":"2022-07-26T22:05:08.045957Z","iopub.execute_input":"2022-07-26T22:05:08.046286Z","iopub.status.idle":"2022-07-26T22:05:08.062291Z","shell.execute_reply.started":"2022-07-26T22:05:08.046247Z","shell.execute_reply":"2022-07-26T22:05:08.061358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def decode_image(filename, label=None, image_size=(512, 512)):\n bits = tf.io.read_file(filename)\n image = tf.image.decode_jpeg(bits, channels=3)\n image = tf.cast(image, tf.float32) / 255.0\n image = tf.image.resize(image, image_size)\n \n if label is None:\n return image\n else:\n return image, label\n\ndef data_augment(image, label=None):\n image = tf.image.random_flip_left_right(image)\n image = tf.image.random_flip_up_down(image)\n \n if label is None:\n return image\n else:\n return image, label","metadata":{"id":"T84Nnc1jpnMW","execution":{"iopub.status.busy":"2022-07-26T22:05:08.063819Z","iopub.execute_input":"2022-07-26T22:05:08.064150Z","iopub.status.idle":"2022-07-26T22:05:08.074598Z","shell.execute_reply.started":"2022-07-26T22:05:08.064111Z","shell.execute_reply":"2022-07-26T22:05:08.073545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Create Dataset objects","metadata":{"id":"tonEhhQ77Knh"}},{"cell_type":"code","source":"train_dataset = (\n tf.data.Dataset\n .from_tensor_slices((train_paths, train_labels))\n .map(decode_image, num_parallel_calls=AUTO)\n .map(data_augment, num_parallel_calls=AUTO)\n .repeat()\n .shuffle(512)\n .batch(BATCH_SIZE)\n .prefetch(AUTO)\n)\n\nvalid_dataset = (\n tf.data.Dataset\n .from_tensor_slices((valid_paths, valid_labels))\n .map(decode_image, num_parallel_calls=AUTO)\n .batch(BATCH_SIZE)\n .cache()\n .prefetch(AUTO)\n)\n\ntest_dataset = (\n tf.data.Dataset\n .from_tensor_slices(test_paths)\n .map(decode_image, num_parallel_calls=AUTO)\n .batch(BATCH_SIZE)\n)","metadata":{"id":"5rkIRCnupnMZ","execution":{"iopub.status.busy":"2022-07-26T22:05:08.076146Z","iopub.execute_input":"2022-07-26T22:05:08.076514Z","iopub.status.idle":"2022-07-26T22:05:08.325104Z","shell.execute_reply.started":"2022-07-26T22:05:08.076470Z","shell.execute_reply":"2022-07-26T22:05:08.324243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Helper functions","metadata":{"id":"xmirtR2L7TDC"}},{"cell_type":"code","source":"def build_lrfn(lr_start=0.00001, lr_max=0.00005, \n lr_min=0.00001, lr_rampup_epochs=5, \n lr_sustain_epochs=0, lr_exp_decay=.8):\n lr_max = lr_max * strategy.num_replicas_in_sync\n\n def lrfn(epoch):\n if epoch < lr_rampup_epochs:\n lr = (lr_max - lr_start) / lr_rampup_epochs * epoch + lr_start\n elif epoch < lr_rampup_epochs + lr_sustain_epochs:\n lr = lr_max\n else:\n lr = (lr_max - lr_min) *\\\n lr_exp_decay**(epoch - lr_rampup_epochs\\\n - lr_sustain_epochs) + lr_min\n return lr\n return lrfn","metadata":{"id":"uiiCB9SdpnMc","execution":{"iopub.status.busy":"2022-07-26T22:05:08.326683Z","iopub.execute_input":"2022-07-26T22:05:08.326995Z","iopub.status.idle":"2022-07-26T22:05:08.334555Z","shell.execute_reply.started":"2022-07-26T22:05:08.326958Z","shell.execute_reply":"2022-07-26T22:05:08.333662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define hyperparameters and callbacks","metadata":{}},{"cell_type":"code","source":"lrfn = build_lrfn()\nSTEPS_PER_EPOCH = train_labels.shape[0] // BATCH_SIZE\nlr_schedule = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2022-07-26T22:05:08.335802Z","iopub.execute_input":"2022-07-26T22:05:08.336081Z","iopub.status.idle":"2022-07-26T22:05:08.345370Z","shell.execute_reply.started":"2022-07-26T22:05:08.336054Z","shell.execute_reply":"2022-07-26T22:05:08.344317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## DenseNet (Camada Densa) \n\nRedes Convolucionais Densamente Conectadas (DenseNets) são um ImageNet popular baseado em CNN usado para uma variedade de aplicações, incluindo classificação, segmentação, localização, etc. A maioria dos modelos anteriores ao DenseNet dependia apenas da profundidade da rede para poder de representação. **Em vez de extrair poder de representação de arquiteturas extremamente profundas ou amplas, os DenseNets exploram o potencial da rede por meio da reutilização de recursos.** Essa foi a principal motivação por trás da arquitetura DenseNet. Agora vamos treinar o DenseNet em imagens de folha e avaliar seu desempenho.\n\nÉ a ultima camada da saída da convolução, normalmente é um MLP.","metadata":{"id":"PRtfn6bJ7qI1"}},{"cell_type":"code","source":"with strategy.scope():\n model = tf.keras.Sequential([DenseNet121(input_shape=(512, 512, 3),\n weights='imagenet',\n include_top=False),\n L.GlobalAveragePooling2D(),\n L.Dense(train_labels.shape[1],\n activation='softmax')])\n \n model.compile(optimizer='adam',\n loss = 'categorical_crossentropy',\n metrics=['categorical_accuracy'])\n model.summary()","metadata":{"id":"7fllHhN9pnMe","outputId":"c030160f-118a-43db-f431-0fd6b4379292","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DenseNet fundamental block","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(Model(model.layers[0].input, model.layers[0].layers[13].output), dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-26T22:05:31.239475Z","iopub.execute_input":"2022-07-26T22:05:31.239708Z","iopub.status.idle":"2022-07-26T22:05:32.630795Z","shell.execute_reply.started":"2022-07-26T22:05:31.239682Z","shell.execute_reply":"2022-07-26T22:05:32.629558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A imagem acima mostra o bloco fundamental na arquitetura DenseNet. A arquitetura envolve principalmente Convolution, Maxpooling, ReLU e concatenação.","metadata":{}},{"cell_type":"markdown","source":"### Arquitetura do modelo\n\nO modelo consiste no DenseNet (sem o topo), seguido de pooling médio global e uma camada densa (com softmax) para gerar probabilidades.","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(model, dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-26T22:05:32.632954Z","iopub.execute_input":"2022-07-26T22:05:32.633336Z","iopub.status.idle":"2022-07-26T22:05:32.768649Z","shell.execute_reply.started":"2022-07-26T22:05:32.633282Z","shell.execute_reply":"2022-07-26T22:05:32.767363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train model","metadata":{"id":"V4yafObR7wIo"}},{"cell_type":"code","source":"history = model.fit(train_dataset,\n epochs=EPOCHS,\n callbacks=[lr_schedule],\n steps_per_epoch=STEPS_PER_EPOCH,\n validation_data=valid_dataset)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"id":"V56dwx6opnMh","outputId":"4fc94f9f-ac22-4386-a9d2-6e3192493b33","execution":{"iopub.status.busy":"2022-07-26T22:05:32.770215Z","iopub.execute_input":"2022-07-26T22:05:32.770497Z","iopub.status.idle":"2022-07-26T22:19:03.774079Z","shell.execute_reply.started":"2022-07-26T22:05:32.770463Z","shell.execute_reply":"2022-07-26T22:19:03.773019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualizar resultados","metadata":{"id":"H32GrB0L7ulh"}},{"cell_type":"code","source":"def display_training_curves(training, validation, yaxis):\n if yaxis == \"loss\":\n ylabel = \"Loss\"\n title = \"Loss vs. Epochs\"\n else:\n ylabel = \"Accuracy\"\n title = \"Accuracy vs. Epochs\"\n \n fig = go.Figure()\n \n fig.add_trace(\n go.Scatter(x=np.arange(1, EPOCHS+1), mode='lines+markers', y=training, marker=dict(color=\"dodgerblue\"),\n name=\"Train\"))\n \n fig.add_trace(\n go.Scatter(x=np.arange(1, EPOCHS+1), mode='lines+markers', y=validation, marker=dict(color=\"darkorange\"),\n name=\"Val\"))\n \n fig.update_layout(title_text=title, yaxis_title=ylabel, xaxis_title=\"Epochs\", template=\"plotly_white\")\n fig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-26T22:19:03.775582Z","iopub.execute_input":"2022-07-26T22:19:03.775830Z","iopub.status.idle":"2022-07-26T22:19:03.783388Z","shell.execute_reply.started":"2022-07-26T22:19:03.775801Z","shell.execute_reply":"2022-07-26T22:19:03.782645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Scatter plots","metadata":{"id":"U2Rmet4p-ot3"}},{"cell_type":"code","source":"display_training_curves(\n history.history['categorical_accuracy'], \n history.history['val_categorical_accuracy'], \n 'accuracy')","metadata":{"_kg_hide-input":true,"id":"dKUl8NckpnMn","outputId":"eb37495e-52fc-4e0f-dc0b-e82e708dcf94","execution":{"iopub.status.busy":"2022-07-26T22:19:03.784460Z","iopub.execute_input":"2022-07-26T22:19:03.784784Z","iopub.status.idle":"2022-07-26T22:19:03.830541Z","shell.execute_reply.started":"2022-07-26T22:19:03.784758Z","shell.execute_reply":"2022-07-26T22:19:03.829155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the above plots, we can see that the losses decrease and accuracies increase quite consistently. The training metrics settle down very fast (after 1 or 2 epochs), whereas the validation metrics much greater volatility and start to settle down only after 7-8 epochs. This is expected because validation data is unseen and more diffcult to make predictions on than training data. ","metadata":{}},{"cell_type":"markdown","source":"### Animation (click ▶️)","metadata":{}},{"cell_type":"code","source":"acc_df = pd.DataFrame(np.transpose([[*np.arange(1, EPOCHS+1).tolist()*3], [\"Train\"]*EPOCHS + [\"Val\"]*EPOCHS + [\"Benchmark\"]*EPOCHS,\n history.history['categorical_accuracy'] + history.history['val_categorical_accuracy'] + [1.0]*EPOCHS]))\nacc_df.columns = [\"Epochs\", \"Stage\", \"Accuracy\"]\nfig = px.bar(acc_df, x=\"Accuracy\", y=\"Stage\", animation_frame=\"Epochs\", title=\"Accuracy vs. Epochs\", color='Stage',\n color_discrete_map={\"Train\":\"dodgerblue\", \"Val\":\"darkorange\", \"Benchmark\":\"seagreen\"}, orientation=\"h\")\n\nfig.update_layout(\n xaxis = dict(\n autorange=False,\n range=[0, 1]\n )\n)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-26T22:19:03.831809Z","iopub.execute_input":"2022-07-26T22:19:03.832030Z","iopub.status.idle":"2022-07-26T22:19:04.328199Z","shell.execute_reply.started":"2022-07-26T22:19:03.832004Z","shell.execute_reply":"2022-07-26T22:19:04.327380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the animations above, we can see the volatility in validation metrics a lot more clearly. The validation metrics oscillate in an erratic fashion until it reaches the 7th epoch and starts to generalize properly.","metadata":{}},{"cell_type":"markdown","source":"### Sample predictions\n\nNow, I will visualize some sample predictions made by the DenseNet model. The red bars represent the model's prediction (maximum probability), the green represent the ground truth (label), and the rest of the bars are blue. When the model predicts correctly, the prediction bar is green.","metadata":{}},{"cell_type":"code","source":"def process(img):\n return cv2.resize(img/255.0, (512, 512)).reshape(-1, 512, 512, 3)\ndef predict(img):\n return model.layers[2](model.layers[1](model.layers[0](process(img)))).numpy()[0]\n\nfig = make_subplots(rows=4, cols=2)\npreds = predict(train_images[2])\n\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Scab\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Multiple diseases\"\n\ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Healthy\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[2], (205, 136))), row=1, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=1, col=2)\nfig.update_layout(height=1200, width=800, title_text=\"DenseNet Predictions\", showlegend=False)\n\npreds = predict(train_images[0])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Multiple diseases\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[0], (205, 136))), row=2, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=2, col=2)\n\npreds = predict(train_images[3])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Rust\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[3], (205, 136))), row=3, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=3, col=2)\n\npreds = predict(train_images[1])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Scab\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[1], (205, 136))), row=4, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=4, col=2)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-26T22:19:04.329669Z","iopub.execute_input":"2022-07-26T22:19:04.330109Z","iopub.status.idle":"2022-07-26T22:19:22.431651Z","shell.execute_reply.started":"2022-07-26T22:19:04.330074Z","shell.execute_reply":"2022-07-26T22:19:22.430667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that DenseNet predicts leaf diseases with great accuracy. No red or blue bars are seen. The probabilities are very polarized (one very high and the rest very low), indicating that the model is making these predictions with great confidence.","metadata":{}},{"cell_type":"markdown","source":"### Generate submission","metadata":{}},{"cell_type":"code","source":"probs_dnn = model.predict(test_dataset, verbose=1)\nsub.loc[:, 'healthy':] = probs_dnn\nsub.to_csv('submission_dnn.csv', index=False)\nsub.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-26T22:19:22.432798Z","iopub.execute_input":"2022-07-26T22:19:22.433015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EfficientNet \n\nEfficientNet is another popular (more recent) CNN-based ImageNet model which achieved the SOTA on several image-based tasks in 2019. EfficientNet performs model scaling in an innovative way to achieve excellent accuracy with significantly fewer parameters. It achieves the same if not greater accuracy than ResNet and DenseNet with a mcuh shallower architecture. Now let us train EfficientNet on leaf images and evaluate its performance.","metadata":{"id":"isy53hSJ72O5"}},{"cell_type":"code","source":"with strategy.scope():\n model = tf.keras.Sequential([efn.EfficientNetB7(input_shape=(512, 512, 3),\n weights='imagenet',\n include_top=False),\n L.GlobalAveragePooling2D(),\n L.Dense(train_labels.shape[1],\n activation='softmax')])\n \n \n \n model.compile(optimizer='adam',\n loss = 'categorical_crossentropy',\n metrics=['categorical_accuracy'])\n model.summary()","metadata":{"id":"x8ELPOLJpnMp","outputId":"03e23ca7-1a98-4a3d-d61b-3edcb1ae0ced","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EfficientNet fundamental block","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(Model(model.layers[0].input, model.layers[0].layers[11].output), dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above image shows the fundamental block in the EfficientNet architecture. This architecture involves more addition and multiplication-based operators than DenseNet. These operations are less parameter-intensive than concatenation, which is much more common in DenseNet. Such transformations help EfficientNet achieve great efficiency (in terms of performance per parameter).","metadata":{}},{"cell_type":"markdown","source":"### Visualize model architecture\n\nThe model consists of the EfficientNet head (without the top), followed by global average pooling and a dense layer (with softmax) to generate probabilities.","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(model, dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train model","metadata":{"id":"DKydIBmq76tf"}},{"cell_type":"code","source":"history = model.fit(train_dataset,\n epochs=EPOCHS,\n callbacks=[lr_schedule],\n steps_per_epoch=STEPS_PER_EPOCH,\n validation_data=valid_dataset)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"id":"MHuypFwdpnMr","outputId":"3fde00ef-c8ac-46b4-bd0f-81a1294bf66f","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualize results","metadata":{"id":"hus0ZMYP8Ehy"}},{"cell_type":"markdown","source":"### Scatter plots","metadata":{"id":"gTrARyKO-aQP"}},{"cell_type":"code","source":"display_training_curves(\n history.history['categorical_accuracy'], \n history.history['val_categorical_accuracy'], \n 'accuracy')","metadata":{"_kg_hide-input":true,"id":"94JJ7UNDpnMu","outputId":"f4331b89-9fbc-47aa-884a-0b5d26df1479","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Animation (click ▶️)","metadata":{}},{"cell_type":"markdown","source":"From the above plots, we can once again see that the losses decrease and accuracies increase quite consistently. The training metrics settle down very fast (after 1 or 2 epochs). In this case, the validation metrics do not show high volatility as compared to the DenseNet model.","metadata":{}},{"cell_type":"code","source":"acc_df = pd.DataFrame(np.transpose([[*np.arange(1, EPOCHS+1).tolist()*3], [\"Train\"]*EPOCHS + [\"Val\"]*EPOCHS + [\"Benchmark\"]*EPOCHS,\n history.history['categorical_accuracy'] + history.history['val_categorical_accuracy'] + [1.0]*EPOCHS]))\nacc_df.columns = [\"Epochs\", \"Stage\", \"Accuracy\"]\nfig = px.bar(acc_df, x=\"Accuracy\", y=\"Stage\", animation_frame=\"Epochs\", title=\"Accuracy vs. Epochs\", color='Stage',\n color_discrete_map={\"Train\":\"dodgerblue\", \"Val\":\"darkorange\", \"Benchmark\":\"seagreen\"}, orientation=\"h\")\n\nfig.update_layout(\n xaxis = dict(\n autorange=False,\n range=[0, 1]\n )\n)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the animations above, we can see that the validation and training metrics do not show great volatility. They steadily rise towards 1.0.","metadata":{}},{"cell_type":"markdown","source":"### Sample predictions\n\nNow, I will visualize some sample predictions made by the EfficientNet model. The red bars represent the model's prediction (maximum probability), the green represent the ground truth (label), and the rest of the bars are blue. When the model predicts correctly, the prediction bar is green.","metadata":{}},{"cell_type":"code","source":"def process(img):\n return cv2.resize(img/255.0, (512, 512)).reshape(-1, 512, 512, 3)\ndef predict(img):\n return model.layers[2](model.layers[1](model.layers[0](process(img)))).numpy()[0]\n\nfig = make_subplots(rows=4, cols=2)\npreds = predict(train_images[2])\n\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Scab\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Multiple diseases\"\n\ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Healthy\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[2], (205, 136))), row=1, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=1, col=2)\nfig.update_layout(height=1200, width=800, title_text=\"EfficientNet Predictions\", showlegend=False)\n\npreds = predict(train_images[0])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Multiple diseases\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[0], (205, 136))), row=2, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=2, col=2)\n\npreds = predict(train_images[3])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Rust\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[3], (205, 136))), row=3, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=3, col=2)\n\npreds = predict(train_images[1])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Scab\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[1], (205, 136))), row=4, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=4, col=2)\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The model predicts the leaf diseases with great accuracy. The level of performance is similar to that of DenseNet, as the green bars are very common. The red and blue bars are more prominent in the last (fourth) leaf labeled \"multiple diseases\". This is probably because leaves with multiple diseases may show symptoms of rust and scab as well, thus slightly confusing the model.","metadata":{}},{"cell_type":"markdown","source":"### Generate submission","metadata":{}},{"cell_type":"code","source":"probs_efn = model.predict(test_dataset, verbose=1)\nsub.loc[:, 'healthy':] = probs_efn\nsub.to_csv('submission_efn.csv', index=False)\nsub.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EfficientNet NoisyStudent \n\nEfficientNet NoisyStudent, released in 2020, is based on EfficientNet and uses semi-supervised learning on noisy images to learn rich visual representation. It outperformed EfficientNet on several tasks and is the SOTA at the time of writing (March 2020). Now let us train EfficientNet NoisyStudent on leaf images and evaluate its performance.","metadata":{}},{"cell_type":"code","source":"with strategy.scope():\n model = tf.keras.Sequential([efn.EfficientNetB7(input_shape=(512, 512, 3),\n weights='noisy-student',\n include_top=False),\n L.GlobalAveragePooling2D(),\n L.Dense(train_labels.shape[1],\n activation='softmax')])\n \n \n \n model.compile(optimizer='adam',\n loss = 'categorical_crossentropy',\n metrics=['categorical_accuracy'])\n model.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### EfficientNet NoisyStudent","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(Model(model.layers[0].input, model.layers[0].layers[11].output), dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above image shows the fundamental block in the EfficientNet NoisyStudent architecture. This model has the same architecture as EfficientNet. Only the weights are different, as they are obtained through semi-supervision.","metadata":{}},{"cell_type":"markdown","source":"### Visualize model architecture\n\nThe model consists of the EfficientNet NoisyStudent head (without the top), followed by global average pooling and a dense layer (with softmax) to generate probabilities.","metadata":{}},{"cell_type":"code","source":"SVG(tf.keras.utils.model_to_dot(model, dpi=70).create(prog='dot', format='svg'))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train model","metadata":{}},{"cell_type":"code","source":"history = model.fit(train_dataset,\n epochs=EPOCHS,\n callbacks=[lr_schedule],\n steps_per_epoch=STEPS_PER_EPOCH,\n validation_data=valid_dataset)","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualize results","metadata":{}},{"cell_type":"markdown","source":"### Scatter plots","metadata":{}},{"cell_type":"code","source":"display_training_curves(\n history.history['categorical_accuracy'], \n history.history['val_categorical_accuracy'], \n 'accuracy')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the above plots, we can see that the losses decrease and accuracies increase quite consistently. The training metrics settle down very fast (after 1 or 2 epochs), whereas the validation metrics much greater volatility and start to settle down only after 12-13 epochs (similar to DenseNet). This is expected because validation data is unseen and more diffcult to make predictions on than training data. ","metadata":{}},{"cell_type":"markdown","source":"### Animation (click ▶️)","metadata":{}},{"cell_type":"code","source":"acc_df = pd.DataFrame(np.transpose([[*np.arange(1, EPOCHS+1).tolist()*3], [\"Train\"]*EPOCHS + [\"Val\"]*EPOCHS + [\"Benchmark\"]*EPOCHS,\n history.history['categorical_accuracy'] + history.history['val_categorical_accuracy'] + [1.0]*EPOCHS]))\nacc_df.columns = [\"Epochs\", \"Stage\", \"Accuracy\"]\nfig = px.bar(acc_df, x=\"Accuracy\", y=\"Stage\", animation_frame=\"Epochs\", title=\"Accuracy vs. Epochs\", color='Stage',\n color_discrete_map={\"Train\":\"dodgerblue\", \"Val\":\"darkorange\", \"Benchmark\":\"seagreen\"}, orientation=\"h\")\n\nfig.update_layout(\n xaxis = dict(\n autorange=False,\n range=[0, 1]\n )\n)\n\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the animations above, we can see the volatility in validation metrics a lot more clearly. The validation metrics oscillate in an erratic fashion until it reaches the 12th epoch and starts to generalize properly.","metadata":{}},{"cell_type":"markdown","source":"### Sample predictions\n\nNow, I will visualize some sample predictions made by the EfficientNet NoisyStudent model. The red bars represent the model's prediction (maximum probability), the green represent the ground truth (label), and the rest of the bars are blue. When the model predicts correctly, the prediction bar is green.","metadata":{}},{"cell_type":"code","source":"def process(img):\n return cv2.resize(img/255.0, (512, 512)).reshape(-1, 512, 512, 3)\ndef predict(img):\n return model.layers[2](model.layers[1](model.layers[0](process(img)))).numpy()[0]\n\nfig = make_subplots(rows=4, cols=2)\npreds = predict(train_images[2])\n\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Scab\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Multiple diseases\"\n\ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Healthy\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[2], (205, 136))), row=1, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=1, col=2)\nfig.update_layout(height=1200, width=800, title_text=\"EfficientNet NoisyStudent Predictions\", showlegend=False)\n\npreds = predict(train_images[0])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Multiple diseases\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[0], (205, 136))), row=2, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=2, col=2)\n\npreds = predict(train_images[3])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Rust\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[3], (205, 136))), row=3, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=3, col=2)\n\npreds = predict(train_images[1])\ncolors = {\"Healthy\":px.colors.qualitative.Plotly[0], \"Scab\":px.colors.qualitative.Plotly[0], \"Rust\":px.colors.qualitative.Plotly[0], \"Multiple diseases\":px.colors.qualitative.Plotly[0]}\nif list.index(preds.tolist(), max(preds)) == 0:\n pred = \"Healthy\"\nif list.index(preds.tolist(), max(preds)) == 1:\n pred = \"Multiple diseases\"\nif list.index(preds.tolist(), max(preds)) == 2:\n pred = \"Rust\"\nif list.index(preds.tolist(), max(preds)) == 3:\n pred = \"Scab\"\n \ncolors[pred] = px.colors.qualitative.Plotly[1]\ncolors[\"Scab\"] = \"seagreen\"\ncolors = [colors[val] for val in colors.keys()]\nfig.add_trace(go.Image(z=cv2.resize(train_images[1], (205, 136))), row=4, col=1)\nfig.add_trace(go.Bar(x=[\"Healthy\", \"Multiple diseases\", \"Rust\", \"Scab\"], y=preds, marker=dict(color=colors)), row=4, col=2)\nfig.update_layout(template=\"plotly_white\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Similar to the DenseNet model, EfficientNet NoisyStudent predicts leaf diseases with great accuracy. No red bars are seen. The probabilities are very polarized (one very high and the rest very low), indicating that the model is making these predictions with great confidence. The semi-supervised weights seem to set this model apart from EfficientNet. The red and blue bars are, once again, more prominent in the last (fourth) leaf labeled \"multiple_diseases\". This is probably because leaves with multiple diseases may show symptoms of rust and scab as well, thus slightly confusing the model.","metadata":{}},{"cell_type":"markdown","source":"### Generate submission","metadata":{}},{"cell_type":"code","source":"probs_efnns = model.predict(test_dataset, verbose=1)\nsub.loc[:, 'healthy':] = probs_efnns\nsub.to_csv('submission_efnns.csv', index=False)\nsub.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Montagem \n\nA montagem envolve a média de vários vectos de previsão para reduzir erros e melhorar a precisão. Agora, vou combinar as previsões do DenseNet e do EfficientNet para (espero) produzir melhores resultados.","metadata":{}},{"cell_type":"code","source":"ensemble_1, ensemble_2, ensemble_3 = [sub]*3\n\nensemble_1.loc[:, 'healthy':] = 0.50*probs_dnn + 0.50*probs_efn\nensemble_2.loc[:, 'healthy':] = 0.25*probs_dnn + 0.75*probs_efn\nensemble_3.loc[:, 'healthy':] = 0.75*probs_dnn + 0.25*probs_efn\n\nensemble_1.to_csv('submission_ensemble_1.csv', index=False)\nensemble_2.to_csv('submission_ensemble_2.csv', index=False)\nensemble_3.to_csv('submission_ensemble_3.csv', index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Takeaways \n\n1. Métodos de processamento e aumento de imagem, como detecção de borda, estimativa de profundidade, inversão, etc., podem ser usados para construir modelos.\n\n2. Vários modelos pré-treinados como DenseNet e EfficientNet podem ser usados para classificar doenças foliares com alta precisão.\n\n3. Técnicas de agrupamento, empilhamento e validação forte podem levar a modelos mais precisos e robustos.","metadata":{}}]}