{"cells":[{"metadata":{},"cell_type":"markdown","source":"\n

Cassava Leaf Disease Classification. Data analysis and visualization.

\n\n
"},{"metadata":{},"cell_type":"markdown","source":"### This competition will challenge you to distinguish between several diseases that cause material harm to the food supply of many African countries. In some cases the main remedy is to burn the infected plants to prevent further spread, which can make a rapid automated turnaround quite useful to the farmers."},{"metadata":{},"cell_type":"markdown","source":"\n\n
\n

Quick navigation

\n\n* [1. train.csv](#1)\n* [2. Image samples](#2)\n* * [2.1 Healthy](#2.1)\n* * [2.2 Cassava Mosaic Disease (CMD)](#2.2)\n* * [2.3 Cassava Green Mottle (CGM)](#2.3)\n* * [2.4 Cassava Brown Streak Disease (CBSD)](#2.4)\n* * [2.5 Cassava Bacterial Blight (CBB)](#2.5)\n* [3. Image size analysis](#3)\n* [4. List of unusual samples for Cassava Brown Streak Disease](#4)\n* [5. List of outliers](#5)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport random\nimport os\n \nimport plotly.express as px\nimport json\n\nimport matplotlib.pyplot as plt\nimport cv2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

1. train.csv

"},{"metadata":{},"cell_type":"markdown","source":"**train.csv**\n\n* image_id the image file name.\n\n* label the ID code for the disease."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/cassava-leaf-disease-classification/train.csv\")\ntrain","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"ds = train['label'].value_counts().reset_index()\nds.columns = [\n 'label', \n 'percent'\n]\n\nds['percent'] /= len(train)\n\nfig = px.pie(\n ds, \n names='label', \n values='percent', \n title='Diseases distribution', \n width=800,\n height=500 \n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**label_num_to_disease_map.json** The mapping between each disease code and the real disease name."},{"metadata":{"trusted":true},"cell_type":"code","source":"with open(\"../input/cassava-leaf-disease-classification/label_num_to_disease_map.json\") as f:\n map_dis = json.loads(f.read())\n\nprint(json.dumps(map_dis, indent=4))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So we can see that the most popular disease is \"Cassava Mosaic Disease (CMD)\".\n\n**Cassava mosaic disease (CMD)** is primarily spread through the dissemination of stem cuttings (A) obtained from cassava plants affected by the disease. Secondary spread can occur within and between fields through the activities of the whitefly vector Bemisia tabaci (B)."},{"metadata":{},"cell_type":"markdown","source":"**[train/test]_images the image files.** The full set of test images will only be available to your notebook when it is submitted for scoring. Expect to see roughly 15,000 images in the test set."},{"metadata":{},"cell_type":"markdown","source":"\n

2. Image samples

"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"def plot_images(class_id, label, images_number):\n \n plot_list = train[train[\"label\"] == class_id].sample(images_number)['image_id'].tolist()\n labels = [label for i in range(len(plot_list))]\n size = np.sqrt(images_number)\n if int(size)*int(size) < images_number:\n size = int(size) + 1\n \n plt.figure(figsize=(20, 20))\n \n for ind, (image_id, label) in enumerate(zip(plot_list, labels)):\n plt.subplot(size, size, ind + 1)\n image = cv2.imread(os.path.join('../input/cassava-leaf-disease-classification/', \"train_images\", image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n plt.title(label, fontsize=12)\n plt.axis(\"off\")\n \n plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

2.1. Healthy

"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_images(\n class_id=4, \n label='Healthy',\n images_number=16\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

2.2. Cassava Mosaic Disease (CMD)

"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_images(\n class_id=3, \n label='Cassava Mosaic Disease (CMD)',\n images_number=16\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

2.3. Cassava Green Mottle (CGM)

"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_images(\n class_id=2, \n label='Cassava Green Mottle (CGM)',\n images_number=16\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

2.4. Cassava Brown Streak Disease (CBSD)

"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_images(\n class_id=1, \n label='Cassava Brown Streak Disease (CBSD)',\n images_number=16\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

2.5. Cassava Bacterial Blight (CBB)

"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_images(\n class_id=0, \n label='Class Cassava Bacterial Blight (CBB)',\n images_number=16\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

3. Image size analysis

"},{"metadata":{},"cell_type":"markdown","source":"Let's check shapes for all of images in training set."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\ncheck_dict = dict()\n\nfor filename in os.listdir('/kaggle/input/cassava-leaf-disease-classification/train_images/'):\n img = cv2.imread('/kaggle/input/cassava-leaf-disease-classification/train_images/' + filename)\n try:\n check_dict[img.shape] += 1\n except:\n check_dict[img.shape] = 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"check_dict","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

4. List of unusual samples for Cassava Brown Streak Disease

"},{"metadata":{"trusted":true},"cell_type":"code","source":"unusual = [\n '1004389140.jpg',\n '1008244905.jpg',\n '1338159402.jpg',\n '1339403533.jpg',\n '159654644.jpg', \n '1010470173.jpg',\n '1014492188.jpg',\n '1359893940.jpg',\n '1366430957.jpg',\n '1689510013.jpg',\n '1726694302.jpg',\n '1770746162.jpg',\n '1773381712.jpg',\n '1848686439.jpg',\n '1905119159.jpg',\n '1917903934.jpg',\n '1960041118.jpg',\n '199112616.jpg',\n '2016389925.jpg',\n '2073193450.jpg',\n '2074713873.jpg',\n '2084868828.jpg',\n '2139839273.jpg',\n '2166623214.jpg',\n '2262263316.jpg',\n '2276509518.jpg',\n '2278166989.jpg',\n '2321669192.jpg',\n '2320471703.jpg',\n '2382642453.jpg',\n '2415837573.jpg',\n '2482667092.jpg',\n '2604713994.jpg',\n '262902341.jpg',\n '2642216511.jpg',\n '2698282165.jpg',\n '2719114674.jpg',\n '274726002.jpg',\n '2925605732.jpg',\n '2981404650.jpg',\n '3040241097.jpg',\n '3043097813.jpg',\n '3123906243.jpg',\n '3126296051.jpg',\n '3199643560.jpg',\n '3251960666.jpg',\n '3252232501.jpg',\n '3425850136.jpg',\n '3435954655.jpg',\n '3477169212.jpg',\n '3609350672.jpg',\n '3652033201.jpg',\n '3810809174.jpg',\n '3838556102.jpg',\n '3881028757.jpg',\n '3892366593.jpg',\n '4060987360.jpg',\n '4089218356.jpg',\n '4134583704.jpg',\n '4203623611.jpg',\n '421035788.jpg',\n '4239074071.jpg',\n '4269208386.jpg',\n '457405364.jpg',\n '549854027.jpg',\n '554488826.jpg',\n '580111608.jpg',\n '600736721.jpg',\n '616718743.jpg',\n '695438825.jpg',\n '723564013.jpg',\n '746746526.jpg',\n '826231979.jpg',\n '847847826.jpg',\n '9224019.jpg',\n '992748624.jpg'\n]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"images_number = 16\n\nplot_list = random.sample(unusual, images_number)\nlabels = ['Unusual Cassava Brown Streak Disease' for i in range(len(plot_list))]\nsize = np.sqrt(images_number)\nif int(size)*int(size) < images_number:\n size = int(size) + 1\n\nplt.figure(figsize=(20, 20))\n\nfor ind, (image_id, label) in enumerate(zip(plot_list, labels)):\n plt.subplot(size, size, ind + 1)\n image = cv2.imread(os.path.join('../input/cassava-leaf-disease-classification/', \"train_images\", image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n plt.title(label, fontsize=12)\n plt.axis(\"off\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

5. List of outliers

"},{"metadata":{},"cell_type":"markdown","source":"**Already processed classes:**\n* 1) Cassava Brown Streak Disease (CBSD)\n* 2) Cassava Bacterial Blight (CBB)\n* 3) Cassava Mosaic Disease (CMD)\n* 4) Cassava Green Mottle (CGM)\n* 5) Healthy"},{"metadata":{"trusted":true},"cell_type":"code","source":"outliers = [\n '156080014.jpg',\n '2182500020.jpg',\n '2489013604.jpg',\n '3129393327.jpg',\n '314640668.jpg',\n '490649765.jpg',\n '1285436512.jpg',\n '1403621003.jpg', # Looks like unusual sample from Cassava Brown Streak Disease but labeled like Cassava Mosaic Disease\n '1819546557.jpg',\n '1841279687.jpg',\n '2088351120.jpg',\n '2161797110.jpg',\n '2602649407.jpg',\n '277532565.jpg',\n '3184864595.jpg',\n '3238801760.jpg',\n '3272750945.jpg',\n '3382391338.jpg',\n '357924077.jpg',\n '4044829046.jpg',\n '4059169921.jpg',\n '4280523848.jpg',\n '449389274.jpg',\n '452420525.jpg',\n '479472063.jpg',\n '612680278.jpg',\n '726377415.jpg',\n '1179237425.jpg',\n '1663857014.jpg',\n '2565638908.jpg',\n '3188953817.jpg',\n '3421208425.jpg',\n '504689064.jpg',\n '597389720.jpg',\n '1119403430.jpg',\n '1774341872.jpg',\n '1886828385.jpg',\n '2484530081.jpg',\n '2632579053.jpg',\n '2839068946.jpg',\n '284130814.jpg',\n '3609986814.jpg',\n '3724956866.jpg',\n '3746679490.jpg',\n '3853597900.jpg',\n '927165736.jpg'\n]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"images_number = 16\n\nplot_list = random.sample(outliers, images_number)\nlabels = ['Outlier' for i in range(len(plot_list))]\nsize = np.sqrt(images_number)\nif int(size)*int(size) < images_number:\n size = int(size) + 1\n\nplt.figure(figsize=(20, 20))\n\nfor ind, (image_id, label) in enumerate(zip(plot_list, labels)):\n plt.subplot(size, size, ind + 1)\n image = cv2.imread(os.path.join('../input/cassava-leaf-disease-classification/', \"train_images\", image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n plt.title(label, fontsize=12)\n plt.axis(\"off\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Work in progress"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}