{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Preface**\n1. This Notebook contains some basic Data Exploration for Cassava Leaf Disease Prediction Problem \n2. It also has VGG16 Trained Model and an Example of using Transfer Learning and using CNN as Feature Extractor.\n3. Please do through Dataset and Code to understand how to make a Cnn for this Problem.\n4. To make submission to this competition just download h5 file of the model and copy paste entire code from title \"Prediction and Making the Submission File\".\n5. Please do provide feedbacks for improvements as it will help me and others learn a lot."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nfrom keras.preprocessing.image import ImageDataGenerator, load_img\nfrom keras.utils import to_categorical\nfrom sklearn.model_selection import train_test_split\nimport matplotlib.pyplot as plt\nimport random\nimport os\nprint(os.listdir(\"../input/cassava-leaf-disease-classification\"))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 1. Problem Definition :"},{"metadata":{},"cell_type":"markdown","source":"1. Cassava is an important plant and source of nutrition in many African Countries . But the problem is Leaf Disease Associated with Cassava Leaves ! If not prevented and Monitored properly the leaf disease can affect yield of Farmers . Current Method involves manuall inspection of Cassava Leaves and Labelling Them In this competition We are tasked to build a Model which can also detect the Disease and Classify Them ! \n\n2. The Leaf Disease Associated with Cassava Leaves can Fall into 5 Categories , where 4 Categories are Associated with a Disease and Fifth Category is Not Associated with a Disease ."},{"metadata":{},"cell_type":"markdown","source":"### 2 . Data Peak"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"## Let us Peek Over Some Data\ntrain = pd.read_csv(\"../input/cassava-leaf-disease-classification/train.csv\")\nss = pd.read_csv(\"../input/cassava-leaf-disease-classification/sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"##### We have Nearly 21k Images in Training Data , the train.csv has two columns :\n1. image_id : Corresponding to name of the image file with train_images\n2. label_id : This is the target We are going to predict !"},{"metadata":{"trusted":true},"cell_type":"code","source":"ss.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ss.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can See Clearly here only 1 row given for Sample Submission ! As specified in the Competition Data [Here](https://www.kaggle.com/c/cassava-leaf-disease-classification/data) , We are not given the Full set of Test Images it Will be available when Kernel is actually submitted as this a Code / Kernels Only Competition."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Lets Map Disease to Their Actual Values \n# The Mapping can be obtained by using label_num_to_disease_map.json\ntrain['label'] = train['label'].map({0:\"Cassava Bacterial Blight (CBB)\",1:\"Cassava Brown Streak Disease (CBSD)\" , \n                   2:\"Cassava Green Mottle (CGM)\" , 3:\"Cassava Mosaic Disease (CMD)\",4:\"Healthy\"})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['label'].value_counts().plot.bar()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Disease CMD is Dominating in the Train Set . CGM , Healthy and CBSD have almost same Number of Appearances . CBB has Least Number of Occurences "},{"metadata":{},"cell_type":"markdown","source":"#### Lets Plot Some Images in Train !"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_cbb = train.loc[train['label'] ==\"Cassava Bacterial Blight (CBB)\"].head(50).reset_index(drop = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Ref : https://www.kaggle.com/parulpandey/melanoma-classification-eda-starter\nimages = df_cbb['image_id'].values\n\n# Extract 9 random images from it\nrandom_images = [np.random.choice(images) for i in range(9)]\nIMAGE_PATH =  \"../input/cassava-leaf-disease-classification\"\n# Location of the image dir\nimg_dir = IMAGE_PATH+'/train_images'\n\nprint('Display Random Images OF CBB ')\n\n# Adjust the size of your images\nplt.figure(figsize=(10,8))\n\n# Iterate and plot random images\nfor i in range(9):\n    plt.subplot(3, 3, i + 1)\n    img = plt.imread(os.path.join(img_dir, random_images[i]))\n    plt.imshow(img, cmap='gray')\n    plt.axis('off')\n    \n# Adjust subplot parameters to give specified padding\nplt.tight_layout()   ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_cmd = train.loc[train['label'] ==\"Cassava Mosaic Disease (CMD)\"].head(50).reset_index(drop = True)\n# Ref : https://www.kaggle.com/parulpandey/melanoma-classification-eda-starter\nimages = df_cmd['image_id'].values\n\n# Extract 9 random images from it\nrandom_images = [np.random.choice(images) for i in range(9)]\nIMAGE_PATH =  \"../input/cassava-leaf-disease-classification\"\n# Location of the image dir\nimg_dir = IMAGE_PATH+'/train_images'\n\nprint('Display Random Images OF CMB ')\n\n# Adjust the size of your images\nplt.figure(figsize=(10,8))\n\n# Iterate and plot random images\nfor i in range(9):\n    plt.subplot(3, 3, i + 1)\n    img = plt.imread(os.path.join(img_dir, random_images[i]))\n    plt.imshow(img, cmap='gray')\n    plt.axis('off')\n    \n# Adjust subplot parameters to give specified padding\nplt.tight_layout()   ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_healthy = train.loc[train['label'] ==\"Healthy\"].head(50).reset_index(drop = True)\n# Ref : https://www.kaggle.com/parulpandey/melanoma-classification-eda-starter\nimages = df_cmd['image_id'].values\n\n# Extract 9 random images from it\nrandom_images = [np.random.choice(images) for i in range(9)]\nIMAGE_PATH =  \"../input/cassava-leaf-disease-classification\"\n# Location of the image dir\nimg_dir = IMAGE_PATH+'/train_images'\n\nprint('Display Random Images OF Healthy ')\n\n# Adjust the size of your images\nplt.figure(figsize=(10,8))\n\n# Iterate and plot random images\nfor i in range(9):\n    plt.subplot(3, 3, i + 1)\n    img = plt.imread(os.path.join(img_dir, random_images[i]))\n    plt.imshow(img, cmap='gray')\n    plt.axis('off')\n    \n# Adjust subplot parameters to give specified padding\nplt.tight_layout()   ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Observation :\n1. We can see that there are wide variety of Images Present for each class , Some Images are focused on single leaf whereas some images have group of leaves focused on . \n2. The resoultion of Images and Camera Angle at which Image is Taken Varies Alot !\n3. It will be critical to do heavy augmentations and useful augmentations so that our model is invariant to transformations ."},{"metadata":{},"cell_type":"markdown","source":"### 3. Modelling : What we have been Waiting for !"},{"metadata":{},"cell_type":"markdown","source":"1. For Modelling I will use VGG16 as a Baseline Pretrained on weights of Imagenet ! I will use VGG16 as a Feature Extractor , Add some Dense Layers and Fine Tune Final Layers !\n\n2. Since it is Multiclass Classification Problem We will Use Number of Units = Number of Classes , in final Dense Layer with Softmax Activation and Will use Categorical Crossentropy !"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Code Credits : Deep Learning with Python By Francois Chollet\n# Code Credits : https://www.kaggle.com/uysimty/keras-cnn-dog-or-cat-classification\nfrom keras.models import Sequential\nfrom keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation, BatchNormalization\nfrom keras import layers\nfrom keras.applications import VGG16\n\nIMAGE_WIDTH = 225\nIMAGE_HEIGHT = 225\nNUM_CHANNELS = 3\nconv_base = VGG16(weights = 'imagenet' , include_top = False , input_shape = (IMAGE_WIDTH , IMAGE_HEIGHT , NUM_CHANNELS))\nconv_base.trainable = False # Freeze VGG16 base\nmodel = Sequential()\nmodel.add(conv_base)\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(512 , activation = \"relu\"))\nmodel.add(layers.Dense(units = 5 , activation = \"softmax\"))\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Lets Import Some Callbacks \n# Callbacks Help to Avoid Overfitting and Makes Training Easy and Efficient\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau\n\nearlystop = EarlyStopping(patience=10)\nlearning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', \n                                            patience=3, \n                                            verbose=1, \n                                            factor=0.5, \n                                            min_lr=0.00001)\ncallbacks = [earlystop, learning_rate_reduction]\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 4. Data preparation "},{"metadata":{},"cell_type":"markdown","source":"1. Now comes Data Preparation most critical and Time Consuming Part of any Deep Learning Project ! Since Image File Names are given in DataFrame we will use flow_from_dataframe() utility from keras but before that let us prepare a Validation Set ! \n2. Preparing Validation Set is important to check model generalizibility ! \n3. Also there are many methods of Validation , You can refer them online\n4. Here I am using 20 % Holdout from Train Data Stratified based on Targets , Please Note that this is not the best validation strategy there may be better validation strategies suitable for this use case , this is just to get started"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df, validate_df = train_test_split(train, test_size=0.20, random_state=42 , stratify = np.array(train['label']))\ntrain_df = train_df.reset_index(drop=True)\nvalidate_df = validate_df.reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Distributions of Label in Train\ntrain_df['label'].value_counts().plot.bar()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"validate_df['label'].value_counts().plot.bar()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see almost similar distributions of targets in both train and validation set ! This is also similar to distribution of entire train set ! Another Experiment Could be to use different seeds while splitting Data ! Build models on Each of this split and average them ! "},{"metadata":{"trusted":true},"cell_type":"code","source":"total_train = train_df.shape[0]\ntotal_validate = validate_df.shape[0]\nbatch_size= 128","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Image Data Generator "},{"metadata":{},"cell_type":"markdown","source":"Let us Define Some Augmentations Which will increase robustness of the model and increase size of Training Data ! \n1. Rotate by 30 degrees\n2. Rescale to 1./255\n3. zoom by factor of 0.2\n4. Flip Horizontally and Vertically\n\nThese are not the only augmentations ! We can get more creative with augmentations and use much more from inspecting images ! Comment Below if You find some useful augmentations."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_datagen = ImageDataGenerator(\n    rotation_range=30,\n    rescale=1./255,\n    zoom_range=0.2,\n    horizontal_flip=True,\n    vertical_flip = True\n)\n\ntrain_generator = train_datagen.flow_from_dataframe(\n    train_df, \n    directory = \"../input/cassava-leaf-disease-classification/train_images/\", \n    x_col='image_id',\n    y_col='label',\n    target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),\n    class_mode='categorical',\n    batch_size=batch_size\n)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Validation Data \nvalidation_datagen = ImageDataGenerator(rescale=1./255)\nvalidation_generator = validation_datagen.flow_from_dataframe(\n    validate_df, \n    directory = \"../input/cassava-leaf-disease-classification/train_images/\", \n    x_col='image_id',\n    y_col='label',\n    target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),\n    class_mode='categorical',\n    batch_size=batch_size\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5. Train Model "},{"metadata":{},"cell_type":"markdown","source":"We will use accuracy as a metric and use RmsProp Optimizer with Learning Rate of 1e-5"},{"metadata":{},"cell_type":"markdown","source":"Training the model for 5 Epochs Nearly Takes 20 mins on Kaggle's GPU environment ! Training For More Epochs Might Improve Performance but need to take care of overfitting by Adding Dropouts Etc"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras import optimizers\nmodel.compile(loss='categorical_crossentropy', optimizer=optimizers.RMSprop(lr=1e-5), metrics=['accuracy'])\nhistory = model.fit_generator(\n    train_generator, \n    epochs=5,\n    validation_data=validation_generator,\n    validation_steps=total_validate//batch_size,\n    steps_per_epoch=total_train//batch_size,\n    callbacks=callbacks\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Save Model For Reproducibility and Inference\nmodel.save(\"Cassava_VGG16Baseline.h5\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets Visualize Training and Validation Loss "},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 12))\nax1.plot(history.history['loss'], color='b', label=\"Training loss\")\nax1.plot(history.history['val_loss'], color='r', label=\"validation loss\")\nax1.set_xticks(np.arange(1, 5, 1)) # 5 corresponds to number of epochs\nax1.set_yticks(np.arange(0, 1, 0.1))\n\nax2.plot(history.history['accuracy'], color='b', label=\"Training accuracy\")\nax2.plot(history.history['val_accuracy'], color='r',label=\"Validation accuracy\")\nax2.set_xticks(np.arange(1, 5, 1)) # 5 corresponds to number of epochs \n\nlegend = plt.legend(loc='best', shadow=True)\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Conclusions :\n1. We are able to Reach Validation Accuracy of 70 % with just 5 Epochs and Finetuning Dense Layer of VGG16.\n2. There are more models like Resnet , Inception Net which can offer significant increase.\n3. We can apply more pre processing and augmentation to improve model further . "},{"metadata":{},"cell_type":"markdown","source":"### 6 Prediction and Making Submission Files"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_filenames = os.listdir(\"../input/cassava-leaf-disease-classification/test_images\")\ntest_df = pd.DataFrame({\n    'image_id': test_filenames\n})\nnb_samples = test_df.shape[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Only 1 Test Image Available remaining will be available when we submit our kernel\ntest_gen = ImageDataGenerator(rescale=1./255)\ntest_generator = test_gen.flow_from_dataframe(\n    test_df, \n    \"../input/cassava-leaf-disease-classification/test_images\", \n    x_col='image_id',\n    y_col=None,\n    class_mode=None,\n    target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),\n    batch_size=batch_size,\n    shuffle=False\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Make Predictions \npredict = model.predict_generator(test_generator, steps=np.ceil(nb_samples/batch_size))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predict","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that Predictions are Probability of Each 5 Classes ! We will simply Pick Maximum of these probability index and Finally replace index with actual label of the disease ! "},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df['label'] = np.argmax(predict, axis=-1)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# class_indices map the index to actual category of the disease \nlabel_map = dict((v,k) for k,v in train_generator.class_indices.items())\ntest_df['label'] = test_df['label'].replace(label_map)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# now lets convert back it into the format required for submission\ntest_df['label'] = test_df['label'].replace({ \"Cassava Bacterial Blight (CBB)\": 0, \"Cassava Brown Streak Disease (CBSD)\": 1 ,\"Cassava Green Mottle (CGM)\": 2 , \"Cassava Mosaic Disease (CMD)\":3 ,\"Healthy\":4})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Making Final Submission\nsubmission_df = test_df.copy()\nsubmission_df.to_csv(\"submission.csv\" ,index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Improvements Ideas:\n1. The VGG16 Model used here is very basic , but it can be improved .\n2. It can be great to try models like Resent and Inception Net.\n3. Using Better preprocessing Techniques Before Feeding Image to Neural Network\n4. Trying to Add Dropouts and regularization to Neural Network\n5. Trying out Image Segmentation and other stuffs (will need to research on these )\n6. Training Model for Longer !"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}