{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Cassava Disesase Classification🌱"},{"metadata":{},"cell_type":"markdown","source":"Hey folks!\n\nThis is a simpler version of the model used for the competition. Here I used a pre-trained model (*EfficientNetB3*) to classify the different types of diseases that affect the Cassava plant in Africa. This is my baseline model but I think that it could be very helpful to people who just started to dive into the deep learning world.\n\nN.B: I'm still a novice in Kaggle competitions, so any suggestion or comment (of course after the ending of the competition) is very welcome!\n\nHappy coding to everyone!"},{"metadata":{},"cell_type":"markdown","source":"### IMPORTING MAIN LIBRARIES"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import os\nimport glob\nimport shutil\nimport json\nimport keras\nimport itertools\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport tensorflow as tf\nfrom PIL import Image\nimport matplotlib.pyplot as plt\nfrom collections import Counter\nfrom keras.preprocessing.image import ImageDataGenerator\n\n# Defining the working directories\n\nwork_dir = '../input/cassava-leaf-disease-classification/'\nos.listdir(work_dir) \ntrain_path = '/kaggle/input/cassava-leaf-disease-classification/train_images'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### IMPORTING DATA"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Importing Train Data\n\ndata = pd.read_csv(work_dir + 'train.csv')\nprint(Counter(data['label'])) # Checking the frequencies of the labels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Importing the json file with labels\n\nf = open(work_dir + 'label_num_to_disease_map.json')\nreal_labels = json.load(f)\nreal_labels = {int(k):v for k,v in real_labels.items()}\n\n# Defining the working dataset\n\ndata['class_name'] = data.label.map(real_labels)\n\n# Splitting the data\n\nfrom sklearn.model_selection import train_test_split\n\ntrain,val = train_test_split(data, test_size = 0.05, random_state = 42, stratify = data['class_name'])\n\n# Importing the data using ImageDataGenerator\n\nIMG_SIZE = 456\nsize = (IMG_SIZE,IMG_SIZE)\nn_CLASS = 5\nBATCH_SIZE = 15\n\ndatagen_train = ImageDataGenerator(\n                    preprocessing_function = tf.keras.applications.efficientnet.preprocess_input,\n                    rotation_range = 40,\n                    width_shift_range = 0.2,\n                    height_shift_range = 0.2,\n                    shear_range = 0.2,\n                    zoom_range = 0.2,\n                    horizontal_flip = True,\n                    vertical_flip = True,\n                    fill_mode = 'nearest')\n\ndatagen_val = ImageDataGenerator(\n                    preprocessing_function = tf.keras.applications.efficientnet.preprocess_input,\n                    )\n\ntrain_set = datagen_train.flow_from_dataframe(train,\n                             directory = train_path,\n                             seed=42,\n                             x_col = 'image_id',\n                             y_col = 'class_name',\n                             target_size = size,\n                             #color_mode=\"rgb\",\n                             class_mode = 'categorical',\n                             interpolation = 'nearest',\n                             shuffle = True,\n                             batch_size = BATCH_SIZE)\n\nval_set = datagen_val.flow_from_dataframe(val,\n                             directory = train_path,\n                             seed=42,\n                             x_col = 'image_id',\n                             y_col = 'class_name',\n                             target_size = size,\n                             #color_mode=\"rgb\",\n                             class_mode = 'categorical',\n                             interpolation = 'nearest',\n                             shuffle = True,\n                             batch_size = BATCH_SIZE)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As mentioned in the discussion part, computing class weights do not increase the accuracy at all, so I silenced this part."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Computing Class Weights\n\n#counter = Counter(train_set.classes)                          \n#max_val = float(max(counter.values()))\n#d_class_weights = {class_id : max_val/num_images for class_id, num_images in counter.items()}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### MODELING PART ☸"},{"metadata":{"trusted":true},"cell_type":"code","source":"# MODEL CREATION \n\nfrom keras.models import Sequential\nfrom keras.layers import GlobalAveragePooling2D, Flatten, Dense, Dropout, BatchNormalization\nfrom keras.optimizers import RMSprop, Adam\nfrom keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\nfrom tensorflow.keras.applications import EfficientNetB3\n\ndef create_model():\n    \n    model = Sequential()\n    # initialize the model with input shape\n    model.add(EfficientNetB3(input_shape = (IMG_SIZE, IMG_SIZE, 3), include_top = False,\n                             weights = 'imagenet',\n                             drop_connect_rate=0.6))\n    #for layer in model.layers[:-40]:  # Training just part of the architecture do not optimize the performance\n    #    layer.trainable = False\n    model.add(GlobalAveragePooling2D())\n    model.add(Flatten())\n    model.add(Dense(256, activation = 'relu', bias_regularizer=tf.keras.regularizers.L1L2(l1=0.01, l2=0.001)))\n    model.add(Dropout(0.5))\n    model.add(Dense(n_CLASS, activation = 'softmax'))\n    \n    return model\n\nleaf_model = create_model()\nleaf_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"keras.utils.plot_model(leaf_model)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"EPOCHS = 50\nSTEP_SIZE_TRAIN = train_set.n//train_set.batch_size\nSTEP_SIZE_VALID = val_set.n//val_set.batch_size","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# FITTING THE MODEL\n\ndef Model_fit():\n    \n    #leaf_model = None\n    \n    leaf_model = create_model()\n    \n    '''Compiling the model'''\n    \n    loss = tf.keras.losses.CategoricalCrossentropy(from_logits = False,\n                                                   label_smoothing=0.0001,\n                                                   name='categorical_crossentropy' )\n    \n    leaf_model.compile(optimizer = Adam(learning_rate = 1e-3),\n                        loss = loss, #'categorical_crossentropy'\n                        metrics = ['categorical_accuracy']) #'acc'\n    \n    # Stop training when the val_loss has stopped decreasing for 3 epochs.\n    es = EarlyStopping(monitor='val_loss', mode='min', patience=3,\n                       restore_best_weights=True, verbose=1)\n    \n    # Save the model with the minimum validation loss\n    checkpoint_cb = ModelCheckpoint(\"Cassava_best_model.h5\",\n                                    save_best_only=True,\n                                    monitor = 'val_loss',\n                                    mode='min')\n    \n    # reduce learning rate\n    reduce_lr = ReduceLROnPlateau(monitor = 'val_loss',\n                                  factor = 0.2,\n                                  patience = 2,\n                                  min_lr = 1e-6,\n                                  mode = 'min',\n                                  verbose = 1)\n    \n    history = leaf_model.fit(train_set,\n                             validation_data = val_set,\n                             epochs= EPOCHS,\n                             batch_size = BATCH_SIZE,\n                             #class_weight = d_class_weights,\n                             steps_per_epoch = STEP_SIZE_TRAIN,\n                             validation_steps = STEP_SIZE_VALID,\n                             callbacks=[es, checkpoint_cb, reduce_lr])\n    \n    leaf_model.save('Cassava_model'+'.h5')  \n    \n    return history","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"results = Model_fit()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# CHECKING THE METRICS\n\nprint('Train_Cat-Acc: ', max(results.history['categorical_accuracy']))\nprint('Val_Cat-Acc: ', max(results.history['val_categorical_accuracy']))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# PLOTTING RESULTS (Train vs Validation FOLDER 1)\n\ndef Train_Val_Plot(acc,val_acc,loss,val_loss):\n    \n    fig, (ax1, ax2) = plt.subplots(1,2, figsize= (15,10))\n    fig.suptitle(\" MODEL'S METRICS VISUALIZATION \", fontsize=20)\n\n    ax1.plot(range(1, len(acc) + 1), acc)\n    ax1.plot(range(1, len(val_acc) + 1), val_acc)\n    ax1.set_title('History of Accuracy', fontsize=15)\n    ax1.set_xlabel('Epochs', fontsize=15)\n    ax1.set_ylabel('Accuracy', fontsize=15)\n    ax1.legend(['training', 'validation'])\n\n\n    ax2.plot(range(1, len(loss) + 1), loss)\n    ax2.plot(range(1, len(val_loss) + 1), val_loss)\n    ax2.set_title('History of Loss', fontsize=15)\n    ax2.set_xlabel('Epochs', fontsize=15)\n    ax2.set_ylabel('Loss', fontsize=15)\n    ax2.legend(['training', 'validation'])\n    plt.show()\n    \n\nTrain_Val_Plot(results.history['categorical_accuracy'],results.history['val_categorical_accuracy'],\n               results.history['loss'],results.history['val_loss'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# EVALUATING THE MODEL\n\nimport keras\n\nfinal_model = keras.models.load_model('Cassava_best_model.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Predictions on the test set with TTA (Test Time Augmentation)"},{"metadata":{},"cell_type":"markdown","source":"Test Time Augmentation (*TTA*) is a specific application of data augmentation to the test set. It involves creating multiple copies of each image in the test set allowing the model to make predictions for each of them to then return an ensemble of those. Augmentations are chosen to give the model the best opportunity for correctly classifying a given image."},{"metadata":{"trusted":true},"cell_type":"code","source":"TEST_DIR = '../input/cassava-leaf-disease-classification/test_images/'\ntest_images = os.listdir(TEST_DIR)\ndatagen = ImageDataGenerator(horizontal_flip=True)\n\n\ndef pred(images):\n    for image in test_images:\n        img = Image.open(TEST_DIR + image)\n        img = img.resize(size)\n        samples = np.expand_dims(img, axis=0)\n        it = datagen.flow(samples, batch_size=10)\n        yhats = final_model.predict_generator(it, steps=10, verbose=0)\n        summed = np.sum(yhats, axis=0)\n    return np.argmax(summed)\n\npredictions = pred(test_images)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Creating the CSV for final submission\n\nsub = pd.DataFrame({'image_id': test_images, 'label': predictions})\ndisplay(sub)\nsub.to_csv('submission.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Thanks for reading this notebook! If you found this notebook helpful, please give it an **upvote**. It is always appreciated!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}