{"cells":[{"metadata":{},"cell_type":"markdown","source":"Loading the packages"},{"metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true},"cell_type":"code","source":"library(knitr)\nlibrary(tidyverse) # Collection of all the good stuff like dplyr, ggplot2 ect.\nlibrary(magrittr)\nlibrary(data.table)\nlibrary(rjson)\nlibrary(jsonlite)\nlibrary(qdapTools)\nlibrary(purrr)\nlibrary(keras)\nlibrary(float)\nlibrary(ggplot2)\nlibrary(corrplot)\nlibrary(quantmod)\nlibrary(keras)\nlibrary(mltools)\nlibrary(onehot)\nlibrary(splitstackshape)\nlibrary(tensorflow)\nSys.setenv(RETICULATE_PYTHON=\"/usr/local/share/.virtualenvs/r-reticulate/bin/\") # Only necessary in kaggle currently","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We will load the images, labels for the images and the sample_submission which is the image that will be the test after getting the models.\nWe can see a head of what image is defined on what desease. What desease is categorised to what number you can see at the below code "},{"metadata":{"trusted":true},"cell_type":"code","source":"image_path<-'/kaggle/input/cassava-leaf-disease-classification//train_images'\nsample_submission <- read_csv(\"/kaggle/input/cassava-leaf-disease-classification/sample_submission.csv\")\nlabels <- fread('../input/cassava-leaf-disease-classification/train.csv',data.table=F)\n#test_labels = read_csv('')\nhead(labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we categorize the labels as different deseases."},{"metadata":{"trusted":true},"cell_type":"code","source":"Y1H <- labels[,\"label\"] %>% factor() %>% keras::to_categorical()\nlabels = cbind(labels, Y1H)\ncolnames(labels)[3:7] <- c('CBB', 'CBSD', 'CGM', 'CMD', 'Healthy')\nhead(labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We only have one dataset and then one image that will be the test. But we have to split the trainingset to a training- and testset. Afterwards we can see the first three images and what they are labeled to "},{"metadata":{"trusted":true},"cell_type":"code","source":"Y_tr <- labels[1:13000,]\nY_te <- labels[13001:21000,]\n\nhead(Y_tr, n = 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we defines the variables for the whole file for the models"},{"metadata":{"trusted":true},"cell_type":"code","source":"w = 224\nh = 224\nbs = 32\nfactor = .4\nreduceFrom = 2\nearly_stop = 4\nb1 = .9\nb2 = .999\nleRa = 0.001\nclip = 5\neps = 20\nlabSm = .0011\nchannels=3\ntarget_size = c(w,h)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets see three of the images in a table which will have a chance of 1-0.0011 chance of being the label"},{"metadata":{"trusted":true},"cell_type":"code","source":"for(i in 3:7){\n    Y_tr[,i] <- ifelse(Y_tr[,i] == 0,labSm,(1-labSm))\n}\n\nhead(Y_tr, n = 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we load the images from the directory. The function turns the imagefiles on disk into batches of preprocessed tensors. We rescale all the images by 1/255. We convert it to floating-point tensors and rescale the pixelvalue to [0,1] interval because neural networks prefer to deal with small input values. We do it for both training-, validation- and testing-dataset. See the comments for more information."},{"metadata":{"trusted":true},"cell_type":"code","source":"#optional data augmentation\ntrain_gen = image_data_generator(\n  rescale = 1/255, #Rescale the images\n  shear_range = 0.2, #Randomly applying shearing transformations\n  zoom_range = 0.2, #Randomly zooming inside pictures\n  horizontal_flip = TRUE, #Flipping randomly half the images horizontally - Here it is relevant when there is no assumptions of horizontal asymmetry\n  fill_mode = \"nearest\", #The strategy used for filling in  newly created pixels which can appear after a rotation or a width/height shift\n  rotation_range = 40, #Value in degrees (0-180) which will randomly rotate pictures \n  width_shift_range = 0.2, #A fraction  of total width or height which will randomly translate pictures vertically or horizontally\n  height_shift_range = 0.2\n \n)\n\n# Validation data shouldn't be augmented! But it should also be scaled.\nvalidation_gen <- image_data_generator(\n  rescale = 1/255\n  )  \n\ntest_gen <- image_data_generator(\n  rescale = 1/255\n  )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Takes the dataframe and the path to a directory and generates batches of augmented/normalized data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_generator <- flow_images_from_dataframe(dataframe = Y_tr, #Takes the trainingset of the labels\n                                              directory = image_path, #Imports the images for different labels\n                                              generator = train_gen, #To use the normalizing/augmentation image data\n                                              class_mode = \"other\", #Array of y_col data\n                                              x_col = \"image_id\",\n                                              y_col = c(\"CBB\",\"CBSD\", \"CGM\", \"CMD\", \"Healthy\"), #List of the columns in the datafram that has the target data\n                                              target_size = c(w,h), #Resizes all the images to 224,224 in height and width\n                                              batch_size = bs,\n                                              shuffle = TRUE,\n                                              drop_duplicates = FALSE,\n                                              interpolation = \"nearest\" #Used to resample the image if the target is different from that of the loaded image\n                                             )\n\n\nvalidation_generator<-flow_images_from_dataframe(dataframe = Y_te, \n                                              directory = image_path,\n                                              generator= validation_gen,\n                                              class_mode = \"other\",\n                                              x_col = \"image_id\",\n                                              y_col = c(\"CBB\",\"CBSD\", \"CGM\", \"CMD\", \"Healthy\"),\n                                              target_size = c(w,h),\n                                              batch_size = bs,\n                                              shuffle = FALSE,\n                                              drop_duplicates = FALSE,\n                                              interpolation = \"nearest\"\n                                              \n                                                  )\n#This is not used anywhere and it is the same as the final generator\ntest_generator <- flow_images_from_dataframe(dataframe = sample_submission, \n                                         directory = '../input/cassava-leaf-\ndisease-classification/test_images',\n                                         class_mode = NULL,\n                                         x_col = \"image_id\",\n                                         y_col = NULL,\n                                         target_size = c(w, h),\n                                         shuffle = FALSE,\n                                         batch_size=1,\n                                         generator = test_gen\n                                         )\nfinal_generator <- flow_images_from_dataframe(dataframe = sample_submission, \n                                         directory = '../input/cassava-leaf-disease-classification/test_images',\n                                         class_mode = NULL,\n                                         x_col = \"image_id\",\n                                         y_col = NULL,\n                                         target_size = c(w, h),\n                                         shuffle = FALSE,\n                                         batch_size=1,\n                                         generator = test_gen\n                                         )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Defining the callbacks which will automate some tasks after every epoch which will controle the proces after every epoch"},{"metadata":{"trusted":true},"cell_type":"code","source":"reduce_lr <- callback_reduce_lr_on_plateau(factor = factor, patience = reduceFrom, verbose = 0, mode = \"auto\", monitor = \"val_accuracy\") #Benefit from reducing the learning rate by a factor once the learning rate stagnates. If there is no improvement for a patience number of epochs the learning rate is reduced\nstop <- callback_early_stopping(monitor = 'val_accuracy', patience = early_stop) #Will stop if there is no improvements after some epochs\ncheck_point <- callback_model_checkpoint(\"model4.h5\", save_best_only = TRUE, verbose = 0, mode = \"auto\") #Saves the model after every epoch","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Normal CNN-model"},{"metadata":{},"cell_type":"markdown","source":"Here we define two normal CNN-models which we will see, so we know what we have to improve. Here we add convolutional, maxpooling layers and so on. We will use the bookexample and a model we have improved by our selves. The example from the book is model_cnn1"},{"metadata":{"trusted":true},"cell_type":"code","source":"input <- layer_input(shape = c(w, h, channels))\noutput <- input %>%\n    layer_conv_2d(filters = 32, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2), stride = 2) %>% \n    layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2), stride = 2) %>% \n    layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    layer_conv_2d(filters = 256, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    layer_conv_2d(filters = 256, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_flatten() %>%layer_batch_normalization() %>%     \n    layer_dense(units = 512, activation=\"relu\")%>%     \n    layer_batch_normalization() %>%     \n    layer_dense(units = 5, activation=\"softmax\")#We have softmax as last-layer activation because it is a multiclass (More than one class), singlelabel classification (Each image has to have one classification)\nmodel_cnn <- keras_model(input, output)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"input1 <- layer_input(shape = c(w, h, channels))\noutput1 <- input1 %>%\n    layer_conv_2d(filters = 32, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2), stride = 2) %>% \n    layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2), stride = 2) %>% \n    layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    #layer_conv_2d(filters = 256, kernel_size = c(3, 3), activation = 'relu') %>%\n    #layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    #layer_conv_2d(filters = 256, kernel_size = c(3, 3), activation = 'relu') %>%\n    #layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    #layer_conv_2d(filters = 256, kernel_size = c(3, 3), activation = 'relu') %>%\n    #layer_dropout(rate=0.25) %>%\n    layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = 'relu') %>%\n    layer_max_pooling_2d(pool_size = c(2, 2)) %>% \n    layer_flatten() %>%#layer_batch_normalization() %>%     \n    layer_dense(units = 512, activation=\"relu\")%>% \n    #layer_flatten %>%\n    #layer_batch_normalization() %>%     \n    layer_dense(units = 5, activation=\"softmax\")#We have softmax as last-layer activation because it is a multiclass (More than one class), singlelabel classification (Each image has to have one classification)\nmodel_cnn1 <- keras_model(input1, output1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we compile the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# compile\nmodel_cnn %>% compile(\n  loss = loss_categorical_crossentropy,#This is chosen because we have multiclass, singlelabel classification\n  optimizer = 'adam',\n  metrics = \"accuracy\"\n#    metrics = c('accuracy')\n)\nmodel_cnn1 %>% compile(\n  loss = loss_categorical_crossentropy,#This is chosen because we have multiclass, singlelabel classification\n  optimizer = 'adam',\n  metrics = \"accuracy\"\n#    metrics = c('accuracy')\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we train the model on the training data and validate on the validationdata. Steps per epoch is defined as the number of images in the trainingset divided by the batch-size (32)."},{"metadata":{"trusted":true},"cell_type":"code","source":"set.seed(1347)\nhistory_cnn1 <- model_cnn1 %>% fit_generator(\n  train_generator,\n  steps_per_epoch = dim(Y_tr)[1] %/% bs,\n  epochs = eps,\n  validation_data = validation_generator,\n  validation_steps = dim(Y_te)[1] %/% bs,\n  #verbose = 2,\n  )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"set.seed(1347)\nhistory_cnn <- model_cnn %>% fit_generator(\n  train_generator,\n  steps_per_epoch = dim(Y_tr)[1] %/% bs,\n  epochs = eps,\n  validation_data = validation_generator,\n  validation_steps = dim(Y_te)[1] %/% bs,\n  #verbose = 2,\n  )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we can see how well the normal cnn-normal is doing and the one we have improved. The accuracy and validation accuracy for the one improved is close to be the same. But 75% accuracy must be a number we can improve. We can also se the plot of the model and we save model. We also validate the model and get 72% accuracy."},{"metadata":{"trusted":true},"cell_type":"code","source":"model_cnn1 %>% save_model_hdf5(\"Disease_detector_cnn1.h5\")\nmodel_cnn1 %>% evaluate_generator(validation_generator, steps = 500)\nhistory_cnn1\nplot(history_cnn1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model_cnn %>% save_model_hdf5(\"Disease_detector_cnn.h5\")\nmodel_cnn %>% evaluate_generator(validation_generator, steps = 500)\nhistory_cnn\nplot(history_cnn)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"num_test_image <- nrow(sample_submission)\npred_cnn<-model_cnn %>% predict_generator(final_generator,steps = num_test_image)\nhead(pred_cnn)\n\nlabel_pred_cnn <- pred_cnn %>% apply(1, function(x) which(x==max(x))-1)\nprediction_cnn <- tibble(image_id = sample_submission$image_id, label = label_pred_cnn)\nhead(prediction_cnn)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Transfer learned models with vgg16 and vgg19"},{"metadata":{},"cell_type":"markdown","source":"In transfer learning we include pretrained models to our own model. These models are pretrained on predefined data and may improve our model"},{"metadata":{"trusted":true},"cell_type":"code","source":"base_model_vgg16 <- application_vgg16(\n  include_top = FALSE,\n  weights='imagenet',\n  input_shape = c(w,h,channels))\nbase_model_vgg19 <- application_vgg19(\n  include_top = FALSE,\n  weights='imagenet',\n  input_shape = c(w,h,channels))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model_vgg16 <-keras_model_sequential() %>%\n  base_model_vgg16 %>% #The add of the pretrained model\n  layer_global_max_pooling_2d() %>% \n  layer_dropout(rate=0.5) %>%\n  layer_dense(units = 512,\n              activation = \"relu\") %>%\n  layer_batch_normalization() %>% \n  layer_dropout(rate=0.5) %>%\n  layer_dense(units=5, activation=\"softmax\")\n\nmodel_vgg19<-keras_model_sequential() %>%\n  base_model_vgg19 %>% \n  layer_global_max_pooling_2d() %>% \n  layer_dropout(rate=0.5) %>%\n  layer_dense(units = 512,\n              activation = \"relu\") %>%\n  layer_batch_normalization() %>% \n  layer_dropout(rate=0.5) %>%\n  layer_dense(units=5, activation=\"softmax\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Freezing layers"},{"metadata":{},"cell_type":"markdown","source":"The layers in the vgg16 and vgg19 that has been trained from the models are freezed so it will not be changed with the new data. If we don't do this, the model will update the weights will be modified during training and the model is unimportant. The things that will learn is the layers we have got on the vgg-models you can see above. So we have the models (the vgg16 and vgg19) and use them with other layers on our data"},{"metadata":{"trusted":true},"cell_type":"code","source":"#first: train only the top layers \nfor (layer in base_model_vgg16$layers)\n  layer$trainable <- FALSE\n#first: train only the top layers \nfor (layer in base_model_vgg19$layers)\n  layer$trainable <- FALSE","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Compile"},{"metadata":{},"cell_type":"markdown","source":"Compile the model again with the new trainable models with some new parameters"},{"metadata":{"trusted":true},"cell_type":"code","source":"model_vgg16 %>% compile(\n  loss = loss_categorical_crossentropy,\n  optimizer = optimizer_adamax(beta_1 = b1, #Parameter between 0 and 1\n                               beta_2 = b2, #Parameter between 0 and 1\n                               clipvalue = clip, #The gradients will be clipped when their absolutte value exceeds this value\n                               lr = leRa), #Float of the learning rate\n  metrics = c('accuracy'))\n\nmodel_vgg19 %>% compile(\n  loss = loss_categorical_crossentropy,\n  optimizer = optimizer_adamax(beta_1 = b1,\n                               beta_2 = b2,\n                               clipvalue = clip,\n                               lr = leRa),\n  metrics = c('accuracy'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Training VGG-models"},{"metadata":{},"cell_type":"markdown","source":"Training the models again but with the base-models attached to them. "},{"metadata":{"trusted":true},"cell_type":"code","source":"set.seed(1337)\nhistory_vgg16 <- model_vgg16 %>%\n              fit_generator(\n              train_generator,\n              steps_per_epoch = dim(Y_tr)[1] %/% bs,\n              validation_data = validation_generator,\n              epochs = eps,\n              verbose = 2,\n              callbacks = c(check_point,\n                            reduce_lr,\n                            stop))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see an accuracy of 63% and 65% on the validation accuracy\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"model_vgg16 %>% evaluate_generator(validation_generator, steps = 500)\nmodel_vgg16 %>% save_model_hdf5(\"Disease_detecter_vgg16.h5\")\nhistory_vgg16","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we first visualizes the layers from vgg16 and chooses to freeze the first layers and then unfreeze the rest to see if that can improve the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# let's visualize layer names and layer indices to see how many layers we should freeze:\nlayers <- base_model_vgg16$layers\nfor (i in 1:length(layers))\n  cat(i, layers[[i]]$name, \"\\n\")\n\n# we chose to train the top 2 inblocks, i.e. we will freeze the first layers and unfreeze the rest:\nfor (i in 1:15)\n  layers[[i]]$trainable <- FALSE\nfor (i in 16:length(layers))\n  layers[[i]]$trainable <- TRUE","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Compile the model with freezed layer"},{"metadata":{"trusted":true},"cell_type":"code","source":"model_vgg16 %>% compile(\n  loss = loss_categorical_crossentropy,\n  optimizer = optimizer_adamax(beta_1 = b1,\n                               beta_2 = b2,\n                               clipvalue = clip,\n                               lr = leRa),\n  metrics = c('accuracy'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we run the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"set.seed(1347)\nhistory_vgg16_freeze <- model_vgg16 %>%\n              fit_generator(\n              train_generator,\n              steps_per_epoch = dim(Y_tr)[1] %/% bs,\n              validation_data = validation_generator,\n              epochs = eps,\n              verbose = 2,\n              callbacks = c(check_point,\n                            reduce_lr,\n                            stop))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see an accuracy of 80% and 75% on the validation accuracy"},{"metadata":{"trusted":true},"cell_type":"code","source":"model_vgg16 %>% evaluate_generator(validation_generator, steps = 500)\nmodel_vgg16 %>% save_model_hdf5(\"Disease_detecter_vgg16_freeze.h5\")\nhistory_vgg16_freeze","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Predictions"},{"metadata":{},"cell_type":"markdown","source":"Here we have the normal cnn-model and the vgg16-model to compare them. In the cnn we have in accuracy 75% and in validation accuracy 72%. In the vgg16 we have an accuracy on 80% and validation accuracy on 75%"},{"metadata":{},"cell_type":"markdown","source":"CNN - Historyplot and predictions"},{"metadata":{"trusted":true},"cell_type":"code","source":"history_cnn\nplot(history_cnn)\n\nnum_test_image <- nrow(sample_submission)\npred_cnn<-model_cnn %>% predict_generator(final_generator,steps = num_test_image)\nhead(pred_cnn)\n\nlabel_pred_cnn <- pred_cnn %>% apply(1, function(x) which(x==max(x))-1)\nprediction_cnn <- tibble(image_id = sample_submission$image_id, label = label_pred_cnn)\nhead(prediction_cnn)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"VGG-16"},{"metadata":{"trusted":true},"cell_type":"code","source":"history_vgg16_freeze\n\nnum_test_image <- nrow(sample_submission)\npred_vgg16_freeze<-model_vgg16 %>% predict_generator(final_generator,steps = num_test_image)\nhead(pred_vgg16_freeze)\n\nlabel_pred_vgg16_freeze <- pred_vgg16_freeze %>% apply(1, function(x) which(x==max(x))-1)\nprediction_vgg16_freeze <- tibble(image_id = sample_submission$image_id, label = label_pred_vgg16_freeze)\nhead(prediction_vgg16_freeze)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history_cnn1\nhistory_cnn\nhistory_vgg16\nhistory_vgg16_freeze","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"What do the models"},{"metadata":{},"cell_type":"markdown","source":"Not all kinds of deep learning-models can show what they are looking for. But here we can do excactly that with different perspectives."},{"metadata":{},"cell_type":"markdown","source":"Here we just visualize a training-image"},{"metadata":{"trusted":true},"cell_type":"code","source":"fnames <- '../input/cassava-leaf-disease-classification/train_images/1000201771.jpg'\nimg_path1 <- fnames\nimg <- image_load(img_path1, target_size = c(w,h))\nimg_array <- image_to_array(img)\nimg_array <- array_reshape(img_array, c(1,w ,h, channels))\naugmentation_generator <- flow_images_from_data(\n    img_array,\n    generator = train_gen,\n    batch_size = 1\n)\nop <- par(mfrow = c(2,2), pty = \"s\", mar = c(1, 0, 1, 0))\nfor (i in 1:4) {\n    batch <- generator_next(augmentation_generator)\n    plot(as.raster(batch[1,,,]))\n}\npar(op)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we visualizes what a certain layer is doing with the image"},{"metadata":{"trusted":true},"cell_type":"code","source":"img_path1 <- '../input/cassava-leaf-disease-classification/train_images/1000201771.jpg'\nimg <- image_load(img_path1, target_size = c(w,h))\nimg_tensor <- image_to_array(img)\nimg_tensor <- array_reshape(img_tensor, c(1, w,h, channels))\nimg_tensor <- img_tensor / 255\n#dim(img_tensor)\n#plot(as.raster(img_tensor[1,,,]))\n\nlayer_outputs <- lapply(model_cnn$layers[1:8], function(layer) layer$output)\nactivation_model <- keras_model(inputs = model_cnn$input, outputs = layer_outputs)\nactivations <- activation_model %>% predict(img_tensor)\nfirst_layer_activation <- activations[[8]]\n                        \n#dim(first_layer_activation)\nplot_channel <- function(channel) {\n    rotate <- function(x) t(apply(x, 2, rev))\n    image(rotate(channel), axes = FALSE, asp = 1,\n         col = terrain.colors(12))\n}\n                        \nplot_channel(first_layer_activation[1,,,8])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we show what classification the model vgg16 is predicting to "},{"metadata":{"trusted":true},"cell_type":"code","source":"# instantiate the model\nmodel <- application_vgg16(weights='imagenet')\n\n# load the image\nimg_path <- '../input/cassava-leaf-disease-classification/train_images/1000201771.jpg'\nimg <- image_load(img_path, target_size = target_size)\nx <- image_to_array(img)\n\n# ensure we have a 4d tensor with single element in the batch dimension,\n# the preprocess the input for prediction using resnet50\nx <- array_reshape(x, c(1, dim(x)))\nx <- imagenet_preprocess_input(x)\n\n# make predictions then decode and print them\npreds <- model %>% predict(x)\nimagenet_decode_predictions(preds, top = 10)[[1]]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lime"},{"metadata":{"trusted":true},"cell_type":"code","source":"library(lime)\nlibrary(magick)\nimg0 <- image_read('../input/cassava-leaf-disease-classification/train_images/1000015157.jpg')\nimg_path0 <- file.path(tempdir(), '1000015157.jpg')\nimage_write(img0, img_path0)\n\nimg1 <- image_read('../input/cassava-leaf-disease-classification/train_images/100042118.jpg')\nimg_path1 <- file.path(tempdir(), '100042118.jpg')\nimage_write(img1, img_path1)\n\nimg2 <- image_read('../input/cassava-leaf-disease-classification/train_images/1000910826.jpg')\nimg_path2 <- file.path(tempdir(), '1000910826.jpg')\nimage_write(img2, img_path2)\n\nimg3 <- image_read('../input/cassava-leaf-disease-classification/train_images/1000201771.jpg')\nimg_path3 <- file.path(tempdir(), '1000201771.jpg')\nimage_write(img3, img_path3)\n\nimg4 <- image_read('../input/cassava-leaf-disease-classification/train_images/1000201771.jpg')\nimg_path4 <- file.path(tempdir(), '1000201771.jpg')\nimage_write(img4, img_path4)\n\nplot(as.raster(img3))\n\nplot_superpixels(img_path3, n_superpixels = 35, weight = 10)\nplot_superpixels(img_path3, n_superpixels = 200, weight = 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model <- application_vgg16(weights = \"imagenet\", include_top = TRUE)\n\nimage_prep <- function(x) {\n  arrays <- lapply(x, function(path) {\n    img <- image_load(path, target_size = c(224,224))\n    x <- image_to_array(img)\n    x <- array_reshape(x, c(1, dim(x)))\n    x <- imagenet_preprocess_input(x)\n  })\n  do.call(abind::abind, c(arrays, list(along = 1)))\n}\n\nres <- predict(model, image_prep(c(img_path0, img_path1, img_path2, img_path3, img_path4#)))\n                                   )))\n#imagenet_decode_predictions(res)\n\nmodel_labels <- readRDS(system.file('extdata', 'imagenet_labels.rds', package = 'lime'))\nexplainer <- lime(c(img_path0, img_path1, img_path2, img_path3, img_path4), \n                  as_classifier(model, model_labels), image_prep)\n\nexplanation <- explain(c(img_path0, img_path1, img_path2, img_path3, img_path4), \n                       explainer, n_labels = 5, n_features = 20)\nexplanation <- as.data.frame(explanation) #It will not plot the image as tbl so we must turn it into dataframe","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#explanation\nexplanation %>%\n  ggplot(aes(x = feature_weight)) +\n    facet_wrap(~ case, scales = \"free\") +\n    geom_density()\n#plot_features(explanation)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CBB <- explanation[explanation$case == \"1000015157.jpg\",]\nCBSD <- explanation[explanation$case == \"100042118.jpg\",]\nCGM <- explanation[explanation$case == \"1000910826.jpg\",]\nCMD <- explanation[explanation$case == \"1000201771.jpg\",]\nHealthy <- explanation[explanation$case == \"1000201771.jpg\",]\nCategory_to_look_at <- CMD\nplot_image_explanation(Category_to_look_at)\nplot_image_explanation(Category_to_look_at, display = 'block', threshold = -0.002)\nplot_image_explanation(Category_to_look_at, threshold = 0, show_negative = TRUE, fill_alpha = 0.6)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"3.6.3"}},"nbformat":4,"nbformat_minor":4}