{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Visualizing the areas of an image used for making predictions: Class Activation Maps on Bengali\n\nThe goal of this notebook is to investigate the use of Class Activation Maps for an EfficientNetB0.\n\nThis notebook is heavily inspired form the work of @cdeotte in his [notebook for the Understanding Clouds competitions](https://www.kaggle.com/cdeotte/unsupervised-masks-cv-0-60) and the work he refers to that can be found [here](https://alexisbcook.github.io/2017/global-average-pooling-layers-for-object-localization/). (This details why the Global Average Pooling is used, and shows a nice implementation for ResNet50). I also wanted to turn towards this after seeing what @pnussbaum did for his [mind reading notebook](https://www.kaggle.com/pnussbaum/grapheme-mind-reader-panv12-nogpu) where we explores what the different conolutional layers look like. His work is a bit different, but still very interesting nonetheless! I'm sure combining filter + convolutional visualizations with class activation maps can provide some really interesting feedback on how models make there predictions, and how to improve them!."},{"metadata":{},"cell_type":"markdown","source":"### Multi-Class Activation Maps\n\nThere doesn't seem to be much word on multi-class activation maps. There has been a lot of dicussions about wether 1 model with 3 outputs, or 3 models with 1 output each is better for this competition, but everytime it comes down to compute time, and local cross-validation scores. People then _assume_ that this or that is better, but it seems like we don't know much. Hopefully methods like this will help give more evidence towards one method or the other."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import sys\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nimport cv2\n\n# Plotting\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\n\n# Modelling\nimport tensorflow\nfrom tensorflow import keras\nfrom tensorflow.keras.layers import Input, Add, Dense, Dropout, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D, AveragePooling2D, MaxPooling2D, GlobalMaxPooling2D, GlobalAveragePooling2D\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.preprocessing import image\nfrom tensorflow.keras.initializers import glorot_uniform\n\n# Metrics & Loss\nfrom tensorflow.keras import optimizers\nfrom tensorflow.keras.losses import categorical_crossentropy\nfrom sklearn.metrics import f1_score\n\nimport os","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Adding the EfficientNet library\n\nsys.path.append(os.path.abspath(\"../input/efficientnet-keras-source-code/repository/qubvel-efficientnet-c993591\"))\nimport efficientnet.tfkeras as efn","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"SEED_VALUE = 42","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"model_nb_output_dictionary = {\n    'grapheme_root': 168,\n    'vowel_diacritic': 11,\n    'consonant_diacritic': 7\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Original image size\nHEIGHT = 137\nWIDTH = 236\n\n# Size of the images that will be resized\nRESIZED_HEIGHT = 75\nRESIZED_WIDTH = 75","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We tweak a little bit how the model is created, by adding a GlobalAveragePooling layer (once again, more on why [here](https://alexisbcook.github.io/2017/global-average-pooling-layers-for-object-localization/)), as well as also creating a CAM_model and keeping the weights of the last convolutional layer. More on that later."},{"metadata":{"trusted":true},"cell_type":"code","source":"def EfficientNetB0_with_CAM(input_shape=(HEIGHT, WIDTH, 1),\n                            classes_dict=model_nb_output_dictionary, \n                            print_last_conv_info = False):\n    \"\"\"\n    EfficientNetB0 implementation\n    \n    Arguments:\n    input_shape -- shape of the images of the dataset\n    classes_dict -- dict of integer, number of classes\n\n    Returns:\n    model -- a Model() instance in Keras\n    \"\"\"\n\n    # Define the input as a tensor with shape input_shape\n    X_input = Input(shape=input_shape)\n    x = Conv2D(3, (3,3), padding='same')(X_input)\n\n    # Base model for transfer learning\n    base_model = efn.EfficientNetB0(weights=None, include_top=False, input_tensor=x)\n\n    x = base_model.output\n    \n    # The Global Average Pooling is important to add here.\n    x = GlobalAveragePooling2D()(x)    \n    \n    # 3 outputs for our three symbols\n    out_root = Dense(classes_dict['grapheme_root'], activation='softmax', name='fc_root', kernel_initializer=glorot_uniform(seed=0))(x)\n    out_vowel = Dense(classes_dict['vowel_diacritic'], activation='softmax', name='fc_vowel', kernel_initializer=glorot_uniform(seed=0))(x)\n    out_consonant = Dense(classes_dict['consonant_diacritic'], activation='softmax', name='fc_consonant', kernel_initializer=glorot_uniform(seed=0))(x)\n\n    # Create model\n    model = Model(inputs=X_input,\n                  outputs=[out_root,\n                           out_vowel,\n                           out_consonant],\n                  name='Base_model')\n    \n    # For CAM\n    last_conv = base_model.layers[-3] # In the EfficientNetB0, the last Conv2D layer is the 3rd to last\n    if print_last_conv_info:\n        print(f'last conv layer: {last_conv}')\n        print(f'\\nConfig of the last conv layer:\\n{last_conv.get_config()}')\n    \n    last_dense_root = model.layers[-3]\n    last_dense_vowel = model.layers[-2]\n    last_dense_consonant = model.layers[-1]\n    last_dense_root_weights = last_dense_root.get_weights()[0]\n    last_dense_vowel_weights = last_dense_vowel.get_weights()[0]\n    last_dense_consonant_weights = last_dense_consonant.get_weights()[0]\n    \n    model_cam = Model(inputs = X_input,\n                     outputs = (last_conv.output,\n                                last_dense_root.output,\n                                last_dense_vowel.output,\n                                last_dense_consonant.output),\n                     name = 'CAM_model')\n    \n    dense_layer_weights_list = [last_dense_root_weights, last_dense_vowel_weights, last_dense_consonant_weights]\n\n    return model, model_cam, dense_layer_weights_list","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model, model_cam, dense_layer_weights_list = EfficientNetB0_with_CAM(\n                                                    input_shape = (RESIZED_HEIGHT, RESIZED_WIDTH, 1),\n                                                    classes_dict=model_nb_output_dictionary,\n                                                    print_last_conv_info=False\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The EfficientNetB0 + Global Average Pooling was trained in another notebook in order to only focus on the Class Activaton Maps in this notebook."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Loading the model trained with GlobalAveragePooling after the EfficientNetB0.\n\nmodel.load_weights('/kaggle/input/efficientnetb0-cam-weights/EfficientNetB0_for_CAM_common_10_epochs_Earlystop (2).h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def crop_and_resize_images(df, resized_df, resize_size = RESIZED_HEIGHT):\n    \"\"\"\n    Function used to resize & center the images.\n    \"\"\"\n    cropped_imgs = {}\n    for img_id in tqdm(range(df.shape[0])):\n        img = resized_df[img_id]\n        _, thresh = cv2.threshold(img, 30, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)\n        contours, _ = cv2.findContours(thresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)[-2:]\n        \n        idx = 0 \n        ls_xmin = []\n        ls_ymin = []\n        ls_xmax = []\n        ls_ymax = []\n        for cnt in contours:\n            idx += 1\n            x,y,w,h = cv2.boundingRect(cnt)\n            ls_xmin.append(x)\n            ls_ymin.append(y)\n            ls_xmax.append(x + w)\n            ls_ymax.append(y + h)\n        xmin = min(ls_xmin)\n        ymin = min(ls_ymin)\n        xmax = max(ls_xmax)\n        ymax = max(ls_ymax)\n\n        roi = img[ymin:ymax,xmin:xmax]\n        resized_roi = cv2.resize(roi, (resize_size, resize_size))\n        cropped_imgs[df.image_id[img_id]] = resized_roi.reshape(-1)\n        \n    resized = pd.DataFrame(cropped_imgs).T.reset_index()\n    resized.columns = resized.columns.astype(str)\n    resized.rename(columns={'index':'image_id'},inplace=True)\n    return resized","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"df = pd.read_parquet(\"/kaggle/input/bengaliai-cv19/train_image_data_0.parquet\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"NUMBER_OF_IMAGES_FOR_CAM = 50","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Resizing the images to 75x75 and centering them\n\ndf_subset = df.head(NUMBER_OF_IMAGES_FOR_CAM)\nresized = df_subset.iloc[:, 1:].values.reshape(-1, HEIGHT, WIDTH)\ncropped_df = crop_and_resize_images(df_subset, resized, RESIZED_HEIGHT)\nindexes = df_subset.index.values\ndf_subset.set_index('image_id', inplace=True)\nresized = cropped_df.iloc[:, 1:].values.reshape(-1, RESIZED_HEIGHT, RESIZED_WIDTH)\nresized = resized.reshape(-1, RESIZED_HEIGHT, RESIZED_WIDTH, 1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We're going to only compute these for a small number of images, in this case 50 (75x75x1) images."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(f'Shape of the resized images : {resized.shape}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Getting the features and the predictions for each of the 3 symbols:\n\nfeatures, preds_root, preds_vowel, preds_consonant = model_cam.predict(resized)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(f'Features shape: {features.shape}')\nprint(f'Root predictions shape: {preds_root.shape}')\nprint(f'Vowel predictions shape: {preds_vowel.shape}')\nprint(f'Consonant predictions shape: {preds_consonant.shape}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If we break down what we have:\n\nfeatures: [ number of images (50) ; height of filters (3) ;  width of filters (3) ; number of filters (1280) ] \n\nThese is the output from the last convolutional 2 layer of the EfficientNetB0 model. However, simply taken alone, those filters don't represent much. We need to multiple them by the different weights of the Global Average Pooling layer, which will then tell us which ones are activated for each image, for each of the 3 symbols."},{"metadata":{},"cell_type":"markdown","source":"# Creating the Class Activation Maps\n\nA Class Activation Map, as its name implies, gives the activation map for a given class. We are interested in seeing the one for each of our 3 symbols. For each of those, we could respectfully create 168, 11 and 7 class activation map, for each of the possible values. However we'll only show the one for the predicted class from the model."},{"metadata":{"trusted":true},"cell_type":"code","source":"# We need to choose 1 image to plot:\n\nIMG_TO_PLOT = 42","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Getting the predictions for each symbol:\nraw_preds_list = [preds_root, preds_vowel, preds_consonant]\n\nroot_preds = np.argmax(preds_root[IMG_TO_PLOT])\nvowel_preds = np.argmax(preds_vowel[IMG_TO_PLOT])\nconsonant_preds = np.argmax(preds_consonant[IMG_TO_PLOT])\n\npreds_for_img_list = [root_preds, vowel_preds, consonant_preds]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# We then can then see the features for this specific image:\nprint(f'Shape of the feature map of a given image: {features[IMG_TO_PLOT,:,:,:].shape}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"However, these are a lot of different 3x3 filters. If we want to see their activations, we need to scale them up to the size of the image"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Upscaling those features to the size of the image:\nimport scipy\n\nscale_factor_height = RESIZED_HEIGHT/features[IMG_TO_PLOT,:,:,:].shape[0]\nscale_factor_width = RESIZED_HEIGHT/features[IMG_TO_PLOT,:,:,:].shape[1]\n\nprint(f'Scale height factor: {scale_factor_height}')\nprint(f'Scale width factor: {scale_factor_width}')\n\nupscaled_features = scipy.ndimage.zoom(features[IMG_TO_PLOT,:,:,:],\n                                       (scale_factor_height, scale_factor_width, 1), order=1)\nprint(f'\\nScaled feature map size for a given image: {upscaled_features.shape}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Finally, in order to visualize which features are activated and which aren't, we need to multiply them by the weights of the final dense layer of the symbol & class we wish to visualise. This is a simple dot product."},{"metadata":{"trusted":true},"cell_type":"code","source":"symbol_to_plot = 0\n\ncam_output = np.dot(upscaled_features, \n                    dense_layer_weights_list[symbol_to_plot][:,preds_for_img_list[symbol_to_plot]])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For visual interpretation, we can plot the original feature map, and the upscaled one, to see how the upscaling works."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"original_cam_output = np.dot(features[IMG_TO_PLOT,:,:,:],\n                                          dense_layer_weights_list[symbol_to_plot][:,preds_for_img_list[symbol_to_plot]])\n\nfig, (ax0, ax1) = plt.subplots(1, 2, figsize = (10, 5))\n\nax0.imshow(original_cam_output)\nax0.set_title('Original CAM output')\n\nax1.imshow(cam_output)\nax1.set_title(\"Rescaled CAM output\")\n\nplt.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"All that is left to do is to perform this for each of the 3 symbols, and plot it above the original image. This will then allow us to have a sense of which areas are mostly used for making the predictions in the image."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def showing_cam(img, \n                img_arrays, \n                features=features, \n                #predicted_img_list=preds_for_img_list, \n                raw_preds_list=raw_preds_list, \n                dense_layer_weights_list=dense_layer_weights_list):\n    \n    features_for_img = features[img,:,:,:]\n    \n    root_preds = np.argmax(raw_preds_list[0][img])\n    vowel_preds = np.argmax(raw_preds_list[1][img])\n    consonant_preds = np.argmax(raw_preds_list[2][img])\n    predicted_img_list = [root_preds, vowel_preds, consonant_preds]\n    \n    preds_root_round = np.round(raw_preds_list[0][img][root_preds], 3)\n    preds_vowel_round = np.round(raw_preds_list[1][img][vowel_preds], 3)\n    preds_consonant_round = np.round(raw_preds_list[2][img][consonant_preds], 3)\n    \n    # Upscaling those features to the size of the image:\n    scale_factor_height = RESIZED_HEIGHT/features[IMG_TO_PLOT,:,:,:].shape[0]\n    scale_factor_width = RESIZED_HEIGHT/features[IMG_TO_PLOT,:,:,:].shape[1]\n    \n    upscaled_features = scipy.ndimage.zoom(features[img,:,:,:], \n                                           (scale_factor_height, scale_factor_width, 1), \n                                           order=1)\n    \n    prediction_for_img = []\n    cam_weights = []\n    cam_output = []\n    \n    for symbol in range(3):\n        prediction_for_img.append(predicted_img_list[symbol])\n        cam_weights.append(dense_layer_weights_list[symbol][:,prediction_for_img[symbol]])\n        cam_output.append(np.dot(upscaled_features, cam_weights[symbol]))\n    \n    fig, (ax0, ax1, ax2, ax3) = plt.subplots(1, 4, figsize=(15, 10))\n    \n    squeezed_img = np.squeeze(img_arrays[img], -1)\n    \n    #fig.suptitle(img_prediction_class)\n    ax0.imshow(squeezed_img, cmap='Greys')\n    ax0.set_title(\"Original image\")\n    \n    ax1.imshow(squeezed_img, cmap='Greys', alpha=0.5)\n    ax1.imshow(cam_output[0], cmap='jet', alpha=0.5)\n    ax1.set_title(f\"CAM - Root ({root_preds} with proba = {preds_root_round})\")\n    #ax1.set_title(f\"CAM - Root\")\n    \n    ax2.imshow(squeezed_img, cmap='Greys', alpha=0.5)\n    ax2.imshow(cam_output[1], cmap='jet', alpha=0.5)\n    ax2.set_title(f\"CAM - Vowel ({vowel_preds} with proba = {preds_vowel_round})\")\n\n    \n    ax3.imshow(squeezed_img, cmap='Greys', alpha=0.5)\n    ax3.imshow(cam_output[2], cmap='jet', alpha=0.5)\n    ax3.set_title(f\"CAM - Consonant ({consonant_preds} with proba = {preds_consonant_round})\")\n\n    \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for img in range(10,15):\n    showing_cam(img, img_arrays=resized)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"These provide some interesting information about which section of the image is used for each image & symbol.\n\nOnce again, it is important to keep in mind these are made from the 3x3 filters from the last Conv2D layer of the model, and so can only provide some limited information about the overage region of the image used for making the prediction, and not the small details. Having a last convolutional layer with bigger features might be a nice addition to try to find more fine-tuned details in the areas used to make the predictions. Choosing a model that have a bigger filter sizes in the last convolutional layer might also be something to take into account when deploying models that require a high degree of explainability."},{"metadata":{},"cell_type":"markdown","source":"### Ideas & improvements to be made\n\nObvioulsy, as is, there isn't that much we can take form these. However, we could look at all the images with the same root in them for example and visualize if the model picks up the same things in the image all the time. Visualizing the different areas in correctly and incorreclty classified image might also provide helpful information!\n\nAs mentioned above, I would also like to try this on models having bigger filter sizes. Maybe it could be interesting to try to add Convolutional layers after the final ones in the EfficientNet, in a similar fashion as a UNet would, to visualize which areas are being used.\n\nI'm also totally open to any feedback & suggestions!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}