{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\n#import os\n#for dirname, _, filenames in os.walk('/kaggle/input'):\n #   for filename in filenames:\n  #      print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## SSD\n##### Single Shot MultiBox Detector model for object detection\n\n![alt](https://pytorch.org/assets/images/ssd_diagram.png)"},{"metadata":{},"cell_type":"markdown","source":"# Model Description\n### This SSD300 model is based on the SSD: Single Shot MultiBox Detector paper, which describes SSD as “a method for detecting objects in images using a single deep neural network”. The input size is fixed to 300x300.[paper link](http://arxiv.org/abs/1512.02325)\n\n### Additional enhancement: \n* The conv5_x, avgpool, fc and softmax layers were removed from the original classification model.\n* All strides in conv4_x are set to 1x1.\n\nThe backbone is followed by 5 additional convolutional layers. In addition to the convolutional layers, we attached 6 detection heads:\n\n* The first detection head is attached to the last conv4_x layer.\n* The other five detection heads are attached to the corresponding 5 additional layers.\n\n#### The main difference between this model and the one described in the paper is in the backbone. Specifically, the VGG model is obsolete and is replaced by the ResNet-50 model.\n\n#### Reference: https://pytorch.org/hub/nvidia_deeplearningexamples_ssd/"},{"metadata":{},"cell_type":"markdown","source":"#### The following code will load an SSD model pretrained on COCO dataset from Torch Hub."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import torch \nprecision = 'fp32'\nssd_model = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_ssd', model_math=precision)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### importing libraries "},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy\nimport scipy\nimport skimage \nimport matplotlib as mt\nfrom skimage import io,transform","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loading a set of utility methods for convenient and comprehensive formatting of input and output of the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_ssd_processing_utils')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ssd_model.to('cuda')\nssd_model.eval()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install torchsummary ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Using torchsummary to understand the deep layers of the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"from torchsummary import summary\n\nsummary(ssd_model, (3, 300, 300))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Collecting the test data "},{"metadata":{"trusted":true},"cell_type":"code","source":"uris = [\n    '/kaggle/input/open-images-2019-object-detection/test/c744be039ce8b59f.jpg',\n    '/kaggle/input/open-images-2019-object-detection/test/6ced51a34b3e6bb5.jpg',\n    '/kaggle/input/open-images-2019-object-detection/test/5a3215a639ea3308.jpg',\n    '/kaggle/input/open-images-2019-object-detection/test/827376834a225c73.jpg',\n    '/kaggle/input/open-images-2019-object-detection/test/9a3de6cd6c83f1e0.jpg'\n]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### For convenient and comprehensive formatting of input and output of the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"inputs = [utils.prepare_input(uri) for uri in uris]\ntensor = utils.prepare_tensor(inputs, precision == 'fp16')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Runingn the SSD network to perform object detection."},{"metadata":{"trusted":true},"cell_type":"code","source":"with torch.no_grad():\n    detections_batch = ssd_model(tensor)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### By default, raw output from SSD network per input image contains 8732 boxes with localization and class probability distribution. Let’s filter this output to only get reasonable detections (confidence>40%) in a more comprehensive format."},{"metadata":{"trusted":true},"cell_type":"code","source":"results_per_input = utils.decode_results(detections_batch)\nbest_results_per_input = [utils.pick_best(results, 0.40) for results in results_per_input]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Downloading annotations as the model was trained on COCO dataset,which we need to access in order to translate class IDs into object names"},{"metadata":{"trusted":true},"cell_type":"code","source":"classes_to_labels = utils.get_coco_object_dictionary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualising the outcome "},{"metadata":{"trusted":true},"cell_type":"code","source":"from matplotlib import pyplot as plt\nimport matplotlib.patches as patches\n\nfor image_idx in range(len(best_results_per_input)):\n    fig, ax = plt.subplots(1)\n    # Show original, denormalized image...\n    image = inputs[image_idx] / 2 + 0.5\n    ax.imshow(image)\n    # ...with detections\n    bboxes, classes, confidences = best_results_per_input[image_idx]\n    for idx in range(len(bboxes)):\n        left, bot, right, top = bboxes[idx]\n        x, y, w, h = [val * 300 for val in [left, bot, right - left, top - bot]]\n        rect = patches.Rectangle((x, y), w, h, linewidth=1, edgecolor='r', facecolor='none')\n        ax.add_patch(rect)\n        ax.text(x, y, \"{} {:.0f}%\".format(classes_to_labels[classes[idx] - 1], confidences[idx]*100), bbox=dict(facecolor='white', alpha=0.5))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}