{"cells":[{"metadata":{},"cell_type":"markdown","source":"The reference is taken from following:<br>\n[Deep Learning based Object Detection using YOLOv3 with OpenCV ( Python / C++ )](https://www.learnopencv.com/deep-learning-based-object-detection-using-yolov3-with-opencv-python-c/)<br>\n[YOLO object detection using Opencv with Python](https://pysource.com/2019/06/27/yolo-object-detection-using-opencv-with-python/)","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np \nimport pandas as pd\nimport os\nimport cv2\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"First need to configure yolov3 model with opencv.\nThe *readNet* function from dnn module detects an original framwork of train model and calls automatically the function *readNetFromDarknet*.\nAnd *readNetFromDarknet* function returns the object that is ready to do forward, throw an exception in failure cases.\n\nFor *readNet* function order for passing the weights and cfg files doesn't matter.\n> So here *readNetFromDarknet* also can be used instead of *readNet*.\nBut the reason for considering readNet is to make it generic. If thr trained model belongs to tensorflow *readNet* automatcally calls *readNetFromTensorflow*.\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"net = cv2.dnn.readNet(\"/kaggle/input/yolov3-weight/yolov3.weights\", \"/kaggle/input/yolov3-weight/yolov3.cfg\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's have a look at different layers using *getLayerNames*. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"layer_names = net.getLayerNames()\nprint(\"layers names:\")\nprint(layer_names)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* conv - convolution layer<br>\n  Convolution layer applies a filter to an input to create a feature map\n* bn - batch normalization layer<br>\n This normalize the input for the hidden layer and also helps to reduce the training time, to reduce the effect of covariate shift and also add regularization effect.\n* relu - relu activation layer\n* shortcut - skip connection or residual connection<br>\nThis helps to improve the accuracy for a large neural network which tends to reduce the accuracy because of vanishing gradients as the network grows.\n* Permute - Permute layer<br>\nThis is used to re-order the dimention of the input according to the given pattern.\n* identity - This layer maps the output of unconnected layer to next input layer. [yolo_84(unconnected layer) --> conv_84]\n* upsample - Convolution layer performs downsampling by filtering input genarate the output of a smaller shape compare to input. Upsample layer performs the reverse opration by repeating rows and columns of input.\n* concat - This merges s list of inputs.\n* yolo - This is an output layer which a list of bounding boxes along with the recognised classes.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's idetify output layers using a function *getUnconnectedOutLayersNames*.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"output_layers = net.getUnconnectedOutLayersNames()\nprint(\"output layers:\")\nprint(output_layers)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"YoloV3 is trained to indetify 80 different types of objects.\nLet's fetch this detail from coco names.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"classes = []\nwith open(\"/kaggle/input/coconames/coco.names\", \"r\") as f:\n    classes = [line.strip() for line in f.readlines()]\n    \ncolors = np.random.uniform(0, 255, size=(len(classes), 3)) #This will be used later to assign colors for the bounding box for the detected objects","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The network requires the image is blob format.<br>\nBlob - Binary Large Objects.<br>\nBlob represents the group of pixels having simmilar values and different from surrounding pixels.<br>\nThe function blobFromImage convets the image in blob.<br>\nWe can scale, resize , subtract the mean from each pixels, change the order of the channels from BGR to RGB using swapRB argument and also crop the image.<br>\nWith the method setInput, the blob of an image is set as input for the network.<br>\nThe forward method propragate the blob of an image through the network and return the predictions.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_objects_predictions(img):\n    height, width = img.shape[:2]\n    blob = cv2.dnn.blobFromImage(img, scalefactor = 1/255, size = (416, 416), mean= (0, 0, 0), swapRB = True, crop=False)\n    net.setInput(blob)\n    predictions = net.forward(output_layers)\n    return predictions,height, width","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The first 4 elements represent the center_x, center_y, width and height. The fifth element represents the confidence that the bounding box encloses an object.<br>\nThe rest of the elements are the confidence associated with each class (i.e. object type). The box is assigned to the class corresponding to the highest score for the box.<br>\nThe highest score for a box is also called its confidence.(here the confidence is set as 0.5). If the confidence of a box is less than the given threshold, the bounding box is dropped and not considered for further processing.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_box_dimentions(predictions,height, width, confThreshold = 0.5):\n    class_ids = []\n    confidences = []\n    boxes = []\n    for out in predictions:\n        for detection in out:\n            scores = detection[5:]\n            class_id = np.argmax(scores)#Identifing the class type of the detected object by checking maximum confidence\n            confidence = scores[class_id]\n            if confidence > confThreshold:\n                # Object detected\n                center_x = int(detection[0] * width) #converting center_x with respect to original image size\n                center_y = int(detection[1] * height)#converting center_y with respect to original image size\n                w = int(detection[2] * width)#converting width with respect to original image size\n                h = int(detection[3] * height)#converting height with respect to original image size\n                # Rectangle coordinates\n                x = int(center_x - w / 2)\n                y = int(center_y - h / 2)\n                boxes.append([x, y, w, h])\n                confidences.append(float(confidence))\n                class_ids.append(class_id)\n    return boxes,confidences,class_ids","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The Non max suppression technique is used to ensure that the obeject is detected only once.<br>\nIn this the bounding box with probability more nmsThresold is considered, other bounding boxes will be dropped out.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def non_max_suppression(boxes,confidences,confThreshold = 0.5, nmsThreshold = 0.4):\n    return cv2.dnn.NMSBoxes(boxes, confidences, confThreshold, nmsThreshold)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's draw the bounding boxes.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def draw_bouding_boxes(img,boxes,confidences,class_ids,nms_indexes,colors):\n    for i in range(len(boxes)):\n        if i in nms_indexes:\n            x, y, w, h = boxes[i]\n            label = str(classes[class_ids[i]]) + ' :' + str(int(confidences[i]*100)) + '%'\n            color = colors[i]\n            cv2.rectangle(img, (x, y), (x + w, y + h), color, 3)\n            cv2.putText(img, label, (x, y - 15),cv2.FONT_HERSHEY_PLAIN ,2, color, 3)\n    return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def detect_objects(img_path):\n    predictions,height, width = get_objects_predictions(img_path)\n    boxes,confidences,class_ids = get_box_dimentions(predictions,height, width)\n    nms_indexes = non_max_suppression(boxes,confidences)\n    img = draw_bouding_boxes(img_path,boxes,confidences,class_ids,nms_indexes,colors)\n    return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"files = ['/kaggle/input/open-images-2019-object-detection/test/' + i for i in os.listdir('/kaggle/input/open-images-2019-object-detection/test')]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's have a look at some images with object detection.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(25,30))\n\nfor i in range(1,13):\n    index = np.random.randint(len(files))\n    plt.subplot(6, 2, i)\n    plt.imshow(detect_objects(cv2.imread(files[index])), cmap='cool')\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}