{"cells":[{"metadata":{"_uuid":"9dff97bbe7a93e72917037853050eb79af1a3f30"},"cell_type":"markdown","source":"

Comprehensive Guide of Object Detection Algorithms(Computer Vision)

\n\n---\n

\"\"

\n\n---\n## **Zero to Hero Guide of Object Detection**\n\n---\n## Part-1 Object Detection\n* [**1.1 What is Object Detection?**](#1.1What-is-Object-Detection?)\n* [**1.2 Object Localization**](#1.2-Object-Localization)\n* [**1.3 Maths Behind Object Localization**](#1.3-Maths-Behind-Object-Localization)\n* [**1.4 Object Localization Regression**](#1.4-Object-Localization-Regression)\n* [**1.5 Comparing bounding box prediction accuracy**](#1.5-Comparing-bounding-box-prediction-accuracy)\n\n## Part-2 Algorithm of Object Detection\n* [**HOG**](#HOG)\n* [**RCNN**](#RCNN)\n * [**2.1 Understanding of RCNN**](#2.1-Understanding-of-RCNN)\n * [**2.2 Problems with RCNN**](#2.2-Problems-with-RCNN)\n* [**SPP-NET**](#SPP-NET)\n* [**FastRCNN**](#FastRCNN)\n * [**2.3 Understanding Fast RCNN**](#2.3-Understanding-Fast-RCNN)\n * [**2.4 Problems with Fast RCNN**](#2.4-Problems-with-Fast-RCNN)\n* [**FasterRCNN**](#FasterRCNN)\n * [**2.5 Understanding Faster RCNN**](#2.5-Understanding-Faster-RCNN)\n * [**2.6 Problems with Faster RCNN**](#2.6-Problems-with-Faster-RCNN)\n* [**Summary of the Algorithms covered**](#Summary-of-the-Algorithms-covered)\n"},{"metadata":{"_uuid":"6cae496954f81f0e58c1201b92e4ebc4713d7098"},"cell_type":"markdown","source":"---\n## **Part-1 Object Detection**\n---\n### **1.1 What is Object Detection?**\n---\n\n\n\n

\"Smiley\n

\n

\n\n"},{"metadata":{"_uuid":"a981dd95d365c79ff66c2740acf53cad40b99abb"},"cell_type":"markdown","source":"### **1.2 Object Localization**\n\n---\n\n\n

\"Smiley\n

\n

\n\n\n\n

A Bounding Box can be initialized using the following parameters:

\n
  • bx, by : coordinates of the center of the bounding box
    • \n
  • bw : width of the bounding box w.r.t the image width
    • \n
  • bh : height of the bounding box w.r.t the image height
    • \n\n### **1.3 Maths Behind Object Localization**\n---\n![](https://storage.googleapis.com/kagglesdsdata/datasets/70591/163612/11.JPG?GoogleAccessId=web-data@kaggle-161607.iam.gserviceaccount.com&Expires=1548055371&Signature=a%2FTlIjiLUSBI70BDaEuKw%2F10wZ1b8KsiOMVPlvWnoAkrqFFSr4qACEbGz8qusPnA5wJ7AtjXFMw2SM8%2FALgzG2jyWcbtqwm14mCtW7nalBpMncdWVTxU669ZYu7qdEb3xBij6gi1FtykEvpgtZW%2FmRofmusZalfrvvZUFs7wGZ9RbFXLkqvolCUkkGelqm8vi9eKxC1%2Btbh74ofnsZkzZxt978YrpIwYyWluT5CXPtsU46amdj75%2BVvCFTRdy8pkJcobe%2BaCTWDccftu9K1u9eABQcIFO3%2FgxrZTZfbgDeWP1P886Et7Xgg5Nd7SPoeKzrLSj%2B5ZXcnTNj27ULppeg%3D%3D)\n![](https://storage.googleapis.com/kagglesdsdata/datasets/70591/163612/22.JPG?GoogleAccessId=web-data@kaggle-161607.iam.gserviceaccount.com&Expires=1548056181&Signature=rQ%2BksrL8z50UAd8s5QF9MxJA5UXnZRtTcWv5aia2c%2F%2FqP60xN1lt0j%2FtqBQlKSAFNnY7IJDtlfSIIUV%2FzHKNk5rKOIeF9soeBHdgzItYvuUTlASconSjrdiADA%2BlPL4vx8jqfIP0obREW0s92UrzPukJ8ZKGFptZZKI4RZoCMXXYd6QD3l25S6NgEb7k4ZI%2Fbxu75hc4CLWRSGxCEm70c7gXeHxxUELftjmwSw10S%2FYSPLoAsPAZLeLREjSXCHs1LV2F%2F7EBvVrFVbmxD57dCx6vlOcZUtd2zN47pY8ZTLyPNvXG6mPy%2BM8Tgf0EXENp9Ub0WNSnAbcJ%2FHOGtQ2qJg%3D%3D)\n\n### **1.4 Object Localization Regression**\n---\n* Regression is about returning a number instead of a class, in our case we're going to return 4 numbers (x0,y0,width,height) that are related to a bounding box. You train this system with an image an a ground truth bounding box, and use L2 distance to calculate the loss between the predicted bounding box and the ground truth.\n![](https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/more_images/LocalizationRegression1.png)\n* Normally what you do is attach another fully connected layer on the last convolution layer\n![](https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/more_images/LocalizationRegression2.png)\n* This will work only for one object at a time. Some people attach the regression part after the last convolution (Overfeat) layer, while others attach after the fully connected layer (RCNN). Both works.\n\n### **1.5 Comparing bounding box prediction accuracy**\n* Basically we need to compare if the Intersect Over Union (ioU) between the prediction and the ground truth is bigger than some threshold **(ex > 0.5)**\n\n![](https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/more_images/Intersection-over-Union-IoU-of.png)"},{"metadata":{"_uuid":"ecdfc75a6270a4ade0cf549dbaa42bf1edb44816"},"cell_type":"markdown","source":"---\n## Part-2 **Algorithm of Object Detection**\n---\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
    AlgorithmFeaturesPrediction time / imageLimitations
    CNNDivides the image into multiple regions and then classifies each region into various classes.–Needs a lot of regions to predict accurately and hence high computation time.
    R-CNNUses selective search to generate regions. Extracts around 2000 regions from each image.40-50 secondsHigh computation time as each region is passed to the CNN separately. Also, it uses three different models for making predictions.
    Fast R-CNNEach image is passed only once to the CNN and feature maps are extracted. Selective search is used on these maps to generate predictions. Combines all the three models used in R-CNN together.2 secondsSelective search is slow and hence computation time is still high.
    Faster R-CNNReplaces the selective search method with region proposal network (RPN) which makes the algorithm much faster.0.2 secondsObject proposal takes time and as there are different systems working one after the other, the performance of systems depends on how the previous system has performed.
    \n\n\n### **HOG**\n* **Object detection** is modeled as a classification problem where we take fixed size windows of an input image in all possible locations and then pass those corrections to an image classifier\n![](https://cv-tricks.com/wp-content/uploads/2017/12/Sliding-window.gif)\n\n![](https://cv-tricks.com/wp-content/uploads/2017/12/pyramid-269x300.png)\n\n* In an **innovative article on the history of computer vision, Navneet Dalal and Bill Triggs presented the features of HOG in 2005.** Hog's features are not economical and can be used to solve problems. Many concrete problems. In each window, obtained by executing the sliding window in the pyramid, we compute the properties of the pig, which are transmitted to a SVM (Suport Vector Machine) to create classifiers. We were able to do this in real-time in pedestrian recognition, face recognition and many other object recognition applications.\n\n### **2.1 Understanding of RCNN**\n---\n![](https://proxy-us4.toolur.com/browse.php?u=5gf%2B7RRDvnm8hsluzHUo6p3Lx5G%2BjNJX%2BZlCqpmPYJTo9877iM2S1bnhEU2gHqxVUTULpN3SbaiBgGYI7ZfmB5t7HouzCZA5C8IdHmWicsg5ijiSQ6gCb%2BYDJwXzCcSqN24e8qiBr6%2Fkw60mwkSu3A5YWM5sbAs4RelChDXissLwvDgKzdTj3w0oXK%2Fu5MgALsNdNzvVOWnWdRQsFRZew5TFDrpMc%2BCQSqpoWcBjSbxywo9sb2tcqqqUXVtmLrYjNQm%2Be%2BMCD0302DnHBPaVd0W4B7EPQERyrR%2BBtFuWzjcv%2F%2FG2nvOysHltKU4JXh%2FKM%2FWmAlZHi8kWqkLIZCrV&b=1)\n* Since we model object recognition as a ***classification problem,*** success depends on the accuracy of the classification. After the **increase in deep learning,** the obvious **idea was to replace HOG-based classifiers with a more precise convolutional classifier based on a neural network.**\n* **Problem with HOG** - However, there was a problem. CNNs were too slow and expensive to calculate. It was impossible to run CNN in so many fleck generated by a sliding window detector.\n* R-CNN solves this problem with an object suggestion algorithm called **Selective Search, which reduces the number of limit frames that are assigned to the classifier to almost 2000 region suggestions.** The **selective search uses local tracks such as texture, intensity, color and / or some privileged information, etc., to generate all the possible positions of the object.** Now we can feed these tables to our CNN-based classifier. Note that the fully connected portion of CNN requires a fixed-size entry. Therefore, we resize (without maintaining the aspect ratio) to a fixed size (224 × 224 for VGG) and pass it to the CNN part. Therefore, there are 3 important parts of R-CNN:\n\n 1. *Execute selective search to generate probable objects.*\n 1. *Feed these patches with CNN, followed by SVM to predict the class of each patch.*\n 1. *Optimize the patches by separately training the regression of the regression chart.*\n\n![](https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/more_images/RCNNSimple.png)\n\n\n#### **RCNN Working**\n\n* ***Step 1 - Train (or download) a classification model (like AlexNet).***\n\n![](https://dzone.com/storage/temp/9814895-screen-shot-2018-07-23-at-114121-am.png)\n\n* ***Step 2 - finetune the model***\n * Change the number of categories from 1000 to 20.\n * Remove the last complete connection layer\n \n![](https://dzone.com/storage/temp/9814896-screen-shot-2018-07-23-at-114132-am.png)\n\n* ***step 3 - Feature extraction:***\n * Extract all candidate fields from the image ([selective search](https://www.cs.cornell.edu/courses/cs7670/2014sp/slides/VisionSeminar14.pdf)/[Official Reference](https://www.koen.me/research/pub/vandesande-iccv2011-poster.pdf)).\n * **For each range:** Correct the size of the range corresponding to the CNN input, perform a forward operation, and dump the fifth pooled level (that is, the features extracted from the candidate field) to disk.\n\n![](https://dzone.com/storage/temp/9814897-screen-shot-2018-07-23-at-114143-am.png)\n\n* ***Step 4***\n * Train an SVM classifier (2 categorías) to determine the category of objects in this candidate box.\n * Determine if the SVM belongs to each of the categories. If so, the result is positive and negative, if not. For example, the following figure shows the SVM for a dog classification.\n \n![](https://dzone.com/storage/temp/9814898-screen-shot-2018-07-23-at-114155-am.png)\n\n* ***Step 5***\n * Use regression to optimize the position of candidate tables. For each class, train a linear regression model to see if the table is optimized.\n \n![](https://dzone.com/storage/temp/9814899-screen-shot-2018-07-23-at-114207-am.png)\n\n\n### **2.2 Problems with RCNN**\n---\n* Extract 2,000 regions for each image based on selective search.\n* Extract features using CNN for each region of the image. Suppose we have N images, then the number of CNN features will be N 2,000\n* The whole object detection process using RCNN has three models:\n * CNN for feature extraction\n * Linear SVM classifier for the identification of objects.\n * Regression model to tighten the bounding boxes.\n\nAll these processes combine to make RCNN very slow. It takes around 40 to 50 seconds to make predictions for each new image, which essentially makes the model cumbersome and virtually impossible to construct when faced with a gigantic data set.\n\n#### This is the good news: we have another object detection technique that corrects most of the limitations we saw in RCNN."},{"metadata":{"_uuid":"a2d08b3011d96dcee5fa58ea721fdfc9227b8caf"},"cell_type":"markdown","source":"***RCNN Practical Implementation : http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/detection.ipynb***"},{"metadata":{"_uuid":"d1f6dd24ef7bdeeb9b91e9bcf1d9c6b5f7cc75d2"},"cell_type":"markdown","source":"### **SPP-NET**\n\n* The idea of Spatial Pyramid Pooling (SPP) has made a significant contribution to the R-CNN. Here we will give a brief introduction of SPP Net.\n\n![](https://img-blog.csdn.net/20180521093651167)\n\n![](https://img-blog.csdn.net/20180521093539391)\n\n**SPP has two characteristics:**\n\n1. Integrate the spatial pyramid method at the entrance of scale to the CNN A full connection layer or a classifier follows the general CNN. Everyone needs a fixed input size, so they have to cut or distort the input data. This preprocessing can cause data loss or geometric distortion. The first contribution of SPP is the addition of the idea of a pyramid to CNN to achieve data entry on multiple scales.\n As shown in the image below, an SPP layer is added between the convolutional layer and the fully connected layer. Now, the scale of any scale. In the SPP layer, the filter of each cluster will be sized according to the input, and the scale of the SPP output will always be set.\n\n![](https://cv-tricks.com/wp-content/uploads/2016/12/CNN.png)\n\n2. Extract the folding features of the original image only once.\n In R-CNN, each candidate box is scaled down to a standard size and then subdivided into the CNN input, thereby reducing efficiency.\n On the other hand, SPP Net applies a single convolution to obtain a complete feature map, then locates the map patch of the zaifeature map of each candidate cell, and treats this patch as the convolution feature entry of the SPP layer and subsequent levels. This saves a lot of computing time because it's a hundred times faster than R-CNN.\n\n### **2.3 Understanding Fast RCNN**\n---\n![](https://proxy-us4.toolur.com/browse.php?u=5gf%2B7RRDvnm8hsluzHUo6p3Lx5G%2BjNJX%2BZlCqpmPM5a6rcz60p%2FF0Lq3EUn6RKRXVz9Qq9qCbaSG1TNX7Ze2B8YqGIziUZA5C8IdHmWicsg5ijiSQ6gCb%2BYDJwXzCcSqN24e8qiBr6%2Fkw60mwkSu3A5YWM5sbAs4RelChDXissLwvDgKzdTj3w0oXK%2Fu5MgALsNdNzvVOWnWdRQsFRZew5TFDrpMc%2BCQSqpoWcBjSblwx45pYGtYqq%2BRWV5jK7MjNQy0e%2B8HCEj22DnCD%2FaXckC4BbEPRUdyrRqHtFyWzjct%2F%2FazmvaysH5tKU4GXh7KN%2FWkAlNHjskWqkLIZCrV&b=1)\n![](https://proxy-us4.toolur.com/browse.php?u=5gf%2B7RRDvnm8hsluzHUo6p3Lx5G%2BjNJX%2BZlCqpmPMsLt%2Bsz9hc2S1%2B28FUykQaVQUjUBr4%2FWPqmC0GAM6M%2FgUZctH93lVpA5C8IdHmWicsg5ijiSQ6gCb%2BYDJwXzCcSqN24e8qiBr6%2Fkw60mwkSu3A5YWM5sbAs4RelChDXissLwvDgKzdTj3w0oXK%2Fu5MgALsNdNzvVOWnWdRQsFRZew5TFDrpMc%2BCQSqpoWcBjSbx2x4lpZWtZqq2RUl5lK7YjNQy%2Be%2B4CDE322DnCDvacd0G9B7EPRU5ypxqFtF2Wzjci%2BvCzkva%2BsHpoJE4JXhzPNPWkB1JHg8kWqkLIZCrV&b=1)\n\n* Fast RCNN uses the ideas of SPP-net and RCNN and solves the key problem of SPP-net, i.e. ***They allowed to train from end to end.*** To propagate the gradients by spatial pooling, a simple back propagation calculation is used that is very similar to the computation of the max-pooling gradient, except that the pooling regions overlap and so the gradients can be present. It consists of several pumped regions\n\n\n* Another thing that FAST RCNN has done quickly has been to add the bounding box regression training to the formation of the neural network. Now, the network had two heads, the classification head and the regression head of the bounding box. This goal of multitasking is an essential feature of speed because classification and location no longer require independent network training. Both of these modifications reduce overall training time and increase the accuracy of the SPP net through CNN's comprehensive learning.\n\n\n* The fast RCNN method receives region suggestions from an external system (selective search). These proposals will be sent to a Roi Pooling level, where all regions with their data will be fixed in size. This step is necessary because the fully connected layer expects all vectors to be the same size.\n\n![](https://docs.microsoft.com/en-us/cognitive-toolkit/tutorial_fastrcnn/rcnnpipeline.jpg)\n\n* What else can we do to reduce the calculation time of an RCNN algorithm? Instead of running 2,000 times CNN per image, we can run it once per image and get all the regions of interest (regions that contain an object)."},{"metadata":{"_uuid":"dd1a6df1de00f7e331162a64a10af65274ddbc9d"},"cell_type":"markdown","source":"We will divide this into steps to simplify the concept:\n\n1. **As with the two previous techniques, we take a picture as input.**\n\n![](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/10/Screenshot-from-2018-10-08-14-59-021.png)\n\n2. **This image is transmitted to a ConvNet network, which in turn generates the regions of interest.**\n\n![](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/10/Screenshot-from-2018-10-08-15-44-03.png)\n\n3. **In all these regions, a RoI pooling leyer is applied to reshape them according to the ConvNet input. Then each region becomes a fully connected network.**\n\n***ROI Layer Processing***\n\n![](https://www.researchgate.net/profile/Xiaochuan_Fan/publication/323142262/figure/fig5/AS:593417776148482@1518493231702/An-illustration-of-ROI-pooling-by-including-masking-operations-a-ROI-pooling-2-2.png)\n\n![](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/10/Screenshot-from-2018-10-08-15-45-26.png)\n\n4. **A Softmax layer is used at the top of the fully connected network to generate the classes. In the Softmax layer, a linear regression layer is also used in parallel to generate the bounding box coordinates for the predicted classes.**\n\n![](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2018/10/Screenshot-from-2018-10-08-15-47-18.png)"},{"metadata":{"_uuid":"eb9f2e26f20c97c8a3471571257fac5c218d7ddc"},"cell_type":"markdown","source":"### **2.4 Problems with Fast RCNN**\n---\n* Fast RCNN also presents some problems. It also uses selective search as a method of suggestion to find regions of interest, which is a lazy process. The recognition of objects takes approximately 2 seconds per image, which is much better compared to RCNN. But if you look at the actual records, even a fast RCNN will not seem that fast. \n\n### **Computation View of FRCNN**\n\n* In F - RCNN, it is not necessary to run CNN each time when image recognition is performed, it is only necessary to cut out the feature region extracted from Region Proposal and give it to all the coupling layers . Compared to the conventional R - CNN running the CNN layer for each image recognition, it can achieve a significant speedup.\n\n* Assuming that RegionProposal was 1000 times, the amount of computation can be 1/1000 of the \n`\nconventional R-CNN: CNN * 1000 + FC * 1000 \nFR-CNN: CNN * 1 + FC * 1000 \nand CNN! \n`\n* In addition, F - RCNN uses a learning technique called Multi - task loss , and has successfully learned the entire model end - to - end including the RegionProposal model. This leads to improved learning efficiency. \n* This image showed that model precision converged properly even if back propagation was applied simultaneously to RP loss and image recognition model loss.\n* As a result, Fast R - CNN realizes 150 × inference speed improvement and 10 × learning speed improvement for R - CNN. Fast as the name !!!!!!!!!\n\n***FAST RCNN Implementation : https://github.com/Liu-Yicheng/Fast-RCNN***\n\n![](https://dzone.com/storage/temp/9814913-screen-shot-2018-07-23-at-114252-am.png)\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505142556821-1286928548.png)"},{"metadata":{"_uuid":"06f29c0e41f3cfa5318f963446b4a469903d639b"},"cell_type":"markdown","source":"### **2.5 Understanding Faster RCNN**\n\n---\n\n* ***The problem with Fast R-CNN:*** There is a bottleneck: Selective search, find all candidate boxes, this is also very time consuming. So can we find a more efficient way to find these candidate boxes? \n* ***Solution:*** Add a neural network that extracts edges, which means that the work of finding the candidate box is also done to the neural network. \n\nTherefore, rgbd introduces ***Region Proposal Network (RPN) instead of Selective Search in Fast R-CNN***, and introduces **anchor box to deal with the change of target shape (anchor is a fixed position and size box, which can be understood as a fixed proposal set in advance).** \n\n> To do this: \n* **RPN (Region proposals): Gives a set of rectangles based on deep convolution layer.**\n* **Fast-RCNN Roi Pooling layer: Classify each proposal, and refining proposal location**\n \n ![](https://img-blog.csdn.net/20180502185307597)\n \n \n #### **Speed Comparision**\n \n ---\n \n ![](https://img-blog.csdn.net/2018050218534282)\n \n ***Notes : The main contribution of Faster R-CNN is to design the network RPN for extracting candidate regions, instead of the time-consuming selective search selective search, which greatly improves the detection speed. ***\n \n \n ### **Working of FasterRCNN**\n \n ---\n \n \n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505140852035-53224161.png)\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505140951802-712034784.png)\n\n\n**Region Proposal Network:**\n\n* The Region Proposal Network (RPN) takes the output feature map of the first convolutional network as input. It slides a 3 × 3 convolution kernel on the feature map to construct a category-independent candidate region using a convolutional network (ZF network as shown below). Other deep networks such as VGG or ResNet can be used for more comprehensive feature extraction, but at the expense of speed. The ZF network will eventually output 256 values, which will be fed to two separate fully connected layers to predict the bounding box and two objectness scores, which measure whether the bounding box contains the target. In fact, we can calculate a single score objectness regression is, but for the sake of brevity, Faster R-CNN use only two types of classifiers: the band have targeted categories and classes without destination .\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505141538370-1305739714.png)\n\n* For each position in the feature map, the RPN will make k predictions. Therefore, the RPN will output 4 x k coordinates and 2 x k scores per position. The figure below shows an 8 × 8 feature map with a 3 × 3 convolution kernel performing the operation, which finally outputs 8 × 8 × 3 ROIs (where k = 3). The figure below (right) shows three candidate areas for a single location.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505142042648-254532648.png)\n\n* There are 3 guesses here, and we will improve them later. Since only one correct guess is needed, our initial guess is best to cover different shapes and sizes. Therefore, Faster R-CNN does not create a random bounding box. Instead, it predicts some offsets (such as x, y) associated with the reference frame named \"Anchor\" in the upper left corner . We limit the values ​​of these offsets, so our guess is still similar to the anchor point.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505142155131-51553556.png)\n\n* To make k predictions for each location, we need k anchor points centered on each location. Each prediction is associated with a specific anchor point, but different locations share anchor points of the same shape.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505142221601-1484881357.png)\n\n* These anchor points are carefully chosen so they are diverse and cover realistic targets with different scales and aspect ratios. This allows us to guide the initial training with better guesses and allows each prediction to be specific to a particular shape. This strategy makes early training more stable and simple.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505142409837-738297303.png)\n\n* Faster R-CNN uses more anchor points. It deploys 9 anchor boxes: 3 anchor boxes of 3 different sizes with different aspect ratios. Each location uses 9 anchor points, each generating 2 x 9 objectness scores and 4 x 9 coordinates.\n\n\n### **Region-based Full Convolutional Neural Network (R-FCN)**\n\n---\n\nSuppose we only have one feature map to detect the right eye. So can we use it to locate people? it should be OK. Because the right eye should be in the upper left corner of the face image, we can use this to locate the entire face.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505142731192-1064481204.png)\n\n- If we have other feature maps for detecting the left eye, nose or mouth, then we can combine the test results to better locate the face. \n- Now let's review all the issues. In Faster R-CNN, the detector uses multiple fully connected layers for prediction. If there are 2000 ROIs, the cost is very high.\n\n```\nFeature_maps = process(image)\nROIs = region_proposal(feature_maps)\n for ROI in ROIs\n Patch = roi_pooling(feature_maps, ROI)\n Class_scores, box = detector(patch) # Expensive! \n class_probabilities = softmax(class_scores)\n```\n\nR-FCN accelerates by reducing the amount of work required for each ROI (with the fully connected layer removed) . The above region-based feature map is independent of the ROI and can be calculated separately from each ROI. The rest of the work is relatively simple, so R-FCN is faster than Faster R-CNN.\n```\nFeature_maps = process(image)\nROIs = region_proposal(feature_maps) \nScore_maps = compute_score_map(feature_maps)\n for ROI in ROIs\n V = region_roi_pool(score_maps, ROI) \n Class_scores, box = average(V) # Much simpler! \n class_probabilities = softmax(class_scores)\n```\n- Now let's take a look at the 5 × 5 feature map M, which contains a blue square inside. We divide the square into 3 × 3 regions equally. Now we have created a new feature map in M ​​to detect the top left corner (TL) of the block. This new feature map is shown below (right). Only the yellow grid cells [2, 2] are active.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505143333476-553828348.png)\n\n* We divided the box into 9 parts, which created 9 feature maps, each for detecting the corresponding target area. These feature maps are called position-sensitive score maps because each graph detects sub-regions of the target (calculating their scores).\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505143440734-1537004993.png)\n\n* The red dotted rectangle in the figure below is the recommended ROI. We split it into 3 × 3 regions and ask what the probability that each region contains the corresponding part of the target. For example, the upper left corner ROI area contains the probability of the left eye. We store the result as a 3 × 3 vote array, as shown in the following figure (right). For example, vote_array[0][0] contains a score for whether the upper left corner contains the corresponding part of the target.\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505143707803-1747244764.png)\n\n* The process of mapping a Feature Map and ROI to a vote array is called a position-sensitive ROI-pool .\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505144039681-1211806338.png)\n\n* 　After calculating all the values ​​of the location-sensitive ROI pooling, the category score is the average of all of its element scores .\n\n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505144213719-1278140951.png)\n\n* Suppose we have C categories to test. We expanded it to C + 1 categories, which added a new category to the background (non-target). There are 3 × 3 score maps for each category, so there are a total of (C + 1) × 3 × 3 score maps. Use the score graph for each category to predict the category score for that category. We then apply the softmax function to these scores to calculate the probability of each category.\n The following is a data flow diagram, in our case, k=3.\n \n![](https://images2018.cnblogs.com/blog/1192699/201805/1192699-20180505144433895-646844741.png)\n"},{"metadata":{"_uuid":"f247e3ca7c2ef23470d1a34f5961117404bb3c48"},"cell_type":"markdown","source":"### **2.6 Problems with Faster RCNN**\n\n---\nAll of the object recognition algorithms described so far use regions to identify the objects. The network does not look at the whole image at once, but focuses on parts of the image sequentially. This leads to two complications:\n\n- The algorithm requires many passes through a single image to extract all objects.\n- Because there are several systems that operate sequentially, the performance of the systems depends on the performance of the previous systems.\n\n---"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true},"cell_type":"markdown","source":"### **References :**\n\n---\n1. https://cv-tricks.com/object-detection/faster-r-cnn-yolo-ssd/\n1. https://towardsdatascience.com/r-cnn-fast-r-cnn-faster-r-cnn-yolo-object-detection-algorithms-36d53571365e\n1. https://towardsdatascience.com/evolution-of-object-detection-and-localization-algorithms-e241021d8bad\n1. https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852\n1. https://www.analyticsvidhya.com/blog/2018/10/a-step-by-step-introduction-to-the-basic-object-detection-algorithms-part-1/\n1. https://www.datacamp.com/community/news/a-step-by-step-introduction-to-the-basic-object-detection-algorithms-part-1-analytics-vidhya-6u92kfhewwm\n1. https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/object_localization_and_detection.html\n1. https://lilianweng.github.io/lil-log/2017/10/29/object-recognition-for-dummies-part-1.html"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}