{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction"},{"metadata":{},"cell_type":"markdown","source":"I am going to introduce Mobilenet SSD face extractor. **It is better than MTCNN/same with MTCNN in accuracy, but still have a competitive speed** I am aware of dual shot detector is better, but it takes too long.\n\nIt is recommended by @harshit_sheoran. FYI, it is used in our best score kernel(0.34LB)."},{"metadata":{},"cell_type":"markdown","source":"We are going to compare blazeface, mtcnn, and Mobilenet Face Extractor(I will call it mobilenet for the rest of this kernel). For this kernel, I am going to take only the most confident face bbox detected from the first frame.\n\nBlazeFace helper funciton taken from [link](https://www.kaggle.com/basharallabadi/yolov2-vs-faced-vs-blazeface-vs-mtcnn), \n\nmtcnn code taken from [link](https://github.com/ipazc/mtcnn), \n\nmobilenet code/weights taken from [link](https://github.com/yeephycho/tensorflow-face-detection)\n\nyolo code taken from [link](https://www.kaggle.com/basharallabadi/yolov2-vs-faced-vs-blazeface-vs-mtcnn)"},{"metadata":{},"cell_type":"markdown","source":"**Here is the helper code for this face extractor(clean version)[link](https://www.kaggle.com/unkownhihi/mobilenet-face-extractor-helper-code)**"},{"metadata":{},"cell_type":"markdown","source":"# Imports"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import sys\nimport torch\nimport matplotlib.pyplot as plt\nimport cv2\nimport time\nimport tensorflow as tf\nimport numpy as np\nimport math\nsys.path.insert(0, \"/kaggle/input/blazeface-pytorch\")\n\nfrom blazeface import BlazeFace","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Initialize BlazeFace"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"blazeface = BlazeFace()\nblazeface.load_weights(\"/kaggle/input/blazeface-pytorch/blazeface.pth\")\nblazeface.load_anchors(\"/kaggle/input/blazeface-pytorch/anchors.npy\")\n\n# Optionally change the thresholds:\nblazeface.min_score_thresh = 0.75\nblazeface.min_suppression_threshold = 0.3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_blaze_boxes(detections, with_keypoints=False):\n    result = []\n    if isinstance(detections, torch.Tensor):\n        detections = detections.cpu().numpy()\n\n    if detections.ndim == 1:\n        detections = np.expand_dims(detections, axis=0)\n\n    img_shape = (128, 128)\n    for i in range(detections.shape[0]):\n        ymin = detections[i, 0] * img_shape[0]\n        xmin = detections[i, 1] * img_shape[1]\n        ymax = detections[i, 2] * img_shape[0]\n        xmax = detections[i, 3] * img_shape[1]\n        result.append((xmin, ymin, xmax, ymax))\n    return result\ndef scale_boxes(boxes, scale_w, scale_h):\n    sb = []\n    for b in boxes:\n        sb.append((b[0] * scale_w, b[1] * scale_h, b[2] * scale_w, b[3] * scale_h))\n    return sb","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Initialize Yolo"},{"metadata":{"trusted":true},"cell_type":"code","source":"from tensorflow.keras.layers import Conv2D, Input, ZeroPadding2D, Dense, Lambda\nfrom tensorflow.keras.models import Model\nfrom tensorflow.keras.applications.mobilenet_v2 import MobileNetV2\ndef load_mobilenetv2_224_075_detector(path):\n    input_tensor = Input(shape=(224, 224, 3))\n    output_tensor = MobileNetV2(weights=None, include_top=False, input_tensor=input_tensor, alpha=0.75).output\n    output_tensor = ZeroPadding2D()(output_tensor)\n    output_tensor = Conv2D(kernel_size=(3, 3), filters=5)(output_tensor)\n\n    model = Model(inputs=input_tensor, outputs=output_tensor)\n    model.load_weights(path)\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mobilenetv2 = load_mobilenetv2_224_075_detector(\"../input/facedetection-mobilenetv2/facedetection-mobilenetv2-size224-alpha0.75.h5\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def transpose_shots(shots):\n    return [(shot[1], shot[0], shot[3], shot[2], shot[4]) for shot in shots]\n\n#That constant describe pieces for 16:9 images\nSHOTS = {\n    # fast less accurate\n    '2-16/9' : {\n        'aspect_ratio' : 16/9,\n        'shots' : [\n             (0, 0, 9/16, 1, 1),\n             (7/16, 0, 9/16, 1, 1)\n        ]\n    },\n    # slower more accurate\n    '10-16/9' : {\n        'aspect_ratio' : 16/9,\n        'shots' : [\n             (0, 0, 9/16, 1, 1),\n             (7/16, 0, 9/16, 1, 1),\n             (0, 0, 5/16, 5/9, 0.5),\n             (0, 4/9, 5/16, 5/9, 0.5),\n             (11/48, 0, 5/16, 5/9, 0.5),\n             (11/48, 4/9, 5/16, 5/9, 0.5),\n             (22/48, 0, 5/16, 5/9, 0.5),\n             (22/48, 4/9, 5/16, 5/9, 0.5),\n             (11/16, 0, 5/16, 5/9, 0.5),\n             (11/16, 4/9, 5/16, 5/9, 0.5),\n        ]\n    }\n}\n\n# 9:16 respectively\nSHOTS_T = {\n    '2-9/16' : {\n        'aspect_ratio' : 9/16,\n        'shots' : transpose_shots(SHOTS['2-16/9']['shots'])\n    },\n    '10-9/16' : {\n        'aspect_ratio' : 9/16,\n        'shots' : transpose_shots(SHOTS['10-16/9']['shots'])\n    }\n}\n\ndef r(x):\n    return int(round(x))\n\ndef sigmoid(x):\n    return 1 / (np.exp(-x) + 1)\n\ndef non_max_suppression(boxes, p, iou_threshold):\n    if len(boxes) == 0:\n        return np.array([])\n\n    x1 = boxes[:, 0]\n    y1 = boxes[:, 1]\n    x2 = boxes[:, 2]\n    y2 = boxes[:, 3]\n\n    indexes = np.argsort(p)\n    true_boxes_indexes = []\n\n    while len(indexes) > 0:\n        true_boxes_indexes.append(indexes[-1])\n\n        intersection = np.maximum(np.minimum(x2[indexes[:-1]], x2[indexes[-1]]) - np.maximum(x1[indexes[:-1]], x1[indexes[-1]]), 0) * np.maximum(np.minimum(y2[indexes[:-1]], y2[indexes[-1]]) - np.maximum(y1[indexes[:-1]], y1[indexes[-1]]), 0)\n        iou = intersection / ((x2[indexes[:-1]] - x1[indexes[:-1]]) * (y2[indexes[:-1]] - y1[indexes[:-1]]) + (x2[indexes[-1]] - x1[indexes[-1]]) * (y2[indexes[-1]] - y1[indexes[-1]]) - intersection)\n\n        indexes = np.delete(indexes, -1)\n        indexes = np.delete(indexes, np.where(iou >= iou_threshold)[0])\n\n    return boxes[true_boxes_indexes]\n\ndef union_suppression(boxes, threshold):\n    if len(boxes) == 0:\n        return np.array([])\n\n    x1 = boxes[:, 0]\n    y1 = boxes[:, 1]\n    x2 = boxes[:, 2]\n    y2 = boxes[:, 3]\n\n    indexes = np.argsort((x2 - x1) * (y2 - y1))\n    result_boxes = []\n\n    while len(indexes) > 0:\n        intersection = np.maximum(np.minimum(x2[indexes[:-1]], x2[indexes[-1]]) - np.maximum(x1[indexes[:-1]], x1[indexes[-1]]), 0) * np.maximum(np.minimum(y2[indexes[:-1]], y2[indexes[-1]]) - np.maximum(y1[indexes[:-1]], y1[indexes[-1]]), 0)\n        min_s = np.minimum((x2[indexes[:-1]] - x1[indexes[:-1]]) * (y2[indexes[:-1]] - y1[indexes[:-1]]), (x2[indexes[-1]] - x1[indexes[-1]]) * (y2[indexes[-1]] - y1[indexes[-1]]))\n        ioms = intersection / (min_s + 1e-9)\n        neighbours = np.where(ioms >= threshold)[0]\n        if len(neighbours) > 0:\n            result_boxes.append([min(np.min(x1[indexes[neighbours]]), x1[indexes[-1]]), min(np.min(y1[indexes[neighbours]]), y1[indexes[-1]]), max(np.max(x2[indexes[neighbours]]), x2[indexes[-1]]), max(np.max(y2[indexes[neighbours]]), y2[indexes[-1]])])\n        else:\n            result_boxes.append([x1[indexes[-1]], y1[indexes[-1]], x2[indexes[-1]], y2[indexes[-1]]])\n\n        indexes = np.delete(indexes, -1)\n        indexes = np.delete(indexes, neighbours)\n\n    return result_boxes\n\nclass FaceDetector():\n    \"\"\"\n    That's API you can easily use to detect faces\n    \n    __init__ parameters:\n    -------------------------------\n    model - model to infer\n    shots - list of aspect ratios that images could be (described earlier)\n    image_size - model's input size (hardcoded for mobilenetv2)\n    grids - model's output size (hardcoded for mobilenetv2)\n    union_threshold - threshold for union of predicted boxes within multiple shots\n    iou_threshold - IOU threshold for non maximum suppression used to merge YOLO detected boxes for one shot,\n                    you do need to change this because there are one face per image as I can see from the samples\n    prob_threshold - probability threshold for YOLO algorithm, you can balance beetween precision and recall using this threshold\n    \n    detect parameters:\n    -------------------------------\n    frame - (1920, 1080, 3) or (1080, 1920, 3) RGB Image\n    returns: list of 4 element tuples (left corner x, left corner y, right corner x, right corner y) of detected boxes within [0, 1] range (see box draw code below)\n    \"\"\"\n    def __init__(self, model=mobilenetv2, shots=[SHOTS['10-16/9'], SHOTS_T['10-9/16']], image_size=224, grids=7, iou_threshold=0.1, union_threshold=0.1):\n        self.model = model\n        self.shots = shots\n        self.image_size = image_size\n        self.grids = grids\n        self.iou_threshold = iou_threshold\n        self.union_threshold = union_threshold\n        self.prob_threshold = 0.7\n        \n    \n    def detect(self, frame, threshold = 0.7):\n        original_frame_shape = frame.shape\n        self.prob_threshold = threshold\n        aspect_ratio = None\n        for shot in self.shots:\n            if abs(frame.shape[1] / frame.shape[0] - shot[\"aspect_ratio\"]) < 1e-9:\n                aspect_ratio = shot[\"aspect_ratio\"]\n                shots = shot\n        \n        assert aspect_ratio is not None\n        \n        c = min(frame.shape[0], frame.shape[1] / aspect_ratio)\n        slice_h_shift = r((frame.shape[0] - c) / 2)\n        slice_w_shift = r((frame.shape[1] - c * aspect_ratio) / 2)\n        if slice_w_shift != 0 and slice_h_shift == 0:\n            frame = frame[:, slice_w_shift:-slice_w_shift]\n        elif slice_w_shift == 0 and slice_h_shift != 0:\n            frame = frame[slice_h_shift:-slice_h_shift, :]\n\n        frames = []\n        for s in shots[\"shots\"]:\n            frames.append(cv2.resize(frame[r(s[1] * frame.shape[0]):r((s[1] + s[3]) * frame.shape[0]), r(s[0] * frame.shape[1]):r((s[0] + s[2]) * frame.shape[1])], (self.image_size, self.image_size), interpolation=cv2.INTER_NEAREST))\n        frames = np.array(frames)\n\n        predictions = self.model.predict(frames, batch_size=len(frames), verbose=0)\n\n        boxes = []\n        prob = []\n        shots = shots['shots']\n        for i in range(len(shots)):\n            slice_boxes = []\n            slice_prob = []\n            for j in range(predictions.shape[1]):\n                for k in range(predictions.shape[2]):\n                    p = sigmoid(predictions[i][j][k][4])\n                    if not(p is None) and p > self.prob_threshold:\n                        px = sigmoid(predictions[i][j][k][0])\n                        py = sigmoid(predictions[i][j][k][1])\n                        pw = min(math.exp(predictions[i][j][k][2] / self.grids), self.grids)\n                        ph = min(math.exp(predictions[i][j][k][3] / self.grids), self.grids)\n                        if not(px is None) and not(py is None) and not(pw is None) and not(ph is None) and pw > 1e-9 and ph > 1e-9:\n                            cx = (px + j) / self.grids\n                            cy = (py + k) / self.grids\n                            wx = pw / self.grids\n                            wy = ph / self.grids\n                            if wx <= shots[i][4] and wy <= shots[i][4]:\n                                lx = min(max(cx - wx / 2, 0), 1)\n                                ly = min(max(cy - wy / 2, 0), 1)\n                                rx = min(max(cx + wx / 2, 0), 1)\n                                ry = min(max(cy + wy / 2, 0), 1)\n\n                                lx *= shots[i][2]\n                                ly *= shots[i][3]\n                                rx *= shots[i][2]\n                                ry *= shots[i][3]\n\n                                lx += shots[i][0]\n                                ly += shots[i][1]\n                                rx += shots[i][0]\n                                ry += shots[i][1]\n\n                                slice_boxes.append([lx, ly, rx, ry])\n                                slice_prob.append(p)\n\n            slice_boxes = np.array(slice_boxes)\n            slice_prob = np.array(slice_prob)\n\n            slice_boxes = non_max_suppression(slice_boxes, slice_prob, self.iou_threshold)\n\n            for sb in slice_boxes:\n                boxes.append(sb)\n\n\n        boxes = np.array(boxes)\n        boxes = union_suppression(boxes, self.union_threshold)\n\n        for i in range(len(boxes)):\n            boxes[i][0] /= original_frame_shape[1] / frame.shape[1]\n            boxes[i][1] /= original_frame_shape[0] / frame.shape[0]\n            boxes[i][2] /= original_frame_shape[1] / frame.shape[1]\n            boxes[i][3] /= original_frame_shape[0] / frame.shape[0]\n\n            boxes[i][0] += slice_w_shift / original_frame_shape[1]\n            boxes[i][1] += slice_h_shift / original_frame_shape[0]\n            boxes[i][2] += slice_w_shift / original_frame_shape[1]\n            boxes[i][3] += slice_h_shift / original_frame_shape[0]\n\n        return list(boxes)\ndef get_boxes_points(boxes, frame_shape):\n    result = []\n    for box in boxes:\n        lx = int(round(box[0] * frame_shape[1]))\n        ly = int(round(box[1] * frame_shape[0]))\n        rx = int(round(box[2] * frame_shape[1]))\n        ry = int(round(box[3] * frame_shape[0]))\n        result.append((lx,rx, ly, ry))\n    return result ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"yolo_model = FaceDetector()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Initialize MTCNN"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install mtcnn","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from mtcnn import MTCNN\nmtcnn = MTCNN()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Initialize MobilenetFace"},{"metadata":{"trusted":true},"cell_type":"code","source":"import tensorflow as tf\ndetection_graph = tf.Graph()\nwith detection_graph.as_default():\n    od_graph_def = tf.compat.v1.GraphDef()\n    with tf.io.gfile.GFile('../input/mobilenet-face/frozen_inference_graph_face.pb', 'rb') as fid:\n        serialized_graph = fid.read()\n        od_graph_def.ParseFromString(serialized_graph)\n        tf.import_graph_def(od_graph_def, name='')\n        config = tf.compat.v1.ConfigProto()\n    config.gpu_options.allow_growth = True\n    sess=tf.compat.v1.Session(graph=detection_graph, config=config)\n    image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')\n    boxes_tensor = detection_graph.get_tensor_by_name('detection_boxes:0')    \n    scores_tensor = detection_graph.get_tensor_by_name('detection_scores:0')\n    num_detections = detection_graph.get_tensor_by_name('num_detections:0')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video='../input/deepfake-detection-challenge/train_sample_videos/bdnaqemxmr.mp4'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cap=cv2.VideoCapture(video)\nret,frame=cap.read()\nframe=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nplt.imshow(frame)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define Helper Fuctions"},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_mtcnn_face(img):\n    start=time.time()\n    bboxes=mtcnn.detect_faces(frame)[0]['box']\n    x,y,w,h=bboxes\n    bboxes=x,x+w,y,y+h\n    return time.time()-start, bboxes\ndef get_blazeface_face(img):\n    start=time.time()\n    scale_w = img.shape[1] / 128.0 \n    scale_h = img.shape[0] / 128.0\n    blaze_output=blazeface.predict_on_image(cv2.resize(frame, (128,128)))\n    blaze_bboxes=scale_boxes(get_blaze_boxes(blaze_output), scale_w, scale_h)\n    if blaze_bboxes==[]:\n        return time.time()-start,[]\n    lx, ly, rx, ry = blaze_bboxes[0]\n    bboxes=int(lx), int(rx), int(ly), int(ry)\n    return time.time()-start, bboxes\ndef get_mobilenet_face(image):\n    start=time.time()\n    global boxes,scores,num_detections\n    (im_height,im_width)=image.shape[:-1]\n    imgs=np.array([image])\n    (boxes, scores) = sess.run(\n        [boxes_tensor, scores_tensor],\n        feed_dict={image_tensor: imgs})\n    max_=np.where(scores==scores.max())[0][0]\n    box=boxes[0][max_]\n    ymin, xmin, ymax, xmax = box\n    (left, right, top, bottom) = (xmin * im_width, xmax * im_width,\n                                ymin * im_height, ymax * im_height)\n    left, right, top, bottom = int(left), int(right), int(top), int(bottom)\n    return time.time()-start,(left, right, top, bottom)\ndef get_yolo_face(image):\n    start=time.time()\n    bbox=yolo_model.detect(frame, 0.7)\n    bbox=get_boxes_points(bbox,frame.shape)[0]\n    return time.time()-start,bbox","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def annotate_image(frame,bbox,color):\n    if bbox==[]:\n        return frame\n    frame=frame.copy()\n    return cv2.rectangle(frame,(bbox[0],bbox[2]),(bbox[1],bbox[3]),color,10)\ndef crop_image(frame,bbox):\n    left, right, top, bottom=bbox\n    return frame[top:bottom,left:right]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Warm up"},{"metadata":{"trusted":true},"cell_type":"code","source":"_=get_blazeface_face(frame)\n_=get_mtcnn_face(frame)\n_=get_mobilenet_face(frame)\n_=get_yolo_face(frame)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Speed Comparison"},{"metadata":{"trusted":true},"cell_type":"code","source":"blaze_time,blaze_bboxes=get_blazeface_face(frame)\nmtcnn_time,mtcnn_bboxes=get_mtcnn_face(frame)\nmobilenet_time,mobilenet_bboxes=get_mobilenet_face(frame)\nyolo_time,yolo_bboxes=get_yolo_face(frame)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"MTCNN Detection Time:\"+str(mtcnn_time))\nprint(\"Yolo Detection Time:\"+str(yolo_time))\nprint(\"Mobilenet Detection Time:\"+str(mobilenet_time))\nprint(\"BlazeFace Detection Time:\"+str(blaze_time))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Mobilenet is '+str(mtcnn_time/mobilenet_time)+'times faster than MTCNN')\nprint('Blazeface is '+str(mobilenet_time/blaze_time)+'times faster than Mobilenet')\nprint('Mobilenet is '+str(yolo_time/mobilenet_time)+'times faster than YOLO')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Ability to Detect Face"},{"metadata":{"trusted":true},"cell_type":"code","source":"if blaze_bboxes==[]:\n    print('⚠️BlazeFace is unable to detect face in this frame.')\nif mtcnn_bboxes==[]:\n    print('⚠️MTCNN is unable to detect face in this frame.')\nif mobilenet_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nif yolo_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Accuracy Comparison"},{"metadata":{},"cell_type":"markdown","source":"## Annotated Images"},{"metadata":{"trusted":true},"cell_type":"code","source":"annotated=annotate_image(frame,mobilenet_bboxes,(255,0,0))\nannotated=annotate_image(annotated,mtcnn_bboxes,(0,255,0))\nannotated=annotate_image(annotated,blaze_bboxes,(0,0,255))\nannotated=annotate_image(annotated,yolo_bboxes,(255,0,255))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(annotated)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Color Key:\n* Blue: BlazeFace\n* Red: Mobilenet\n* Green: MTCNN\n* Purple: YOLO"},{"metadata":{},"cell_type":"markdown","source":"## Cropped Images"},{"metadata":{},"cell_type":"markdown","source":"### BlazeFace"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(crop_image(frame,blaze_bboxes))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### MTCNN"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(crop_image(frame,mtcnn_bboxes))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Mobilenet"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(crop_image(frame,mobilenet_bboxes))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### YOLO"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(crop_image(frame,yolo_bboxes))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Extra Comparisons"},{"metadata":{"trusted":true},"cell_type":"code","source":"video='../input/deepfake-detection-challenge/train_sample_videos/eqnoqyfquo.mp4'\ncap=cv2.VideoCapture(video)    \nret,frame=cap.read()\nframe=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nblaze_time,blaze_bboxes=get_blazeface_face(frame)\nmtcnn_time,mtcnn_bboxes=get_mtcnn_face(frame)\nmobilenet_time,mobilenet_bboxes=get_mobilenet_face(frame)\nyolo_time,yolo_bboxes=get_yolo_face(frame)  \nprint(\"MTCNN Detection Time:\"+str(mtcnn_time))\nprint(\"Yolo Detection Time:\"+str(yolo_time))\nprint(\"Mobilenet Detection Time:\"+str(mobilenet_time))\nprint(\"BlazeFace Detection Time:\"+str(blaze_time))\nprint('Mobilenet is '+str(mtcnn_time/mobilenet_time)+'times faster than MTCNN')\nprint('Blazeface is '+str(mobilenet_time/blaze_time)+'times faster than Mobilenet')\nprint('Mobilenet is '+str(yolo_time/mobilenet_time)+'times faster than YOLO')\nif blaze_bboxes==[]:\n    print('⚠️BlazeFace is unable to detect face in this frame.')\nif mtcnn_bboxes==[]:\n    print('⚠️MTCNN is unable to detect face in this frame.')\nif mobilenet_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nif yolo_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nannotated=annotate_image(frame,mobilenet_bboxes,(255,0,0))\nannotated=annotate_image(annotated,mtcnn_bboxes,(0,255,0))\nannotated=annotate_image(annotated,blaze_bboxes,(0,0,255))\nannotated=annotate_image(annotated,yolo_bboxes,(255,0,255))\nplt.imshow(annotated)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video='../input/deepfake-detection-challenge/train_sample_videos/eqjscdagiv.mp4'\ncap=cv2.VideoCapture(video)    \nret,frame=cap.read()\nframe=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nblaze_time,blaze_bboxes=get_blazeface_face(frame)\nmtcnn_time,mtcnn_bboxes=get_mtcnn_face(frame)\nmobilenet_time,mobilenet_bboxes=get_mobilenet_face(frame)\nyolo_time,yolo_bboxes=get_yolo_face(frame)  \nprint(\"MTCNN Detection Time:\"+str(mtcnn_time))\nprint(\"Yolo Detection Time:\"+str(yolo_time))\nprint(\"Mobilenet Detection Time:\"+str(mobilenet_time))\nprint(\"BlazeFace Detection Time:\"+str(blaze_time))\nprint('Mobilenet is '+str(mtcnn_time/mobilenet_time)+'times faster than MTCNN')\nprint('Blazeface is '+str(mobilenet_time/blaze_time)+'times faster than Mobilenet')\nprint('Mobilenet is '+str(yolo_time/mobilenet_time)+'times faster than YOLO')\nif blaze_bboxes==[]:\n    print('⚠️BlazeFace is unable to detect face in this frame.')\nif mtcnn_bboxes==[]:\n    print('⚠️MTCNN is unable to detect face in this frame.')\nif mobilenet_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nif yolo_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nannotated=annotate_image(frame,mobilenet_bboxes,(255,0,0))\nannotated=annotate_image(annotated,mtcnn_bboxes,(0,255,0))\nannotated=annotate_image(annotated,blaze_bboxes,(0,0,255))\nannotated=annotate_image(annotated,yolo_bboxes,(255,0,255))\nplt.imshow(annotated)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video='../input/deepfake-detection-challenge/train_sample_videos/emgjphonqb.mp4'\ncap=cv2.VideoCapture(video)    \nret,frame=cap.read()\nframe=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nblaze_time,blaze_bboxes=get_blazeface_face(frame)\nmtcnn_time,mtcnn_bboxes=get_mtcnn_face(frame)\nmobilenet_time,mobilenet_bboxes=get_mobilenet_face(frame)\nyolo_time,yolo_bboxes=get_yolo_face(frame)  \nprint(\"MTCNN Detection Time:\"+str(mtcnn_time))\nprint(\"Yolo Detection Time:\"+str(yolo_time))\nprint(\"Mobilenet Detection Time:\"+str(mobilenet_time))\nprint(\"BlazeFace Detection Time:\"+str(blaze_time))\nprint('Mobilenet is '+str(mtcnn_time/mobilenet_time)+'times faster than MTCNN')\nprint('Blazeface is '+str(mobilenet_time/blaze_time)+'times faster than Mobilenet')\nprint('Mobilenet is '+str(yolo_time/mobilenet_time)+'times faster than YOLO')\nif blaze_bboxes==[]:\n    print('⚠️BlazeFace is unable to detect face in this frame.')\nif mtcnn_bboxes==[]:\n    print('⚠️MTCNN is unable to detect face in this frame.')\nif mobilenet_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nif yolo_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nannotated=annotate_image(frame,mobilenet_bboxes,(255,0,0))\nannotated=annotate_image(annotated,mtcnn_bboxes,(0,255,0))\nannotated=annotate_image(annotated,blaze_bboxes,(0,0,255))\nannotated=annotate_image(annotated,yolo_bboxes,(255,0,255))\nplt.imshow(annotated)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video='../input/deepfake-detection-challenge/train_sample_videos/cffffbcywc.mp4'\ncap=cv2.VideoCapture(video)    \nret,frame=cap.read()\nframe=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nblaze_time,blaze_bboxes=get_blazeface_face(frame)\nmtcnn_time,mtcnn_bboxes=get_mtcnn_face(frame)\nmobilenet_time,mobilenet_bboxes=get_mobilenet_face(frame)\nyolo_time,yolo_bboxes=get_yolo_face(frame)  \nprint(\"MTCNN Detection Time:\"+str(mtcnn_time))\nprint(\"Yolo Detection Time:\"+str(yolo_time))\nprint(\"Mobilenet Detection Time:\"+str(mobilenet_time))\nprint(\"BlazeFace Detection Time:\"+str(blaze_time))\nprint('Mobilenet is '+str(mtcnn_time/mobilenet_time)+'times faster than MTCNN')\nprint('Blazeface is '+str(mobilenet_time/blaze_time)+'times faster than Mobilenet')\nprint('Mobilenet is '+str(yolo_time/mobilenet_time)+'times faster than YOLO')\nif blaze_bboxes==[]:\n    print('⚠️BlazeFace is unable to detect face in this frame.')\nif mtcnn_bboxes==[]:\n    print('⚠️MTCNN is unable to detect face in this frame.')\nif mobilenet_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nif yolo_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nannotated=annotate_image(frame,mobilenet_bboxes,(255,0,0))\nannotated=annotate_image(annotated,mtcnn_bboxes,(0,255,0))\nannotated=annotate_image(annotated,blaze_bboxes,(0,0,255))\nannotated=annotate_image(annotated,yolo_bboxes,(255,0,255))\nplt.imshow(annotated)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video='../input/deepfake-detection-challenge/train_sample_videos/byfenovjnf.mp4'\ncap=cv2.VideoCapture(video)    \nret,frame=cap.read()\nframe=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nblaze_time,blaze_bboxes=get_blazeface_face(frame)\nmtcnn_time,mtcnn_bboxes=get_mtcnn_face(frame)\nmobilenet_time,mobilenet_bboxes=get_mobilenet_face(frame)\nyolo_time,yolo_bboxes=get_yolo_face(frame)  \nprint(\"MTCNN Detection Time:\"+str(mtcnn_time))\nprint(\"Yolo Detection Time:\"+str(yolo_time))\nprint(\"Mobilenet Detection Time:\"+str(mobilenet_time))\nprint(\"BlazeFace Detection Time:\"+str(blaze_time))\nprint('Mobilenet is '+str(mtcnn_time/mobilenet_time)+'times faster than MTCNN')\nprint('Blazeface is '+str(mobilenet_time/blaze_time)+'times faster than Mobilenet')\nprint('Mobilenet is '+str(yolo_time/mobilenet_time)+'times faster than YOLO')\nif blaze_bboxes==[]:\n    print('⚠️BlazeFace is unable to detect face in this frame.')\nif mtcnn_bboxes==[]:\n    print('⚠️MTCNN is unable to detect face in this frame.')\nif mobilenet_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nif yolo_bboxes==[]:\n    print('⚠️mobilenet is unable to detect face in this frame.')\nannotated=annotate_image(frame,mobilenet_bboxes,(255,0,0))\nannotated=annotate_image(annotated,mtcnn_bboxes,(0,255,0))\nannotated=annotate_image(annotated,blaze_bboxes,(0,0,255))\nannotated=annotate_image(annotated,yolo_bboxes,(255,0,255))\nplt.imshow(annotated)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Conclusion"},{"metadata":{},"cell_type":"markdown","source":"After all, blaze face is indeed fast, but not that accurate and also can't detect face. mobilenet face have very close results to MTCNN/YOLO but faster. Feel free to fork this kernel and play around with a other train sample videos. \n\nSorry for bad coding. Might have some kind of bug in these code(especially in blazeface). Please kindly point out if you found out.\n\nPlease upvote this kernel and the associated dataset if you found this helpful."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}