{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Face detection with BlazeFace\n\nThis notebook shows how to use the model from [BlazeFace PyTorch](https://www.kaggle.com/humananalog/blazeface-pytorch) for detecting faces in images."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os, sys\nimport numpy as np\nimport torch\nimport cv2\n\n%matplotlib inline\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For the best results, enable GPU in the notebook. But CPU should work fine as well."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"print(\"PyTorch version:\", torch.__version__)\nprint(\"CUDA version:\", torch.version.cuda)\nprint(\"cuDNN version:\", torch.backends.cudnn.version())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gpu = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\ngpu","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Helper code for making plots:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_detections(img, detections, with_keypoints=True):\n    fig, ax = plt.subplots(1, figsize=(10, 10))\n    ax.grid(False)\n    ax.imshow(img)\n    \n    if isinstance(detections, torch.Tensor):\n        detections = detections.cpu().numpy()\n\n    if detections.ndim == 1:\n        detections = np.expand_dims(detections, axis=0)\n\n    print(\"Found %d faces\" % detections.shape[0])\n        \n    for i in range(detections.shape[0]):\n        ymin = detections[i, 0] * img.shape[0]\n        xmin = detections[i, 1] * img.shape[1]\n        ymax = detections[i, 2] * img.shape[0]\n        xmax = detections[i, 3] * img.shape[1]\n\n        rect = patches.Rectangle((xmin, ymin), xmax - xmin, ymax - ymin,\n                                 linewidth=1, edgecolor=\"r\", facecolor=\"none\", \n                                 alpha=detections[i, 16])\n        ax.add_patch(rect)\n\n        if with_keypoints:\n            for k in range(6):\n                kp_x = detections[i, 4 + k*2    ] * img.shape[1]\n                kp_y = detections[i, 4 + k*2 + 1] * img.shape[0]\n                circle = patches.Circle((kp_x, kp_y), radius=0.5, linewidth=1, \n                                        edgecolor=\"lightskyblue\", facecolor=\"none\", \n                                        alpha=detections[i, 16])\n                ax.add_patch(circle)\n        \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"import sys\nsys.path.insert(0, \"/kaggle/input/blazeface-pytorch\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from blazeface import BlazeFace\n\nnet = BlazeFace().to(gpu)\nnet.load_weights(\"/kaggle/input/blazeface-pytorch/blazeface.pth\")\nnet.load_anchors(\"/kaggle/input/blazeface-pytorch/anchors.npy\")\n\n# Optionally change the thresholds:\nnet.min_score_thresh = 0.75\nnet.min_suppression_threshold = 0.3","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load an image\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"input_dir = \"/kaggle/input/deepfake-detection-challenge/test_videos\"\nvideo_path = os.path.join(input_dir, np.random.choice(os.listdir(input_dir)))\nvideo_path","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video_path = \"/kaggle/input/deepfake-detection-challenge/test_videos/uhrqlmlclw.mp4\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The input image should be 128x128. BlazeFace will not automatically resize the image, you have to do this yourself!\n\nThe images from the Deepfake Detection Challenge dataset are 1920x1080, so resizing to 128x128 will squash them a bit. The version of BlazeFace we're using here doesn't work very well on small faces, so you may need to be more clever about how you resize/crop the input images."},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_frame(video_path):\n    capture = cv2.VideoCapture(video_path)\n    ret, frame = capture.read()\n    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n    frame = cv2.resize(frame, (128, 128))\n    capture.release()\n    return frame","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frame = read_frame(video_path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(frame)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Make the prediction"},{"metadata":{"trusted":true},"cell_type":"code","source":"%time detections = net.predict_on_image(frame)\ndetections.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This returns a PyTorch tensor of shape `(num_faces, 17)`. \n\nHow to interpret these 17 numbers:\n\n- The first 4 numbers describe the bounding box corners: \n    - `ymin, xmin, ymax, xmax`\n    - These are normalized coordinates (between 0 and 1).\n    - Note that y comes before x here!\n- The next 12 numbers are the x,y-coordinates of the 6 facial landmark keypoints:\n    - `right_eye_x, right_eye_y`\n    - `left_eye_x, left_eye_y`\n    - `nose_x, nose_y`\n    - `mouth_x, mouth_y`\n    - `right_ear_x, right_ear_y`\n    - `left_ear_x, left_ear_y`\n    - Tip: these labeled as seen from the perspective of the person, so their right is your left.\n- The final number is the confidence score that this detection really is a face.\n\nIf no faces are found, the tensor has shape `(0, 17)`."},{"metadata":{"trusted":true},"cell_type":"code","source":"detections","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_detections(frame, detections)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Prediction on a batch"},{"metadata":{"trusted":true},"cell_type":"code","source":"frame1 = read_frame(\"/kaggle/input/deepfake-detection-challenge/test_videos/jyfvaequfg.mp4\")\nframe2 = read_frame(\"/kaggle/input/deepfake-detection-challenge/test_videos/gkutjglghz.mp4\")\n\nbatch = np.stack([frame1, frame2])\nbatch.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%time detections = net.predict_on_batch(batch)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The batch prediction returns a list of PyTorch tensors, one for each image in the batch."},{"metadata":{"trusted":true},"cell_type":"code","source":"len(detections)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"[x.shape[0] for x in detections]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_detections(frame1, detections[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_detections(frame2, detections[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}