{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Detecting faces in the video using OpenCV\n\nThis kernel is a very simple update from the original Kernel https://www.kaggle.com/robikscube/kaggle-deepfake-detection-introduction showing that it is not necessary to install the package `face_recognition` in order to identify faces. We can simply import the XML model which is already available in the OpenCV directory:\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import cv2, os\nhaar_path = '/opt/conda/lib/python3.6/site-packages/cv2/data'\n!ls {haar_path}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Nothe that besides `frontal_face` there are many other features that are already available to be identified. In our example, let's import `haarcascade_frontalface_alt2.xml`:"},{"metadata":{"trusted":true},"cell_type":"code","source":"xml_name = 'haarcascade_frontalface_alt2.xml'\nxml_path = os.path.join(haar_path, xml_name)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Based on the XML path, let's declare a CascadeClassifier:"},{"metadata":{"trusted":true},"cell_type":"code","source":"clf = cv2.CascadeClassifier(xml_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The following section is the same of the original kernel, I just added two new lines, one to convert the image into gray scale and the other to identify the face locations:\n```\ngray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\nface_locations = clf.detectMultiScale(gray)\n```"},{"metadata":{"trusted":true},"cell_type":"code","source":"import cv2 as cv\nimport os\nimport matplotlib.pylab as plt\n\nfig, ax = plt.subplots(1,1, figsize=(15, 15))\nvideo_file = '/kaggle/input/deepfake-detection-challenge/train_sample_videos/akxoopqjqz.mp4'\ncap = cv.VideoCapture(video_file)\nsuccess, image = cap.read()\n\n# Convert image into gray scale and classify images\ngray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\nface_locations = clf.detectMultiScale(gray)\n\n# Continue with the original code\nimage = cv.cvtColor(image, cv.COLOR_BGR2RGB)\ncap.release()   \nax.imshow(image)\nax.xaxis.set_visible(False)\nax.yaxis.set_visible(False)\nax.title.set_text(f\"FRAME 0: {video_file.split('/')[-1]}\")\nplt.grid(False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Locating a face within an image\nAs a difference from `face_recognition`, instead of identifying top, left, bottom and right, OpenCV identify the X and Y coordinates and their respectives width and height."},{"metadata":{"trusted":true},"cell_type":"code","source":"from PIL import Image\n\nprint(\"I found {} face(s) in this photograph.\".format(len(face_locations)))\n\nfor face_location in face_locations:\n\n    # Print the location of each face in this image\n    x, y, w, h = face_location\n    print(\"A face is located at pixel location X: {}, Y: {}, Width: {}, Height: {}\".format(x, y, w, h))\n\n    # You can access the actual face itself like this:\n    face_image = image[y:y+h, x:x+w]\n    fig, ax = plt.subplots(1,1, figsize=(5, 5))\n    plt.grid(False)\n    ax.xaxis.set_visible(False)\n    ax.yaxis.set_visible(False)\n    ax.imshow(face_image)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Done! Thanks Rob Mulla for the [great kernel]( https://www.kaggle.com/robikscube/kaggle-deepfake-detection-introduction)!"},{"metadata":{},"cell_type":"markdown","source":"## UPDATE #1\nLet's also visualize the other atributes that can be extracted:"},{"metadata":{"trusted":true},"cell_type":"code","source":"import glob\nall_xmls = glob.glob(haar_path + '/*.xml')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Option of XML features:\")\nprint([xml.split('/')[-1] for xml in all_xmls])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for xml in all_xmls:\n    if xml.split(\"/\")[-1]=='haarcascade_licence_plate_rus_16stages.xml': # Skipping. This attribute is throwing an error\n        print(f\"Skipping {xml}\")\n        continue \n        \n    clf = cv2.CascadeClassifier(xml)\n    locations = clf.detectMultiScale(gray)\n\n    name_xml = xml.split(\"/\")[-1].split(\".\")[0].replace(\"haarcascade_\", \"\")\n    print('='*80)\n    print(f'Feature to be extracted: {name_xml}')\n    print(f\"I found {len(locations)} {name_xml} in this photograph.\")\n\n    for location in locations:\n\n        # Print the location of each face in this image\n        x, y, w, h = location\n        print(f\"A {name_xml} is located at pixel location X: {x}, Y: {y}, Width: {w}, Height: {h}\")\n\n        # You can access the actual face itself like this:\n        attribute_image = image[y:y+h, x:x+w]\n        fig, ax = plt.subplots(1,1, figsize=(5, 5))\n        plt.grid(False)\n        ax.xaxis.set_visible(False)\n        ax.yaxis.set_visible(False)\n        ax.imshow(attribute_image)\n        plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Conclusion\nWe can observe that some of XMLs performs very badly, for example smile was identified 30 times in the image (!?!?)"},{"metadata":{},"cell_type":"markdown","source":"## Update #2: MTCNN\nBased on [this discussion](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121523), I will also test the usage of the package MTCNN. It will be installed offline using the dataset provided by Kaggler unkownhihi: https://www.kaggle.com/unkownhihi/mtcnn-package"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install ../input/mtcnn-package/mtcnn-0.1.0-py3-none-any.whl","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from mtcnn import MTCNN\ndetector = MTCNN()\nresult = detector.detect_faces(image); result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's declare variables with those attributes and visualize:"},{"metadata":{"trusted":true},"cell_type":"code","source":"x, y, w, h = result[0]['box']\nright_eye = result[0]['keypoints']['right_eye']\nnose = result[0]['keypoints']['nose']\nmouth_left = result[0]['keypoints']['mouth_left']\nmouth_right = result[0]['keypoints']['mouth_right']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video_file = '/kaggle/input/deepfake-detection-challenge/train_sample_videos/akxoopqjqz.mp4'\ncap = cv.VideoCapture(video_file)\nsuccess, image = cap.read()\nimage = cv.cvtColor(image, cv.COLOR_BGR2RGB)\ncap.release()\n\ncv2.rectangle(image, (x, y), (x+w, y+h), (0, 0, 255), thickness=5)\nfig, ax = plt.subplots(1,1, figsize=(15, 15))\nplt.grid(False)\nax.xaxis.set_visible(False)\nax.yaxis.set_visible(False)\nax.imshow(image)\nax.plot(right_eye[0], right_eye[1], 'go') # Right eye in green\nax.plot(nose[0], nose[1], 'yo') # Nose in yellow\nax.plot(mouth_left[0], mouth_left[1], 'ro') # Left and right mouth in red\nax.plot(mouth_right[0], mouth_right[1], 'ro')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}