{"cells":[{"metadata":{},"cell_type":"markdown","source":"By skipping frames and using trackers, we can extract facial bounding boxes from video in the Deepfake Detection Challenge.\n\nTo begin, we install MTCNN, which has a dependency of opencv-python. \n\nThen we uninstall opencv-python so we can install opencv-contrib-python to use opencv's trackers."},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install /kaggle/input/dfdc-packages/mtcnn-0.1.0-py3-none-any.whl\n!pip uninstall numpy -y\n!pip uninstall opencv-python -y\n!pip install --upgrade --force-reinstall /kaggle/input/dfdc-packages/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl\n!pip install --upgrade --force-reinstall --no-deps /kaggle/input/dfdc-packages/opencv_contrib_python-4.2.0.32-cp36-cp36m-manylinux1_x86_64.whl","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import os\nimport sys\nimport cv2\nimport csv\nimport time\nimport string\nimport random\nfrom mtcnn.mtcnn import MTCNN\nfrom datetime import timedelta\nfrom joblib import Parallel, delayed","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We introduce a helper function to generate ids for each trackable face"},{"metadata":{"trusted":true},"cell_type":"code","source":"def id_gen(chars=string.ascii_uppercase, id_len=6):\n    return ''.join(random.choice(chars) \n             for x in range(id_len)).lower()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Finally, the main function iterates through frames in the source video.\n\nTo process videos more quickly, we grab frames and only load every skip_frame to perform face detection.\n\nSince trackers drift, we reinitialize trackers after an expiration period.\n\nWe write the frame number and corresponding bounding box to files for each trackable face."},{"metadata":{"trusted":true},"cell_type":"code","source":"def main(source, skip_frames=5, expiration=30):\n    cap = cv2.VideoCapture(source)\n    name = os.path.basename(source).replace('.mp4', '.csv')\n    n_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))\n    for idx in range(n_frames):\n        ret = cap.grab()\n        if ret:\n            if not idx % skip_frames:\n                ret, frame = cap.retrieve()\n                if not idx % expiration:\n                    file_dict, tracker_dict = {}, {}\n                    faces = [tuple(face['box']) for face in detector.detect_faces(frame)]\n                    for box in faces:\n                        tracker_id = id_gen()\n                        tracker = cv2.TrackerCSRT_create()\n                        tracker.init(frame, box)\n                        tracker_dict[tracker] = tracker_id\n                        dataFile = open(DATA_DIR + '{}_{}'.format(tracker_id, name), 'w')\n                        file_dict[tracker_id] = csv.writer(dataFile)\n                else:\n                    for tracker in list(tracker_dict.keys()):\n                        (success, box) = tracker.update(frame)\n                        box = list(map(int, box))\n                        file_dict[tracker_dict[tracker]].writerow([idx] + box)\n        else:\n            break\n    cap.release()\n    return              ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Initialize face detector\ndetector = MTCNN()\n\nINPUT_DIR = \"/kaggle/input/deepfake-detection-challenge/test_videos/\"\nDATA_DIR = '/kaggle/working/'\n\nstart_time = time.time()\nfor vid_fl in os.listdir(INPUT_DIR):\n    main(INPUT_DIR + vid_fl)\nelapsed = time.time() - start_time\nprint(\"Elapsed time to process test set: \", str(timedelta(seconds=elapsed)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Processing video files sequentially is time consuming and inefficient. To speed up our processing we can use the ```Parallel``` module from the ```joblib``` library and take advantage of all cores available to process chunks of videos."},{"metadata":{"trusted":true},"cell_type":"code","source":"!mkdir -p /kaggle/working/output_parallel/\n\nDATA_DIR = '/kaggle/working/output_parallel/'\ntest_videos = os.listdir(INPUT_DIR)\ntest_videos = [INPUT_DIR + fl for fl in test_videos]\n\nstart_time = time.time()\nParallel(n_jobs=4)(delayed(main)(fl) for fl in test_videos)\nelapsed = time.time() - start_time\nprint(\"Elapsed time to process test set: \", str(timedelta(seconds=elapsed)))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}