{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport cv2\nimport matplotlib.pyplot as plt\nimport os\nfrom tqdm import tqdm\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nprint(os.getcwd())\nfor dirname, dirs, _ in os.walk('../input'):\n    for d in dirs:\n        print(os.path.join(dirname, d))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"# The size of a video\nIn order to train a model, we first need to see the input properties. What is the size of a frame ? it is equal in all the videos ? How many frames exist in all the vodies ? Are they equal ?"},{"metadata":{"trusted":true},"cell_type":"code","source":"def video_prop(reader):\n    w = 0\n    h = 0\n\n    success, image = reader.read()\n    h = image.shape[0]\n    w = image.shape[1]\n    nFrames = int(reader.get(cv2.CAP_PROP_FRAME_COUNT))\n    return h, w, nFrames\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def video_size_counter(path):\n    video_sizes = dict()\n    for dirname, _, filenames in os.walk(path):\n        for filename in filenames:\n            if filename.endswith('.mp4'):\n                video_filename = os.path.join(dirname, filename)\n                reader = cv2.VideoCapture(video_filename)\n                h, w, nFrames = video_prop(reader)\n                if (h, w, nFrames) in video_sizes.keys():\n                    video_sizes[(h, w, nFrames)] += 1\n                else:\n                    video_sizes[(h, w, nFrames)] = 1\n    return video_sizes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"video_sizes_train = video_size_counter('/kaggle/input/deepfake-detection-challenge/train_sample_videos')\nvideo_sizes_test = video_size_counter('/kaggle/input/deepfake-detection-challenge/test_videos')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(video_sizes_train)\nprint(video_sizes_test)\nsizes1 = set([k for k in video_sizes_train.keys()])\nsizes2 = set([k for k in video_sizes_test.keys()])\nsizes = sizes1.union(sizes2)\nsizes_str = [str(s) for s in sizes]\ny_pos = [3*i for i in range(len(sizes_str))]\n\nn_accurance = []\nfor s in sizes:\n    if s in video_sizes_train.keys():\n        n_accurance.append(video_sizes_train[s])\n    else:\n        n_accurance.append(0)\nfig = plt.figure(1)\nplt.bar(y_pos, n_accurance, width=1)\nplt.xticks(y_pos, sizes_str)\nplt.title('Video sizes distribution over the sampled training data')\nplt.show()\n\nn_accurance = []\nfor s in sizes:\n    if s in video_sizes_test.keys():\n        n_accurance.append(video_sizes_test[s])\n    else:\n        n_accurance.append(0)\nfig = plt.figure(2)\nplt.bar(y_pos, n_accurance, width=1)\nplt.xticks(y_pos, sizes_str)\nplt.title('Video sizes distribution over the sampled test data')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Note that the analysis was done only on the 400 videos of the sampled training and the sampled test data. It should be done on all the training data\n\nAs you can see:\n* Most of the vidoes have size of 1920x1080, but there are videos that are 1080x1920. If you want to train a model that contians spatial CNN, you have to think what do do with the 1080x1920 videos.\n* The vidoes have almost the same number of frames. You can use RNN by frames or 3D CNN and use ony the 299 first frames on each video. I do not think that taking 299 of 300 frames in a video will change the result a lot"},{"metadata":{},"cell_type":"markdown","source":"# Looking at the labels\nIn order to know if the training set is balanced, lets look at the metadata"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_sample_metadata = pd.read_json('/kaggle/input/deepfake-detection-challenge/train_sample_videos/metadata.json').T\ntrain_sample_metadata.head(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_count = train_sample_metadata.groupby('label')['label'].count()\nprint(labels_count)\nlabels_count.plot(kind='bar', title='Distribution of Labels in the Training Set')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can see, the sampled data is not balanced. Of course, we have to check it on the full data. But if it is like that in the full data, we will have to deal with it in our model"},{"metadata":{},"cell_type":"markdown","source":"# Simple face detection results.\nRob Mulla in https://www.kaggle.com/robikscube/kaggle-deepfake-detection-introduction, Gabriel Preda in https://www.kaggle.com/gpreda/deepfake-starter-kit and many others suggested that in fake videos, it is harder to detect faces.\nLets look at the videos and count in how many frames a face was detected."},{"metadata":{},"cell_type":"markdown","source":"the following code is copied from https://www.kaggle.com/gpreda/deepfake-starter-kit"},{"metadata":{"trusted":true},"cell_type":"code","source":"class ObjectDetector:\n    # Class for Object Detection\n    def __init__(self,object_cascade_path):\n        '''\n        param: object_cascade_path - path for the *.xml defining the parameters for {face, eye, smile, profile}\n        detection algorithm\n        source of the haarcascade resource is: https://github.com/opencv/opencv/tree/master/data/haarcascades\n        '''\n        self.objectCascade = cv2.CascadeClassifier(object_cascade_path)\n\n    def detect(self, image, scale_factor=1.3, min_neighbors=5, min_size=(20,20)):\n        '''\n        Function return rectangle coordinates of object for given image\n        param: image - image to process\n        param: scale_factor - scale factor used for object detection\n        param: min_neighbors - minimum number of parameters considered during object detection\n        param: min_size - minimum size of bounding box for object detected\n        '''\n        rects=self.objectCascade.detectMultiScale(image, scaleFactor=scale_factor, minNeighbors=min_neighbors,\n                                                  minSize=min_size)\n        return rects","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"FACE_DETECTION_FOLDER = '/kaggle/input/haar-cascades-for-face-detection'\nfrontal_cascade_path = os.path.join(FACE_DETECTION_FOLDER, 'haarcascade_frontalface_default.xml')\nfront_detector = ObjectDetector(frontal_cascade_path)\nprofile_cascade_path = os.path.join(FACE_DETECTION_FOLDER, 'haarcascade_profileface.xml')\nprofile_detector = ObjectDetector(profile_cascade_path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def count_faces(video_filename):\n    reader = cv2.VideoCapture(video_filename)\n    nFrames = int(reader.get(cv2.CAP_PROP_FRAME_COUNT))\n    nDetected = 0\n    for i in range(nFrames):\n        _, image = reader.read()\n        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n        n_profile = 0\n        n_front = len(front_detector.detect(gray))\n        if (n_front==0):\n            n_profile = len(profile_detector.detect(gray))\n        if n_front > 0 or n_profile > 0:\n            nDetected += 1\n    reader.release()\n    return nDetected / nFrames","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"n_faces_real_videos = []\nn_faces_fake_videos = []\n\nfor dir_name, _, file_names in os.walk('../input/deepfake-detection-challenge/train_sample_videos/'):\n    for filename in file_names:\n        if filename.endswith('.mp4'):\n            print(filename)\n            video_filename = os.path.join(dir_name, filename)\n            detection_ratio = count_faces(video_filename)\n            if train_sample_metadata.loc[filename]['label'] == 'REAL':\n                n_faces_real_videos.append(detection_ratio)\n            else:\n                n_faces_fake_videos.append(detection_ratio)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Plot an histogram of detections.\nplt.figure(1)\nplt.hist(n_faces_real_videos, bins=21, range=(0, 1))\nplt.title('Face detection ratio of real videos')\nplt.xlabel('Number of videos')\nplt.ylabel('Detection ratio')\nplt.figure(2)\nplt.hist(n_faces_fake_videos, bins=20, range=(0, 1))\nplt.title('Face detection ratio of fake videos')\nplt.xlabel('Number of videos')\nplt.ylabel('Detection ratio')\nplt.show()\n","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}