{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Deepfake Detection Challenge\n- Identify videos with facial or voice manipulations"},{"metadata":{},"cell_type":"markdown","source":"In this [competition](https://deepfakedetectionchallenge.ai/) produced in cooperation with Amazon, Microsoft, the nonprofit Partnership on AI, and academics from eight universities—researchers around the world are vying to create automated tools that can spot fraudulent media. The competition was announced at the AI conference NeurIPS, and will accept entries through March 2020. Facebook has dedicated more than US $10 million for awards and grants.\n\n\nDeepfake techniques, which present realistic AI-generated videos of people doing and saying fictional things, have the potential to have a significant impact on how people determine the legitimacy of information presented online. These content generation and modification technologies may affect the quality of public discourse and the safeguarding of human rights—especially given that deepfakes may be used maliciously as a source of misinformation, manipulation, harassment, and persuasion. Identifying manipulated media is a technically demanding and rapidly evolving challenge that requires collaborations across the entire tech industry and beyond."},{"metadata":{},"cell_type":"markdown","source":"![](https://spectrum.ieee.org/image/MzQyNjU1OQ.jpeg)"},{"metadata":{},"cell_type":"markdown","source":"DeepFake uses AI (artificial intelligence) and machine learning to manipulate videos or any other form of digital representations. They result in images, videos or just audios that appear to be real. Since DeepFake uses AI and machine learning, the tech analyzes the videos and images of the target person from all angles. Thereafter, the technology accurately mimics the behavior and speech of the target person."},{"metadata":{},"cell_type":"markdown","source":"# Import Packages"},{"metadata":{"trusted":true},"cell_type":"code","source":"SAMPLE_SUB = \"../input/deepfake-detection-challenge/sample_submission.csv\"\nTRAIN_VIDEOS = \"../input/deepfake-detection-challenge/train_sample_videos\"\nTEST_VIDEOS = \"../input/deepfake-detection-challenge/test_videos\"\nTRAIN_JSON_PATH = \"../input/deepfake-detection-challenge/train_sample_videos/metadata.json\"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#import packages\n\nimport pandas as pd\nimport numpy as np\nimport cv2\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport os\n\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Files\n\n* **train_sample_videos.zip** - a ZIP file containing a sample set of training videos and a metadata.json with labels. the full set of training videos is available through the links provided above.\n* **sample_submission.csv** - a sample submission file in the correct format.\n* **test_videos.zip** - a zip file containing a small set of videos to be used as a public validation set."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# check the number of videos present in the train and test data.\n\nn_train_videos = len(os.listdir(TRAIN_VIDEOS))\nn_test_videos = len(os.listdir(TEST_VIDEOS))\n\ntrain_videos = os.listdir(TRAIN_VIDEOS)\ntest_videos = os.listdir(TEST_VIDEOS)\n\nprint(\"Number of training vidoes: \", n_train_videos - 1)\nprint(\"Number of testing videos: \", n_test_videos)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> The data is comprised of .mp4 files, split into compressed sets of ~10GB apiece. A **metadata.json** accompanies each set of .mp4 files, and contains `filename`, `label` (REAL/FAKE), `original` and `split` columns, listed below under Columns"},{"metadata":{"trusted":true},"cell_type":"code","source":"#read the json file\n\ndeepfake_labels = pd.read_json(TRAIN_JSON_PATH).T\n\ndeepfake_labels.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Analysis - MetaData File"},{"metadata":{"trusted":true},"cell_type":"code","source":"#start with the basic analysis - Missing value analysis\n\nmissing_df = pd.DataFrame({\"Missing_Count\": deepfake_labels.isnull().sum(),\n                          \"Missing_Percent\": round(deepfake_labels.isnull().mean(),2)})\nmissing_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Around 20% of the videos present doesn't have a original video associated with it. It could mean that these 77 videos might be REAL so that's why original video column is empty for these videos."},{"metadata":{"trusted":true},"cell_type":"code","source":"#check the distribution of labels\nplt.style.use(\"ggplot\")\nplt.rcParams['figure.figsize'] = 14,7\n\ndeepfake_labels.label.value_counts(normalize = True).plot(kind = \"barh\")\nplt.xlabel(\"Percentage of labels\")\nplt.title(\"Distribution of labels for videos\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Training data is skewed. More than 80% of the data consisting of `FAKE` videos"},{"metadata":{},"cell_type":"markdown","source":"# Exploratory Data Analysis - Video Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# we have already created a list of training data and test data videos\n\nprint(\"Training videos: \" ,train_videos[:5])\nprint(\"Testing videos: \", test_videos[:5])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- We will look at the some of the FAKE and REAL Videos. First we will get the names of the FAKE and REAL videos into seperate lists for training and test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_fake_lst = deepfake_labels[deepfake_labels[\"label\"] == \"FAKE\"].index.tolist()\ntrain_real_lst = deepfake_labels[deepfake_labels[\"label\"] == \"REAL\"].index.tolist()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- To display a single frame from the video. We will use the function from the kernel **\"[Basic EDA Face Detection, split video and ROI](https://www.kaggle.com/marcovasquez/basic-eda-face-detection-split-video-and-roi)\"**"},{"metadata":{"trusted":true},"cell_type":"code","source":"#display frame\n\ndef display_frame(video, axis):\n    cap = cv2.VideoCapture(video)  \n    ret, frame = cap.read()\n    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n    axis.imshow(frame)\n    axis.grid(False)\n    video_name = video.split(\"/\")[-1]\n    axis.set_title(\"Video: \" + video_name, size = 30)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Analysis of Training - FAKE Videos\n- Look at the FAKE videos present in training data"},{"metadata":{"trusted":true},"cell_type":"code","source":"random_num = np.random.randint(0,len(train_fake_lst))\nfig,axs = plt.subplots(nrows = 1, ncols=3, figsize=(50,40))\nfor i in range(3):\n    display_frame(os.path.join(TRAIN_VIDEOS, train_fake_lst[random_num]), axs[i])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"random_num = np.random.randint(0,len(train_fake_lst))\nfig,axs = plt.subplots(nrows = 1, ncols=3, figsize=(50,40))\nfor i in range(3):\n    display_frame(os.path.join(TRAIN_VIDEOS, train_fake_lst[random_num]), axs[i])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Analysis of Training - REAL Videos\n- Look at the REAL videos present in training data"},{"metadata":{"trusted":true},"cell_type":"code","source":"random_num = np.random.randint(0,len(train_real_lst))\nfig,axs = plt.subplots(nrows = 1, ncols=3, figsize=(50,40))\nfor i in range(3):\n    display_frame(os.path.join(TRAIN_VIDEOS, train_real_lst[random_num]), axs[i])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Another REAL Video"},{"metadata":{"trusted":true},"cell_type":"code","source":"random_num = np.random.randint(0,len(train_real_lst))\nfig,axs = plt.subplots(nrows = 1, ncols=3, figsize=(50,40))\nfor i in range(3):\n    display_frame(os.path.join(TRAIN_VIDEOS, train_real_lst[random_num]), axs[i])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"random_num = np.random.randint(0,len(train_real_lst))\nfig,axs = plt.subplots(nrows = 1, ncols=3, figsize=(50,40))\nfor i in range(3):\n    display_frame(os.path.join(TRAIN_VIDEOS, train_real_lst[random_num]), axs[i])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"#checking the label distribution\n\ndeepfake_labels.label.value_counts(normalize = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Since our data is imbalanced (80% of data contains FAKE Video's). So we will assume that test data might also follows same distribution for sample submission."},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_df = pd.read_csv(SAMPLE_SUB)\nsample_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_df[\"label\"] = 0.65\nsample_df.to_csv(\"submission.csv\", index = False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}