![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2219085%2F76e1bb275d9fc5448c31ebdb92162728%2Fvid17.png?generation=1576540142161397&alt=media\")
Are faces the only thing modified in the video? Are they modified for the entire video or just a portion of it? To find out I prepared some comparison videos using the training data. Going through the training data there are a few examples that have a fake and real video to compare with. This script will process two videos into a comparison of the two.
\n\nThis first comparison is between ahbweevwpv.mp4 and dkuayagnmc.mp4. The trees in the background are likely different due to compression, you can clearly see the differences in the face.
\n\n
Video: https://youtu.be/T1IvNnKXw1c
\n\nThe second comparison is between adylbeequz.mp4 and dlpoieqvfb.mp4. A farther away face and you can see only the face being different. The manipulation method appears to be changed which was mentioned in the preview paper.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2219085%2Ffe47c444964f1f03d7fb4a447835fe94%2Fvid11.png?generation=1576540510691366&alt=media\")
Video: https://youtu.be/xzzzIA8V6gI
\n\nCode:
\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom skimage.measure import compare_ssim\nimport imutils\nimport cv2\nfrom tqdm import tqdm\n\nTRAIN_VIDEOS='../input/deepfake-detection-challenge/train_sample_videos'\nTEST_VIDEOS='../input/deepfake-detection-challenge/test_videos'\nALL_VIDEOS='../input/dfdc_train_all'\nTRAIN_JSON=TRAIN_VIDEOS+'/metadata.json'\n\ndef get_meta_from_json(path):\n df = pd.read_json(path)\n df = df.T\n df.reset_index(level=0, inplace=True)\n df['file']=df['index']\n return df\n\nmeta_train_df = get_meta_from_json(TRAIN_JSON)\n\n#build balanced validation data - the samples give as an example\nnum_reals = len(meta_train_df[meta_train_df['label'] == 'REAL'])\nreals_idx = meta_train_df[meta_train_df['label'] == 'REAL'].index\nfake_idx = meta_train_df[meta_train_df['label'] == 'FAKE'].index\nrandom_idx = np.random.choice(fake_idx,num_reals, replace=False)\nbalanced_idx = np.concatenate([reals_idx,random_idx])\nbalanced_df = meta_train_df.loc[balanced_idx]\n\n\n\n\ndef diff_videos(input_path,input_path2,output_path):\n reader1 = cv2.VideoCapture(input_path)\n reader2 = cv2.VideoCapture(input_path2)\n frame_count1 = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n frame_count2 = int(reader2.get(cv2.CAP_PROP_FRAME_COUNT))\n print(frame_count1,frame_count2)\n ### get format of input\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,1)\n video_FourCC = int(reader1.get(cv2.CAP_PROP_FOURCC))\n video_fps = reader1.get(cv2.CAP_PROP_FPS)\n video_size = (int(reader1.get(cv2.CAP_PROP_FRAME_WIDTH)),\n int(reader1.get(cv2.CAP_PROP_FRAME_HEIGHT)))\n video_msec = int(reader1.get(cv2.CAP_PROP_POS_MSEC))\n video_frames = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,0)\n ###\n\n ### setup outputs\n if reader1.isOpened() and reader2.isOpened():\n out = cv2.VideoWriter(output_path, video_FourCC, video_fps, video_size)\n try:\n pbar = tqdm(total=frame_count1)\n while reader1.isOpened() and reader2.isOpened():\n _, image1 = reader1.read()\n _, image2 = reader2.read()\n pbar.update(1)\n try:\n (score, diff) = compare_ssim(image1, image2, full=True, multichannel=True)\n diff = (diff * 255).astype(\"uint8\")\n except:\n print(\"error diffing\")\n break\n\n out.write(diff) \n finally:\n out.release()\n else:\n print(\"Failed to open both inputs\")\n\n\nfake_df = meta_train_df[meta_train_df['label']=='FAKE']\nfor idx,input1 in enumerate(fake_df['file']):\n input2 = fake_df['original'].iloc[idx]\n file_path1 = TRAIN_VIDEOS +'/'+input1\n file_path2 = TRAIN_VIDEOS +'/'+input2\n print(file_path1,file_path2)\n diff_videos(file_path1,file_path2,\"vid\"+str(idx)+\".mp4\")\n\n\n", "messages": [ { "id": "696684", "postDate": "12/17/2019 00:00:05", "content": "
Are faces the only thing modified in the video? Are they modified for the entire video or just a portion of it? To find out I prepared some comparison videos using the training data. Going through the training data there are a few examples that have a fake and real video to compare with. This script will process two videos into a comparison of the two.
\n\nThis first comparison is between ahbweevwpv.mp4 and dkuayagnmc.mp4. The trees in the background are likely different due to compression, you can clearly see the differences in the face.
\n\n
Video: https://youtu.be/T1IvNnKXw1c
\n\nThe second comparison is between adylbeequz.mp4 and dlpoieqvfb.mp4. A farther away face and you can see only the face being different. The manipulation method appears to be changed which was mentioned in the preview paper.
\n\n
Video: https://youtu.be/xzzzIA8V6gI
\n\nCode:
\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom skimage.measure import compare_ssim\nimport imutils\nimport cv2\nfrom tqdm import tqdm\n\nTRAIN_VIDEOS='../input/deepfake-detection-challenge/train_sample_videos'\nTEST_VIDEOS='../input/deepfake-detection-challenge/test_videos'\nALL_VIDEOS='../input/dfdc_train_all'\nTRAIN_JSON=TRAIN_VIDEOS+'/metadata.json'\n\ndef get_meta_from_json(path):\n df = pd.read_json(path)\n df = df.T\n df.reset_index(level=0, inplace=True)\n df['file']=df['index']\n return df\n\nmeta_train_df = get_meta_from_json(TRAIN_JSON)\n\n#build balanced validation data - the samples give as an example\nnum_reals = len(meta_train_df[meta_train_df['label'] == 'REAL'])\nreals_idx = meta_train_df[meta_train_df['label'] == 'REAL'].index\nfake_idx = meta_train_df[meta_train_df['label'] == 'FAKE'].index\nrandom_idx = np.random.choice(fake_idx,num_reals, replace=False)\nbalanced_idx = np.concatenate([reals_idx,random_idx])\nbalanced_df = meta_train_df.loc[balanced_idx]\n\n\n\n\ndef diff_videos(input_path,input_path2,output_path):\n reader1 = cv2.VideoCapture(input_path)\n reader2 = cv2.VideoCapture(input_path2)\n frame_count1 = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n frame_count2 = int(reader2.get(cv2.CAP_PROP_FRAME_COUNT))\n print(frame_count1,frame_count2)\n ### get format of input\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,1)\n video_FourCC = int(reader1.get(cv2.CAP_PROP_FOURCC))\n video_fps = reader1.get(cv2.CAP_PROP_FPS)\n video_size = (int(reader1.get(cv2.CAP_PROP_FRAME_WIDTH)),\n int(reader1.get(cv2.CAP_PROP_FRAME_HEIGHT)))\n video_msec = int(reader1.get(cv2.CAP_PROP_POS_MSEC))\n video_frames = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,0)\n ###\n\n ### setup outputs\n if reader1.isOpened() and reader2.isOpened():\n out = cv2.VideoWriter(output_path, video_FourCC, video_fps, video_size)\n try:\n pbar = tqdm(total=frame_count1)\n while reader1.isOpened() and reader2.isOpened():\n _, image1 = reader1.read()\n _, image2 = reader2.read()\n pbar.update(1)\n try:\n (score, diff) = compare_ssim(image1, image2, full=True, multichannel=True)\n diff = (diff * 255).astype(\"uint8\")\n except:\n print(\"error diffing\")\n break\n\n out.write(diff) \n finally:\n out.release()\n else:\n print(\"Failed to open both inputs\")\n\n\nfake_df = meta_train_df[meta_train_df['label']=='FAKE']\nfor idx,input1 in enumerate(fake_df['file']):\n input2 = fake_df['original'].iloc[idx]\n file_path1 = TRAIN_VIDEOS +'/'+input1\n file_path2 = TRAIN_VIDEOS +'/'+input2\n print(file_path1,file_path2)\n diff_videos(file_path1,file_path2,\"vid\"+str(idx)+\".mp4\")\n\n\n", "rawMarkdown": "Are faces the only thing modified in the video? Are they modified for the entire video or just a portion of it? To find out I prepared some comparison videos using the training data. Going through the training data there are a few examples that have a fake and real video to compare with. This script will process two videos into a comparison of the two.\n\nThis first comparison is between **ahbweevwpv.mp4** and **dkuayagnmc.mp4**. The trees in the background are likely different due to compression, you can clearly see the differences in the face.\n\n\n\nVideo: [https://youtu.be/T1IvNnKXw1c](https://youtu.be/T1IvNnKXw1c)\n\nThe second comparison is between **adylbeequz.mp4** and **dlpoieqvfb.mp4**. A farther away face and you can see only the face being different. The manipulation method appears to be changed which was mentioned in the preview paper.\n\n\n\nVideo: [https://youtu.be/xzzzIA8V6gI](https://youtu.be/xzzzIA8V6gI)\n\nCode:\n\n
import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom skimage.measure import compare_ssim\nimport imutils\nimport cv2\nfrom tqdm import tqdm\n\nTRAIN_VIDEOS='../input/deepfake-detection-challenge/train_sample_videos'\nTEST_VIDEOS='../input/deepfake-detection-challenge/test_videos'\nALL_VIDEOS='../input/dfdc_train_all'\nTRAIN_JSON=TRAIN_VIDEOS+'/metadata.json'\n\ndef get_meta_from_json(path):\n df = pd.read_json(path)\n df = df.T\n df.reset_index(level=0, inplace=True)\n df['file']=df['index']\n return df\n\nmeta_train_df = get_meta_from_json(TRAIN_JSON)\n\n#build balanced validation data - the samples give as an example\nnum_reals = len(meta_train_df[meta_train_df['label'] == 'REAL'])\nreals_idx = meta_train_df[meta_train_df['label'] == 'REAL'].index\nfake_idx = meta_train_df[meta_train_df['label'] == 'FAKE'].index\nrandom_idx = np.random.choice(fake_idx,num_reals, replace=False)\nbalanced_idx = np.concatenate([reals_idx,random_idx])\nbalanced_df = meta_train_df.loc[balanced_idx]\n\n\n\n\ndef diff_videos(input_path,input_path2,output_path):\n reader1 = cv2.VideoCapture(input_path)\n reader2 = cv2.VideoCapture(input_path2)\n frame_count1 = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n frame_count2 = int(reader2.get(cv2.CAP_PROP_FRAME_COUNT))\n print(frame_count1,frame_count2)\n ### get format of input\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,1)\n video_FourCC = int(reader1.get(cv2.CAP_PROP_FOURCC))\n video_fps = reader1.get(cv2.CAP_PROP_FPS)\n video_size = (int(reader1.get(cv2.CAP_PROP_FRAME_WIDTH)),\n int(reader1.get(cv2.CAP_PROP_FRAME_HEIGHT)))\n video_msec = int(reader1.get(cv2.CAP_PROP_POS_MSEC))\n video_frames = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,0)\n ###\n\n ### setup outputs\n if reader1.isOpened() and reader2.isOpened():\n out = cv2.VideoWriter(output_path, video_FourCC, video_fps, video_size)\n try:\n pbar = tqdm(total=frame_count1)\n while reader1.isOpened() and reader2.isOpened():\n _, image1 = reader1.read()\n _, image2 = reader2.read()\n pbar.update(1)\n try:\n (score, diff) = compare_ssim(image1, image2, full=True, multichannel=True)\n diff = (diff * 255).astype(\"uint8\")\n except:\n print(\"error diffing\")\n break\n\n out.write(diff) \n finally:\n out.release()\n else:\n print(\"Failed to open both inputs\")\n \n \nfake_df = meta_train_df[meta_train_df['label']=='FAKE']\nfor idx,input1 in enumerate(fake_df['file']):\n input2 = fake_df['original'].iloc[idx]\n file_path1 = TRAIN_VIDEOS +'/'+input1\n file_path2 = TRAIN_VIDEOS +'/'+input2\n print(file_path1,file_path2)\n diff_videos(file_path1,file_path2,\"vid\"+str(idx)+\".mp4\")\n \n \n", "votes": null }, { "id": "696895", "postDate": "12/17/2019 07:47:55", "content": "
In my experiments, fake versions are using at least 4 different deepfake models (or methods) to modify their original version. Sometimes they failed to find faces in original video, so I could find a small rectangular patch, which is not located on faces, in some of fake videos.
", "rawMarkdown": "In my experiments, fake versions are using at least 4 different deepfake models (or methods) to modify their original version. Sometimes they failed to find faces in original video, so I could find a small rectangular patch, which is not located on faces, in some of fake videos.", "votes": null }, { "id": "696951", "postDate": "12/17/2019 09:29:41", "content": "Great info, I haven't yet run this on the full dataset to see the different types. One idea I have is to use these pairings to train a network that is faster at picking out faces from a video.
", "rawMarkdown": "Great info, I haven't yet run this on the full dataset to see the different types. One idea I have is to use these pairings to train a network that is faster at picking out faces from a video.", "votes": null }, { "id": "733632", "postDate": "01/31/2020 11:20:28", "content": "Thanks very much
", "rawMarkdown": "Thanks very much", "votes": null } ], "comments": [ { "id": 696895, "author_name": "gowithsky", "author_url": "", "post_date": "12/17/2019 07:47:55", "content": "In my experiments, fake versions are using at least 4 different deepfake models (or methods) to modify their original version. Sometimes they failed to find faces in original video, so I could find a small rectangular patch, which is not located on faces, in some of fake videos.
", "votes": null, "replies": [ { "id": 696951, "author_name": "ldm314", "author_url": "", "post_date": "12/17/2019 09:29:41", "content": "Great info, I haven't yet run this on the full dataset to see the different types. One idea I have is to use these pairings to train a network that is faster at picking out faces from a video.
", "votes": null, "replies": [] } ] }, { "id": 733632, "author_name": "beeaware", "author_url": "", "post_date": "01/31/2020 11:20:28", "content": "Thanks very much
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "696684": "Are faces the only thing modified in the video? Are they modified for the entire video or just a portion of it? To find out I prepared some comparison videos using the training data. Going through the training data there are a few examples that have a fake and real video to compare with. This script will process two videos into a comparison of the two.\n\nThis first comparison is between **ahbweevwpv.mp4** and **dkuayagnmc.mp4**. The trees in the background are likely different due to compression, you can clearly see the differences in the face.\n\n\n\nVideo: [https://youtu.be/T1IvNnKXw1c](https://youtu.be/T1IvNnKXw1c)\n\nThe second comparison is between **adylbeequz.mp4** and **dlpoieqvfb.mp4**. A farther away face and you can see only the face being different. The manipulation method appears to be changed which was mentioned in the preview paper.\n\n\n\nVideo: [https://youtu.be/xzzzIA8V6gI](https://youtu.be/xzzzIA8V6gI)\n\nCode:\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom skimage.measure import compare_ssim\nimport imutils\nimport cv2\nfrom tqdm import tqdm\n\nTRAIN_VIDEOS='../input/deepfake-detection-challenge/train_sample_videos'\nTEST_VIDEOS='../input/deepfake-detection-challenge/test_videos'\nALL_VIDEOS='../input/dfdc_train_all'\nTRAIN_JSON=TRAIN_VIDEOS+'/metadata.json'\n\ndef get_meta_from_json(path):\n df = pd.read_json(path)\n df = df.T\n df.reset_index(level=0, inplace=True)\n df['file']=df['index']\n return df\n\nmeta_train_df = get_meta_from_json(TRAIN_JSON)\n\n#build balanced validation data - the samples give as an example\nnum_reals = len(meta_train_df[meta_train_df['label'] == 'REAL'])\nreals_idx = meta_train_df[meta_train_df['label'] == 'REAL'].index\nfake_idx = meta_train_df[meta_train_df['label'] == 'FAKE'].index\nrandom_idx = np.random.choice(fake_idx,num_reals, replace=False)\nbalanced_idx = np.concatenate([reals_idx,random_idx])\nbalanced_df = meta_train_df.loc[balanced_idx]\n\n\n\n\ndef diff_videos(input_path,input_path2,output_path):\n reader1 = cv2.VideoCapture(input_path)\n reader2 = cv2.VideoCapture(input_path2)\n frame_count1 = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n frame_count2 = int(reader2.get(cv2.CAP_PROP_FRAME_COUNT))\n print(frame_count1,frame_count2)\n ### get format of input\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,1)\n video_FourCC = int(reader1.get(cv2.CAP_PROP_FOURCC))\n video_fps = reader1.get(cv2.CAP_PROP_FPS)\n video_size = (int(reader1.get(cv2.CAP_PROP_FRAME_WIDTH)),\n int(reader1.get(cv2.CAP_PROP_FRAME_HEIGHT)))\n video_msec = int(reader1.get(cv2.CAP_PROP_POS_MSEC))\n video_frames = int(reader1.get(cv2.CAP_PROP_FRAME_COUNT))\n reader1.set(cv2.CAP_PROP_POS_AVI_RATIO,0)\n ###\n\n ### setup outputs\n if reader1.isOpened() and reader2.isOpened():\n out = cv2.VideoWriter(output_path, video_FourCC, video_fps, video_size)\n try:\n pbar = tqdm(total=frame_count1)\n while reader1.isOpened() and reader2.isOpened():\n _, image1 = reader1.read()\n _, image2 = reader2.read()\n pbar.update(1)\n try:\n (score, diff) = compare_ssim(image1, image2, full=True, multichannel=True)\n diff = (diff * 255).astype(\"uint8\")\n except:\n print(\"error diffing\")\n break\n\n out.write(diff) \n finally:\n out.release()\n else:\n print(\"Failed to open both inputs\")\n \n \nfake_df = meta_train_df[meta_train_df['label']=='FAKE']\nfor idx,input1 in enumerate(fake_df['file']):\n input2 = fake_df['original'].iloc[idx]\n file_path1 = TRAIN_VIDEOS +'/'+input1\n file_path2 = TRAIN_VIDEOS +'/'+input2\n print(file_path1,file_path2)\n diff_videos(file_path1,file_path2,\"vid\"+str(idx)+\".mp4\")\n \n \n", "696895": "In my experiments, fake versions are using at least 4 different deepfake models (or methods) to modify their original version. Sometimes they failed to find faces in original video, so I could find a small rectangular patch, which is not located on faces, in some of fake videos.", "696951": "Great info, I haven't yet run this on the full dataset to see the different types. One idea I have is to use these pairings to train a network that is faster at picking out faces from a video.", "733632": "Thanks very much" }, "source": "meta" }