{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Can we use simple features to distinguish DeepFakes?  ##\n\nThis kernel uses the techniques outlined in the paper Unmasking DeepFakes with simple Features by Ricard Durall, Margret Keuper, Franz-Josef Pfreundt and Janis Keuper(https://arxiv.org/pdf/1911.00686v2.pdf) to use power transforms as an input into logistic regression. To keep things simple the model was trained on the sample videos rather than the whole data set. \n\nBelow we show how to make predictions on the submission set using a model that is pre-trained on the sample data. It will be interesting to see if training of the full dataset yields much better results. \n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"collapsed":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nimport pickle\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\n\n#pip install glob\nimport glob\n#conda install -c menpo opencv\nimport cv2\n#pip install PIL\nfrom PIL import Image\nfrom matplotlib import pyplot as plt\nimport os\nfrom scipy.interpolate import griddata\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Basic idea # \nReal and fake faces tend to have different footprints when we apply some signal processing. The exact processing is well outside my domain (read the paper!) but imporantly below, when we plot the average signal from images taken from real and fake videos they look different! The process then becomes: find face, calculate features from the processed images, and feed these into the logistic regression trained on the sample data. "},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":false},"cell_type":"code","source":"#The top three lines save the processed data from the training videos. For this I'm reading it back\n#res = pd.DataFrame(X)\n#res['label']=Y\n#res.to_csv('/kaggle/working/power_average.csv', Index = True )\nres = pd.read_csv(\"/kaggle/input/powerdata/power_average.csv\")\nF_mean = np.array(res[res['label']=='FAKE'].iloc[:,1:-1]).mean(0)\nF_sd = np.array(res[res['label']=='FAKE'].iloc[:,1:-1]).std(0)\nR_mean = np.array(res[res['label']=='REAL'].iloc[:,1:-1]).mean(0)\nR_sd = np.array(res[res['label']=='REAL'].iloc[:,1:-1]).std(0)\n\nx = np.arange(0, 300, 1)\nfig, ax = plt.subplots(figsize=(15, 9))\nax.plot(x, F_mean, alpha=0.5, color='red', label='fake', linewidth =2.0)\nax.fill_between(x, F_mean - F_sd, F_mean + F_sd, color='red', alpha=0.2)\n\nax.plot(x, R_mean, alpha=0.5, color='blue', label='real', linewidth = 2.0)\nax.fill_between(x, R_mean - R_sd, R_mean + R_sd, color='blue', alpha=0.2)\n\nax.set_title('1D Power Spectrum for real/fake faces taken from the sample videos ',size=20)\nplt.xlabel('Spatial Frequency', fontsize=20)\nplt.ylabel('Power Spectrum', fontsize=20)\nplt.tick_params(axis='x', labelsize=20)\nplt.tick_params(axis='y', labelsize=20)\nax.legend(loc='best', prop={'size': 20})\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Two helper functions: ###\n**ROI:** detects and returns a face from an image. This handy function was inspired by https://www.kaggle.com/marcovasquez/basic-eda-face-detection-split-video-and-roi\n\n**Azimuthal average:** From the cited article - 'We apply azimuthal averaging to compute a\nrobust 1D representation of the FFT power spectrum'. The aforementioned paper has a relevant repo https://github.com/cc-hpc-itwm/DeepFakeDetection where this function come from. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def ROI(img):\n    \n    offset = 50\n    face_img = img.copy()\n  \n    face_rects = face_cascade.detectMultiScale(face_img,scaleFactor=1.3, minNeighbors=5) \n    \n    for (x,y,w,h) in face_rects: \n        roi = face_img[y-offset:y+h+offset,x-offset:x+w+offset] \n           \n    return roi\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def azimuthalAverage(image, center=None):\n    \"\"\"\n    Calculate the azimuthally averaged radial profile.\n    image - The 2D image\n    center - The [x,y] pixel coordinates used as the center. The default is \n             None, which then uses the center of the image (including \n             fracitonal pixels).\n    \n    \"\"\"\n    \n    # Calculate the indices from the image\n    y, x = np.indices(image.shape)\n\n    if not center:\n        center = np.array([(x.max()-x.min())/2.0, (x.max()-x.min())/2.0])\n\n    r = np.hypot(x - center[0], y - center[1])\n\n    # Get sorted radii\n    ind = np.argsort(r.flat)\n    r_sorted = r.flat[ind]\n    i_sorted = image.flat[ind]\n\n    # Get the integer part of the radii (bin size = 1)\n    r_int = r_sorted.astype(int)\n\n    # Find all pixels that fall within each radial bin.\n    deltar = r_int[1:] - r_int[:-1]  # Assumes all radii represented\n    rind = np.where(deltar)[0]       # location of changed radius\n    nr = rind[1:] - rind[:-1]        # number of radius bin\n    \n    # Cumulative sum to figure out sums for each radius bin\n    csim = np.cumsum(i_sorted, dtype=float)\n    tbin = csim[rind[1:]] - csim[rind[:-1]]\n\n    radial_prof = tbin / nr\n\n    return radial_prof","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#train_filenames = glob.glob('/kaggle/input/deepfake-detection-challenge/train_sample_videos/*.mp4')\ntest_filenames = glob.glob('/kaggle/input/deepfake-detection-challenge/test_videos/*.mp4')\n#json = glob.glob('/kaggle/input/deepfake-detection-challenge/train_sample_videos/*.json')\n#metadata = pd.read_json(json[0])\n\nface_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + \"haarcascade_frontalface_default.xml\")\n\nwith open(\"/kaggle/input/pre-trained-balanced/logreg_model_bal.sav\", 'rb') as file:\n   lr_classifier = pickle.load(file)\n\n# Number of frames to sample (evenly spaced) from each video\nn_frames = 10\nsubmission = []\nX = []\n#Y = []","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"for i, filename in enumerate(test_filenames): #change to train_filenames for training\n    try:\n        print(f'Processing {i+1:5n} of {len(filenames):5n} videos\\r', end='')\n\n        # Create video reader and find length\n        v_cap = cv2.VideoCapture(filename)\n        v_len = int(v_cap.get(cv2.CAP_PROP_FRAME_COUNT))\n\n        # Pick 'n_frames' evenly spaced frames to sample\n        sample = np.linspace(0, v_len - 1, n_frames).round().astype(int)\n        faces = []\n\n        for j in range(v_len):\n            success, vframe = v_cap.read()\n            vframe = cv2.cvtColor(vframe, cv2.COLOR_BGR2RGB)\n            \n            if j in sample:\n                try: \n                    face = ROI(vframe)\n                    face = cv2.cvtColor(face, cv2.COLOR_RGB2GRAY)\n                    faces.append(face)\n                except: \n                    pass \n        v_cap.release()\n\n        prob = []\n        for face in faces:\n            f = np.fft.fft2(face) #Create the FFT\n            fshift = np.fft.fftshift(f) #Shift the FFT\n            magnitude_spectrum = 20*np.log(np.abs(fshift)) #Magnitude spectrum\n            psd1D = azimuthalAverage(magnitude_spectrum)\n\n        # Interpolation\n            points = np.linspace(0,300,num=psd1D.size) \n            xi = np.linspace(0,300,num=300) \n            interpolated = griddata(points,psd1D,xi,method='cubic')\n\n            # Normalization\n            interpolated /= interpolated[0]\n            X.append(interpolated)\n            #Y.append(metadata.at['label',os.path.basename(filename)]) #training labels\n            prob.append(lr_classifier.predict_proba(interpolated.reshape(1,-1))[:,0][0])\n        \n        submission.append([os.path.basename(filename), sum(prob)/len(prob)]) #average probs over all images\n            \n    except:\n        submission.append([os.path.basename(filename), 0.5]) #sometimes bad things can happen, if that occurs just guess 0.5!\n        #pass\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.DataFrame(submission, columns=['filename', 'label'])\nsubmission.sort_values('filename').to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If you're interested on how the model was trained see below"},{"metadata":{"trusted":true,"_kg_hide-output":false},"cell_type":"code","source":"'''\n#This segment is used on the trainings set. Cross validation to estimate accuracy then training set.\nLR = 0\nfor i in range(5):\n\n    X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size = 0.2)\n    from sklearn.linear_model import LogisticRegression\n    logreg = LogisticRegression(solver='liblinear', max_iter=1000, class_weight = 'balanced')\n    logreg.fit(X_train, y_train)\n    LR+=logreg.score(X_test,y_test)\n    prob = logreg.predict_proba(np.array(X_test[0]).reshape(1,-1))[:,0]\n\nmodel = logreg.fit(X,Y)\nfilename = '/kaggle/working/logreg_model_bal.sav'\npickle.dump(model, open(filename, 'wb'))\nprint(F'Accuracy is {round(LR/5,2)}')\n'''\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Thats it! This is one of my first notebooks so let me know if you found it useful, as well as any ideas on how it could be improved. "}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}