{"cells":[{"metadata":{},"cell_type":"markdown","source":"Deeply grateful to the sponsors and participants. Even though I ended up with a low score, I felt it was time well spent - went from zero to epsilon on a wide range of topics in a short amount of time.\n\nThis is a compilation of interesting things I had learnt and implemented as part of this competition - mainly for my own reference. Hope others may find them useful as well in the future.  \n\n**- Video reading using DALI and CV2**\n\n**- Extracting MEL Spectrograms**\n\n**- Fake vs Real Image Analysis**\n\n**- Training and inference with fastai** \n\n**- General lessons learned**\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install imagehash","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport sys\nimport cv2\nimport glob\nimport fastai\nimport PIL\nfrom functools import partial\nfrom fastai import *\nfrom fastai.vision import *\nfrom fastai.callbacks import *\nfrom fastai.basics import *\nfrom fastai.vision import learner\n\nfrom tqdm import tqdm\nimport imagehash\nfrom skimage.metrics import structural_similarity\nimport ipywidgets as widgets\nfrom ipywidgets import interact, interact_manual\n\ntqdm.pandas()\n\nINPUT_PATH = \"../input/deepfake-detection-challenge\"\nVERBOSE = False\nEPS = 1e-5\nRUN_NOTEBOOK=False\n\nFACES_PATH = 'faces'\nos.makedirs(FACES_PATH, exist_ok=True)\nos.makedirs(\"mels\", exist_ok=True)\n\nfrom dfdc_face_extractor import *\nfrom dfdc_fastai_reusables import *\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"######################################################################################################\n# Audio Extraction from Video using FFMPEG\n\nhttps://librosa.github.io/librosa/auto_examples/plot_vocal_separation.html\n\nhttps://www.kaggle.com/rakibilly/extract-audio-starter\n\n\nAudio training on this competition was mostly a waste of time. I could only find a few hundred videos where the fake video spectrogram was significantly different from the real one. Training on those  led to complete overfitting and did not boost lb score at all."},{"metadata":{"trusted":true},"cell_type":"code","source":"#!pip install ffmpeg - does not puth ffmpeg on path for librosa\n\n!tar xvf ../input/ffmpeg-static-build/ffmpeg-git-amd64-static.tar.xz\n\n#replace 20191209 with current version of wheel installed from ffmpeg-static-build\n!mv ffmpeg-git-20191209-amd64-static/ffmpeg ffmpeg-git-20191209-amd64-static/ffprobe /usr/local/bin/\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import librosa\nimport soundfile as sf\nfrom skimage import io\n\ndef apply_mel_transforms(mels):\n    \"Normalizes and inverts mel spectrogram\"\n    mels = np.log(mels + EPS) \n    mels = mels - mels.min()\n    img = mels*(255.0/(mels.max()-mels.min()))\n    img = img.astype(np.uint8)\n    img = np.flip(img, axis=0)\n    return img\n\n# https://librosa.github.io/librosa/auto_examples/plot_vocal_separation.html\ndef extract_voice(y, sr):\n    \"Extracts voice from background\"\n    S_full, phase = librosa.magphase(librosa.stft(y))\n\n    S_filter = librosa.decompose.nn_filter(S_full,\n                                           aggregate=np.median,\n                                           metric='cosine',\n                                           width=int(librosa.time_to_frames(2, sr=sr)))\n\n    S_filter = np.minimum(S_full, S_filter)\n    margin_i, margin_v = 2, 10\n    power = 2\n\n    mask_i = librosa.util.softmask(S_filter,\n                                   margin_i * (S_full - S_filter),\n                                   power=power)\n\n    mask_v = librosa.util.softmask(S_full - S_filter,\n                                   margin_v * S_filter,\n                                   power=power)\n\n    S_foreground = mask_v * S_full\n    S_background = mask_i * S_full\n    return S_foreground\n\n\ndef extract_mels(video_path, basename):\n    \"Extracts mel spectrograms from video file\"\n    try:\n        command = f\"ffmpeg -i {video_path} -ab 192000 -ac 2 -ar 44100 -vn mels/{basename}.wav\"\n        subprocess.call(command, shell=True)\n        audio_data, sample_rate = librosa.load( 'mels/' + basename + '.wav')\n        #audio_data, sample_rate = librosa.load(video_path)\n        voice_data = extract_voice(audio_data, sample_rate)\n        hop_length = 512 # number of samples per time-step in spectrogram\n        n_mels = 224 # number of bins in spectrogram. Height of image\n        time_steps = 224 # number of time-steps. Width of image\n\n        mel_data = librosa.feature.melspectrogram(S=voice_data, n_mels=n_mels,\n                                        n_fft=hop_length*2, hop_length=hop_length)\n\n        mel_image=apply_mel_transforms(mel_data)\n        io.imsave(\"mels/%s.jpg\" % (basename), mel_image, quality=100)\n    except:\n        print(f\"Error reading audio from {video_path}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"######################################################################################################\n# Image and Audio Extraction from Videos"},{"metadata":{"trusted":true},"cell_type":"code","source":"if RUN_NOTEBOOK==True:\n    pair_df = pd.read_csv(f'../input/fakereal-pairs-in-dfdc-test-videos/dfdc_test_video_pairs.csv')\n\n    extractor = DFDCVideoFaceExtractor(backend='CV2')\n    pair_df = pair_df[:20]\n    for index, row in tqdm(pair_df.iterrows(), total=len(pair_df)):\n        video_filename = row[\"filename\"]\n        basename, _ = basename_and_ext(video_filename)\n        video_path=f'{INPUT_PATH}/test_videos/{video_filename}'\n        extractor.extract_faces(video_path, seq_length=10,stride=1, faces_path=\"faces\", margin=1)\n        extract_mels(video_path, basename)\n        \n        video_filename = row[\"original\"]\n        basename, _ = basename_and_ext(video_filename)\n        video_path=f'{INPUT_PATH}/test_videos/{video_filename}'\n        extractor.extract_faces(video_path, seq_length=10,stride=1, faces_path=\"faces\", margin=1)\n        extract_mels(video_path, basename)\n        ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"######################################################################################################\n# Image Analysis\n\nUsing structural_similarity, mean square error and imagehash to analyze difference between fake and real images and ultimately select images for training.\n\nAlso tried applying various cv2 filters to see if anyone of them make the fake aspects of an image pop.\n\nhttps://scikit-image.org/docs/dev/auto_examples/transform/plot_ssim.html\n\nhttps://stackoverflow.com/questions/20271479/what-does-it-mean-to-get-the-mse-mean-error-squared-for-2-images\n\nhttps://github.com/JohannesBuchner/imagehash\n\nhttps://towardsdatascience.com/computer-vision-for-beginners-part-2-29b3f9151874"},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndef concat(pils):\n    assert (x == pils[0].width for x.width in pils)\n    assert (x == pils[0].height for x.height in pils)\n    l = len(pils)\n    w = pils[0].width\n    h = pils[0].height\n    dst = PIL.Image.new('RGB', (l*w , h))\n    left = 0\n    for i in range(l):\n        im = pils[i]\n        dst.paste(im, (i*w, 0))\n    return dst\n\ndef image_mse(imageA, imageB):\n    \"Computes mean squared error between two images\"\n    err = np.sum((imageA.astype(\"float\") - imageB.astype(\"float\")) ** 2)\n    err /= float(imageA.size)\n    return err\n\ndef adaptiveThreshold(rgbimg):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2GRAY)\n    return cv2.adaptiveThreshold(image, 255, \n                                    cv2.ADAPTIVE_THRESH_GAUSSIAN_C, \n                                    cv2.THRESH_BINARY, 3, 2)\n\ndef otsuThreshold(rgbimg):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2GRAY)\n    blur = cv2.GaussianBlur(image,(5,5),0)\n    _, img = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)\n    return img\n\n\ndef blur(rgbimg):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2HSV) # convert to HSV\n    image = cv2.blur(image, (9,9))\n    return cv2.cvtColor(image, cv2.COLOR_HSV2RGB)\n\ndef gaussian(rgbimg):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2HSV) # convert to HSV\n    image = cv2.GaussianBlur(image, (9,9),0)\n    return cv2.cvtColor(image, cv2.COLOR_HSV2RGB)\n\ndef apply_filter(rgbimg, func=None):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2HSV) # convert to HSV\n    image = func(image)\n    return cv2.cvtColor(image, cv2.COLOR_HSV2RGB)\n    \ndef median(rgbimg):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2HSV) # convert to HSV\n    image = cv2.medianBlur(image, 9)\n    return cv2.cvtColor(image, cv2.COLOR_HSV2RGB)\n\ndef adaptiveThreshold(rgbimg):\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2GRAY)\n    return cv2.adaptiveThreshold(image, 255, \n                                    cv2.ADAPTIVE_THRESH_GAUSSIAN_C, \n                                    cv2.THRESH_BINARY, 3, 2)\n    \n\ndef sobelxy(rgbimg) :\n    image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2GRAY) \n    sobel_x = cv2.Sobel(image.astype('float32'), cv2.CV_32F, dx = 1, dy = 0, ksize = 1)\n    sobel_y = cv2.Sobel(image.astype('float32'), cv2.CV_32F, dx = 0, dy = 1, ksize = 1)\n    blended = cv2.addWeighted(src1=sobel_x, alpha=0.5, src2=sobel_y,\n                          beta=0.5, gamma=0)\n    return blended.astype('uint8')\n\ndef laplacian(rgbimg):\n    #image = cv2.cvtColor(rgbimg, cv2.COLOR_RGB2GRAY) \n    image = cv2.Laplacian(rgbimg.astype('float32'),cv2.CV_32F)\n    return image.astype('uint8')\n    \ndef dft1(c1):\n    dft = cv2.dft(np.float32(c1),flags = cv2.DFT_COMPLEX_OUTPUT)\n    # shift the zero-frequncy component to the center of the spectrum\n    dft_shift = np.fft.fftshift(dft)\n    # save image of the image in the fourier domain.\n    magnitude_spectrum = 20*np.log(cv2.magnitude(dft_shift[:,:,0],dft_shift[:,:,1]))\n    return magnitude_spectrum\n\ndef dft(rgbimg):\n    r = rgbimg[:,:,0]\n    g = rgbimg[:,:,1]\n    b = rgbimg[:,:,1]\n    r,g,b=cv2.split(rgbimg)\n    r=dft1(r)\n    g=dft1(g)\n    b=dft1(b)\n    merged= cv2.merge((r,g,b))\n    return merged.astype('uint8')\n\ndef show_pairs(real_faces, fake_faces, real_videoname, fake_videoname, func=None ):\n    l = len(real_faces)\n    fig,axes = plt.subplots(l,1,figsize=(600,20))\n    count = 0\n    for i in range(len(real_faces)):\n        pil_im1 = real_faces[i]\n        im1 = np.array(pil_im1)\n        if func != None:\n            im1=func(im1)\n        h,w = im1.shape[0], im1.shape[1]\n        pil_im2 = fake_faces[i]\n        im2 = np.array(pil_im2)\n        if func != None:\n            im2=func(im2)\n        mse = image_mse(im1, im2) \n        skssim, diff = structural_similarity(im1,im2,multichannel=True, full=True)\n        diff = (diff * 255).astype('uint8')\n        pil_im1 = PIL.Image.fromarray(im1)\n        pil_im2 = PIL.Image.fromarray(im2)\n        diff = PIL.Image.fromarray(diff)\n        im3 = concat([pil_im1,pil_im2, diff])\n        im1hash = imagehash.whash(pil_im1)\n        im2hash = imagehash.whash(pil_im2)\n        hash_diff = abs(im1hash-im2hash)\n        title = 'real: %s / fake:%s  / mse: %.2f / skssim: %.2f / hash_diff: %d'% \\\n        (real_videoname, fake_videoname, mse, skssim,hash_diff)\n        axes[count].set_title(title)\n        axes[count].axis('off')\n        axes[count].imshow(im3)\n        count+=1\n        ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def compare_faces(rownum, func=None):\n    pair_df = pd.read_csv(f'../input/fakereal-pairs-in-dfdc-test-videos/dfdc_test_video_pairs.csv')\n    if rownum > len(pair_df):\n        return\n    extractor = DFDCVideoFaceExtractor(backend='CV2')\n    row = pair_df.iloc[rownum]\n    video_filename = row[\"filename\"]\n    fake_basename, _ = basename_and_ext(video_filename)\n    video_path=f'{INPUT_PATH}/test_videos/{video_filename}'\n    fake_frames = extractor.extract_frames(video_path, seq_length=5,stride=1)\n\n    video_filename = row[\"original\"]\n    real_basename, _ = basename_and_ext(video_filename)\n    video_path=f'{INPUT_PATH}/test_videos/{video_filename}'\n    real_frames = extractor.extract_frames(video_path, seq_length=5,stride=1)\n\n    realfaces, fakefaces, framenums = extractor.extract_face_pairs(real_frames, fake_frames, real_basename, margin=.5)\n    \n    show_pairs(realfaces, fakefaces, real_basename, fake_basename, func=func)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"K = 20 \n@interact_manual\ndef compare_faces_interactive():\n    global K\n    compare_faces(K, func=median)\n    K += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"######################################################################################################\n# Training with fastai\n\nFantastic library and course for deep-learning beginners.\n\nhttps://www.fast.ai/\n\n### Data loading\n\nPrimary method of feeding image data into a fastai learner is ImageList and ImageDataBunch. Below are different extensions I had tried to attack unique aspects of DFDC."},{"metadata":{"_kg_hide-input":true},"cell_type":"markdown","source":"## Binary Classification\n### Training"},{"metadata":{"trusted":true},"cell_type":"code","source":"if RUN_NOTEBOOK==True:\n    data = get_deepfakeimagelist_data()\n    data.show_batch(dstype=DatasetType.Valid)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if RUN_NOTEBOOK==True:\n    # Resnet50 Pretrained\n\n    model_dir = 'models/dfdc-resnet50'\n    os.makedirs(model_dir, exist_ok=True)\n    learn = cnn_learner(data,\n                        models.resnet50,\n                        bn_final=True,\n                        loss_func=BCEWithLogitsFlat(),\n                        pretrained=True,\n                        metrics=[DFDCAUROC(),RealLoss(),FakeLoss()],\n                        model_dir=model_dir,\n                        concat_pool = True\n                       )\n    #.to_fp16(); requires GPU\n    learn.fit(1)\n    #learn.fit_one_cycle(3,max_lr=3e-4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Inference"},{"metadata":{"trusted":true},"cell_type":"code","source":"def logits(p):\n    return log( p / (1-p))\n\n\ndef gsigmoid( t):\n    'Gentle sigmoid function that spreads out the predictions'\n    return (t / (1 + abs(t)) + 1.) / 2.\n\ndef get_fake_preds(testfaces):\n    count = 0\n    fakeness = 0.5\n    preds = torch.tensor(())\n    preds.new_empty((0,2),dtype=torch.float32)\n\n    nf = len(testfaces)\n\n    if(nf > 0):\n        faces = pd.Series(testfaces)\n        with warnings.catch_warnings():\n          warnings.simplefilter(\"ignore\", UserWarning)\n          test_data = (DeepFakeImageList.from_pils(faces)\n                 .split_none()\n                 .label_from_array(np.arange(nf))\n                 .transform(TFMS)\n                 .databunch(bs=nf))\\\n                 .normalize(imagenet_stats)\n        preds = get_preds_from_learner(learn, test_data, sigmoid=False)\n        return preds\n\ndef get_preds_from_learner(lrnr, test_data, sigmoid=False):\n    if (DEVICE != 'cpu'):\n        lrnr.to_fp32()\n    lrnr.data = test_data\n    lrnr.data.valid_dl = lrnr.data.train_dl\n    if (DEVICE != 'cpu'):\n        lrnr.to_fp16()\n    preds,y = lrnr.get_preds(ds_type=DatasetType.Valid)\n    preds = preds[np.argsort(y)]\n    preds = preds.detach().float().cpu()\n    preds = preds.squeeze()\n    # Depending on loss function used get_preds returns probability or logits\n    if sigmoid == True:\n        preds = torch.sigmoid(preds)\n    preds = torch.clamp(preds, EPS, 1-EPS)\n    return preds.numpy()\n\ndef aggregate(preds, framenums):\n    fp = pd.DataFrame(columns=['n','label'])\n    fp['n'] = framenums\n    fp['label'] = preds\n\n    # get fakest face from each frame\n    frame_preds = fp.groupby(['n']).agg({'label': ['max']})\n    fp_df = pd.DataFrame(frame_preds)\n    fp_df.reset_index(inplace=True)\n    fp_df.columns = ['n','label']\n    fakeness = fp_df['label'].mean()\n    \n    # spread out the probabilities to eliminate over-confident predictions\n    fakeness = gsigmoid(logits(fakeness))\n    return fakeness\n\nif RUN_NOTEBOOK==True:\n    learn.model.eval();    \n    test_df = pd.read_csv(f'{INPUT_PATH}/sample_submission.csv')\n    test_df = test_df[:10]\n    submission_df = pd.DataFrame(columns=['filename', 'label'])\n    total = len(test_df)\n    for index, row in tqdm(test_df.iterrows(), total=total):\n        video_filename = row['filename']\n        basename, _ = basename_and_ext(video_filename)\n        video_path=f'{INPUT_PATH}/test_videos/{video_filename}'\n        faces, framenums = extract_faces_with_cv2(video_path, basename, seq_length=10,stride=1, output_path=None)\n        if(len(faces) == 0):\n            submission_df = submission_df.append({'filename': video_filename, 'label': 0.5}, ignore_index = True)\n        else:\n            preds = get_fake_preds(faces)\n            label = aggregate(preds, framenums)\n            submission_df = submission_df.append({'filename': video_filename, 'label': label}, ignore_index = True)\n    submission_df.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Other Models"},{"metadata":{"trusted":true},"cell_type":"code","source":"if RUN_NOTEBOOK==True:\n    # GAN Critic\n    from fastai.vision.gan import *\n    def dfdc_critic(in_size:int, n_channels:int, n_features:int=16, n_extra_layers:int=0, **conv_kwargs):\n        \"Based on fastai basic_critic\"\n        layers = [conv_layer(n_channels, n_features, 5, 2, 1, leaky=0.2, **conv_kwargs)]#norm_type=None?\n        cur_size, cur_ftrs = in_size//2, n_features\n        layers.append(nn.Sequential(*[conv_layer(cur_ftrs, cur_ftrs, 3, 1, leaky=0.2, **conv_kwargs) for _ in range(n_extra_layers)]))\n        while cur_size > 4:\n            layers.append(conv_layer(cur_ftrs, cur_ftrs*2, 3, 2, 1, leaky=0.2, **conv_kwargs))\n            cur_ftrs *= 2 ; cur_size //= 2\n        layers += [conv_layer(cur_ftrs, 1, 4, padding=0,leaky=0.2, **conv_kwargs), Flatten()]\n        return nn.Sequential(*layers)\n\n    critic = dfdc_critic(224,3,64,1 )\n\n    apply_init(critic, nn.init.kaiming_normal_)\n    model_dir = 'models/dfdc-critic'\n    os.makedirs(model_dir, exist_ok=True)\n\n    critic_learn = Learner(data=data, model=critic, loss_func=FocalLoss(), \n                    model_dir=model_dir,\n                    metrics=[DFDCAUROC(), RealLoss(),FakeLoss(),DFDCBceLoss()],\n                    )# .to_fp16()\n\n    critic_learn.fit(1)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if RUN_NOTEBOOK==True:\n    # EfficientNet\n    !pip install efficientnet_pytorch\n    from efficientnet_pytorch import EfficientNet\n    effnet = EfficientNet.from_pretrained('efficientnet-b5',num_classes=1)\n    model_dir = 'models/dfdc-effnet'\n    os.makedirs(model_dir, exist_ok=True)\n\n    effnet_learn = Learner(data=data, model=effnet, loss_func=BCEWithLogitsFlat(), \n                    model_dir=model_dir,\n                    metrics=[DFDCAUROC(), RealLoss(),FakeLoss()],\n                    )# .to_fp16()\n    effnet_learn.fit(1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"################################################################################################################\n# General lessons learned\n\n1. Run controlled experiments\n    - Log each experiment\n    - Change one thing at a time\n    - Think about each experiment even the failed ones \n2. Rely on own hardware\n    - Could not figure out how to access the free-TPUs. Lost a few cycles trying to get that going.\n    - Tried uploading face crops to google drive and run training in parallel from Colab - too slow.\n    - Many top scorers seem to have relied on their own GPUs.\n    - My GEFORCE RTX 2060s is fast but limited to 6GB memory\n3.  Find a good partner/team\n    - Just did not have time to run thorough experiments for all the different ideas. Had to abandon most ideas after a few training epochs.\n    - Good partner => sounding board, divide and conquer, potential for ensembles\n4. Learn to read papers and decipher dense math\n  - All the math behind CNNs\n  - How to select an architecture appropriate to the problem?\n  - Hyperparameter selection\n  - Interpreting tensorboard metrics\n5. Some basic questions\n  - How can humans learn with so few examples and yet neural networks require hours and hours of training and then can be used only for a narrow domain?\n  - What will it take to build neural networks that can handle a variety of tasks?\n6. Papers and references\n  - [Age and Gender Recognition from Human Facial Images, Tizita Nesibu Shewaye](https://arxiv.org/pdf/1304.0019.pdf)\n  - [Detection of Deepfake Video Manipulation, Marissa Koopman et al](https://www.researchgate.net/publication/329814168_Detection_of_Deepfake_Video_Manipulation)\n  - [PRNU-based Detection of Morphed Face Images, Luca Debiasi et al](https://www.christoph-busch.de/files/Debiasi-PRNUMorphDetection-IWBF-2018.pdf)\n  - [Deep Learning for Deepfakes Creation and Detection, Thanh Thi Nguyen et al](https://arxiv.org/pdf/1909.11573.pdf)\n  - [CNN ARCHITECTURES FOR LARGE-SCALE AUDIO CLASSIFICATION, Shawn Hershey et al](https://paperswithcode.com/paper/cnn-architectures-for-large-scale-audio)\n  - [USE OF A CAPSULE NETWORK TO DETECT FAKE IMAGES AND VIDEOS, Huy H. Nguyen et al](https://arxiv.org/pdf/1910.12467.pdf)\n  - [Exposing DeepFake Videos By Detecting Face Warping Artifacts, Yuezun Li et al](http://openaccess.thecvf.com/content_CVPRW_2019/papers/Media%20Forensics/Li_Exposing_DeepFake_Videos_By_Detecting_Face_Warping_Artifacts_CVPRW_2019_paper.pdf)\n  - [MesoNet: a Compact Facial Video Forgery Detection Network, Darius Afchar et al](https://arxiv.org/pdf/1809.00888.pdf)\n7. Other articles\n  - https://becominghuman.ai/siamese-networks-algorithm-applications-and-pytorch-implementation-4ffa3304c18\n  - https://towardsdatascience.com/image-forgery-detection-2ee6f1a65442\n8. Code\n  - https://www.kaggle.com/humananalog/inference-demo\n  - https://github.com/polimi-ispl/prnu-python\n  - https://github.com/PPPW/deep-learning-random-explore/blob/master/CNN_archs/cnn_archs.ipynb\n  - https://github.com/locuslab/TCN/blob/master/TCN/poly_music/music_test.py\n  - https://github.com/radekosmulski/whale/blob/master/siamese_network_prototype.ipynb\n\n"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Good luck to all top scorers. On to the next."},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}