{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Guide to Google Cloud Training, Ensembling & Learning Learning Rates\n\nThe objectives of this kernel are to first provide a quick overview of the steps I followed to set up virtual machine (VM) instance on Google Cloud Platform, to download the required competition files, to train the models, how I changed the parameters to obtain the public score of <0.45, and also how to leave the instance running upon exiting the terminal.\n\nIn particular, this will be my **last public kernel (with scores) until the competition ends**. The end of the competition is in about a month, and I figured that some insights on the tweeks to the existing training scripts could help those who have yet to make some sort of breakthrough at this point. \n\n\nBest regards,\n\nWei Hao"},{"metadata":{},"cell_type":"markdown","source":"# Google Cloud Platform\n\nGoogle Cloud Platform (GCP), offered by Google, is a suite of cloud computing services that runs on the same infrastructure that Google uses internally for its end-user products, such as Google Search and YouTube.[1] Alongside a set of management tools, it provides a series of modular cloud services including computing, data storage, data analytics and machine learning. Registration requires a credit card or bank account details. Google Cloud Platform provides infrastructure as a service, platform as a service, and serverless computing environments.\n\nIn April 2008, Google announced App Engine, a platform for developing and hosting web applications in Google-managed data centers, which was the first cloud computing service from the company. The service became generally available in November 2011. Since the announcement of App Engine, Google added multiple cloud services to the platform.\n\nGoogle Cloud Platform is a part of Google Cloud, which includes the Google Cloud Platform public cloud infrastructure, as well as G Suite, enterprise versions of Android and Chrome OS, and application programming interfaces (APIs) for machine learning and enterprise mapping services.\n\n**Source:** Wikipedia"},{"metadata":{},"cell_type":"markdown","source":"## Free Credits\n\nIf you are a first time user, you can get around 300 USD free credits to play around with, which I personally feel is quite decent! To-date, I've only used 100 USD, since Jan 2020. "},{"metadata":{},"cell_type":"markdown","source":"## Compute Engine\n\nCompute Engine delivers configurable virtual machines running in Google’s data centers with access to high-performance networking infrastructure and block storage. It caters to:\n- Select the right VM for your needs, whether general purpose, or workload optimized, in predefined or custom machine sizes\n- Integrate compute with other Google Cloud services such as AI/ML and data analytics\n- Make reservations to help ensure your applications have the capacity they need as they scale. Save money just for running compute with sustained-use discounts, and achieve greater savings when you use committed-use discounts\n\n**Source:** https://cloud.google.com/compute"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from IPython.display import Image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Launching and Navigating the Virtual Machine (VM) Instance\n\nOnce you have created your VM instance with the desired CPU & GPU resources, go to 'VM Instances' and you should see something like this:"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/google_cloud_compute_engine_launch_vm.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Once you are able to see that, click on the dropdown icon beside SSH and click 'Open in broswer window' (see above). A console like the following should appear:"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/google_cloud_vm.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can refer to https://github.com/Kaggle/kaggle-api, where the README describes the CLI and API commands required to download the competition data and files into the VM's directory."},{"metadata":{},"cell_type":"markdown","source":"# Model Training - Parameter Tuning\n\nFor model training, I am using GreatGameDota's script found at https://www.kaggle.com/greatgamedota/xception-classifier-w-ffhq-training-lb-537."},{"metadata":{},"cell_type":"markdown","source":"## Epochs, Factor, Learning Rates & Patience (IMPORTANT EXPERIMENTAL NOTES)\n\nI've tried various learning rates `lr` (0.001, 0.0015, 0.002, 0.004), `epochs` (10, 12, 20, 42), `patience` (2, 5, 7) and `factor` (0.1, 0.2, 0.5, 0.7). The result from these is the score of 0.50480 for the Xception single model, and 0.43846 for the Resnext + Xception esemble. Further tweeking of these parameters do help increase the public LB score by a bit. Also, yet interestingly, as the score of the single model Xception decreases, the higher scoring esembles have equal weights, i.e. 0.5 for Resnext and 0.5 for Xception. \n\nFeel free to tweek these parameters further! Note that the total time for each run of 30 epochs or more on Google Cloud Compute took **more than 20 hours**. So it may be wise to plan your parameters properly based on the insights gathered from past trainings!"},{"metadata":{},"cell_type":"markdown","source":"## Some Xception (Single Model) Training Outputs"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"!ls ../input/deepfake-kernel-data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Learning Rate: 0.0015, Epochs: 42, Patience: 5"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/lr_15e-2_epochs_42_patience_5.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Learning Rate: 0.002, Epochs: 10, Patience: 5"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/lr_2e-3_epochs_10_patience_5.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Learning Rate: 0.002, Epochs: 20, Patience: 5"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/lr_2e-3_epochs_20_patience_5.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Learning Rate: 0.004, Epochs: 12, Patience: 2"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/lr_4e-3_epochs_12_patience_2.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Learning Rate: 0.004, Epochs: 30, Patience: 2"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/lr_4e-3_epochs_30_patience_2.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Some Training Outputs in the VM"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image('../input/deepfake-kernel-data/google_cloud_vm_deepfake_training_screenshot.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Running the Instance in the Backend with Screen"},{"metadata":{},"cell_type":"markdown","source":"To leave the VM instance running in the backend, you can use `screen`. For installation details and how to execute it, instructions can be found in the source link below, and at https://stackoverflow.com/questions/48221807/google-cloud-instance-terminate-after-close-browser.\n\nScreen or GNU Screen is a terminal multiplexer. In other words, it means that you can start a screen session and then open any number of windows (virtual terminals) inside that session. Processes running in Screen will continue to run when their window is not visible even if you get disconnected.\n\n**Source:** https://linuxize.com/post/how-to-use-linux-screen/"},{"metadata":{},"cell_type":"markdown","source":"# Resnext & Xception Ensemble\n\n- The following have been taken from https://www.kaggle.com/khoongweihao/xception-resnext-ensemble-inference"},{"metadata":{},"cell_type":"markdown","source":"## Resnext Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os, sys, time\nimport cv2\nimport numpy as np\nimport pandas as pd\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n%matplotlib inline\nimport matplotlib.pyplot as plt\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_dir = \"/kaggle/input/deepfake-detection-challenge/test_videos/\"\n\ntest_videos = sorted([x for x in os.listdir(test_dir) if x[-4:] == \".mp4\"])\nframe_h = 5\nframe_l = 5\nlen(test_videos)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"PyTorch version:\", torch.__version__)\nprint(\"CUDA version:\", torch.version.cuda)\nprint(\"cuDNN version:\", torch.backends.cudnn.version())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gpu = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\ngpu","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import sys\nsys.path.insert(0, \"/kaggle/input/blazeface-pytorch\")\nsys.path.insert(0, \"/kaggle/input/deepfakes-inference-demo\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from blazeface import BlazeFace\nfacedet = BlazeFace().to(gpu)\nfacedet.load_weights(\"/kaggle/input/blazeface-pytorch/blazeface.pth\")\nfacedet.load_anchors(\"/kaggle/input/blazeface-pytorch/anchors.npy\")\n_ = facedet.train(False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from helpers.read_video_1 import VideoReader\nfrom helpers.face_extract_1 import FaceExtractor\n\nframes_per_video = 64 #frame_h * frame_l\nvideo_reader = VideoReader()\nvideo_read_fn = lambda x: video_reader.read_frames(x, num_frames=frames_per_video)\nface_extractor = FaceExtractor(video_read_fn, facedet)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"input_size = 224","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from torchvision.transforms import Normalize\n\nmean = [0.485, 0.456, 0.406]\nstd = [0.229, 0.224, 0.225]\nnormalize_transform = Normalize(mean, std)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def isotropically_resize_image(img, size, resample=cv2.INTER_AREA):\n    h, w = img.shape[:2]\n    if w > h:\n        h = h * size // w\n        w = size\n    else:\n        w = w * size // h\n        h = size\n\n    resized = cv2.resize(img, (w, h), interpolation=resample)\n    return resized\n\n\ndef make_square_image(img):\n    h, w = img.shape[:2]\n    size = max(h, w)\n    t = 0\n    b = size - h\n    l = 0\n    r = size - w\n    return cv2.copyMakeBorder(img, t, b, l, r, cv2.BORDER_CONSTANT, value=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch.nn as nn\nimport torchvision.models as models\n\nclass MyResNeXt(models.resnet.ResNet):\n    def __init__(self, training=True):\n        super(MyResNeXt, self).__init__(block=models.resnet.Bottleneck,\n                                        layers=[3, 4, 6, 3], \n                                        groups=32, \n                                        width_per_group=4)\n        self.fc = nn.Linear(2048, 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"checkpoint = torch.load(\"/kaggle/input/deepfakes-inference-demo/resnext.pth\", map_location=gpu)\n\nmodel = MyResNeXt().to(gpu)\nmodel.load_state_dict(checkpoint)\n_ = model.eval()\n\ndel checkpoint","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def predict_on_video(video_path, batch_size):\n    try:\n        # Find the faces for N frames in the video.\n        faces = face_extractor.process_video(video_path)\n\n        # Only look at one face per frame.\n        face_extractor.keep_only_best_face(faces)\n        \n        if len(faces) > 0:\n            # NOTE: When running on the CPU, the batch size must be fixed\n            # or else memory usage will blow up. (Bug in PyTorch?)\n            x = np.zeros((batch_size, input_size, input_size, 3), dtype=np.uint8)\n\n            # If we found any faces, prepare them for the model.\n            n = 0\n            for frame_data in faces:\n                for face in frame_data[\"faces\"]:\n                    # Resize to the model's required input size.\n                    # We keep the aspect ratio intact and add zero\n                    # padding if necessary.                    \n                    resized_face = isotropically_resize_image(face, input_size)\n                    resized_face = make_square_image(resized_face)\n\n                    if n < batch_size:\n                        x[n] = resized_face\n                        n += 1\n                    else:\n                        print(\"WARNING: have %d faces but batch size is %d\" % (n, batch_size))\n                    \n                    # Test time augmentation: horizontal flips.\n                    # TODO: not sure yet if this helps or not\n                    #x[n] = cv2.flip(resized_face, 1)\n                    #n += 1\n\n            if n > 0:\n                x = torch.tensor(x, device=gpu).float()\n\n                # Preprocess the images.\n                x = x.permute((0, 3, 1, 2))\n\n                for i in range(len(x)):\n                    x[i] = normalize_transform(x[i] / 255.)\n\n                # Make a prediction, then take the average.\n                with torch.no_grad():\n                    y_pred = model(x)\n                    y_pred = torch.sigmoid(y_pred.squeeze())\n                    return y_pred[:n].mean().item()\n\n    except Exception as e:\n        print(\"Prediction error on video %s: %s\" % (video_path, str(e)))\n\n    return 0.5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from concurrent.futures import ThreadPoolExecutor\n\ndef predict_on_video_set(videos, num_workers):\n    def process_file(i):\n        filename = videos[i]\n        y_pred = predict_on_video(os.path.join(test_dir, filename), batch_size=frames_per_video)\n        return y_pred\n\n    with ThreadPoolExecutor(max_workers=num_workers) as ex:\n        predictions = ex.map(process_file, range(len(videos)))\n\n    return list(predictions)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"speed_test = False  # you have to enable this manually","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if speed_test:\n    start_time = time.time()\n    speedtest_videos = test_videos[:5]\n    predictions = predict_on_video_set(speedtest_videos, num_workers=4)\n    elapsed = time.time() - start_time\n    print(\"Elapsed %f sec. Average per video: %f sec.\" % (elapsed, elapsed / len(speedtest_videos)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = predict_on_video_set(test_videos, num_workers=4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df_resnext = pd.DataFrame({\"filename\": test_videos, \"label\": predictions})\nsubmission_df_resnext.to_csv(\"submission_resnext.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Xception Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install ../input/deepfake-xception-trained-model/pytorchcv-0.0.55-py2.py3-none-any.whl --quiet","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_dir = \"/kaggle/input/deepfake-detection-challenge/test_videos/\"\n\ntest_videos = sorted([x for x in os.listdir(test_dir) if x[-4:] == \".mp4\"])\nlen(test_videos)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gpu = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\ngpu","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import sys\nsys.path.insert(0, \"/kaggle/input/blazeface-pytorch\")\nsys.path.insert(0, \"/kaggle/input/deepfakes-inference-demo\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from blazeface import BlazeFace\nfacedet = BlazeFace().to(gpu)\nfacedet.load_weights(\"/kaggle/input/blazeface-pytorch/blazeface.pth\")\nfacedet.load_anchors(\"/kaggle/input/blazeface-pytorch/anchors.npy\")\n_ = facedet.train(False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from helpers.read_video_1 import VideoReader\nfrom helpers.face_extract_1 import FaceExtractor\n\nframes_per_video = 64 # originally 4\n\nvideo_reader = VideoReader()\nvideo_read_fn = lambda x: video_reader.read_frames(x, num_frames=frames_per_video)\nface_extractor = FaceExtractor(video_read_fn, facedet)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"input_size = 150","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from torchvision.transforms import Normalize\n\nmean = [0.485, 0.456, 0.406]\nstd = [0.229, 0.224, 0.225]\nnormalize_transform = Normalize(mean, std)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def isotropically_resize_image(img, size, resample=cv2.INTER_AREA):\n    h, w = img.shape[:2]\n    if w > h:\n        h = h * size // w\n        w = size\n    else:\n        w = w * size // h\n        h = size\n\n    resized = cv2.resize(img, (w, h), interpolation=resample)\n    return resized\n\n\ndef make_square_image(img):\n    h, w = img.shape[:2]\n    size = max(h, w)\n    t = 0\n    b = size - h\n    l = 0\n    r = size - w\n    return cv2.copyMakeBorder(img, t, b, l, r, cv2.BORDER_CONSTANT, value=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/deepfake-xception-trained-model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls ../input/deepfake-kernel-data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from pytorchcv.model_provider import get_model\nmodel = get_model(\"xception\", pretrained=False)\nmodel = nn.Sequential(*list(model.children())[:-1]) # Remove original output layer\n\nclass Pooling(nn.Module):\n  def __init__(self):\n    super(Pooling, self).__init__()\n    \n    self.p1 = nn.AdaptiveAvgPool2d((1,1))\n    self.p2 = nn.AdaptiveMaxPool2d((1,1))\n\n  def forward(self, x):\n    x1 = self.p1(x)\n    x2 = self.p2(x)\n    return (x1+x2) * 0.5\n\nmodel[0].final_block.pool = nn.Sequential(nn.AdaptiveAvgPool2d((1,1)))\n\nclass Head(torch.nn.Module):\n  def __init__(self, in_f, out_f):\n    super(Head, self).__init__()\n    \n    self.f = nn.Flatten()\n    self.l = nn.Linear(in_f, 512)\n    self.d = nn.Dropout(0.5)\n    self.o = nn.Linear(512, out_f)\n    self.b1 = nn.BatchNorm1d(in_f)\n    self.b2 = nn.BatchNorm1d(512)\n    self.r = nn.ReLU()\n\n  def forward(self, x):\n    x = self.f(x)\n    x = self.b1(x)\n    x = self.d(x)\n\n    x = self.l(x)\n    x = self.r(x)\n    x = self.b2(x)\n    x = self.d(x)\n\n    out = self.o(x)\n    return out\n\nclass FCN(torch.nn.Module):\n  def __init__(self, base, in_f):\n    super(FCN, self).__init__()\n    self.base = base\n    self.h1 = Head(in_f, 1)\n  \n  def forward(self, x):\n    x = self.base(x)\n    return self.h1(x)\n\nnet = []\nmodel = FCN(model, 2048)\nmodel = model.cuda()\nmodel.load_state_dict(torch.load('../input/deepfake-kernel-data/model_50epochs_lr0001_patience5_factor01_batchsize32.pth')) # new, updated\nnet.append(model)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Prediction Loop"},{"metadata":{"trusted":true},"cell_type":"code","source":"def predict_on_video(video_path, batch_size):\n    try:\n        # Find the faces for N frames in the video.\n        faces = face_extractor.process_video(video_path)\n\n        # Only look at one face per frame.\n        face_extractor.keep_only_best_face(faces)\n        \n        if len(faces) > 0:\n            # NOTE: When running on the CPU, the batch size must be fixed\n            # or else memory usage will blow up. (Bug in PyTorch?)\n            x = np.zeros((batch_size, input_size, input_size, 3), dtype=np.uint8)\n\n            # If we found any faces, prepare them for the model.\n            n = 0\n            for frame_data in faces:\n                for face in frame_data[\"faces\"]:\n                    # Resize to the model's required input size.\n                    # We keep the aspect ratio intact and add zero\n                    # padding if necessary.                    \n                    resized_face = isotropically_resize_image(face, input_size)\n                    resized_face = make_square_image(resized_face)\n\n                    if n < batch_size:\n                        x[n] = resized_face\n                        n += 1\n                    else:\n                        print(\"WARNING: have %d faces but batch size is %d\" % (n, batch_size))\n                    \n                    # Test time augmentation: horizontal flips.\n                    # TODO: not sure yet if this helps or not\n                    #x[n] = cv2.flip(resized_face, 1)\n                    #n += 1\n\n            if n > 0:\n                x = torch.tensor(x, device=gpu).float()\n\n                # Preprocess the images.\n                x = x.permute((0, 3, 1, 2))\n\n                for i in range(len(x)):\n                    x[i] = normalize_transform(x[i] / 255.)\n#                     x[i] = x[i] / 255.\n\n                # Make a prediction, then take the average.\n                with torch.no_grad():\n                    y_pred = model(x)\n                    y_pred = torch.sigmoid(y_pred.squeeze())\n                    return y_pred[:n].mean().item()\n\n    except Exception as e:\n        print(\"Prediction error on video %s: %s\" % (video_path, str(e)))\n\n    return 0.5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from concurrent.futures import ThreadPoolExecutor\n\ndef predict_on_video_set(videos, num_workers):\n    def process_file(i):\n        filename = videos[i]\n        y_pred = predict_on_video(os.path.join(test_dir, filename), batch_size=frames_per_video)\n        return y_pred\n\n    with ThreadPoolExecutor(max_workers=num_workers) as ex:\n        predictions = ex.map(process_file, range(len(videos)))\n\n    return list(predictions)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"speed_test = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if speed_test:\n    start_time = time.time()\n    speedtest_videos = test_videos[:5]\n    predictions = predict_on_video_set(speedtest_videos, num_workers=4)\n    elapsed = time.time() - start_time\n    print(\"Elapsed %f sec. Average per video: %f sec.\" % (elapsed, elapsed / len(speedtest_videos)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nmodel.eval()\npredictions = predict_on_video_set(test_videos, num_workers=4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df_xception = pd.DataFrame({\"filename\": test_videos, \"label\": predictions})\nsubmission_df_xception.to_csv(\"submission_xception.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df_resnext.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df_xception.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Ensemble of Resnext & Xception\n\n- Resnext single model public score: 0.46441\n- Xception single model public score: 0.50480"},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df = pd.DataFrame({\"filename\": test_videos})\nsubmission_df[\"label\"] = 0.5*submission_df_resnext[\"label\"] + 0.5*submission_df_xception[\"label\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Thanks for checking this out and I hope it will help in some way. May Gauss bless us all."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}