{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n# Table of Contents\n1. [Introduction](#introduction)\n1. [Configure hyper-parameters](#configure_hyper_parameters)\n1. [Import libraries](#import_libraries)\n1. [Define useful classes](#define_useful_classes)\n1. [Define helper-functions](#define_helper_functions)\n1. [Get data](#get_data)\n1. [Train the classifier](#train_the_classifier)\n1. [Validate the model](#validate_the_model)\n1. [Conclusion](#conclusion)","metadata":{}},{"cell_type":"markdown","source":"\n# Introduction\n\nIn this step, we will build a simple LogisticRegression model using PyTorch to classify `distances` extracted from REAL and FAKE videos.\n\n---\n## Baseline's diagram\n\n\n---\n## Pipeline\nThis end-to-end solution includes 3 steps:\n1. [*Data Preparation*](https://www.kaggle.com/phunghieu/deepfake-detection-data-preparation-baseline)\n1. *Training* <- **you're here**\n1. [*Inference*](https://www.kaggle.com/phunghieu/deepfake-detection-inference-baseline)\n\n---\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"markdown","source":"\n# Configure hyper-parameters\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"TRAIN_PATH = '/kaggle/input/deepfake-detection-data-preparation/train.csv'\nSAVE_PATH = '/kaggle/working/model.pth'\n\nTEST_SIZE = 0.3\nRANDOM_STATE = 128\nEPOCHS = 200\nBATCH_SIZE = 64\nLR = 1e-4","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Import libraries\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom tqdm.notebook import tqdm\n\ndevice = 'cuda:0' if torch.cuda.is_available() else 'cpu'\nprint(f'Running on device: {device}')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Define useful classes\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"class LogisticRegression(nn.Module):\n def __init__(self, D_in=1, D_out=1):\n super(LogisticRegression, self).__init__()\n self.linear = nn.Linear(D_in, D_out)\n \n def forward(self, x):\n y_pred = self.linear(x)\n\n return y_pred\n \n def predict(self, x):\n result = self.forward(x)\n\n return torch.sigmoid(result)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Define helper-functions\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"def shuffle_data(X, y):\n assert len(X) == len(y)\n \n p = np.random.permutation(len(X))\n \n return X[p], y[p]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Get data\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(TRAIN_PATH)\ntrain_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label_count = train_df.groupby('label').count()['filename']\nprint(label_count)\n\n# Use pos_weight value to overcome imbalanced dataset.\n# https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss\npos_weight = torch.ones([1]) * label_count[0]/label_count[1]\nprint('pos_weight:', pos_weight)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = train_df['distance'].to_numpy()\ny = train_df['label'].to_numpy()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=TEST_SIZE, random_state=RANDOM_STATE, stratify=y)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train = torch.tensor(X_train).to(device).unsqueeze(dim=1).float()\nX_val = torch.tensor(X_val).to(device).unsqueeze(dim=1).float()\ny_train = torch.tensor(y_train).to(device).unsqueeze(dim=1).float()\ny_val = torch.tensor(y_val).to(device).unsqueeze(dim=1).float()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Train the classifier\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"classifier = LogisticRegression()\ncriterion = nn.BCEWithLogitsLoss(reduction='mean', pos_weight=pos_weight) # Improve stability\noptimizer = optim.Adam(classifier.parameters(), lr=LR)\n\nn_batches = np.ceil(len(X_train) / BATCH_SIZE).astype(int)\nlosses = np.zeros(EPOCHS)\nval_losses = np.zeros(EPOCHS)\nbest_val_loss = 1e7\n\nfor e in tqdm(range(EPOCHS)):\n batch_losses = np.zeros(n_batches)\n pbar = tqdm(range(n_batches))\n pbar.desc = f'Epoch {e+1}'\n classifier.train()\n \n # Shuffle training data\n X_train, y_train = shuffle_data(X_train, y_train)\n\n for i in pbar:\n # Get batch.\n X_batch = X_train[i*BATCH_SIZE:min(len(X_train), (i+1)*BATCH_SIZE)]\n y_batch = y_train[i*BATCH_SIZE:min(len(y_train), (i+1)*BATCH_SIZE)]\n\n # Make prediction.\n y_pred = classifier(X_batch)\n\n # Compute loss.\n loss = criterion(y_pred, y_batch)\n batch_losses[i] = loss\n\n # Zero gradients, perform a backward pass, and update the weights.\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n \n # Compute batch loss (average)\n losses[e] = batch_losses.mean()\n \n # Compute val loss\n classifier.eval()\n y_val_pred = classifier(X_val)\n val_losses[e] = criterion(y_val_pred, y_val)\n \n # Save model based on the best (lowest) val loss.\n if val_losses[e] < best_val_loss:\n print('Found a better checkpoint!')\n torch.save(classifier.state_dict(), SAVE_PATH)\n best_val_loss = val_losses[e]\n \n \n # Display some information in progress-bar.\n pbar.set_postfix({\n 'loss': losses[e],\n 'val_loss': val_losses[e]\n })","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(16, 8))\nax = fig.add_axes([0, 0, 1, 1])\n\nax.plot(np.arange(EPOCHS), losses)\nax.plot(np.arange(EPOCHS), val_losses)\nax.set_xlabel('epoch', fontsize='xx-large')\nax.set_ylabel('log loss', fontsize='xx-large')\nax.legend(\n ['loss', 'val loss'],\n loc='upper right',\n fontsize='xx-large',\n shadow=True\n)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Validate the model\n[Back to Table of Contents](#toc)","metadata":{}},{"cell_type":"code","source":"without_weight_criterion = nn.BCELoss(reduction='mean')\n\nclassifier.eval()\nwith torch.no_grad():\n y_val_pred = classifier.predict(X_val)\n val_loss = without_weight_criterion(y_val_pred, y_val)\n\nprint('val loss:', val_loss.detach().numpy())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.hist(y_val_pred.squeeze(dim=-1).detach())\nplt.plot()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Conclusion\nSo, we have done the second step in the whole pipeline. Let's move on to the final step -> [*Deepfake Detection - Inference*](https://www.kaggle.com/phunghieu/deepfake-detection-inference-baseline).\n\nIf you have any questions or suggestions, feel free to move to the `comments` section below.\n\nPlease upvote this kernel if you think it is worth reading; and remember to upvote `@timesler`'s [*kernel*](https://www.kaggle.com/timesler/facial-recognition-model-in-pytorch), too. Thank you so much!\n\n---\n[Back to Table of Contents](#toc)","metadata":{}}]}