{"cells":[{"metadata":{},"cell_type":"markdown","source":"\"Google\n"},{"metadata":{},"cell_type":"markdown","source":"#
PLUG AND PLAY
"},{"metadata":{},"cell_type":"markdown","source":"[Image credits](https://www.kdnuggets.com/2020/06/google-colab-deep-learning.html)"},{"metadata":{},"cell_type":"markdown","source":"# Table of contents \n\n* [Introduction](#0)\n* [STEP 1 ](#1)\n* [STEP 2 ](#2)\n* [STEP 3 ](#3)\n* [STEP 4 ](#4)\n* [STEP 5 ](#5)\n* [Checkpoint](#6)\n* [Full Notebook Overview with checkpoint](#7)\n* [Problems Faced](#8)\n* [Reference](#9)"},{"metadata":{},"cell_type":"markdown","source":"# Introduction\n[Table of contents](#0.1)\n\n**Training on your local machine has its own perks. When it comes to train neural networks with huge dataset most of us find it very hard, especially, beginners and many of us Kagglers do not have access to high end hardware machines. Training takes huge time and hard work. Here in this notebook I will demonstrate how easily you can train your model with huge batch size. So let's get started...**\n\n**COPY AND PASTE CODE FROM THIS NOTEBOOK TO YOUR COLAB NOTEBOOK AND RUN THE CELLS. YOUR OUTPUT SHOULD LOOK SIMILAR TO THE IMAGES I HAVE PROVIDED BELOW IN EACH STEP**.\n\n**IT TOOK ME 7 DAYS TO TRAIN WITH SUCH A HUGE BATCH SIZE OF 64 USING COLAB. TRAINING DURATION DEPENDS ON SIZE OF YOUR IMAGE TOO.**\n\n**PLEASE DO READ THIS NOTEBOOK ONCE BEFORE FOLLOWING STEPS MENTIONED IN THIS NOTEBOOK**.\n\n**FOR FULL OVERVIEW OF CODE. CLICK [HERE](#7).**\n\n### **We are going to use bigger batch size of 64 for training. This will take long time. It took me 7 days to complete training for 30,000 steps (9 hours per day). Don't let colab disconnect for more than 10 minutes. We will be using checkpoint to resume training to save our progress and hard work. Let's move ahead.**\n\n### **I HAVE ALSO MENTIONED ABOUT THE PROBLEM I FACED WHILE TRAINING [HERE](#9)**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import os\nfrom IPython.display import Image\n\n# IMAGE DIR\nPATH = '../input/lyftl5googlecolab/'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# STEP 1\n[Table of contents](#0.1)\n\nFirst and the most important thing you need to add [curlwget](https://chrome.google.com/webstore/detail/curlwget/jmocjfidanebdlinpbcdkcmgdifblncg?hl=en) extention from chrome web store. Then you can see it in top right corner of your browser. You will see how easy it is to download huge datasets from **kaggle** in just few minutes. \n\n### **In the image below you can see the icon of curlwget with a > sign on top right corner of the browser with other extensions**."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH,'curl.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# STEP 2\n[Table of contents](#0.1)\n\nFirst we need to create separate directory for our project. You will be creating this inside host machine. Also, Mount your google drive. Copy and paste below code in your colab notebook cell.\n\n```\n \nimport os \n\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/')\n\n%cd '/content/lyft-motion-prediction-autonomous-vehicles/'\n!pwd\n```"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(filename='../input/lyftl5googlecolab/Screenshot (117).png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# STEP 3\n[Table of contents](#0.1)\n\nPaste this code cell in your colab notebook but don't run it. We need to generate link so that you can download the dataset (NOT WHOLE SINCE COLAB HAS LIMITED SPACE).\n\n```\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/scenes/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/scenes/')\n\n!wget --header=\"Host: storage.googleapis.com\" --header=\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36\" --header=\"Accept: .........3Dscenes.zip\" -c -O 'scenes.zip'\n!unzip ./scenes.zip \n ```"},{"metadata":{},"cell_type":"markdown","source":"# STEP 4\n[Table of contents](#0.1)\n\n* Now click on scenes folder in the [data section](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/data). \n* You will see the download icon. Click on it.\n* Your files will start downloading. "},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'scene.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* As it starts downloading cancel the download and click on curlwget icon. You will see the link click on it. Copy it and paste it in your colab (replace with wget link [here](#3) in STEP 3 with the wget link you copied). "},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'scene_curl.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* run the code cell in given in [STEP 3](#3) inside your colab notebook.\n* This will create **scenes** folder inside your **[lyft-motion-prediction-autonomous-vehicles](#2)** root directory."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'download.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* As you can see **scene folder is created inside your root directory**.\n* See in the image below it will start unzipping your scene data after downloading it. This step will take some time so please wait for its completion."},{"metadata":{"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'inf.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* You will see warning message click on ignore and follow next steps. Check the below image."},{"metadata":{"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'ignore.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* This will almost fill up your storage and in order to proceed further you need to copy and paste below code in three separate cells. Here we are removing **scenes.zip** in order to free some space.\n\n```\n%cd '/content/lyft-motion-prediction-autonomous-vehicles/scenes/'\n!pwd\n!ls\n```\n```\n%rm -r validate.zarr/\n%rm -r sample.zarr/\n%rm scenes.zip\n```\n```\n%cd ..\n!pwd\n!ls\n```"},{"metadata":{},"cell_type":"markdown","source":"* After running the code cells your output should looks something similar to output in image given below."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'normal.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Copy the code given below and paste it your colab notebook cell. Don't run it we need to generate wget link first for **aerial_map** data.\n\n```\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/aerial_map/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/aerial_map/')\n\n!wget --header=\"Host: storage.googleapis.com\" --header=\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36\" --header=\"Accept: ... filename%3Daerial_map.zip\" -c -O 'aerial_map.zip'\n!unzip ./aerial_map.zip\n```\n\n* Now we will follow similar procejure to download **aerial_map** folder. Go to the [data section](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/data) of the competition page. You need to click on aerial_map folder and then click on the download icon and your downloading will start. As soon as your download starts cancel it then click on curlwget icon copy the link and paste it here (replace with wget line in the code cell above) and run the cell. It will create **aerial_map** folder inside your **[lyft-motion-prediction-autonomous-vehicles](#2)** root directory. Check the image below to get the idea."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'aerial.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* After the cell execute copy and paste these two code cells in your colab notebook.\n\n```\n%rm -r aerial_map.zip\n```\n* After running this code cell below you will see aerial_map and scenes folder inside your **[lyft-motion-prediction-autonomous-vehicles](#2)** root directory. See in the image below for reference.\n\n```\n%cd ..\n!pwd\n!ls\n```"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'aerial_dir.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Copy and paste the code cell below in your notebook. We need to generate wget link before moving ahead. Follow the same steps as mentioned [above](#0.2) and paste (replace) the link with one given in !wget line and run the cell. This will create **semantic_map** folder inside your **[lyft-motion-prediction-autonomous-vehicles](#2)** root directory.\n\n```\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/semantic_map/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/semantic_map/')\n\n!wget --header=\"Host: storage.googleapis.com\" --header=\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36\" --header=\"Accept: .......semantic_map.zip\" -c -O 'semantic_map.zip'\n!unzip ./semantic_map.zip\n```\n\n* Now copy the code below in your colab notebook.\n\n```\n%rm -r semantic_map.zip\n%cd ..\n!pwd\n!ls\n```"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'fin.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Upload meta.json in our **[lyft-motion-prediction-autonomous-vehicles](#2)** root directory.\n\n## **IF EVERYTHING GOES RIGHT YOUR FILE STRUCTURE SHOULD LOOK LIKE THIS**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'structure.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Now after fetching all the necessary files and arranging them properly in your colab. We are ready to train our model. Copy and paste this code there in your colab and run it. This will install the L5Kit dependencies and L5kit in your colab. \n\n```\n# this script transports l5kit and dependencies\n!pip -q install pymap3d==2.1.0 \n!pip -q install protobuf==3.12.2 \n!pip -q install transforms3d \n!pip -q install zarr \n!pip -q install ptable\n\n!pip -q install --no-dependencies l5kit\n```"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"Image(os.path.join(PATH, 'dependency.png'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# STEP 5\n[Table of contents](#0.1)\n\nWe need to upload the **meta.json** file in our **[lyft-motion-prediction-autonomous-vehicles](#2)** root directory. After that copy and paste all the code cells in your notebook and execute step by step and run them.\n\n```\n# import packages\nfrom google.colab import files\nimport numpy as np\nimport torch\nimport gc, os\n\nfrom torch import nn, optim\nfrom torch.utils.data import DataLoader\nfrom torchvision.models.resnet import resnet18, resnet34, resnet50\nfrom torchvision.models.densenet import densenet121\nfrom tqdm import tqdm\nfrom typing import Dict\n\nfrom l5kit.data import LocalDataManager, ChunkedDataset\nfrom l5kit.dataset import AgentDataset, EgoDataset\nfrom l5kit.rasterization import build_rasterizer\n```\n\nThis is our root path.\n\n```\nINPUT_DIR = '/content/lyft-motion-prediction-autonomous-vehicles/'\n\n```\n# Configuration\n\nOur configuration file. This is controlling certain parameters. Please keep **num_workers=0** otherwise you will see memory error in colab. Also, we will save our checkpoint after every 1000 steps to ensure that progress isn't lost.\n\n```\ncfg = {\n 'format_version': 4,\n 'model_params': {\n 'model_architecture': 'resnet18',\n 'history_num_frames': 10,\n 'history_step_size': 1,\n 'history_delta_time': 0.1,\n 'future_num_frames': 50,\n 'future_step_size': 1,\n 'future_delta_time': 0.1\n },\n\n 'raster_params': {\n 'raster_size': [300, 300],\n 'pixel_size': [0.5, 0.5],\n 'ego_center': [0.25, 0.5],\n 'map_type': 'py_semantic',\n 'satellite_map_key': 'aerial_map/aerial_map.png',\n 'semantic_map_key': 'semantic_map/semantic_map.pb',\n 'dataset_meta_key': 'meta.json',\n 'filter_agents_threshold': 0.5\n },\n \n 'train_data_loader': {\n 'key': 'scenes/train.zarr',\n 'batch_size': 64,\n 'shuffle': True,\n 'num_workers': 0\n },\n \n 'train_params': {\n 'max_num_steps': 30000,\n 'checkpoint_every_n_steps': 1000,\n }\n}\n```\n\n```\n# set env variable for data\nos.environ[\"L5KIT_DATA_FOLDER\"] = INPUT_DIR\ndm = LocalDataManager(None)\n\n# get config\nprint(cfg)\n```\n# Intialize dataset\n\n```\ntrain_cfg = cfg[\"train_data_loader\"]\n\n# Rasterizer\nrasterizer = build_rasterizer(cfg, dm)\n\n# Train dataset/dataloader\ntrain_zarr = ChunkedDataset(dm.require(train_cfg[\"key\"])).open()\ntrain_dataset = AgentDataset(cfg, train_zarr, rasterizer)\ntrain_dataloader = DataLoader(train_dataset,\n shuffle=train_cfg[\"shuffle\"],\n batch_size=train_cfg[\"batch_size\"],\n num_workers=train_cfg[\"num_workers\"])\n\nprint(train_dataset)\n```\n\n```\ngc.collect()\n```\n# Model: resnet18\n\n```\nclass LyftModel(nn.Module):\n \n def __init__(self, cfg: dict):\n super().__init__()\n \n self.backbone = resnet18(pretrained=True, progress=True)\n \n num_history_channels = (cfg[\"model_params\"][\"history_num_frames\"] + 1) * 2\n num_in_channels = 3 + num_history_channels\n\n self.backbone.conv1 = nn.Conv2d(\n num_in_channels,\n self.backbone.conv1.out_channels,\n kernel_size=self.backbone.conv1.kernel_size,\n stride=self.backbone.conv1.stride,\n padding=self.backbone.conv1.padding,\n bias=False,\n )\n \n backbone_out_features = 512\n\n # X, Y coords for the future positions (output shape: Bx50x2)\n num_targets = 2 * cfg[\"model_params\"][\"future_num_frames\"]\n\n self.head = nn.Sequential(\n # nn.Dropout(0.2),\n nn.Linear(in_features=backbone_out_features, out_features=4096),\n )\n\n self.logit = nn.Linear(4096, out_features=num_targets)\n \n def forward(self, x):\n x = self.backbone.conv1(x)\n x = self.backbone.bn1(x)\n x = self.backbone.relu(x)\n x = self.backbone.maxpool(x) \n\n x = self.backbone.layer1(x)\n x = self.backbone.layer2(x)\n x = self.backbone.layer3(x)\n x = self.backbone.layer4(x)\n\n x = self.backbone.avgpool(x)\n x = torch.flatten(x, 1)\n \n x = self.head(x)\n x = self.logit(x)\n\n return x\n```\n# Intialize model \n\n```\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\nmodel = LyftModel(cfg)\nmodel.to(device)\noptimizer = optim.Adam(model.parameters(), lr=1e-3)\n\n# Later we have to filter the invalid steps.\ncriterion = nn.MSELoss(reduction=\"none\")\n```\n\n```\ndevice\n```\n# Start Training\n\nWhile training we will be saving model checkpoint along with epoch and optimizer state. We will be needing these three in order to resume our training process.\n\n```\ntr_it = iter(train_dataloader)\n\nprogress_bar = tqdm(range(cfg[\"train_params\"][\"max_num_steps\"]))\nlosses_train = []\n\nfor itr in progress_bar:\n\n try:\n data = next(tr_it)\n except StopIteration:\n tr_it = iter(train_dataloader)\n data = next(tr_it)\n\n model.train()\n torch.set_grad_enabled(True)\n \n # Forward pass\n inputs = data[\"image\"].to(device)\n target_availabilities = data[\"target_availabilities\"].unsqueeze(-1).to(device)\n targets = data[\"target_positions\"].to(device)\n \n outputs = model(inputs).reshape(targets.shape)\n loss = criterion(outputs, targets)\n\n # not all the output steps are valid, but we can filter them out from the loss using availabilities\n loss = loss * target_availabilities\n loss = loss.mean()\n\n # Backward pass\n optimizer.zero_grad()\n loss.backward() \n optimizer.step()\n\n losses_train.append(loss.item())\n\n if (itr+1) % cfg['train_params']['checkpoint_every_n_steps'] == 0:\n torch.save({'epoch': itr + 1,\n 'model_state_dict': model.state_dict(),\n 'optimizer_state_dict': optimizer.state_dict()},\n f'/content/drive/My Drive/Lyft L5 Motion Prediction/resnet18_300x300_model_state_{itr}.pth') \n progress_bar.set_description(f\"loss: {loss.item()} loss(avg): {np.mean(losses_train[-100:])}\")\n```"},{"metadata":{},"cell_type":"markdown","source":"# Checkpoint\n[Table of contents](#0.1)\n\nThis is the most important part in whole training procedure. You need to add this in your notebook if somehow it crashes to resume training from last checkpoint. Give the path to your .pth file and load the last successful checkpoint.\n\n### **AlSO, YOU NEED TO MAKE CHANGES IN YOUR TRAINING LOOP IN THIS LINE. ADD epoch in range function.**\n\n### **progress_bar = tqdm(range(epoch, cfg[\"train_params\"][\"max_num_steps\"]))**\n\n### **SEE THE FULL COMPLETE OVERVIEW BELOW TO GET THE IDEA.**\n\n```\nWEIGHT_FILE = '/content/drive/My Drive/PATH TO YOUR .pth file'\ncheckpoint = torch.load(WEIGHT_FILE, map_location=device)\nepoch = checkpoint['epoch']\nmodel.load_state_dict(checkpoint['model_state_dict'])\noptimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n```"},{"metadata":{},"cell_type":"markdown","source":"# Full Notebook Overview with Checkpoint\n[Table of contents](#0.1)\n\nYou can take help from this section to check if everything is correct with respect to your colab notebook.\n```\nfrom google.colab import drive\ndrive.mount('/content/drive')\n```\n\n```\nimport os \n\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/')\n\n%cd '/content/lyft-motion-prediction-autonomous-vehicles/'\n!pwd\n```\n\n```\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/scenes/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/scenes/')\n\n!wget --header=\"Host: storage.googleapis.com\" --header=\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36\" --header=\"Accept: ...........................Dscenes.zip\" -c -O 'scenes.zip'\n!unzip ./scenes.zip\n```\n\n```\n%cd '/content/lyft-motion-prediction-autonomous-vehicles/scenes/'\n!pwd\n!ls\n```\n\n```\n%rm -r validate.zarr/\n%rm -r sample.zarr/\n%rm scenes.zip\n```\n\n```\n%cd ..\n!pwd\n!ls\n```\n\n```\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/aerial_map/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/aerial_map/')\n\n!wget --header=\"Host: storage.googleapis.com\" --header=\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36\" --header=\"Accept: ................Daerial_map.zip\" -c -O 'aerial_map.zip'\n!unzip ./aerial_map.zip\n```\n\n```\n%rm -r aerial_map.zip\n```\n\n```\n%cd ..\n!pwd\n!ls\n```\n\n```\nos.makedirs('/content/lyft-motion-prediction-autonomous-vehicles/semantic_map/', exist_ok=True)\nos.chdir('/content/lyft-motion-prediction-autonomous-vehicles/semantic_map/')\n\n!wget --header=\"Host: storage.googleapis.com\" --header=\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36\" --header=\"Accept: ..............semantic_map.zip\" -c -O 'semantic_map.zip'\n!unzip ./semantic_map.zip\n```\n\n```\n%rm -r semantic_map.zip\n%cd ..\n!pwd\n!ls\n```\n\n```\n## this script transports l5kit and dependencies\n!pip -q install pymap3d==2.1.0 \n!pip -q install protobuf==3.12.2 \n!pip -q install transforms3d \n!pip -q install zarr \n!pip -q install ptable\n\n!pip -q install --no-dependencies l5kit\n```\n\n```\n# import packages\nfrom google.colab import files\nimport numpy as np\nimport torch\nimport gc, os\n\nfrom torch import nn, optim\nfrom torch.utils.data import DataLoader\nfrom torchvision.models.resnet import resnet18, resnet34, resnet50\nfrom torchvision.models.densenet import densenet121\nfrom tqdm import tqdm\nfrom typing import Dict\n\nfrom l5kit.data import LocalDataManager, ChunkedDataset\nfrom l5kit.dataset import AgentDataset, EgoDataset\nfrom l5kit.rasterization import build_rasterizer\n```\n\n```\nINPUT_DIR = '/content/lyft-motion-prediction-autonomous-vehicles/'\n```\n\n```\ncfg = {\n 'format_version': 4,\n 'model_params': {\n 'model_architecture': 'resnet18',\n 'history_num_frames': 10,\n 'history_step_size': 1,\n 'history_delta_time': 0.1,\n 'future_num_frames': 50,\n 'future_step_size': 1,\n 'future_delta_time': 0.1\n },\n\n 'raster_params': {\n 'raster_size': [400, 400],\n 'pixel_size': [0.5, 0.5],\n 'ego_center': [0.25, 0.5],\n 'map_type': 'py_semantic',\n 'satellite_map_key': 'aerial_map/aerial_map.png',\n 'semantic_map_key': 'semantic_map/semantic_map.pb',\n 'dataset_meta_key': 'meta.json',\n 'filter_agents_threshold': 0.5\n },\n \n 'train_data_loader': {\n 'key': 'scenes/train.zarr',\n 'batch_size': 64,\n 'shuffle': True,\n 'num_workers': 0\n },\n \n 'train_params': {\n 'max_num_steps': 30000,\n 'checkpoint_every_n_steps': 1000,\n }\n}\n```\n\n```\n# set env variable for data\nos.environ[\"L5KIT_DATA_FOLDER\"] = INPUT_DIR\ndm = LocalDataManager(None)\n\n# get config\nprint(cfg)\n```\n\n```\n####################\n#INITIALIZE DATASET#\n####################\n\ntrain_cfg = cfg[\"train_data_loader\"]\n\n# Rasterizer\nrasterizer = build_rasterizer(cfg, dm)\n\n# Train dataset/dataloader\ntrain_zarr = ChunkedDataset(dm.require(train_cfg[\"key\"])).open()\ntrain_dataset = AgentDataset(cfg, train_zarr, rasterizer)\ntrain_dataloader = DataLoader(train_dataset,\n shuffle=train_cfg[\"shuffle\"],\n batch_size=train_cfg[\"batch_size\"],\n num_workers=train_cfg[\"num_workers\"])\n\nprint(train_dataset)\n```\n\n```\ngc.collect()\n```\n\n```\nclass LyftModel(nn.Module):\n \n def __init__(self, cfg: Dict):\n super().__init__()\n \n self.backbone = resnet18(pretrained=True, progress=True)\n \n num_history_channels = (cfg[\"model_params\"][\"history_num_frames\"] + 1) * 2\n num_in_channels = 3 + num_history_channels\n\n self.backbone.conv1 = nn.Conv2d(\n num_in_channels,\n self.backbone.conv1.out_channels,\n kernel_size=self.backbone.conv1.kernel_size,\n stride=self.backbone.conv1.stride,\n padding=self.backbone.conv1.padding,\n bias=False,\n )\n \n backbone_out_features = 512\n\n # X, Y coords for the future positions (output shape: Bx50x2)\n num_targets = 2 * cfg[\"model_params\"][\"future_num_frames\"]\n\n self.head = nn.Sequential(\n # nn.Dropout(0.2),\n nn.Linear(in_features=backbone_out_features, out_features=4096),\n )\n\n self.logit = nn.Linear(4096, out_features=num_targets)\n \n def forward(self, x):\n x = self.backbone.conv1(x)\n x = self.backbone.bn1(x)\n x = self.backbone.relu(x)\n x = self.backbone.maxpool(x) \n\n x = self.backbone.layer1(x)\n x = self.backbone.layer2(x)\n x = self.backbone.layer3(x)\n x = self.backbone.layer4(x)\n\n x = self.backbone.avgpool(x)\n x = torch.flatten(x, 1)\n \n x = self.head(x)\n x = self.logit(x)\n\n return x\n```\n\n```\n##################\n#INITIALIZE MODEL#\n##################\n\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\nmodel = LyftModel(cfg)\nmodel.to(device)\noptimizer = optim.Adam(model.parameters(), lr=1e-3)\n\n# Later we have to filter the invalid steps.\ncriterion = nn.MSELoss(reduction=\"none\")\n```\n\n```\ndevice\n```\n## **ADD THIS CELL TO RESUME TRAINING FROM LAST CHECKPOINT**\n\n```\nWEIGHT_FILE = '/content/drive/My Drive/Kaggle/Lyft L5 Motion Prediction/resnet18_400x400_model_state_15999.pth'\ncheckpoint = torch.load(WEIGHT_FILE, map_location=device)\nepoch = checkpoint['epoch']\nmodel.load_state_dict(checkpoint['model_state_dict'])\noptimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n```\n\n```\n#################\n# TRAINING LOOP #\n#################\n\ntr_it = iter(train_dataloader)\n\nprogress_bar = tqdm(range(epoch, cfg[\"train_params\"][\"max_num_steps\"]))\nlosses_train = []\n\nfor itr in progress_bar:\n\n try:\n data = next(tr_it)\n except StopIteration:\n tr_it = iter(train_dataloader)\n data = next(tr_it)\n\n model.train()\n torch.set_grad_enabled(True)\n \n # Forward pass\n inputs = data[\"image\"].to(device)\n target_availabilities = data[\"target_availabilities\"].unsqueeze(-1).to(device)\n targets = data[\"target_positions\"].to(device)\n \n outputs = model(inputs).reshape(targets.shape)\n loss = criterion(outputs, targets)\n\n # not all the output steps are valid, but we can filter them out from the loss using availabilities\n loss = loss * target_availabilities\n loss = loss.mean()\n\n # Backward pass\n optimizer.zero_grad()\n loss.backward() \n optimizer.step()\n\n losses_train.append(loss.item())\n\n if (itr+1) % cfg['train_params']['checkpoint_every_n_steps'] == 0:\n torch.save({'epoch': itr + 1,\n 'model_state_dict': model.state_dict(),\n 'optimizer_state_dict': optimizer.state_dict()},\n f'/content/drive/My Drive/Lyft L5 Motion Prediction/resnet18_300x300_model_state_{itr}.pth') \n progress_bar.set_description(f\"loss: {loss.item()} loss(avg): {np.mean(losses_train[-100:])}\")\n```"},{"metadata":{},"cell_type":"markdown","source":"# Problems Faced\n[Table of contents](#0.1)\n\n* You will get GPU Cooldown message while training for more than 2 sessions after utilizing your free GPU Quota. In this case you can either wait for few hours or can access the notebook using another account😛 by turning on the sharing.\n* Your notebook may suddenly stop. This happends in very rare case but you can again reconnect and everything will work fine.\n* Training with batch size of 64 takes lots of time. So please keep patience.\n* While resuming training again keep in mind to load correct checkpoint and don't forget to delete checkpoints which are of no use to you."},{"metadata":{},"cell_type":"markdown","source":"# Reference\n[Table of contents](#0.1)\n\n* https://www.kaggle.com/pestipeti/pytorch-baseline-inference\n* https://www.kaggle.com/pestipeti/pytorch-baseline-train\n* https://www.kaggle.com/pestipeti/lyft-l5kit-unofficial-fix"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}