{"cells":[{"metadata":{},"cell_type":"markdown","source":"<span style=\"color:White;font-size:28px;background-image: url(&quot;https://storage.googleapis.com/kaggle-competitions/kaggle/19990/logos/header.png?t=2020-08-17-20-10-07&quot;);\" > Lyft Motion Prediction for Autonomous Vehicles</span>\n\n\n\n### WORK IN PROGRESS ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Code Reference - https://github.com/lyft/l5kit/blob/master/examples/\n                 https://github.com/lyft/l5kit","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Autonomous vehicles (AVs) are expected to dramatically redefine the future of transportation. However, there are still significant engineering challenges to be solved before one can fully realize the benefits of self-driving cars. One such challenge is building models that reliably predict the movement of traffic agents around the AV, such as cars, cyclists, and pedestrians.\n\nThe ridesharing company Lyft started Level 5 to take on the self-driving challenge and build a full self-driving system (they’re hiring!). Their previous competition tasked participants with identifying 3D objects, an important step prior to detecting their movement. Now, they’re challenging you to predict the motion of these traffic agents.\n\nIn this competition, you’ll apply your data science skills to build motion prediction models for self-driving vehicles. You'll have access to the largest Prediction Dataset ever released to train and test your models. Your knowledge of machine learning will then be required to predict how cars, cyclists,and pedestrians move in the AV's environment.\n\nLyft’s mission is to improve people’s lives with the world’s best transportation. They believe in a future where self-driving cars make transportation safer, environment-friendly and more accessible for everyone. Their goal is to accelerate development across the industry by sharing data with researchers. As a result of your participation, you can have a hand in propelling the industry forward and helping people around the world benefit from self-driving cars sooner.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"<span style='color:blue;font-size:22px' > Please upvote this kernel if you like it and leave your comments for further improvement ! </span>\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import HTML","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"HTML('<iframe  width=\"850\" height=\"450\" src=\"https://www.youtube.com/embed/K0H43N-Hx7w\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"### !pip install zarr\n!pip install pymap3d==2.1.0\n!pip install -U l5kit","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport random\nimport seaborn as sns\nimport cv2\nimport os\n#import zarr\n# General packages\nimport pandas as pd\nimport numpy as np\nimport matplotlib\nimport matplotlib.pyplot as plt\n\n\nimport torch\nfrom torch import nn, optim\nfrom torch.utils.data import DataLoader\nfrom torchvision.models.resnet import resnet18\nfrom typing import Dict\n\nfrom l5kit.data import ChunkedDataset, LocalDataManager\nfrom l5kit.dataset import EgoDataset, AgentDataset\nfrom l5kit.rasterization import build_rasterizer\nfrom l5kit.configs import load_config_data\n\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"DIR_PATH = '../input/lyft-motion-prediction-autonomous-vehicles'\nos.listdir(DIR_PATH)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Understanding the dataset","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"os.environ[\"L5KIT_DATA_FOLDER\"] = DIR_PATH\ndm = LocalDataManager(None)\n\n# get config\ncfg = load_config_data(\"../input/lyft-config-files/agent_motion_config.yaml\")\nrasterizer = build_rasterizer(cfg, dm)\n\n#train\ntrain_dataset = ChunkedDataset(dm.require(cfg[\"train_data_loader\"][\"key\"])).open()\ntrain_dataloader = DataLoader(AgentDataset(cfg, train_dataset, rasterizer),\n                              shuffle=cfg[\"train_data_loader\"][\"shuffle\"],\n                              batch_size=cfg[\"train_data_loader\"][\"batch_size\"],\n                              num_workers=cfg[\"train_data_loader\"][\"num_workers\"])\n\n# validation\nval_dataset = ChunkedDataset(dm.require(cfg[\"val_data_loader\"][\"key\"])).open()\nval_dataloader = DataLoader(AgentDataset(cfg, val_dataset, rasterizer),\n                              shuffle=cfg[\"val_data_loader\"][\"shuffle\"],\n                              batch_size=cfg[\"val_data_loader\"][\"batch_size\"],\n                              num_workers=cfg[\"val_data_loader\"][\"num_workers\"])\n\n\n\n\nprint(train_dataset)\nprint(\" \\n \")\nprint(cfg)\nprint(\" \\n \")\nprint(val_dataset)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# reference taken from - https://www.kaggle.com/pestipeti/pytorch-baseline-train/data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class LyftModel(nn.Module):\n    \n    def __init__(self, cfg: Dict):\n        super().__init__()\n        \n        self.backbone = resnet18(pretrained=True, progress=True)\n        \n        num_history_channels = (cfg[\"model_params\"][\"history_num_frames\"] + 1) * 2\n        num_in_channels = 3 + num_history_channels\n\n        self.backbone.conv1 = nn.Conv2d(\n            num_in_channels,\n            self.backbone.conv1.out_channels,\n            kernel_size=self.backbone.conv1.kernel_size,\n            stride=self.backbone.conv1.stride,\n            padding=self.backbone.conv1.padding,\n            bias=False,\n        )\n        \n        # This is 512 for resnet18 and resnet34;\n        # And it is 2048 for the other resnets\n        backbone_out_features = 512\n\n        # X, Y coords for the future positions (output shape: Bx50x2)\n        num_targets = 2 * cfg[\"model_params\"][\"future_num_frames\"]\n\n        # You can add more layers here.\n        self.head = nn.Sequential(\n            # nn.Dropout(0.2),\n            nn.Linear(in_features=backbone_out_features, out_features=4096),\n        )\n\n        self.logit = nn.Linear(4096, out_features=num_targets)\n        \n    def forward(self, x):\n        x = self.backbone.conv1(x)\n        x = self.backbone.bn1(x)\n        x = self.backbone.relu(x)\n        x = self.backbone.maxpool(x)\n\n        x = self.backbone.layer1(x)\n        x = self.backbone.layer2(x)\n        x = self.backbone.layer3(x)\n        x = self.backbone.layer4(x)\n\n        x = self.backbone.avgpool(x)\n        x = torch.flatten(x, 1)\n        \n        x = self.head(x)\n        x = self.logit(x)\n        \n        return x","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ==== INIT MODEL\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\nmodel = LyftModel(cfg)\nmodel.to(device)\noptimizer = optim.Adam(model.parameters(), lr=1e-2)\ncriterion = nn.MSELoss(reduction=\"none\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tr_it = iter(train_dataloader)\n\nfor itr in range(cfg[\"train_params\"][\"max_num_steps\"]):\n\n    try:\n        data = next(tr_it)\n    except StopIteration:\n        tr_it = iter(train_dataloader)\n        data = next(tr_it)\n\n    model.train()\n    torch.set_grad_enabled(True)\n    \n    # Forward pass\n    inputs = data[\"image\"].to(device)\n    target_availabilities = data[\"target_availabilities\"].unsqueeze(-1).to(device)\n    targets = data[\"target_positions\"].to(device)\n    \n    outputs = model(inputs).reshape(targets.shape)\n    loss = criterion(outputs, targets)\n\n    # not all the output steps are valid, but we can filter them out from the loss using availabilities\n    loss = loss * target_availabilities\n    loss = loss.mean()\n\n    # Backward pass\n    optimizer.zero_grad()\n    loss.backward()\n    optimizer.step()\n\ndel tr_it","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.eval()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import gc\nfuture_coords_offsets_pd = []\ntimestamps = []\nagent_ids = []\n\nwith torch.no_grad():  \n    \n    for data in val_dataloader:\n\n        inputs = data[\"image\"].to(device)\n        target_availabilities = data[\"target_availabilities\"].unsqueeze(-1).to(device)\n        targets = data[\"target_positions\"].to(device)\n\n        outputs = model(inputs).reshape(targets.shape)\n        \n        future_coords_offsets_pd.append(outputs.cpu().numpy().copy())\n        timestamps.append(data[\"timestamp\"].numpy().copy())\n        agent_ids.append(data[\"track_id\"].numpy().copy())\n        \n        del inputs ,target_availabilities , outputs , targets\n        gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#submission\nfrom l5kit.evaluation import write_pred_csv\nwrite_pred_csv('submission.csv',\n               timestamps=np.concatenate(timestamps),\n               track_ids=np.concatenate(agent_ids),\n               coords=np.concatenate(future_coords_offsets_pd))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}