\n\n🚗Build motion prediction models for self-driving vehicles🚗 ","metadata":{}},{"cell_type":"markdown","source":"# Table of contents🗃\n\n- [0. Introduction](#0)\n\n- [1. Import Packages](#1)\n\n- [2. Utility Functions](#2)\n- [3. Basic EDA](#3)\n - [3.1 Data Formats](#3-1)\n - [3.1.1 Introduction](#3-1-1)\n - [3.1.2 Zarr Format](#3-1-2)\n - [3.1.3 Train, Validation And Test Zarr](#3-1-3)\n - [3.2 Checking Configuration Fields](#3-2)\n - [3.3 Loading Data](#3-3)\n - [3.4 Data Overview](#3-4)\n - [3.4.1 Agents](#3-4-1)\n - [3.4.2 Scenes](#3-4-2)\n - [3.4.3 Frames](#3-4-3)\n - [3.5 Dataset Package](#3-5)\n - [3.5.1 ChunckedDataset](#3-5-1)\n - [3.6 Visualize Autonomous Vehicle](#3-6) \n - [3.6.1 Visualizing Various Rasterizer Objects](#3-6-1) \n - [3.6.2 Visualize Trajectory: Semantic View](#3-6-2)\n - [3.6.3 Visualize Trajectory: Satellite View](#3-6-3)\n - [3.6.4 Visualize Agent](#3-6-4)\n - [3.6.5 Visualize Individual Scene: Semantic](#3-6-5)\n - [3.6.6 Visualize Individual Scene: Satellite](#3-6-6)\n \n- [4. Pytorch Baseline](#4)\n - [4.1 Configuration](#4-1)\n - [4.2 Loading Training Data](#4-2)\n - [4.3 Training DataLoader](#4-3)\n - [4.4 Model: resnet50 (Pytorch)](#4-4)\n - [4.5 Compilation](#4-5)\n - [4.6 Training](#4-6)\n - [4.7 Saving Model](#4-7)\n \n- [5. Prediction and Results](#5)\n - [5.1 Test DataLoader](#5-1)\n - [5.2 Getting Predictions](#5-2)\n - [5.2 Submission](#5-3)\n \n- [6. Reference](#6)","metadata":{}},{"cell_type":"markdown","source":"# 0. Introduction📃\n\n[Table of Contents](#0.1)\n\nThis competition is hosted by ridesharing company [Lyft](https://www.lyft.com/) which started [Level 5](https://self-driving.lyft.com/level5/), self-driving division to tackle the challenges in the field of self-driving cars. \n\nIn this competition our task is to build motion prediction models for self-driving vehicles. Model which can predict the movement of traffic agents around the Autonomous Vehicles such as cars, cyclists, and pedestrians etc. We are required to predict how these different agents move in Autonomous Vehicles's environment.\n\n## What is Autonomous Vehicle?\n\nAn autonomous vehicle, or a driverless vehicle, is one that is able to operate itself and perform necessary functions without any human intervention, through ability to sense its surroundings.\n\nAn autonomous vehicle utilises a fully automated driving system in order to allow the vehicle to respond to external conditions that a human driver would manage.\n\n## Competition Data\n\nThis [dataset](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/data) is the largest collection of the traffic agent motion data. This dataset includes the logs of movement of cars, cyclists, pedestrians, and other traffic agents encountered by Lyft's autonomous fleet. These logs come from processing raw lidar, camera, and radar data through our team’s perception systems and are ideal for training motion prediction models. The dataset includes:\n\n* 1000+ hours of traffic agent movement\n* 16k miles of data from 23 vehicle\n* 15k semantic map annotations\n\nHere is the reserach paper of [Prediction Dataset](https://paperswithcode.com/paper/one-thousand-and-one-hours-self-driving).\n\nThe dataset consists of 170,000 scenes capturing the environment around the autonomous vehicle. Each scene encodes the state of the vehicle’s surroundings at a given point in time. [source](https://self-driving.lyft.com/level5/prediction/)\n\n\n![](\"https://self-driving.lyft.com/wp-content/uploads/2020/06/motion_dataset_lrg_redux.gif\")
\n![](\"https://self-driving.lyft.com/wp-content/uploads/2020/06/motion_dataset_2-1.png\")
\n\n
\n\nThis baseline solution is trained on over 2 million samples from the agent locations contained within the dataset. The model predicts a single agent at a time. First, a raster generates a bird’s eye view (BEV) top-down raster, which encodes all agents and the map. The network infers the future coordinates of the agent-based upon this raster.\n\n![](\"https://self-driving.lyft.com/wp-content/uploads/2020/06/diagram-prediction-1.jpg\")
\n
\n \n## What we are predicting?\n\nOur task in the competition is to predict the motion of external objects such as cars, cyclist, pedestrains etc in order to assist the self-driving car. We have to predict the location of objects agents in the next 50 frames.\n\n## Evaluation Metric: Negative log-likelihood\n\nWe calculate the negative log-likelihood of the ground truth data given the multi-modal predictions. You can get more information [here](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/overview/evaluation).\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"## Lyft's Autonomous Vehicle (AV)\n\n**Lyft's** AV Introducton video. ","metadata":{}},{"cell_type":"markdown","source":"# 1. Import Packages📚\n\n[Table of Contents](#0.1)\n\nWe are required to use [L5Kit toolkit](https://github.com/lyft/l5kit) provided by the competition host to prepare/preprocess data, trian and evaluate the model. Please add this [utility script](https://www.kaggle.com/pestipeti/lyft-l5kit-unofficial-fix) provided by [Peter's](https://www.kaggle.com/pestipeti) first by clicking on the [+ Add data]() section inside your notebook before starting. The [L5Kit toolkit](https://github.com/lyft/l5kit) has some issue right now. You can check [this](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/177125) discussion here for more information. \n\n## L5Kit is a library which lets you:\n\n* Load driving scenes from zarr files\n* Read semantic maps\n* Read aerial maps\n* Create birds-eye-view (BEV) images which represent a scene around an AV or another vehicle\n* Sample data\n* Train neural networks\n* Visualize results","metadata":{}},{"cell_type":"code","source":"from IPython.display import HTML\n\nHTML('
\nmodel_state = torch.load(WEIGHT_FILE, map_location=device)\n
\nmodel.load_state_dict(model_state['model_state_dict'])","metadata":{}},{"cell_type":"code","source":"device","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-31T01:11:52.411763Z","iopub.status.idle":"2025-05-31T01:11:52.412092Z","shell.execute_reply":"2025-05-31T01:11:52.411927Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We need to run below two cell to make predictions and generate **submission.csv**. I am going to use my submission.csv here since I am getting memory error while making predictions. ","metadata":{}},{"cell_type":"code","source":"\nmodel.eval()\ntorch.set_grad_enabled(False)\n\n# store information for evaluation\nfuture_coords_offsets_pd = []\ntimestamps = []\n\nagent_ids = []\nprogress_bar = tqdm(test_dataloader)\nfor data in progress_bar:\n \n inputs = data[\"image\"].to(device)\n target_availabilities = data[\"target_availabilities\"].unsqueeze(-1).to(device)\n targets = data[\"target_positions\"].to(device)\n\n outputs = model(inputs).reshape(targets.shape)\n \n future_coords_offsets_pd.append(outputs.cpu().numpy().copy())\n timestamps.append(data[\"timestamp\"].numpy().copy())\n agent_ids.append(data[\"track_id\"].numpy().copy())","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-05-31T02:17:02.440764Z","iopub.execute_input":"2025-05-31T02:17:02.441058Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5.3 Submission\n\n[Table of Contents](#0.1)","metadata":{}},{"cell_type":"code","source":"# submission.csv\n'''\nwrite_pred_csv('submission.csv',\n timestamps=np.concatenate(timestamps),\n track_ids=np.concatenate(agent_ids),\n coords=np.concatenate(future_coords_offsets_pd),\n )\n \n'''\nmodel_sub = pd.read_csv('/kaggle/input/lyft-l5-inference-batch64-resnet18/submission.csv')\nmodel_sub.to_csv('submission.csv', index = False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-31T01:11:52.414032Z","iopub.status.idle":"2025-05-31T01:11:52.414414Z","shell.execute_reply":"2025-05-31T01:11:52.414225Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 6. References📎\n\n[Table of Contents](#0.1)\n\n* https://www.twi-global.com/technical-knowledge/faqs/what-is-an-autonomous-vehicle\n* [Negative log-likelihood](https://github.com/lyft/l5kit/blob/master/competition.md)\n* https://www.kaggle.com/pestipeti/pytorch-baseline-train\n* https://self-driving.lyft.com/level5/prediction/\n* https://www.kaggle.com/corochann/lyft-deep-into-the-l5kit-library\n* https://www.kaggle.com/corochann/lyft-deep-into-the-l5kit-library#1.-Understanding-Rasterizer-class\n* https://www.kaggle.com/gpreda/lyft-first-data-exploration","metadata":{}}]}
","metadata":{}},{"cell_type":"markdown","source":"# Table of contents🗃\n\n- [0. Introduction](#0)\n\n- [1. Import Packages](#1)\n\n- [2. Utility Functions](#2)\n- [3. Basic EDA](#3)\n - [3.1 Data Formats](#3-1)\n - [3.1.1 Introduction](#3-1-1)\n - [3.1.2 Zarr Format](#3-1-2)\n - [3.1.3 Train, Validation And Test Zarr](#3-1-3)\n - [3.2 Checking Configuration Fields](#3-2)\n - [3.3 Loading Data](#3-3)\n - [3.4 Data Overview](#3-4)\n - [3.4.1 Agents](#3-4-1)\n - [3.4.2 Scenes](#3-4-2)\n - [3.4.3 Frames](#3-4-3)\n - [3.5 Dataset Package](#3-5)\n - [3.5.1 ChunckedDataset](#3-5-1)\n - [3.6 Visualize Autonomous Vehicle](#3-6) \n - [3.6.1 Visualizing Various Rasterizer Objects](#3-6-1) \n - [3.6.2 Visualize Trajectory: Semantic View](#3-6-2)\n - [3.6.3 Visualize Trajectory: Satellite View](#3-6-3)\n - [3.6.4 Visualize Agent](#3-6-4)\n - [3.6.5 Visualize Individual Scene: Semantic](#3-6-5)\n - [3.6.6 Visualize Individual Scene: Satellite](#3-6-6)\n \n- [4. Pytorch Baseline](#4)\n - [4.1 Configuration](#4-1)\n - [4.2 Loading Training Data](#4-2)\n - [4.3 Training DataLoader](#4-3)\n - [4.4 Model: resnet50 (Pytorch)](#4-4)\n - [4.5 Compilation](#4-5)\n - [4.6 Training](#4-6)\n - [4.7 Saving Model](#4-7)\n \n- [5. Prediction and Results](#5)\n - [5.1 Test DataLoader](#5-1)\n - [5.2 Getting Predictions](#5-2)\n - [5.2 Submission](#5-3)\n \n- [6. Reference](#6)","metadata":{}},{"cell_type":"markdown","source":"# 0. Introduction📃\n\n[Table of Contents](#0.1)\n\nThis competition is hosted by ridesharing company [Lyft](https://www.lyft.com/) which started [Level 5](https://self-driving.lyft.com/level5/), self-driving division to tackle the challenges in the field of self-driving cars. \n\nIn this competition our task is to build motion prediction models for self-driving vehicles. Model which can predict the movement of traffic agents around the Autonomous Vehicles such as cars, cyclists, and pedestrians etc. We are required to predict how these different agents move in Autonomous Vehicles's environment.\n\n## What is Autonomous Vehicle?\n\nAn autonomous vehicle, or a driverless vehicle, is one that is able to operate itself and perform necessary functions without any human intervention, through ability to sense its surroundings.\n\nAn autonomous vehicle utilises a fully automated driving system in order to allow the vehicle to respond to external conditions that a human driver would manage.\n\n## Competition Data\n\nThis [dataset](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/data) is the largest collection of the traffic agent motion data. This dataset includes the logs of movement of cars, cyclists, pedestrians, and other traffic agents encountered by Lyft's autonomous fleet. These logs come from processing raw lidar, camera, and radar data through our team’s perception systems and are ideal for training motion prediction models. The dataset includes:\n\n* 1000+ hours of traffic agent movement\n* 16k miles of data from 23 vehicle\n* 15k semantic map annotations\n\nHere is the reserach paper of [Prediction Dataset](https://paperswithcode.com/paper/one-thousand-and-one-hours-self-driving).\n\nThe dataset consists of 170,000 scenes capturing the environment around the autonomous vehicle. Each scene encodes the state of the vehicle’s surroundings at a given point in time. [source](https://self-driving.lyft.com/level5/prediction/)\n\n\n\n\n\n
\n\nThis baseline solution is trained on over 2 million samples from the agent locations contained within the dataset. The model predicts a single agent at a time. First, a raster generates a bird’s eye view (BEV) top-down raster, which encodes all agents and the map. The network infers the future coordinates of the agent-based upon this raster.\n\n\n
\n \n## What we are predicting?\n\nOur task in the competition is to predict the motion of external objects such as cars, cyclist, pedestrains etc in order to assist the self-driving car. We have to predict the location of objects agents in the next 50 frames.\n\n## Evaluation Metric: Negative log-likelihood\n\nWe calculate the negative log-likelihood of the ground truth data given the multi-modal predictions. You can get more information [here](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/overview/evaluation).\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"## Lyft's Autonomous Vehicle (AV)\n\n**Lyft's** AV Introducton video. ","metadata":{}},{"cell_type":"markdown","source":"# 1. Import Packages📚\n\n[Table of Contents](#0.1)\n\nWe are required to use [L5Kit toolkit](https://github.com/lyft/l5kit) provided by the competition host to prepare/preprocess data, trian and evaluate the model. Please add this [utility script](https://www.kaggle.com/pestipeti/lyft-l5kit-unofficial-fix) provided by [Peter's](https://www.kaggle.com/pestipeti) first by clicking on the [+ Add data]() section inside your notebook before starting. The [L5Kit toolkit](https://github.com/lyft/l5kit) has some issue right now. You can check [this](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/177125) discussion here for more information. \n\n## L5Kit is a library which lets you:\n\n* Load driving scenes from zarr files\n* Read semantic maps\n* Read aerial maps\n* Create birds-eye-view (BEV) images which represent a scene around an AV or another vehicle\n* Sample data\n* Train neural networks\n* Visualize results","metadata":{}},{"cell_type":"code","source":"from IPython.display import HTML\n\nHTML('
\nmodel_state = torch.load(WEIGHT_FILE, map_location=device)\n
\nmodel.load_state_dict(model_state['model_state_dict'])","metadata":{}},{"cell_type":"code","source":"device","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-31T01:11:52.411763Z","iopub.status.idle":"2025-05-31T01:11:52.412092Z","shell.execute_reply":"2025-05-31T01:11:52.411927Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We need to run below two cell to make predictions and generate **submission.csv**. I am going to use my submission.csv here since I am getting memory error while making predictions. ","metadata":{}},{"cell_type":"code","source":"\nmodel.eval()\ntorch.set_grad_enabled(False)\n\n# store information for evaluation\nfuture_coords_offsets_pd = []\ntimestamps = []\n\nagent_ids = []\nprogress_bar = tqdm(test_dataloader)\nfor data in progress_bar:\n \n inputs = data[\"image\"].to(device)\n target_availabilities = data[\"target_availabilities\"].unsqueeze(-1).to(device)\n targets = data[\"target_positions\"].to(device)\n\n outputs = model(inputs).reshape(targets.shape)\n \n future_coords_offsets_pd.append(outputs.cpu().numpy().copy())\n timestamps.append(data[\"timestamp\"].numpy().copy())\n agent_ids.append(data[\"track_id\"].numpy().copy())","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-05-31T02:17:02.440764Z","iopub.execute_input":"2025-05-31T02:17:02.441058Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5.3 Submission\n\n[Table of Contents](#0.1)","metadata":{}},{"cell_type":"code","source":"# submission.csv\n'''\nwrite_pred_csv('submission.csv',\n timestamps=np.concatenate(timestamps),\n track_ids=np.concatenate(agent_ids),\n coords=np.concatenate(future_coords_offsets_pd),\n )\n \n'''\nmodel_sub = pd.read_csv('/kaggle/input/lyft-l5-inference-batch64-resnet18/submission.csv')\nmodel_sub.to_csv('submission.csv', index = False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-31T01:11:52.414032Z","iopub.status.idle":"2025-05-31T01:11:52.414414Z","shell.execute_reply":"2025-05-31T01:11:52.414225Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 6. References📎\n\n[Table of Contents](#0.1)\n\n* https://www.twi-global.com/technical-knowledge/faqs/what-is-an-autonomous-vehicle\n* [Negative log-likelihood](https://github.com/lyft/l5kit/blob/master/competition.md)\n* https://www.kaggle.com/pestipeti/pytorch-baseline-train\n* https://self-driving.lyft.com/level5/prediction/\n* https://www.kaggle.com/corochann/lyft-deep-into-the-l5kit-library\n* https://www.kaggle.com/corochann/lyft-deep-into-the-l5kit-library#1.-Understanding-Rasterizer-class\n* https://www.kaggle.com/gpreda/lyft-first-data-exploration","metadata":{}}]}
\n\n