{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"\n\n

\n

Lyft Motion Prediction for Autonomous Vehicles - EDA⛐

\n \n# 1. Introduction πŸƒ \n \n### 1.1 What is Autonomous Vehicle?\n* [Autonomous Vehicle also known as self-driving car is a vehicle that is capable of sensing its environment and moving safely with little or no human input](http://https://en.wikipedia.org/wiki/Self-driving_car). Self-driving cars combine a variety of sensors to perceive their surroundings, such as radar, lidar, sonar, GPS, odometry and inertial measurement units. Advanced control systems interpret sensory information to identify appropriate navigation paths, as well as obstacles and relevant signage.\n* In this competition, you’ll predict the motion of traffic agents. It is important to predict the movement of traffic agents around the AV such as cars, cyclists, and pedestrians. The challenge is to use machine learning techniques to make prediction for how cars, cyclists,and pedestrians move in the AV's environment.\n\n* You'll predict the motion of the objects in a given scene. For test, you will have 99 frames of objects moving around will be asked to predict their location in the next 50.\n \nI have linked below an informative video.\n","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"from IPython.display import HTML\nHTML('
')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 1.2 What is Lyft Motion Prediction for Autonomous Vehicles Competition?\n- In this competition, you’ll predict the motion of traffic agents. It is important to predict the movement of traffic agents around the AV such as cars, cyclists, and pedestrians. The challenge is to use machine learning techniques to make prediction for how cars, cyclists,and pedestrians move in the AV's environment.\n\n\n- You'll predict the motion of the objects in a given scene. For test, you will have 99 frames of objects moving around will be asked to predict their location in the next 50.\n\n
\n\n\n\n
\n \n \n### 1.3 What we need to do? Observation\nThe goal of this competition is to predict the trajectories of other traffic participants. You can predict up to 3 trajectories for each agent in the test set. Every agent is identified by its `track_id` and its `timestamp`. Each trajectory holds 50 2D `(X,Y)` predictions. \n\n- The leaderboard of this competition is calculated with approximately 50% of the test data. The final results will be based on the other 50%, so the final standings may be different.\n \n### 1.4 Metric: Negative Log-likelihood of the Ground Truth Data\n![](https://latex.codecogs.com/gif.latex?%5Cbg_white%20%5Clarge%20L%20%3D%20-%20%5Clog%20p%28x_%7B1%2C%20%5Cldots%2C%20T%7D%2C%20y_%7B1%2C%20%5Cldots%2C%20T%7D%7Cc%5E%7B1%2C%20%5Cldots%2C%20K%7D%2C%20%5Cbar%7Bx%7D_%7B1%2C%20%5Cldots%2C%20T%7D%5E%7B1%2C%20%5Cldots%2C%20K%7D%2C%20%5Cbar%7By%7D_%7B1%2C%20%5Cldots%2C%20T%7D%5E%7B1%2C%20%5Cldots%2C%20K%7D%29)\n![](https://latex.codecogs.com/gif.latex?%5Cbg_white%20%5Clarge%20%3D%20-%20%5Clog%20%5Csum_k%20e%5E%7B%5Clog%28c%5Ek%29%20-%5Cfrac%7B1%7D%7B2%7D%20%5Csum_t%20%28%5Cbar%7Bx%7D_t%5Ek%20-%20x_t%29%5E2%20+%20%28%5Cbar%7By%7D_t%5Ek%20-%20y_t%29%5E2%7D)\n- Image Credits: https://github.com/lyft/l5kit/blob/master/competition.md\n \nThe evaluation metric of this competition is the negative log-likelihood of the ground truth data given the multi-modal predictions. Because the format is a CSV file, all 3 trajectories fields must have a value, even if your prediction is single-modal. However, each one of the three trajectory has its own confidence, and you can set it 0 to completely ignore one or more trajectories during evaluation. The 3 confidences must sum to 1.\n\n\nRead more about it on the [Evaluation Page](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/overview/evaluation) and [Metrics Page in L5Kit repository](http://https://github.com/lyft/l5kit/blob/master/competition.md).\n\nIf you feel this was something new and fresh, and it added some value to you, please consider upvoting, it motivates to keep writing good kernels. πŸ˜„","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Contents \n\n\n* [Basic Exploratory Data Analysis](#1) \n * [Getting started - Importing libraries]()\n * [Reading the sample_submission]()\n \n \n* [Data Exploration](#2) \n * [Introduction of L5Kit Data]()\n * [How to see L5Kit for Data]()\n * [Load the data]()\n * [Inside the dataset]()\n * [Metadata exploration]()\n\n \n* [Visualising ](#3) \n * [Visualising Autonomous Vehicles]()\n * [Visualising Satellite View]()\n * [Visualising Agent]()\n * [Visualising Individual Scene]()\n ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 2. Importing the necessary librariesπŸ“— ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfrom os import listdir\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport seaborn as sns\nimport numpy as np\n\n#color\nfrom colorama import Fore, Back, Style\n\n# Suppress warnings\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. Reading the submission.csv πŸ“š","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# List files available\nlist(os.listdir(\"../input/lyft-motion-prediction-autonomous-vehicles\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"single_mode_sample_submission = pd.read_csv('../input/lyft-motion-prediction-autonomous-vehicles/multi_mode_sample_submission.csv')\nmulti_mode_sample_submission = pd.read_csv('../input/lyft-motion-prediction-autonomous-vehicles/single_mode_sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.YELLOW + 'Sample submission for single mode shape: ',Style.RESET_ALL,single_mode_sample_submission.shape)\nsingle_mode_sample_submission.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.BLUE + 'Sample submission for multi mode shape: ',Style.RESET_ALL,multi_mode_sample_submission.shape)\nmulti_mode_sample_submission.head(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 4. Basic Data Exploration πŸ•΅β€","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## General Info of Sample Submission","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Null values and Data types\nprint(Fore.YELLOW + 'Single Mode Sample Submission !!',Style.RESET_ALL)\nprint(single_mode_sample_submission.info())\nprint('-------------')\nprint(Fore.BLUE + 'Multi Mode Sample Submission !!',Style.RESET_ALL)\nprint(multi_mode_sample_submission.info())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`float64`: The type of columns except for 2\n

\n \n`int64` : 2 columns - timestamp, track_id","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 5. Introduction of L5Kit for Data πŸ„ ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Reading the data is also complex - please refer to Lyft's [L5Kit](http://https://github.com/lyft/l5kit) module and sample notebooks to properly load the data and use it for training. Further Kaggle-specific sample notebooks will follow shortly if you need more.\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"- Please check [L5Kit github page](http://https://github.com/lyft/l5kit).","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Note also that this competition requires that submissions be made from kernels, and that internet must be turned off in your submission kernels. For your convenience, Lyft's l5kit module is provided via a utility script called [kaggle_l5kit](http://https://www.kaggle.com/philculliton/kaggle-l5kit). Just attach it to your kernel, and the latest version of l5kit and all dependencies will be available.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"- https://github.com/lyft/l5kit\n\n- https://github.com/lyft/l5kit/blob/master/examples/visualisation/visualise_data.ipynb","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Install Kaggle_L5Kit","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Original kaggle l5kit code is below.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"!pip install --upgrade pip > /dev/null \n!pip uninstall typing -y > /dev/null \n!pip install --ignore-installed --target=/kaggle/working l5kit > /dev/null ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## L5Kit Cores","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"I will explain the visualization utility of L5Kit. The core packages for visualisation are:\n- `Rasterization`\n- `visualization`","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Rasterization","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"It contains classes for getting visual data as multi-channel tensors and turning them into interpretable RGB images. Every class has at least a rasterize method to get the tensor and a to_rgb method to convert it into an image. A few examples are:\n\n- `BoxRasterizer`: this object renders agents (e.g. vehicles or pedestrians) as oriented 2D boxes\n- `SatelliteRasterizer`: this object renders an oriented crop from a satellite map","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from l5kit.rasterization import build_rasterizer","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualization","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"It contains utilities to draw additional information (e.g. trajectories) onto RGB images. These utilities are commonly used after a to_rgb call to add other information to the final visualisation.\n- `draw_trajectory`: this function draws 2D trajectories from coordinates and yaws offset on an image","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from l5kit.visualization import draw_trajectory, TARGET_POINTS_COLOR","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And I import some other packages in L5Kit.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 6. How to see L5Kit for Data πŸ™†β€β™‚ ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Import Packages","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"I'll import some other packages in L5Kit including above 2 packages.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from l5kit.data import ChunkedDataset, LocalDataManager\nfrom l5kit.dataset import EgoDataset, AgentDataset\n\nfrom l5kit.rasterization import build_rasterizer\nfrom l5kit.configs import load_config_data\nfrom l5kit.visualization import draw_trajectory, TARGET_POINTS_COLOR\nfrom l5kit.geometry import transform_points\nfrom tqdm import tqdm\nfrom collections import Counter\nfrom l5kit.data import PERCEPTION_LABELS\nfrom prettytable import PrettyTable","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We need to config - `cfg` for visualization. \n

\nYou can see this:\n\nhttps://github.com/lyft/l5kit/blob/master/examples/visualisation/visualisation_config.yaml","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"cfg = {\n #'format_version': 4, \n 'raster_params': {\n 'raster_size': [224, 224],\n 'pixel_size': [0.5, 0.5],\n 'ego_center': [0.25, 0.5],\n 'map_type': 'py_semantic',\n 'satellite_map_key': 'aerial_map/aerial_map.png',\n 'semantic_map_key': 'semantic_map/semantic_map.pb',\n 'dataset_meta_key': 'meta.json',\n 'filter_agents_threshold': 0.5\n },\n \n 'val_data_loader': {\n 'key': 'scenes/train.zarr',\n 'batch_size': 12,\n 'shuffle': True,\n 'num_workers': 16\n },\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f'current raster_param:\\n')\nfor k,v in cfg[\"raster_params\"].items():\n print(Fore.YELLOW + f\"{k}\",Style.RESET_ALL + f\":{v}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- `raster_size`: the image plane size\n- `pixel_size`: how many meters correspond to a pixel\n- `ego_center`: our raster is centered around an agent, we can move the agent in the image plane with this param\n- `map_type`: the rasterizer to be employed. We currently support a satellite-based and a semantic-based one. We will look at the differences further down in this script\n- `filter_agents_threshold` : e.g. 0.0 include every obstacle, 0.5 show those obstacles with >0.5 probability of being one of the classes we care about (cars, bikes, peds, etc.), >=1.0 filter all other agents.\n- `Others - map_key` : the keys are relative to the dataset environment variable.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Load the data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# set env variable for data\nos.environ[\"L5KIT_DATA_FOLDER\"] = \"../input/lyft-motion-prediction-autonomous-vehicles\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dm = LocalDataManager()\ndataset_path = dm.require(cfg[\"val_data_loader\"][\"key\"])\nzarr_dataset = ChunkedDataset(dataset_path)\nzarr_dataset.open()\nprint(zarr_dataset)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- `ChunkedDataset`\n\n\nA dataset that lives on disk in compressed chunks, it has easy to use data loading and writing interfaces that involves making numpy-like slices.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Inside the dataset","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"First, I check zarr format.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import zarr\ntrain_zarr = zarr.open(\"../input/lyft-motion-prediction-autonomous-vehicles/scenes/train.zarr\")\n\nprint(type(train_zarr))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_zarr.info","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- `Arrays`: scenes, frames, agents, traffic_light_faces\n- `Groups`: agents_mask","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"And then I'll show you more information for zzaz.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fields = [\n \"Num Scenes\",\n \"Num Frames\",\n \"Num Agents\",\n \"Total Time (hr)\",\n \"Avg Frames per Scene\",\n \"Avg Agents per Frame\",\n \"Avg Scene Time (sec)\",\n \"Avg Frame frequency\",\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if len(zarr_dataset.frames) > 1:\n times = zarr_dataset.frames[1:50][\"timestamp\"] - zarr_dataset.frames[0:49][\"timestamp\"]\n frequency = np.mean(1 / (times / 1e9)) # from nano to sec\nelse:\n print(f\"warning, not enough frames({len(zarr_dataset.frames)}) to read the frequency, 10 will be set\")\n frequency = 10","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"values = [\n len(zarr_dataset.scenes),\n len(zarr_dataset.frames),\n len(zarr_dataset.agents),\n len(zarr_dataset.frames) / max(frequency, 1) / 3600,\n len(zarr_dataset.frames) / max(len(zarr_dataset.scenes), 1),\n len(zarr_dataset.agents) / max(len(zarr_dataset.frames), 1),\n len(zarr_dataset.frames) / max(len(zarr_dataset.scenes), 1) / frequency,\n frequency,\n]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - scenes","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[0]])\ntable.add_row([values[0]])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.YELLOW + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can use `get_string` for printing `Num Scenes` columns.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.YELLOW + table.get_string(fields=[\"Num Scenes\"]) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`16265` : `scenes`- driving episodes acquired from a given vehicle","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - frames","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[1]])\ntable.add_row([values[1]])\nprint(Fore.BLUE + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`4039527` : `frames` - snapshots in time of the pose of the vehicle","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - agents","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[2]])\ntable.add_row([values[2]])\nprint(Fore.YELLOW + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`320124624` : `agents` - a generic entity captured by the vehicle's sensors. Note that only 4 of the 17 possible agent label_probabilities are present in this dataset.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - Total Time (hr)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[3]])\ntable.float_format = \".2\"\ntable.add_row([values[3]])\nprint(Fore.BLUE + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`112.19`hr : Total Time","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - Avg Frames per Scene","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[4]])\ntable.float_format = \".2\"\ntable.add_row([values[4]])\nprint(Fore.YELLOW + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`248.36` : Avg Frames per Scene","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - Avg Scene Time (sec)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[5]])\ntable.float_format = \".2\"\ntable.add_row([values[5]])\nprint(Fore.BLUE + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`79.25`sec : Avg Scene Time (sec)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Zarr Dataset - Avg Frame frequency","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"table = PrettyTable(field_names=[fields[6]])\ntable.float_format = \".2\"\ntable.add_row([values[6]])\nprint(Fore.YELLOW + str(table) + Style.RESET_ALL)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`24.83` : Avg Frame frequency","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Metadata Exploration","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"I found [Lyft Dataset for csv](http://https://www.kaggle.com/kneroma/lyft-motion-prediction-autonomous-vehicles-as-csv). So, Let's try EDA using this.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents = pd.read_csv('../input/lyft-motion-prediction-autonomous-vehicles-as-csv/agents_0_10019001_10019001.csv')\nagents","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This include familiar features like:\n\n- `x, y, and z` coords\n- `yaw`\n- `probabilites` of other extraneous factors.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-output":true,"_kg_hide-input":false},"cell_type":"code","source":"cont_feats = [\"centroid_x\", \"centroid_y\", \"extent_x\", \"extent_y\", \"extent_z\", \"yaw\"]\nfig = px.imshow(agents[cont_feats].corr(),\n labels=dict(x=\"Correlation of features\", y=\"\", color=\"Correlation\"),\n x=[\"centroid_x\", \"centroid_y\", \"extent_x\", \"extent_y\", \"extent_z\", \"yaw\"],\n y=[\"centroid_x\", \"centroid_y\", \"extent_x\", \"extent_y\", \"extent_z\", \"yaw\"]\n )\nplt.figure(figsize=(16,12));\nfig.update_xaxes(side=\"top\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* `centroid_x` and `centroid_y` : negative correlations\n* `extent_z` and `extent_x` : correlations","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### centroid_x and centroid_y","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(16, 12));\nsns.distplot(agents['centroid_x'], color='steelblue');\nsns.distplot(agents['centroid_y'], color='red');\nplt.title(\"Distributions of Centroid X and Y\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is different between `centriod_x` and `centroid_y` in distbution plot.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### extent_x, extent_y and extent_z","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(16, 12));\nsns.distplot(agents['extent_x'], color='steelblue');\nsns.distplot(agents['extent_y'], color='red');\n\nplt.title(\"Distributions of Extents X and Y\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(16, 12));\nsns.distplot(agents['extent_z'], color='blue');\n\nplt.title(\"Distributions of Extents z\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"they seems right-skewed distribution.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### yaw","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(16, 12));\nsns.distplot(agents['yaw'], color='blue');\n\nplt.title(\"Distributions of Extents z\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### ego_rotatations","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"frms = pd.read_csv(\"../input/lyft-motion-prediction-autonomous-vehicles-as-csv/frames_0_124167_124167.csv\")\nfrms.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`9`: ego rotation columns corresponding to each","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import seaborn as sns\ncolormap = plt.cm.magma\ncont_feats = [\"ego_rotation_xx\", \"ego_rotation_xy\", \"ego_rotation_xz\", \"ego_rotation_yx\", \"ego_rotation_yy\", \"ego_rotation_yz\", \"ego_rotation_zx\", \"ego_rotation_zy\", \"ego_rotation_zz\"]\nplt.figure(figsize=(16,12));\nplt.title('Pearson correlation of features', y=1.05, size=15);\nsns.heatmap(frms[cont_feats].corr(),linewidths=0.1,vmax=1.0, square=True, \n cmap=colormap, linecolor='white', annot=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"cont_feats = [\"ego_rotation_xx\", \"ego_rotation_xy\", \"ego_rotation_xz\", \"ego_rotation_yx\", \"ego_rotation_yy\", \"ego_rotation_yz\", \"ego_rotation_zx\", \"ego_rotation_zy\", \"ego_rotation_zz\"]\nfig = px.imshow(frms[cont_feats].corr(),\n labels=dict(x=\"Correlation of features\", y=\"\", color=\"Correlation\"),\n x=[\"ego_rotation_xx\", \"ego_rotation_xy\", \"ego_rotation_xz\", \"ego_rotation_yx\", \"ego_rotation_yy\", \"ego_rotation_yz\", \"ego_rotation_zx\", \"ego_rotation_zy\", \"ego_rotation_zz\"],\n y=[\"ego_rotation_xx\", \"ego_rotation_xy\", \"ego_rotation_xz\", \"ego_rotation_yx\", \"ego_rotation_yy\", \"ego_rotation_yz\", \"ego_rotation_zx\", \"ego_rotation_zy\", \"ego_rotation_zz\"]\n )\nplt.figure(figsize=(16,12));\nfig.update_xaxes(side=\"top\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* The rotation coordinates with y and z : uncorrelated","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### binary features","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nzero_count_list, one_count_list = [], []\ncols_list = [\"label_probabilities_PERCEPTION_LABEL_UNKNOWN\",\"label_probabilities_PERCEPTION_LABEL_CAR\",\"label_probabilities_PERCEPTION_LABEL_CYCLIST\",\"label_probabilities_PERCEPTION_LABEL_PEDESTRIAN\"]\nfor col in cols_list:\n zero_count_list.append((agents[col]==0).sum())\n one_count_list.append((agents[col]==1).sum())\n\nN = len(cols_list)\nind = np.arange(N)\nwidth = 0.35\n\nplt.figure(figsize=(6,10))\np1 = plt.barh(ind, zero_count_list, width, color='red')\np2 = plt.barh(ind, one_count_list, width, left=zero_count_list, color=\"blue\")\nplt.yticks(ind, cols_list)\nplt.legend((p1[0], p2[0]), ('Zero count', 'One Count'))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 6. Visualization πŸŠβ€β™€ ","execution_count":null},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#plotly\n!pip install chart_studio\nimport plotly.express as px\nimport chart_studio.plotly as py\nimport plotly.graph_objs as go\nfrom plotly.offline import iplot\nimport cufflinks\ncufflinks.go_offline()\ncufflinks.set_config_file(world_readable=True, theme='pearl')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`.zarr` files support most of the traditional `numpy array` operations. In the following cell we iterate over the frames to get a scatter plot of the AV locations:","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"For plot `train.zarr`, It takes a lot of time. So, I'll use `sample.zarr` in this section.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We just change `scenes/train.zarr` to `scenes/sample.zarr` in cfg and re-load dataset.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"cfg = {\n 'raster_params': {\n 'raster_size': [224, 224],\n 'pixel_size': [0.5, 0.5],\n 'ego_center': [0.25, 0.5],\n 'map_type': 'py_semantic',\n 'satellite_map_key': 'aerial_map/aerial_map.png',\n 'semantic_map_key': 'semantic_map/semantic_map.pb',\n 'dataset_meta_key': 'meta.json',\n 'filter_agents_threshold': 0.5\n },\n \n 'val_data_loader': {\n 'key': 'scenes/sample.zarr',\n 'batch_size': 12,\n 'shuffle': True,\n 'num_workers': 16\n },\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dm = LocalDataManager()\ndataset_path = dm.require(cfg[\"val_data_loader\"][\"key\"])\nzarr_dataset = ChunkedDataset(dataset_path)\nzarr_dataset.open()\nprint(zarr_dataset)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can see that `Num Scenes` is just `100`.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Agents","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"* https://www.kaggle.com/gpreda/lyft-first-data-exploration/data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents = zarr_dataset.agents\nagents_df = pd.DataFrame(agents)\nagents_df.columns = [\"data\"]; features = ['centroid', 'extent', 'yaw', 'velocity', 'track_id', 'label_probabilities']\n\nfor i, feature in enumerate(features):\n agents_df[feature] = agents_df['data'].apply(lambda x: x[i])\nagents_df.drop(columns=[\"data\"],inplace=True)\nprint(f\"agents dataset: {agents_df.shape}\")\nagents_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The fields in the agents dataset are the following:\n\n* `centroid` - the agent position (in plane - two dimmensions)\n* `extent` - the agent dimmensions (three dimmensions, let's called length, width, height)\n* `yaw` - the agent oscilation/twist about the vertical plane\n* `velocity` - the speed of the agent - in euclidian space\n* `track_id` - index of track associated to the agent\n* `label_probabilities` - gives the probability for the agent to belong to one of 17 different agent type; we will explore these labels in a moment","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Centroid distribution","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents_df['cx'] = agents_df['centroid'].apply(lambda x: x[0])\nagents_df['cy'] = agents_df['centroid'].apply(lambda x: x[1])\n\nfig, ax = plt.subplots(1,1,figsize=(8,8))\nplt.scatter(agents_df['cx'], agents_df['cy'], marker='+')\nplt.xlabel('x', fontsize=11); plt.ylabel('y', fontsize=11)\nplt.title(\"Centroids distribution\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Extent distribution","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents_df['ex'] = agents_df['extent'].apply(lambda x: x[0])\nagents_df['ey'] = agents_df['extent'].apply(lambda x: x[1])\nagents_df['ez'] = agents_df['extent'].apply(lambda x: x[2])\n\nsns.set_style('whitegrid')\n\nfig, ax = plt.subplots(1,3,figsize=(16,5))\nplt.subplot(1,3,1)\nplt.scatter(agents_df['ex'], agents_df['ey'], marker='+')\nplt.xlabel('ex', fontsize=11); plt.ylabel('ey', fontsize=11)\nplt.title(\"Extent: ex-ey\")\nplt.subplot(1,3,2)\nplt.scatter(agents_df['ey'], agents_df['ez'], marker='+', color=\"red\")\nplt.xlabel('ey', fontsize=11); plt.ylabel('ez', fontsize=11)\nplt.title(\"Extent: ey-ez\")\nplt.subplot(1,3,3)\nplt.scatter(agents_df['ez'], agents_df['ex'], marker='+', color=\"green\")\nplt.xlabel('ez', fontsize=11); plt.ylabel('ex', fontsize=11)\nplt.title(\"Extent: ez-ex\")\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Velocity","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents_df['vx'] = agents_df['velocity'].apply(lambda x: x[0])\nagents_df['vy'] = agents_df['velocity'].apply(lambda x: x[1])\n\nfig, ax = plt.subplots(1,1,figsize=(8,8))\nplt.title(\"Velocity distribution\")\nplt.scatter(agents_df['vx'], agents_df['vy'], marker='.', color=\"red\")\nplt.xlabel('vx', fontsize=11); plt.ylabel('vy', fontsize=11)\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Scenes","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"scenes = zarr_dataset.scenes\nscenes_df = pd.DataFrame(scenes)\nscenes_df.columns = [\"data\"]; features = ['frame_index_interval', 'host', 'start_time', 'end_time']\nfor i, feature in enumerate(features):\n scenes_df[feature] = scenes_df['data'].apply(lambda x: x[i])\nscenes_df.drop(columns=[\"data\"],inplace=True)\nprint(f\"scenes dataset: {scenes_df.shape}\")\nscenes_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax = plt.subplots(1,1, figsize=(6,4))\nsns.countplot(scenes_df.host)\nplt.xlabel('Host')\nplt.ylabel(f'Count')\nplt.title(\"Scenes host count distribution\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scenes_df['frame_index_start'] = scenes_df['frame_index_interval'].apply(lambda x: x[0])\nscenes_df['frame_index_end'] = scenes_df['frame_index_interval'].apply(lambda x: x[1])\nscenes_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Centroid to plot trajectory","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"frames = zarr_dataset.frames\ncoords = np.zeros((len(frames), 2))\nfor idx_coord, idx_data in enumerate(tqdm(range(len(frames)), desc=\"getting centroid to plot trajectory\")):\n frame = zarr_dataset.frames[idx_data]\n coords[idx_coord] = frame[\"ego_translation\"][:2]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = go.Figure(data=go.Scatter(x=coords[:, 0], y=coords[:, 1], mode='lines'))\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = go.Figure()\n\nfig.add_trace(go.Scatter(x=coords[:, 0], y=coords[:, 1], mode='lines', name='lines'))\nfig.add_trace(go.Scatter(x=coords[:, 0], y=coords[:, 1], mode='markers', name='lines+markers'))\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frames_df = pd.DataFrame(zarr_dataset.frames)\nframes_df.columns = [\"data\"]; features = ['timestamp', 'agent_index_interval', 'traffic_light_faces_index_interval', \n 'ego_translation','ego_rotation']\nfor i, feature in enumerate(features):\n frames_df[feature] = frames_df['data'].apply(lambda x: x[i])\nframes_df.drop(columns=[\"data\"],inplace=True)\nprint(f\"frames dataset: {frames_df.shape}\")\nframes_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frames_df['dx'] = frames_df['ego_translation'].apply(lambda x: x[0])\nframes_df['dy'] = frames_df['ego_translation'].apply(lambda x: x[1])\nframes_df['dz'] = frames_df['ego_translation'].apply(lambda x: x[2])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set_style('whitegrid')\nplt.figure()\n\nfig, ax = plt.subplots(1,3,figsize=(16,5))\n\nplt.subplot(1,3,1)\nplt.scatter(frames_df['dx'], frames_df['dy'], marker='+')\nplt.xlabel('dx', fontsize=11); plt.ylabel('dy', fontsize=11)\nplt.title(\"Translations: dx-dy\")\nplt.subplot(1,3,2)\nplt.scatter(frames_df['dy'], frames_df['dz'], marker='+', color=\"red\")\nplt.xlabel('dy', fontsize=11); plt.ylabel('dz', fontsize=11)\nplt.title(\"Translations: dy-dz\")\nplt.subplot(1,3,3)\nplt.scatter(frames_df['dz'], frames_df['dx'], marker='+', color=\"green\")\nplt.xlabel('dz', fontsize=11); plt.ylabel('dx', fontsize=11)\nplt.title(\"Translations: dz-dx\")\n\nfig.suptitle(\"Ego translations in 2D planes of the 3 components (dx,dy,dz)\")\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(3,3,figsize=(12,12))\ncolors = ['magenta', 'orange', 'darkblue', 'black', 'cyan', 'darkgreen', 'red', 'blue', 'green']\nfor i in range(0,3):\n for j in range(0,3):\n df = frames_df['ego_rotation'].apply(lambda x: x[i][j])\n plt.subplot(3,3,i * 3 + j + 1)\n sns.distplot(df, hist=False, color = colors[ i * 3 + j ])\n plt.xlabel(f'r[ {i + 1} ][ {j + 1} ]')\nfig.suptitle(\"Ego rotation angles distribution\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Using below codes, we can `label_probabilities` for agents.\n\n- If you use `train.zaar`, your kernel will be restarted because OOM.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents = zarr_dataset.agents\nprobabilities = agents[\"label_probabilities\"]\nlabels_indexes = np.argmax(probabilities, axis=1)\ncounts = []\nfor idx_label, label in enumerate(PERCEPTION_LABELS):\n counts.append(np.sum(labels_indexes == idx_label))\n \ntable = PrettyTable(field_names=[\"label\", \"counts\"])\nfor count, label in zip(counts, PERCEPTION_LABELS):\n table.add_row([label, count])\nprint(table)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualising the Autonomous Vehicle (AV)\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We need `model_params` for Visualising the Autonomous Vehicle.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"cfg = {\n 'model_params': {\n 'model_architecture': 'resnet50',\n 'history_num_frames': 10,\n 'history_step_size': 1,\n 'history_delta_time': 0.1,\n 'future_num_frames': 50,\n 'future_step_size': 1,\n 'future_delta_time': 0.1\n },\n 'raster_params': {\n 'raster_size': [224, 224],\n 'pixel_size': [0.5, 0.5],\n 'ego_center': [0.25, 0.5],\n 'map_type': 'py_semantic',\n 'satellite_map_key': 'aerial_map/aerial_map.png',\n 'semantic_map_key': 'semantic_map/semantic_map.pb',\n 'dataset_meta_key': 'meta.json',\n 'filter_agents_threshold': 0.5\n }, \n 'val_data_loader': {\n 'key': 'scenes/sample.zarr',\n 'batch_size': 12,\n 'shuffle': True,\n 'num_workers': 16\n },\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"rast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from l5kit.geometry import transform_points\n\nfrom l5kit.visualization import (draw_trajectory, # draws 2D trajectories from coordinates and yaws offset on an image\n TARGET_POINTS_COLOR)\n\ndata = dataset[50]\n\nim = data[\"image\"].transpose(1, 2, 0)\nim = dataset.rasterizer.to_rgb(im)\ntarget_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\ndraw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n\nfig = px.imshow(im[::-1])\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualizing Satellite View","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"cfg[\"raster_params\"][\"map_type\"] = \"py_satellite\"\nrast = build_rasterizer(cfg, dm)\n\n# EgoDataset object\ndataset = EgoDataset(cfg, zarr_dataset, rast)\ndata = dataset[50]\n\nim = data[\"image\"].transpose(1, 2, 0)\nim = dataset.rasterizer.to_rgb(im)\ntarget_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\ndraw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n\nfig = px.imshow(im[::-1], title='Satellite View: Ground Truth Trajectory of Autonomous Vehicle')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualizing Agent","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset = AgentDataset(cfg, zarr_dataset, rast)\ndata = dataset[50]\n\nim = data[\"image\"].transpose(1, 2, 0)\nim = dataset.rasterizer.to_rgb(im)\ntarget_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\ndraw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n\nfig = px.imshow(im[::-1])\nfig.update_layout(\n title={\n 'text': \"Agent\",\n 'y':0.9,\n 'x':0.5,\n 'xanchor': 'center',\n 'yanchor': 'top'})\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* `Green box` : AV agent\n* `Blue boxes` : Entities which we are captured by the sensors\n\n

\n\nWe want to predict the motion of these entities so that our AV can more effectively predict its path.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Visualize Individual Scene: Semantic","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import display, clear_output\nfrom IPython.display import HTML\n\nimport PIL\nfrom matplotlib import animation","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":false,"_kg_hide-output":true},"cell_type":"code","source":"cfg[\"raster_params\"][\"map_type\"] = \"py_satellite\"\nrast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)\nscene_idx = 34\nindexes = dataset.get_scene_indices(scene_idx)\nimages = []\n\nfig = plt.figure(figsize = (10,10))\n\nfor idx in indexes:\n \n data = dataset[idx]\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n target_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\n center_in_pixels = np.asarray(cfg[\"raster_params\"][\"ego_center\"]) * cfg[\"raster_params\"][\"raster_size\"]\n draw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n clear_output(wait=True)\n im = plt.imshow(PIL.Image.fromarray(im[::-1]), animated=True)\n plt.axis(\"off\")\n images.append([im])\nani = animation.ArtistAnimation(fig, images, interval=100, blit=False, repeat_delay=1000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"HTML(ani.to_jshtml())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"cfg[\"raster_params\"][\"map_type\"] = \"py_semantic\"\nrast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)\nscene_idx = 34\nindexes = dataset.get_scene_indices(scene_idx)\nimages = []\n\nfig = plt.figure(figsize = (10,10))\n\nfor idx in indexes:\n data = dataset[idx]\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n target_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\n center_in_pixels = np.asarray(cfg[\"raster_params\"][\"ego_center\"]) * cfg[\"raster_params\"][\"raster_size\"]\n draw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n clear_output(wait=True)\n im = plt.imshow(PIL.Image.fromarray(im[::-1]), animated=True)\n plt.axis(\"off\")\n images.append([im])\nani = animation.ArtistAnimation(fig, images, interval=100, blit=False, repeat_delay=1000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"HTML(ani.to_jshtml())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In this animation, you can see the Autonomous Vehicle is moving on path.\n\n

\nThere is intersection of roads.\n\n- `green box` : Our AV agent and the arrow on top of it represent its motion\n- `blue boxes` : Agents such as cars, cyclists, predestrians\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Visualize Individual Scene: Satellite##","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"for more detailed understanding, we can visualize the satellite view.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# satellite view\ncfg[\"raster_params\"][\"map_type\"] = \"py_satellite\"\nrast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)\nscene_idx = 34\nindexes = dataset.get_scene_indices(scene_idx)\nimages = []\n\nfig = plt.figure(figsize = (10,10))\n\nfor idx in indexes:\n \n data = dataset[idx]\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n target_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\n center_in_pixels = np.asarray(cfg[\"raster_params\"][\"ego_center\"]) * cfg[\"raster_params\"][\"raster_size\"]\n draw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n clear_output(wait=True)\n im = plt.imshow(PIL.Image.fromarray(im[::-1]), animated=True)\n plt.axis(\"off\")\n images.append([im])\nani = animation.ArtistAnimation(fig, images, interval=100, blit=False, repeat_delay=1000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"HTML(ani.to_jshtml())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- `Green` : Autonomous Vehicle(AV) agent\n- `Blue ` : Entities (cars, bicycles and pedestrians)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# General View of the Street\n\n* https://www.kaggle.com/t3nyks/lyft-working-with-map-api","execution_count":null},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"from l5kit.data.map_api import MapAPI\nfrom l5kit.rasterization.rasterizer_builder import _load_metadata\n\nsemantic_map_filepath = dm.require(cfg[\"raster_params\"][\"semantic_map_key\"])\ndataset_meta = _load_metadata(cfg[\"raster_params\"][\"dataset_meta_key\"], dm)\nworld_to_ecef = np.array(dataset_meta[\"world_to_ecef\"], dtype=np.float64)\n\nmap_api = MapAPI(semantic_map_filepath, world_to_ecef)\nMAP_LAYERS = [\"junction\", \"node\", \"segment\", \"lane\"]\n\n\ndef element_of_type(elem, layer_name):\n return elem.element.HasField(layer_name)\n\n\ndef get_elements_from_layer(map_api, layer_name):\n return [elem for elem in map_api.elements if element_of_type(elem, layer_name)]\n\n\nclass MapRenderer:\n \n def __init__(self, map_api):\n self._color_map = dict(drivable_area='#a6cee3',\n road_segment='#1f78b4',\n road_block='#b2df8a',\n lane='#474747')\n self._map_api = map_api\n \n def render_layer(self, layer_name):\n fig = plt.figure(figsize=(10, 10))\n ax = fig.add_axes([0, 0, 1, 1])\n \n def render_lanes(self):\n all_lanes = get_elements_from_layer(self._map_api, \"lane\")\n fig = plt.figure(figsize=(10, 10))\n ax = fig.add_axes([0, 0, 1, 1])\n for lane in all_lanes:\n self.render_lane(ax, lane)\n return fig, ax\n \n def render_lane(self, ax, lane):\n coords = self._map_api.get_lane_coords(MapAPI.id_as_str(lane.id))\n self.render_boundary(ax, coords[\"xyz_left\"])\n self.render_boundary(ax, coords[\"xyz_right\"])\n \n def render_boundary(self, ax, boundary):\n xs = boundary[:, 0]\n ys = boundary[:, 1] \n ax.plot(xs, ys, color=self._color_map[\"lane\"], label=\"lane\")\n \n \nrenderer = MapRenderer(map_api)\nfig, ax = renderer.render_lanes()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def visualize_rgb_image(dataset, index, title=\"\", ax=None):\n \"\"\"Visualizes Rasterizer's RGB image\"\"\"\n data = dataset[index]\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n\n if ax is None:\n fig, ax = plt.subplots()\n if title:\n ax.set_title(title)\n ax.imshow(im[::-1])\n# Prepare all rasterizer and EgoDataset for each rasterizer\nrasterizer_dict = {}\ndataset_dict = {}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"rasterizer_type_list = [\"py_satellite\", \"satellite_debug\", \"py_semantic\", \"semantic_debug\", \"box_debug\", \"stub_debug\"]\n\nfor i, key in enumerate(rasterizer_type_list):\n # print(\"key\", key)\n cfg[\"raster_params\"][\"map_type\"] = key\n rasterizer_dict[key] = build_rasterizer(cfg, dm)\n dataset_dict[key] = EgoDataset(cfg, zarr_dataset, rasterizer_dict[key])\nfig, axes = plt.subplots(2, 3, figsize=(15, 10))\naxes = axes.flatten()\nfor i, key in enumerate([\"stub_debug\", \"satellite_debug\", \"semantic_debug\", \"box_debug\", \"py_satellite\", \"py_semantic\"]):\n visualize_rgb_image(dataset_dict[key], index=0, title=f\"{key}: {type(rasterizer_dict[key]).__name__}\", ax=axes[i])\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## If this kernel is useful, please upvote!\n- See you next time and I will update it soon!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}