![](\"https://self-driving.lyft.com/wp-content/uploads/2020/06/motion_dataset_lrg_redux.gif\")
\n![](\"https://self-driving.lyft.com/wp-content/uploads/2020/06/motion_dataset_2-1.png\")
\n\n The dataset consists of 170,000 scenes capturing the environment around the autonomous vehicle. Each scene encodes the state of the vehicle’s surroundings at a given point in time.\n\n\n
\n\nThe goal of this competition is to predict other car/cyclist/pedestrian (called \"agent\")'s motion.
\n\n![](\"https://self-driving.lyft.com/wp-content/uploads/2020/06/diagram-prediction-1.jpg\")
","metadata":{}},{"cell_type":"markdown","source":"# Environment setup\n\n - Please add \"[philculliton/kaggle-l5kit](https://www.kaggle.com/mathurinache/kaggle-l5kit)\" as utility script\n - Please add [lyft-config-files](https://www.kaggle.com/jpbremer/lyft-config-files) as dataset\n \nSee previous kernel [Lyft: Comprehensive guide to start competition](https://www.kaggle.com/corochann/lyft-comprehensive-guide-to-start-competition) for details.","metadata":{}},{"cell_type":"markdown","source":"## import","metadata":{}},{"cell_type":"code","source":"import gc\nimport os\nfrom pathlib import Path\nimport random\nimport sys\n\nfrom tqdm.notebook import tqdm\nimport numpy as np\nimport pandas as pd\nimport scipy as sp\n\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom IPython.core.display import display, HTML\n\n# --- plotly ---\nfrom plotly import tools, subplots\nimport plotly.offline as py\npy.init_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport plotly.express as px\nimport plotly.figure_factory as ff\nimport plotly.io as pio\npio.templates.default = \"plotly_dark\"\n\n# --- models ---\nfrom sklearn import preprocessing\nfrom sklearn.model_selection import KFold\nimport lightgbm as lgb\nimport xgboost as xgb\nimport catboost as cb\n\n# --- setup ---\npd.set_option('max_columns', 50)","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:27.388467Z","iopub.execute_input":"2025-06-01T04:10:27.388866Z","iopub.status.idle":"2025-06-01T04:10:37.343152Z","shell.execute_reply.started":"2025-06-01T04:10:27.388744Z","shell.execute_reply":"2025-06-01T04:10:37.341941Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import zarr\n\nimport l5kit\nfrom l5kit.data import ChunkedDataset, LocalDataManager\nfrom l5kit.dataset import EgoDataset, AgentDataset\n\nfrom l5kit.rasterization import build_rasterizer\nfrom l5kit.configs import load_config_data\nfrom l5kit.visualization import draw_trajectory, TARGET_POINTS_COLOR\nfrom l5kit.geometry import transform_points\nfrom tqdm import tqdm\nfrom collections import Counter\nfrom l5kit.data import PERCEPTION_LABELS\nfrom prettytable import PrettyTable\n\nfrom matplotlib import animation, rc\nfrom IPython.display import HTML\n\nrc('animation', html='jshtml')\nprint(\"l5kit version:\", l5kit.__version__)","metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:37.344945Z","iopub.execute_input":"2025-06-01T04:10:37.345299Z","iopub.status.idle":"2025-06-01T04:10:47.224647Z","shell.execute_reply.started":"2025-06-01T04:10:37.345263Z","shell.execute_reply":"2025-06-01T04:10:47.223443Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from IPython.display import display, clear_output\nimport PIL\n\n\n# Originally from https://www.kaggle.com/jpbremer/lyft-scene-visualisations by @jpbremer\n# Modified following:\n# - Added to show timestamp\n# - Do not show image, to only show animation.\n# - Use blit=True.\n\ndef animate_solution(images, timestamps=None):\n def animate(i):\n changed_artifacts = [im]\n im.set_data(images[i])\n if timestamps is not None:\n time_text.set_text(timestamps[i])\n changed_artifacts.append(im)\n return tuple(changed_artifacts)\n\n \n fig, ax = plt.subplots()\n im = ax.imshow(images[0])\n if timestamps is not None:\n time_text = ax.text(0.02, 0.95, \"\", transform=ax.transAxes)\n\n anim = animation.FuncAnimation(fig, animate, frames=len(images), interval=60, blit=True)\n \n # To prevent plotting image inline.\n plt.close()\n return anim","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:47.226644Z","iopub.execute_input":"2025-06-01T04:10:47.227172Z","iopub.status.idle":"2025-06-01T04:10:47.240920Z","shell.execute_reply.started":"2025-06-01T04:10:47.227137Z","shell.execute_reply":"2025-06-01T04:10:47.239731Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Initial setup\n\n## Word definition\n - **\"Ego\"** is the host car which is recording/measuring the dataset.\n - **\"Agent\"** is the surronding car except \"Ego\" car.\n - **\"Frame\"** is the 1 image snapshot, where **\"Scene\"** is made of multiple frames of contious-time (video).\n\n## Class diagram\n\n![](\"https://storage.googleapis.com/kaggle-forum-message-attachments/987047/16744/l5kit_class.png\")
\n\n
","metadata":{}},{"cell_type":"markdown","source":"## Loading data\n\nHere we will only use the first dataset from the sample set. (sample.zarr data is used for visualization, please use train.zarr / validate.zarr / test.zarr for actual model training/validation/prediction.)
\nWe're building a `LocalDataManager` object. This will resolve relative paths from the config using the `L5KIT_DATA_FOLDER` env variable we have just set.","metadata":{}},{"cell_type":"code","source":"# set env variable for data\nos.environ[\"L5KIT_DATA_FOLDER\"] = \"/kaggle/input/lyft-motion-prediction-autonomous-vehicles\"\n# get config\ncfg = load_config_data(\"/kaggle/input/lyft-config-files/visualisation_config.yaml\")\nprint(cfg)","metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:47.242616Z","iopub.execute_input":"2025-06-01T04:10:47.242972Z","iopub.status.idle":"2025-06-01T04:10:47.276342Z","shell.execute_reply.started":"2025-06-01T04:10:47.242937Z","shell.execute_reply":"2025-06-01T04:10:47.275056Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dm = LocalDataManager()\ndataset_path = dm.require('scenes/sample.zarr')\nzarr_dataset = ChunkedDataset(dataset_path)\nzarr_dataset.open()\nprint(zarr_dataset)","metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:52.887880Z","iopub.execute_input":"2025-06-01T04:10:52.888273Z","iopub.status.idle":"2025-06-01T04:10:53.517191Z","shell.execute_reply.started":"2025-06-01T04:10:52.888237Z","shell.execute_reply":"2025-06-01T04:10:53.515873Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n\n# 1. Understanding Rasterizer class\n\n## Rasterizer class\n\nThe first topic I will introduce is \"Rasterizer\". This class supports 2 methods (See base [Rasterizer](https://github.com/lyft/l5kit/blob/master/l5kit/l5kit/rasterization/rasterizer.py) class).\n\n - `rasterize` method: to create (ch, height, width) format image. Basically this can be used for the input of prediciton model. It can have any number of channels.\n - `to_rgb` method: to convert image made by rasterize method into RGB image (ch=3, height, width).\n \n`l5kit` already provides several kinds of Rasterizer, each can be instantiated via [build_rasterizer](https://github.com/lyft/l5kit/blob/1e235b8617488e818be30cd7193d43588125bbab/l5kit/l5kit/rasterization/rasterizer_builder.py#L99) method with `cfg`. Let's see each class's role.","metadata":{}},{"cell_type":"code","source":"def visualize_rgb_image(dataset, index, title=\"\", ax=None):\n \"\"\"Visualizes Rasterizer's RGB image\"\"\"\n data = dataset[index]\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n\n if ax is None:\n fig, ax = plt.subplots()\n if title:\n ax.set_title(title)\n ax.imshow(im[::-1])","metadata":{"_kg_hide-input":true,"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:55.858397Z","iopub.execute_input":"2025-06-01T04:10:55.858761Z","iopub.status.idle":"2025-06-01T04:10:55.865260Z","shell.execute_reply.started":"2025-06-01T04:10:55.858725Z","shell.execute_reply":"2025-06-01T04:10:55.863995Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Prepare all rasterizer and EgoDataset for each rasterizer\nrasterizer_dict = {}\ndataset_dict = {}\n\nrasterizer_type_list = [\"py_satellite\", \"satellite_debug\", \"py_semantic\", \"semantic_debug\", \"box_debug\", \"stub_debug\"]\n\nfor i, key in enumerate(rasterizer_type_list):\n # print(\"key\", key)\n cfg[\"raster_params\"][\"map_type\"] = key\n rasterizer_dict[key] = build_rasterizer(cfg, dm)\n dataset_dict[key] = EgoDataset(cfg, zarr_dataset, rasterizer_dict[key])","metadata":{"_kg_hide-input":true,"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T04:10:57.900038Z","iopub.execute_input":"2025-06-01T04:10:57.900391Z","iopub.status.idle":"2025-06-01T04:11:16.707485Z","shell.execute_reply.started":"2025-06-01T04:10:57.900359Z","shell.execute_reply":"2025-06-01T04:11:16.706274Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig, axes = plt.subplots(2, 3, figsize=(15, 10))\naxes = axes.flatten()\nfor i, key in enumerate([\"stub_debug\", \"satellite_debug\", \"semantic_debug\", \"box_debug\", \"py_satellite\", \"py_semantic\"]):\n visualize_rgb_image(dataset_dict[key], index=0, title=f\"{key}: {type(rasterizer_dict[key]).__name__}\", ax=axes[i])\nfig.show()","metadata":{"_kg_hide-input":true,"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T04:11:16.709262Z","iopub.execute_input":"2025-06-01T04:11:16.709615Z","iopub.status.idle":"2025-06-01T04:11:18.007303Z","shell.execute_reply.started":"2025-06-01T04:11:16.709582Z","shell.execute_reply":"2025-06-01T04:11:18.005699Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We see that\n - `StubRasterizer` is just for debugging, creates all black image with specified (height, width).\n - `BoxRasterizer` creates Ego (host car) as green box, and Agent as blue box.\n - `SatelliteRasterizer` draws satellite map.\n - `SemanticRasterizer` draws semantic map which contains lane & crosswalk information\n - `SatBoxRasterizer` = SatelliteRasterizer + BoxRasterizer\n - `SemBoxRasterizer` = SemanticRasterizer + BoxRasterizer\n\n\nNote that I guess Satellite image is NOT taken at the same time when host car moves, so the car on satellite image does NOT match with the car on drawn in BoxRasterizer!
\nSatellite image is useful to get detailed information about the current place, but does not represent the current situation of car or traffic light etc.","metadata":{}},{"cell_type":"markdown","source":"## Meaning of color in semantic map?\n\nLooking the code, I see that\n\n - default lane color is \"light yellow\" (255, 217, 82). [code](https://github.com/lyft/l5kit/blob/1e235b8617488e818be30cd7193d43588125bbab/l5kit/l5kit/rasterization/semantic_rasterizer.py#L198)\n - green, yellow, red color on lane is to show trafic light condition. [code](https://github.com/lyft/l5kit/blob/1e235b8617488e818be30cd7193d43588125bbab/l5kit/l5kit/rasterization/semantic_rasterizer.py#L199-L201)\n - orange box represents crosswalk. [code](https://github.com/lyft/l5kit/blob/1e235b8617488e818be30cd7193d43588125bbab/l5kit/l5kit/rasterization/semantic_rasterizer.py#L204-L211)\n\nPlease refer below animation to verify it.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport PIL.Image\nimport matplotlib.pyplot as plt\nimport cv2\n\nTARGET_POINTS_COLOR = (255, 0, 255) # Magenta\n\ndef draw_trajectory(on_image, positions, yaws=None, rgb_color=(255, 0, 0), radius=2):\n for i, pos in enumerate(positions):\n x, y = int(pos[0]), int(pos[1])\n cv2.circle(on_image, (x, y), radius, rgb_color, -1)\n\n # If yaw is provided, draw arrow\n if yaws is not None:\n yaw = yaws[i]\n arrow_length = 10\n dx = int(arrow_length * np.cos(yaw))\n dy = -int(arrow_length * np.sin(yaw)) # flip y-axis\n\n # Draw arrow from (x, y) to (x+dx, y+dy)\n cv2.arrowedLine(\n on_image,\n (x, y),\n (x + dx, y + dy),\n rgb_color,\n 1,\n tipLength=0.4\n )\n\ndef create_animate_for_indexes(dataset, indexes, draw_yaw=False, max_frames=45):\n images = []\n timestamps = []\n\n if max_frames is not None:\n indexes = indexes[:max_frames]\n\n for idx in indexes:\n data = dataset[idx]\n\n # Convert image to RGB\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n\n # Convert target positions from world to pixel space\n world_target_positions = data[\"target_positions\"] + data[\"centroid\"][:2]\n pixel_target_positions = transform_points(world_target_positions, data[\"world_to_image\"])\n\n # Draw the trajectory\n yaws = data[\"target_yaws\"] if draw_yaw and \"target_yaws\" in data else None\n draw_trajectory(\n on_image=im,\n positions=pixel_target_positions,\n yaws=yaws,\n rgb_color=TARGET_POINTS_COLOR,\n radius=2\n )\n\n images.append(PIL.Image.fromarray(im))\n timestamps.append(data[\"timestamp\"])\n\n anim = animate_solution(images, timestamps)\n return anim\n\ndef create_animate_for_scene(dataset, scene_idx, draw_yaw=False, max_frames=45):\n indexes = dataset.get_scene_indices(scene_idx)\n return create_animate_for_indexes(dataset, indexes, draw_yaw=draw_yaw, max_frames=max_frames)\n","metadata":{"_kg_hide-input":true,"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T05:25:47.590030Z","iopub.execute_input":"2025-06-01T05:25:47.590413Z","iopub.status.idle":"2025-06-01T05:25:47.604296Z","shell.execute_reply.started":"2025-06-01T05:25:47.590378Z","shell.execute_reply":"2025-06-01T05:25:47.603055Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import numpy as np\nimport cv2\nimport PIL.Image\nfrom IPython.display import clear_output\n\nTARGET_POINTS_COLOR = (255, 0, 255)\n\ndef transform_points(points, transform_matrix):\n \"\"\"Applies 3x3 affine transform to Nx2 points.\"\"\"\n points_h = np.hstack((points, np.ones((len(points), 1))))\n transformed = (transform_matrix @ points_h.T).T\n return transformed[:, :2]\n\ndef transform_yaws(yaws, transform_matrix):\n \"\"\"Transforms yaw angles to image frame (taking care of y-flip).\"\"\"\n rot = transform_matrix[:2, :2]\n transformed = []\n for yaw in yaws:\n # Direction in world coordinates\n vec = np.array([np.cos(yaw), np.sin(yaw)])\n # Transform direction to image space\n img_vec = rot @ vec\n # Flip Y since image coordinates increase downward\n img_yaw = np.arctan2(-img_vec[1], img_vec[0])\n transformed.append(img_yaw)\n return np.array(transformed)\n\ndef draw_trajectory(image, positions, yaws, color=(255, 0, 0), radius=2, arrow_length=10):\n for i, pos in enumerate(positions):\n x, y = int(pos[0]), int(pos[1])\n cv2.circle(image, (x, y), radius, color, -1)\n\n if yaws is not None:\n yaw = yaws[i]\n dx = int(arrow_length * np.cos(yaw))\n dy = int(arrow_length * np.sin(yaw))\n # Flip y-delta for image coordinates\n cv2.arrowedLine(image, (x, y), (x + dx, y - dy), color, 1, tipLength=0.4)\n\ndef create_animate_for_indexes(dataset, indexes):\n images = []\n timestamps = []\n\n for idx in indexes:\n data = dataset[idx]\n im = data[\"image\"].transpose(1, 2, 0)\n im = dataset.rasterizer.to_rgb(im)\n\n # Transform positions and yaws to image space\n target_positions_world = data[\"target_positions\"] + data[\"centroid\"][:2]\n target_positions_pixels = transform_points(target_positions_world, data[\"world_to_image\"])\n target_yaws_pixels = transform_yaws(data[\"target_yaws\"], data[\"world_to_image\"])\n\n draw_trajectory(im, target_positions_pixels, target_yaws_pixels, TARGET_POINTS_COLOR)\n\n clear_output(wait=True)\n images.append(PIL.Image.fromarray(im)) # ✅ no vertical flip\n timestamps.append(data[\"timestamp\"])\n\n anim = animate_solution(images, timestamps)\n return anim\n\ndef create_animate_for_scene(dataset, scene_idx):\n indexes = dataset.get_scene_indices(scene_idx)\n return create_animate_for_indexes(dataset, indexes)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T05:54:40.493386Z","iopub.execute_input":"2025-06-01T05:54:40.494212Z","iopub.status.idle":"2025-06-01T05:54:40.515976Z","shell.execute_reply.started":"2025-06-01T05:54:40.494153Z","shell.execute_reply":"2025-06-01T05:54:40.514464Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Car was stopping during traffic light is red, and starts once traffic becomes green.","metadata":{}},{"cell_type":"code","source":"dataset = dataset_dict[\"py_semantic\"]\nscene_idx = 34\nanim = create_animate_for_scene(dataset, scene_idx)\nprint(\"scene_idx\", scene_idx)\nHTML(anim.to_jshtml())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T06:00:17.173189Z","iopub.execute_input":"2025-06-01T06:00:17.173536Z","iopub.status.idle":"2025-06-01T06:00:49.282697Z","shell.execute_reply.started":"2025-06-01T06:00:17.173505Z","shell.execute_reply":"2025-06-01T06:00:49.281542Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"scene_idx = 0\nprint(\"scene_idx\", scene_idx)\nanim = create_animate_for_scene(dataset, scene_idx)\ndisplay(HTML(anim.to_jshtml()))","metadata":{"_kg_hide-input":true,"trusted":true,"execution":{"iopub.status.busy":"2025-06-01T04:33:43.161570Z","iopub.execute_input":"2025-06-01T04:33:43.162004Z","iopub.status.idle":"2025-06-01T04:33:43.287739Z","shell.execute_reply.started":"2025-06-01T04:33:43.161958Z","shell.execute_reply":"2025-06-01T04:33:43.285269Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We can see car comes to the red traffic light and stopped.","metadata":{}},{"cell_type":"code","source":"scene_idx = 1\nprint(\"scene_idx\", scene_idx)\nanim = create_animate_for_scene(dataset, scene_idx)\ndisplay(HTML(anim.to_jshtml()))","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Traffic light is always red in the scene. But agent car which turns right moves, which is allowed in US.","metadata":{}},{"cell_type":"code","source":"scene_idx = 2\nprint(\"scene_idx\", scene_idx)\nanim = create_animate_for_scene(dataset, scene_idx)\ndisplay(HTML(anim.to_jshtml()))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Observation\n\nWe understand that `SemanticRasterizer` creates each lane, traffic light status or car's box as **\"RGB\" image**.
\nHowever it is not necessary for CNN model to input RGB image. Any information representation for each channel is allowed.\n\nI guess different representation may boost the prediction model's performance, which you need to write your own rasterizer.
\nFor example represent own car, other car, lane and traffic light in different channel.","metadata":{}},{"cell_type":"markdown","source":"\n# 2. Understanding EgoDataset/AgentDataset class\n\nInstead of working with raw data, L5Kit provides PyTorch ready datasets.\nIt's much easier to use this wrapped dataset class to access data.\n\n2 dataset class is implemented.\n\n - **EgoDataset**: this dataset iterates over the AV (Autonomous Vehicle) annotations\n - **AgentDataset**: this dataset iterates over other agents annotations\n\nLet's see each class in detail. What kind of attributes/methods they have? What kind of data structure for each attributes?\n\n\nAs written in [Class diagram](#class_diagram), both classes are instantiated by:\n - `cfg`: configuration file\n - `ChunkedDataset`: Internal data class which holds 4 raw data `scenes`, `frames`, `agents` and `tl_faces` (described later).\n - `rasterizer`: Rasterizer converts raw data into image.","metadata":{}},{"cell_type":"markdown","source":"## EgoDataset\n\n\nThe implementation code is in [ego.py](https://github.com/lyft/l5kit/blob/master/l5kit/l5kit/dataset/ego.py).
\n\n### Internal data structure\n![](\"https://storage.googleapis.com/kaggle-forum-message-attachments/990934/16784/l5kit_ego_dataset.png\")
\n\n`EgoDataset` consists of multiple scenes. Each scene is usually 25 sec consecutive events, consists of multiple frames.
\nA frame reprsents specific time's snapshot. Snapshot is taken in 0.1 sec interval, so usually 1 scene is made of about 250 frames.\n\nBlue box represents each scene, orange box represents each frame as well as each data index.\n\nWhen we access dataset by index i, i-th frame is returned. Frames are concatenated by multiple scenes, we different i-th index points different scene.\nThe point where the scene will change is represented by `cumulative_sizes`.","metadata":{}},{"cell_type":"code","source":"semantic_rasterizer = rasterizer_dict[\"py_semantic\"]\ndataset = dataset_dict[\"py_semantic\"]","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# It shows the split point of each scene.\nprint(\"cumulative_sizes\", dataset.cumulative_sizes)\n\n# How's the length of each scene?\nprint(\"Each scene's length\", dataset.cumulative_sizes[1:] - dataset.cumulative_sizes[:-1])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"It seems 1 scene usually consists of 248, 249 frames.\n\nNow Let's check each method:\n\n### getitem, get_frame\n\nWhen we access data by index as `dataset[i]`, `__getitem__` is called and l5kit internally calls `get_frame`.
\nThis method preprocesses the data and returns many features as dict format.","metadata":{}},{"cell_type":"code","source":"data = dataset[0]\n\nprint(\"dataset[0]=data is \", type(data))\n\ndef _describe(value):\n if hasattr(value, \"shape\"):\n return f\"{type(value).__name__:20} shape={value.shape}\"\n else:\n return f\"{type(value).__name__:20} value={value}\"\n\nfor key, value in data.items():\n print(\" \", f\"{key:25}\", _describe(value))","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Each attribute represents follows (The data structure is same for `AgentDataset`, and I included explanation for `AgentDataset` as well):\n\n - image: image drawn by Rasterizer. As you saw on the top of this kernel. This is usually be the **input image for CNN**\n - target_positions: The \"Ego car\" or \"Agent (car/cyclist/pedestrian etc)\"'s future position. This is **the value to predict in this competition (not for Ego car's, but for Agents)**.\n - target_yaws: The Ego car's future yaw, to represent heading direction.\n - target_availabilities: flag to represent this is valid or not. Only flag=1 is used for competition evaluation.\n - history_positions: Past positions\n - history_yaws: Past yaws\n - history_availabilities:\n - world_to_image: 3x3 transformation matrix to convert world-coordinate into pixel-coordinate.\n - track_id: Unique ID for each Agent. `None` for Ego car.\n - timestamp: timestamp for current frame.\n - centroid: current center position\n - yaw: current direction\n - extent: Ego car or Agent's size. The car is not represented as point, but should be cared as dot box to include size information on the map.","metadata":{}},{"cell_type":"markdown","source":"If you don't know what is \"yaw\", please refer [wikipedia](https://en.wikipedia.org/wiki/Yaw_(rotation)).\n\n
![](\"https://upload.wikimedia.org/wikipedia/commons/5/54/Flight_dynamics_with_text.png\")
\n![](\"https://upload.wikimedia.org/wikipedia/commons/9/96/Aileron_yaw.gif\")
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F518134%2F9ea7ae3af0037edb95cc0effe361568f%2Fl5kit_agent_dataset.png?generation=1598749570093014&alt=media\")
\n\n`AgentDataset` consists of same multiple scenes as `EgoDataset`.\nAlways 1 host Ego car exists in each frame, however the number of agent differ.![](\"https://storage.googleapis.com/kaggle-forum-message-attachments/991027/16786/l5kit_chunked_dataset.png\")
\n\n### scenes\n\n```\nSCENE_DTYPE = [\n (\"frame_index_interval\", np.int64, (2,)),\n (\"host\", \"