{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Zarr files and L5kit data for dummies","execution_count":null},{"metadata":{},"cell_type":"markdown","source":">When I saw this competition for the first time, I said to myself: great, I'm going to have a lot of fun :) . And very quickly, I signed up. But, in fact I hadn't seen the real data yet. The data, although it seems quite rich to me, is presented in a format that makes you want to go kill yourself rather than touch it (I'm exaggerating a bit of course).\n\n>The few public notebooks that try to approach this ogre, all use the same logic that totally prevents you from understanding the the data. Indeed, they all use the L5kit API which is a great API (I think it is optimized from an application point of view). But let's not hide it, this data format is far from the one we are used to caress with Pandas. So all is lost? Well no! This is what I will try to show in this work.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In this notebook, we will discover together, step by step, what is a **zarr** file and as a use-case, we will apply our discoveries to the **L5kit zarr dataset**. Let's start right away !","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To achieve our goals, we will need some ingredients:\n* **Zarr** : as you may have guessed, it is the main package for handling **zarr** files\n* **Numpy** :  zarr files are built in front of **Numpy arrays**\n* **Pandas**:  our zarr files will be parsed into **Pandas DataFrames** for further analysis","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"try:\n    import zarr\nexcept  ModuleNotFoundError:\n    ! pip install zarr > /dev/null\n    ! pip install ipytree > /dev/null","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import zarr\nimport pandas as pd, numpy as np\nimport itertools as it # I will be using the `itertools.chain` function\nfrom pathlib import Path # for better file/path operations management","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h5 style=\"color:blue;text-align:center;\">Please upvote the kernel if you find it useful. You'll motivate me to go through the junky documentations in order to make this competition Great Again :) !</h5>","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 1. Off-topic zarr tutorial for dummies\n> This part is off-topic :), but not the less. Here, I will make a general introduction to zarr files. If you can't wait to get down to business, feel free to jump tho #part2 .","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Zarr provides classes and functions for working with N-dimensional arrays that behave like NumPy arrays but whose data is divided into chunks and each chunk is compressed. If you are already familiar with HDF5 then Zarr arrays provide similar functionality, but with some additional flexibility.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.1 Creating an array","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Zarr has several functions for creating arrays. For example:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z = zarr.zeros((10000, 10000), chunks=(1000, 1000), dtype='i4')\nz","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The code above creates a 2-dimensional array of 32-bit integers with 10000 rows and 10000 columns, divided into chunks where each chunk has 1000 rows and 1000 columns (and so there will be 100 chunks in total).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.2 Reading and writing data","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Zarr arrays support a similar interface to NumPy arrays for reading and writing data. For example, the entire array can be filled with a scalar value:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z[:] = 42","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Regions of the array can also be written to, e.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z[0, :] = np.arange(10000)\nz[:, 0] = np.arange(10000)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The contents of the array can be retrieved by slicing, which will load the requested region into memory as a NumPy array, e.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z[0, 0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z[-1, -1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z[0, :]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z[:]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.3 Persistent arrays","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"In the examples above, compressed data for each chunk of the array was stored in main memory. Zarr arrays can also be stored on a file system, enabling persistence of data between sessions. For example:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z1 = zarr.open('data/example.zarr', mode='w', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')\nz1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":">The array above will store its configuration metadata and all compressed chunk data in a directory called ‘data/example.zarr’ relative to the current working directory. The zarr.convenience.open() function provides a convenient way to create a new persistent array or continue working with an existing array. Note that although the function is called “open”, there is no need to close an array: data are automatically flushed to disk, and files are automatically closed whenever an array is modified.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Persistent arrays support the same interface for reading and writing data, e.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z1[:] = 42","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z1[0, :] = np.arange(10000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z1[:, 0] = np.arange(10000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Check that the data have been written and can be read again:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z2 = zarr.open('data/example.zarr', mode='r')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.all(z1[:] == z2[:])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If you are just looking for a fast and convenient way to save NumPy arrays to disk then load back into memory later, the functions zarr.convenience.save() and zarr.convenience.load() may be useful. E.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"a = np.arange(10)\nzarr.save('data/example.zarr', a)\nzarr.load('data/example.zarr')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.4 Groups","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Zarr supports hierarchical organization of arrays via groups. As with arrays, groups can be stored in memory, on disk, or via other storage systems that support a similar interface.\n\nTo create a group, use the zarr.group() function:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"root = zarr.group()\nroot","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Groups have a similar API to the Group class from h5py. For example, groups can contain other groups:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"foo = root.create_group('foo')\nbar = foo.create_group('bar')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Groups can also contain arrays, e.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"z1 = bar.zeros('baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='i4')\nz1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Members of a group can be accessed via the suffix notation, e.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"root['foo']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The ‘/’ character can be used to access multiple levels of the hierarchy in one call, e.g.:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"root['foo/bar']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"root['foo/bar/baz']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The zarr.hierarchy.Group.tree() method can be used to print a tree representation of the hierarchy,","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(root.tree(expand=True))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.5 User attributes","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Zarr arrays and groups support custom key/value attributes, which can be useful for storing application-specific metadata. For example:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"root = zarr.group()\nroot.attrs['foo'] = 'bar'\nz = root.zeros('zzz', shape=(10000, 10000))\nz.attrs['baz'] = 42\nz.attrs['qux'] = [1, 4, 7, 12]\nsorted(root.attrs)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'foo' in root.attrs","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"root.attrs['foo']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sorted(z.attrs)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z.attrs['baz']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"z.attrs['qux']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Here is the for our Zarr tutorial. For more ressources, you can visit [the official documentation](https://zarr.readthedocs.io/en/stable/tutorial.html#creating-an-array) where I took most of the above examples.**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. The L5Kit dataset","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 2.1 Introduction","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The `L5Kit data` is stored in **zarr** format which is basically a set of numpy structured arrays. Conceptually, it is similar to a set of CSV files with records and different columns as we've seen it above.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"As for any **zarr** file, there must be a root folder. In our case, the root folder would likely look like **<...>/lyft-motion-prediction-autonomous-vehicles**. I set mine in the **DATA_ROOT** global variable as below :","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Set your root path to data\nDATA_ROOT = Path(\"../input/lyft-motion-prediction-autonomous-vehicles\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5 = zarr.open(DATA_ROOT.joinpath(\"scenes/sample.zarr\").as_posix(), mode=\"r\")\nzl5","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, the **L5Kit** data consists of groups of zarr datasets. We take a look by doing:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5.info","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> The dataset name is obivously equal to \"**/**\" as we're in the root folder. The has 5 memebers, namely:\n* **Scenes** :  a collection of frames\n* **Frames** :  a collection of agents (the host agents + other agents)\n* **Agents** : Any object in circulation with the automatic vehicle (AV)\n* **Traffic_light_faces** : traffic lights and their faces (bulbs)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can also see the dataset's **tree** by doing :","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(zl5.tree(expand=True))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So there are:\n * 100 **scenes** in the sample dataset\n * 24838 **frames**\n * 1893736 **agents**\n * 316008 **traffic_light_faces**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.2 The L5Kit dataset: scenes","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's get more info from the **scenes** :","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5.scenes.info","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 2.2.1. What is a scene ?","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's take a look into a **scene**'s `dtype`:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5.scenes.dtype","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, a **scene**  consists of  3 block of things :\n* **Frames** : a scene has a list of frames that start from ***scene.frame_index_interval\\[0\\]*** and ends at ***scene.frame_index_interval\\[1\\]***\n* **Host** : a scene has a ***host*** which is the AV that films the scene.\n* **Timestamps**: a scene has a ***start_time***  and an ***end_time***","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**I will make a small function which take in a scene and outputs those components as a `dict`.**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def parse_scene(scene):\n    scene_dict = {\n            \"frame_index_interval_start\": scene[0][0],\n            \"frame_index_interval_end\": scene[0][1],\n            \"host\":  scene[1],\n            \"start_time\": scene[2],\n            \"end_time\": scene[3]\n        }\n    return scene_dict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scene = zl5.scenes[0]\nscene","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"parse_scene(scene)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Nice one ! A scene is less ugly now !**. We can just iterate over all the scenes and got them into a pandas DataFrame where we could make deeper analysis and create more features to train a good model.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Instead of making a simple function that naively iterates over the scenes, I will expose a robust interface that takes into account the fact that accessing (indexing) a **zarr** file is a somehow **expensive operation**  as **Zarr** needs to unpack the compressed chunk before taking the right index. In place and lieu of taking a single index, I will take a range (slice) in order to make **Zarr** faster.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### 2.2.2 Parsing scenes into a Pandas DataFrame","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class BaseParser:\n    \"\"\"\n    A robust and fast interface to load l5kit data into  Pandas dataframes.\n\n    Parameters\n    ----------\n    chunk_size: int, default=1000\n        How many items do you want in a single slice. The larger the better;\n        as long as you have enough memory. Nevertheless, chunk sizes above `10_000` won't lead to\n        significant speed gain as the original zarr files was chunked at 10_000.\n\n    max_chunks: int, default=10\n        How many chunks do you want to read from memory.\n\n    root:\n        Zarr data root path\n\n    zarr_path:\n        relative path or key to the data.\n    \"\"\"\n    \n    field = \"scenes\"\n    dtypes = {}\n    \n    def __init__(self, start=0, end=None, chunk_size=1000, max_chunks=10, root=DATA_ROOT, zarr_path=\"scenes/sample.zarr\"):\n        \n        self.start = start\n        self.end = end\n        self.chunk_size = chunk_size\n        self.max_chunks = max_chunks\n        \n\n        self.root = Path(root)\n        assert self.root.exists(), \"There is nothing at {}!\".format(self.root)\n        self.zarr_path = Path(zarr_path)\n        \n     \n    def parse(self):\n        raise NotImplementedError\n        \n    def to_pandas(self, start=0, end=None, chunk_size=None, max_chunks=None):\n        start = start or self.start\n        end = end or self.end\n        chunk_size = chunk_size or self.chunk_size\n        max_chunks = max_chunks or self.max_chunks\n        \n        if not chunk_size or  not max_chunks: # One shot load, suitable for small zarr files\n            df = zarr.load(self.root.joinpath(self.zarr_path).as_posix()).get(self.field)\n            df = df[start:end]\n            df = map(self.parse, df) \n        else: # Chunked load, suitable for large zarr files\n            df = []\n            with zarr.open(self.root.joinpath(self.zarr_path).as_posix(), \"r\") as zf:\n                end = start+max_chunks*chunk_size if end is None else min(end, start+max_chunks*chunk_size)\n                for i_start in range(start, end, chunk_size ):\n                    items = zf[self.field][i_start: min(i_start + chunk_size,end)]\n                    items = map(self.parse, items)\n                    df.append(items)\n            df = it.chain(*df)\n            \n        df = pd.DataFrame.from_records(df)\n        for col, col_dtype in self.dtypes.items():\n            df[col] = df[col].astype(col_dtype, copy=False)\n        return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class SceneParser(BaseParser):\n    field = \"scenes\"\n    \n    @staticmethod\n    def parse(scene):\n        scene_dict = {\n            \"frame_index_interval_start\": scene[0][0],\n            \"frame_index_interval_end\": scene[0][1],\n            \"host\":  scene[1],\n            \"start_time\": scene[2],\n            \"end_time\": scene[3]\n        }\n        return scene_dict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sp = SceneParser(chunk_size=None, max_chunks=None, zarr_path=\"scenes/sample.zarr\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scenes = sp.to_pandas()\nscenes.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scenes.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scenes[\"duration\"] = (scenes[\"end_time\"] -  scenes[\"start_time\"])/1e9\nscenes[\"num_frames\"] = scenes[\"frame_index_interval_end\"] - scenes[\"frame_index_interval_start\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scenes.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.3 The L5Kit dataset: frames","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"As we said it, **scenes** are made of **frames**. Each scene holds a reference to its frames whicht starts at ***frame_index_interval_start*** and ends at ***frame_index_interval_end***.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"scene = scenes.iloc[-1]\nscene","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5.frames","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scene_frames = zl5.frames[scene.frame_index_interval_start:scene.frame_index_interval_end]\nframe = scene_frames[0]\nframe","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A **frame** consists of :\n* ***timestamp***: the timesatamp at which the state of the worl was filmed\n* ***agents***: the agent detected by the host (just a reference)\n* ***traffic lights***\n* ***informations about the host***: translation, rotation","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def parse_frame(frame):\n    frame_dict = {\n        'timestamp': frame[0],\n        'agent_index_interval_start': frame[1][0],\n        'agent_index_interval_start': frame[1][1],\n        'traffic_light_faces_index_interval_start': frame[2][0],\n        'traffic_light_faces_index_interval_end': frame[2][1],\n        'ego_translation_x': frame[3][0],\n        'ego_translation_y': frame[3][1],\n        'ego_translation_z': frame[3][2],\n        'ego_rotation_xx': frame[4][0][0],\n        'ego_rotation_xy': frame[4][0][1],\n        'ego_rotation_xz': frame[4][0][2],\n        'ego_rotation_yx': frame[4][1][0],\n        'ego_rotation_yy': frame[4][1][1],\n        'ego_rotation_yz': frame[4][1][2],\n        'ego_rotation_zx': frame[4][2][0],\n        'ego_rotation_zy': frame[4][2][1],\n        'ego_rotation_zz': frame[4][2][2],\n        \n    }\n    return frame_dict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"parse_frame(frame)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class FrameParser(BaseParser):\n    field = \"frames\"\n    \n    @staticmethod\n    def parse(frame):\n        frame_dict = {\n            'timestamp': frame[0],\n            'agent_index_interval_start': frame[1][0],\n            'agent_index_interval_end': frame[1][1],\n            'traffic_light_faces_index_interval_start': frame[2][0],\n            'traffic_light_faces_index_interval_end': frame[2][1],\n            'ego_translation_x': frame[3][0],\n            'ego_translation_y': frame[3][1],\n            'ego_translation_z': frame[3][2],\n            'ego_rotation_xx': frame[4][0][0],\n            'ego_rotation_xy': frame[4][0][1],\n            'ego_rotation_xz': frame[4][0][2],\n            'ego_rotation_yx': frame[4][1][0],\n            'ego_rotation_yy': frame[4][1][1],\n            'ego_rotation_yz': frame[4][1][2],\n            'ego_rotation_zx': frame[4][2][0],\n            'ego_rotation_zy': frame[4][2][1],\n            'ego_rotation_zz': frame[4][2][2],\n\n        }\n        return frame_dict\n\n    def to_pandas(self, start=0, end=None, chunk_size=None, max_chunks=None, scene=None):\n        if scene is not None:\n            start = scene.frame_index_interval_start\n            end = scene.frame_index_interval_end\n        \n        df = super().to_pandas(start=start, end=end, chunk_size=chunk_size, max_chunks=max_chunks)\n        return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fp = FrameParser()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scene","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frames = fp.to_pandas(scene=scene)\n# frames = fp.to_pandas(scene=None)\nframes.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frames.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frame = frames.iloc[0]\nframe","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.4 The L5Kit dataset: agents","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"An agent is actually an object which is in move with the host (AV).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5.agents","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"frame_agents = zl5.agents[int(frame.agent_index_interval_start):int(frame.agent_index_interval_end)]\nagent = frame_agents[0]\nagent","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PERCEPTION_LABELS = [\n    \"PERCEPTION_LABEL_NOT_SET\",\n    \"PERCEPTION_LABEL_UNKNOWN\",\n    \"PERCEPTION_LABEL_DONTCARE\",\n    \"PERCEPTION_LABEL_CAR\",\n    \"PERCEPTION_LABEL_VAN\",\n    \"PERCEPTION_LABEL_TRAM\",\n    \"PERCEPTION_LABEL_BUS\",\n    \"PERCEPTION_LABEL_TRUCK\",\n    \"PERCEPTION_LABEL_EMERGENCY_VEHICLE\",\n    \"PERCEPTION_LABEL_OTHER_VEHICLE\",\n    \"PERCEPTION_LABEL_BICYCLE\",\n    \"PERCEPTION_LABEL_MOTORCYCLE\",\n    \"PERCEPTION_LABEL_CYCLIST\",\n    \"PERCEPTION_LABEL_MOTORCYCLIST\",\n    \"PERCEPTION_LABEL_PEDESTRIAN\",\n    \"PERCEPTION_LABEL_ANIMAL\",\n    \"AVRESEARCH_LABEL_DONTCARE\",\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class AgentParser(BaseParser):\n    field = \"agents\"\n    \n    @staticmethod\n    def parse(agent):\n        frame_dict = {\n            'centroid_x': agent[0][0],\n            'centroid_y': agent[0][1],\n            'extent_x': agent[1][0],\n            'extent_y': agent[1][1],\n            'extent_z': agent[1][2],\n            'yaw': agent[2],\n            \"velocity_x\":  agent[3][0],\n            \"velocity_y\":  agent[3][1],\n            \"track_id\":  agent[4],\n        }\n        for p_label, p in zip(PERCEPTION_LABELS, agent[5]):\n            frame_dict[\"label_probabilities_{}\".format(p_label)] = p\n        return frame_dict\n\n    def to_pandas(self, start=0, end=None, chunk_size=None, max_chunks=None, frame=None):\n        if frame is not None:\n            start = int(frame.agent_index_interval_start)\n            end = int(frame.agent_index_interval_end)\n        \n        df = super().to_pandas(start=start, end=end, chunk_size=chunk_size, max_chunks=max_chunks)\n        return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ap = AgentParser()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"agents = ap.to_pandas(frame=frame)\n# agents = ap.to_pandas(frame=None)\nagents.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"agents.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.5 The L5Kit dataset: traffic lights","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"zl5.traffic_light_faces","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class TrafficLightParser(BaseParser):\n    field = \"traffic_light_faces\"\n    \n    @staticmethod\n    def parse(light):\n        frame_dict = {\n            'face_id': light[0],\n            'traffic_light_id': light[1],\n            'traffic_light_face_status_0': light[2][0],\n            'traffic_light_face_status_1': light[2][1],\n            'traffic_light_face_status_2': light[2][2],\n        }\n        return frame_dict\n\n    def to_pandas(self, start=0, end=None, chunk_size=None, max_chunks=None, frame=None):\n        if frame is not None:\n            start = int(frame.traffic_light_faces_index_interval_start)\n            end = int(frame.traffic_light_faces_index_interval_end)\n        \n        df = super().to_pandas(start=start, end=end, chunk_size=chunk_size, max_chunks=max_chunks)\n        return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tlp = TrafficLightParser()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lights = tlp.to_pandas(frame = frame)\n# lights = tlp.to_pandas(frame = None)\nlights.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lights.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h2 style=\"color:blue;text-align:center;\">Kkiller</h2>","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}