{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":19990,"databundleVersionId":1472735,"sourceType":"competition"},{"sourceId":1440112,"sourceType":"datasetVersion","datasetId":843920}],"dockerImageVersionId":29994,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Lyft: Working with Map API\n\nThis is a quick tutorial to show the different underlying concepts behind HD map from Lyft dataset. After this short tutorial, you should be able to understand the underlying concepts as well as how to query for different map layers. Feel free to contribute visualization for each layer in this notebook.","metadata":{}},{"cell_type":"code","source":"!pip install --upgrade pip\n!pip install pymap3d==2.1.0\n!pip install -U l5kit","metadata":{"collapsed":true,"_kg_hide-output":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport l5kit, os\nimport matplotlib.pyplot as plt\nfrom l5kit.rasterization import build_rasterizer\nfrom l5kit.configs import load_config_data\nfrom l5kit.visualization import draw_trajectory, TARGET_POINTS_COLOR\nfrom l5kit.geometry import transform_points\nfrom tqdm import tqdm\nfrom collections import Counter\nfrom l5kit.data import PERCEPTION_LABELS\nfrom prettytable import PrettyTable\n# set env variable for data\nos.environ[\"L5KIT_DATA_FOLDER\"] = \"../input/lyft-motion-prediction-autonomous-vehicles\"\n# get config\ncfg = load_config_data(\"../input/lyft-config-files/visualisation_config.yaml\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load The Dataset","metadata":{}},{"cell_type":"code","source":"from l5kit.data import ChunkedDataset, LocalDataManager\nfrom l5kit.dataset import EgoDataset, AgentDataset\ndm = LocalDataManager()\ndataset_path = dm.require(cfg[\"val_data_loader\"][\"key\"])\nzarr_dataset = ChunkedDataset(dataset_path)\nzarr_dataset.open()","metadata":{"_kg_hide-output":true,"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load the MapAPI","metadata":{}},{"cell_type":"code","source":"from l5kit.data.map_api import MapAPI\nfrom l5kit.rasterization.rasterizer_builder import _load_metadata\n\nsemantic_map_filepath = dm.require(cfg[\"raster_params\"][\"semantic_map_key\"])\ndataset_meta = _load_metadata(cfg[\"raster_params\"][\"dataset_meta_key\"], dm)\nworld_to_ecef = np.array(dataset_meta[\"world_to_ecef\"], dtype=np.float64)\n\nmap_api = MapAPI(semantic_map_filepath, world_to_ecef)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MAP_LAYERS = [\"junction\", \"node\", \"segment\", \"lane\"]\n\n\ndef element_of_type(elem, layer_name):\n    return elem.element.HasField(layer_name)\n\n\ndef get_elements_from_layer(map_api, layer_name):\n    return [elem for elem in map_api.elements if element_of_type(elem, layer_name)]\n\n\nclass MapRenderer:\n    \n    def __init__(self, map_api):\n        self._color_map = dict(drivable_area='#a6cee3',\n                               road_segment='#1f78b4',\n                               road_block='#b2df8a',\n                               lane='#474747')\n        self._map_api = map_api\n    \n    def render_layer(self, layer_name):\n        fig = plt.figure(figsize=(10, 10))\n        ax = fig.add_axes([0, 0, 1, 1])\n        \n    def render_lanes(self):\n        all_lanes = get_elements_from_layer(self._map_api, \"lane\")\n        fig = plt.figure(figsize=(10, 10))\n        ax = fig.add_axes([0, 0, 1, 1])\n        for lane in all_lanes:\n            self.render_lane(ax, lane)\n        return fig, ax\n        \n    def render_lane(self, ax, lane):\n        coords = self._map_api.get_lane_coords(MapAPI.id_as_str(lane.id))\n        self.render_boundary(ax, coords[\"xyz_left\"])\n        self.render_boundary(ax, coords[\"xyz_right\"])\n        \n    def render_boundary(self, ax, boundary):\n        xs = boundary[:, 0]\n        ys = boundary[:, 1] \n        ax.plot(xs, ys, color=self._color_map[\"lane\"], label=\"lane\")\n        \n        ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"renderer = MapRenderer(map_api)\nfig, ax = renderer.render_lanes()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### There are 7 types of map elements:\n\n- Junction\n- RoadNetworkSegment\n- RoadNetworkNode\n- Lane\n- TrafficControlElement\n- SegmentSequence \n- AnnotatedShape \n\nLet's dive into each one.","metadata":{}},{"cell_type":"markdown","source":"## Junction\n\nJunction is conceptually a \"big\" intersection. This can consist of multiple smaller \"intersections\" which could be lane merges.","metadata":{}},{"cell_type":"code","source":"def is_junction(elem, map_api):\n    return elem.element.HasField(\"junction\")\n\ndef get_junctions(map_api):\n    return [elem for elem in map_api.elements if is_junction(elem, map_api)]\n\nall_junctions = get_junctions(map_api)\nall_junctions[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Road Network Node\n\nRoad network node are map intersection connecting several road segments. Each road network node belongs to one junction. If we think about the road network as a graph, this will be a node in the graph where a road segment would be an edge connecting two nodes.","metadata":{}},{"cell_type":"code","source":"def is_node(elem, map_api):\n    return elem.element.HasField(\"node\")\n\ndef get_nodes(map_api):\n    return [elem for elem in map_api.elements if is_junction(elem, map_api)]\n\nall_nodes = get_nodes(map_api)\nall_nodes[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Road Network Segment\n\nThis is the drivable surface connecting two intersections (nodes). If we think about the road network as a graph, and each intersection as a node, then a road segment is an edge connecting two nodes.","metadata":{}},{"cell_type":"code","source":"def is_segment(elem, map_api):\n    return elem.element.HasField(\"segment\")\n\ndef get_segments(map_api):\n    return [elem for elem in map_api.elements if is_segment(elem, map_api)]\n\nall_segments = get_segments(map_api)\nall_segments[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Lane\n\nEach road segment is built from multiple lanes. It is worth noting that the structure of lane allows for one to traverse the lane graph by using lane.lanes_ahead and lane.adjacent_lane_change_left  and lane.adjacent_lane_change_right","metadata":{}},{"cell_type":"code","source":"def is_lane(elem, map_api):\n    return elem.element.HasField(\"lane\")\n\ndef get_lanes(map_api):\n    return [elem for elem in map_api.elements if is_lane(elem, map_api)]\n\nall_lanes = get_lanes(map_api)\nall_lanes[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Traffic Control Element\n\nTraffic signals, e.g. traffic lights or individual \"faces\" of traffic lights, stop signs, yield signs, etc. controlling the exit from the lane onto one of the lanes ahead.","metadata":{}},{"cell_type":"code","source":"def is_traffic_element(elem, map_api):\n    return elem.element.HasField(\"traffic_control_element\")\n\ndef get_traffic_elements(map_api):\n    return [elem for elem in map_api.elements if is_traffic_element(elem, map_api)]\n\nall_traffic_elements = get_traffic_elements(map_api)\nall_traffic_elements[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Segment Sequence\n\nAn ordered collection of connected segments, indicating that it is either illegal, or conversely, the only option, to proceed along that path when starting from the first segment. Typically used to represent turn restrictions, with 2 or more segments in the sequence. For complicated intersections, the sequence can have more segments, with the intermediate ones just joining nodes that are all in the intersection. In some instances, it is more concise to represent the only path that can be taken, as opposed to all the paths that cannot be taken. The reason for keeping orientation  is to present the directionality; one example is where a turn restriction is on the same from and to segment, and the restriction is only on one side of the segment. With only list of segments, it's not possible to represent this restriction, as we have to distinguish one end from the other end.\n\nBut the small version of the dataset doesn't seem to have any segment sequences.","metadata":{}},{"cell_type":"code","source":"def is_segment_sequence(elem, map_api):\n    return elem.element.HasField(\"segment_sequence\")\n\ndef get_segment_sequences(map_api):\n    return [elem for elem in map_api.elements if is_segment_sequence(elem, map_api)]\n\nall_segment_sequences = get_segment_sequences(map_api)\nprint(len(all_segment_sequences))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Annotated Shape\n\nRepresents polygonal data that are either physical (e.g. buildings) or imaginary (e.g. political boundaries) and their associated properties.","metadata":{}},{"cell_type":"code","source":"def is_annotated_shape(elem, map_api):\n    return elem.element.HasField(\"annotated_shape\")\n\ndef get_annotated_shapes(map_api):\n    return [elem for elem in map_api.elements if is_annotated_shape(elem, map_api)]\n\nall_annotated_shapes = get_annotated_shapes(map_api)\nall_annotated_shapes[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}