{"cells":[{"metadata":{},"cell_type":"markdown","source":"# About the competitions","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Experiment with the largest-ever self-driving Prediction Dataset to build motion prediction model.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Prediction Dataset sample","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"![](http://https://self-driving.lyft.com/wp-content/uploads/2020/06/motion_dataset_2-1.png)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"![](https://self-driving.lyft.com/wp-content/uploads/2020/06/motion_dataset_2-1.png)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# What is autonomous vehicle ?\nA self-driving car, also known as an autonomous vehicle (AV), connected and autonomous vehicle (CAV), driverless car, robo-car, or robotic car,(automated vehicles and fully automated vehicles in the European Union) is a vehicle that is capable of sensing its environment and moving safely with little or no human input.\n\nSelf-driving cars combine a variety of sensors to perceive their surroundings, such as radar, lidar, sonar, GPS, odometry and inertial measurement units. Advanced control systems interpret sensory information to identify appropriate navigation paths, as well as obstacles and relevant signage\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Example prediction solution","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"![](https://self-driving.lyft.com/wp-content/uploads/2020/06/diagram-prediction-1.jpg)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Scene Visualisation","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Code is taken from: \n\nhttps://github.com/lyft/l5kit\n\nhttps://github.com/lyft/l5kit/blob/master/examples/visualisation/visualise_data.ipynb\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!pip install --upgrade pip\n!pip install pymap3d==2.1.0\n!pip install -U l5kit","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Imports","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nimport numpy as np\nimport pandas as pd\n\nfrom l5kit.data import ChunkedDataset, LocalDataManager\nfrom l5kit.dataset import EgoDataset, AgentDataset\n\nfrom l5kit.rasterization import build_rasterizer\nfrom l5kit.configs import load_config_data\nfrom l5kit.visualization import draw_trajectory, TARGET_POINTS_COLOR\nfrom l5kit.geometry import transform_points\nfrom tqdm import tqdm\nfrom collections import Counter\nfrom l5kit.data import PERCEPTION_LABELS\nfrom prettytable import PrettyTable\n\nimport os\n\nfrom matplotlib import animation, rc\nfrom IPython.display import HTML\n\nrc('animation', html='jshtml')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"def animate_solution(images):\n\n    def animate(i):\n        im.set_data(images[i])\n \n    fig, ax = plt.subplots()\n    im = ax.imshow(images[0])\n    \n    return animation.FuncAnimation(fig, animate, frames=len(images), interval=60)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# set env variable for data\nos.environ[\"L5KIT_DATA_FOLDER\"] = \"../input/lyft-motion-prediction-autonomous-vehicles\"\n# get config\ncfg = load_config_data(\"../input/lyft-config-files/visualisation_config.yaml\")\nprint(cfg)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load the data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dm = LocalDataManager()\ndataset_path = dm.require(cfg[\"val_data_loader\"][\"key\"])\nzarr_dataset = ChunkedDataset(dataset_path)\nzarr_dataset.open()\nprint(zarr_dataset)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# DATA FORMAT\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":" The dataset is provided in zarr format. The zarr files are flat, compact, and highly performant for loading. To read the dataset please use our new Python software kit.\n The dataset consists of frames and agent states. A frame is a snapshot in time which consists of ego pose, time, and multiple agent states. Each agent state describes the position, orientation, bounds, and type.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# EXAMPLE","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import zarr","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_zarr = zarr.open(\"../input/lyft-motion-prediction-autonomous-vehicles/scenes/train.zarr\")\n\nprint(type(train_zarr))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_zarr.info","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The 2020 Lyft competition dataset is stored in four structured arrays: scenes, frames, agents and tl_faces.\n *Note: in the following all _interval fields assume that information is stored consecutively in the arrays. This means that if frame_index_interval for scene_0 is (0, 100), frames from scene_1 will start from index 100 in the frames array.*","execution_count":null},{"metadata":{},"cell_type":"markdown","source":" Reading from a zarr array is as easy as slicing from it like you would any numpy array. The return value is an ordinary numpy array. Zarr takes care of determining which chunks to read from.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_zarr.agents[:5]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Overview","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"agents = zarr_dataset.agents\nprobabilities = agents[\"label_probabilities\"]\nlabels_indexes = np.argmax(probabilities, axis=1)\ncounts = []\nfor idx_label, label in enumerate(PERCEPTION_LABELS):\n    counts.append(np.sum(labels_indexes == idx_label))\n    \ntable = PrettyTable(field_names=[\"label\", \"counts\"])\nfor count, label in zip(counts, PERCEPTION_LABELS):\n    table.add_row([label, count])\nprint(table)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"rast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Autonomous Vehicle with Trajectory","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"data = dataset[80]\n\nim = data[\"image\"].transpose(1, 2, 0)\nim = dataset.rasterizer.to_rgb(im)\ntarget_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\ndraw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\nfigsize = plt.subplots(figsize = (10,10))\nplt.title('Autonomous Vehicle with Trajectory',fontsize=20)\n\nplt.imshow(im[::-1])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Satellite View","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"cfg[\"raster_params\"][\"map_type\"] = \"py_satellite\"\nrast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)\ndata = dataset[80]\n\nim = data[\"image\"].transpose(1, 2, 0)\nim = dataset.rasterizer.to_rgb(im)\ntarget_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\ndraw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\nfigsize = plt.subplots(figsize = (10,10))\nplt.title('Satellite View',fontsize=20)\nplt.imshow(im[::-1])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset = AgentDataset(cfg, zarr_dataset, rast)\ndata = dataset[80]\n\nim = data[\"image\"].transpose(1, 2, 0)\nim = dataset.rasterizer.to_rgb(im)\ntarget_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\ndraw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n\nfig = plt.subplots(figsize=(10,10))\nplt.imshow(im[::-1])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" Green represents the autonomous vehicle (AV), and blue is primarily all the other cars/vehicles/exogenous factors we need to predict for.\n If we are able to accurately predict the path the vehicles go through, it will make it easier for an AV to compute its trajectory on the fly.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"#  Entire Scene","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Semantic","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"What is semantic map?\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import display, clear_output\nimport PIL\n \ncfg[\"raster_params\"][\"map_type\"] = \"py_semantic\"\nrast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)\nscene_idx = 34\nindexes = dataset.get_scene_indices(scene_idx)\nimages = []\n\nfor idx in indexes:\n    \n    data = dataset[idx]\n    im = data[\"image\"].transpose(1, 2, 0)\n    im = dataset.rasterizer.to_rgb(im)\n    target_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\n    center_in_pixels = np.asarray(cfg[\"raster_params\"][\"ego_center\"]) * cfg[\"raster_params\"][\"raster_size\"]\n    draw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n    clear_output(wait=True)\n    images.append(PIL.Image.fromarray(im[::-1]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"anim = animate_solution(images)\nHTML(anim.to_jshtml())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Satellite","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from IPython.display import display, clear_output\nimport PIL\n \ncfg[\"raster_params\"][\"map_type\"] = \"py_satellite\"\nrast = build_rasterizer(cfg, dm)\ndataset = EgoDataset(cfg, zarr_dataset, rast)\nscene_idx = 34\nindexes = dataset.get_scene_indices(scene_idx)\nimages = []\n\n\nfor i, idx in enumerate(indexes):\n    data = dataset[idx]\n    im = data[\"image\"].transpose(1, 2, 0)\n    im = dataset.rasterizer.to_rgb(im)\n    target_positions_pixels = transform_points(data[\"target_positions\"] + data[\"centroid\"][:2], data[\"world_to_image\"])\n    center_in_pixels = np.asarray(cfg[\"raster_params\"][\"ego_center\"]) * cfg[\"raster_params\"][\"raster_size\"]\n    draw_trajectory(im, target_positions_pixels, data[\"target_yaws\"], TARGET_POINTS_COLOR)\n    clear_output(wait=True)\n    images.append(PIL.Image.fromarray(im[::-1]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"anim = animate_solution(images)\nHTML(anim.to_jshtml())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Sample Submission","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"DIR_INPUT = \"/kaggle/input/lyft-motion-prediction-autonomous-vehicles\"\n\nSINGLE_MODE_SUBMISSION = f\"{DIR_INPUT}/single_mode_sample_submission.csv\"\nMULTI_MODE_SUBMISSION = f\"{DIR_INPUT}/multi_mode_sample_submission.csv\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Single mode\nsample_submission = pd.read_csv(SINGLE_MODE_SUBMISSION)\n\n# Multi mode\n# sample_submission = pd.read_csv(MULTI_MODE_SUBMISSION)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.head()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}