{
  "id": 109482,
  "title": "A doubt in render_sample_data",
  "url": "/competitions/3d-object-detection-for-autonomous-vehicles/discussion/109482",
  "author_name": "",
  "post_date": "2019-09-19T14:01:01.238888700Z",
  "votes": 6,
  "comment_count": 2,
  "views": 0,
  "content": "<p>SDK's <a href=\"https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/lyftdataset.py#L857\">render_sample_data</a> function converts lidar cloud points to vehicles frame of reference using following transformation matrices:</p>\n\n<p>```</p>\n\n<h1>Compute transformation matrices for lidar point cloud</h1>\n\n<p>cs_record = self.lyftd.get(\"calibrated_sensor\", sd_record[\"calibrated_sensor_token\"])\npose_record = self.lyftd.get(\"ego_pose\", sd_record[\"ego_pose_token\"])\nvehicle_from_sensor = np.eye(4)\nvehicle_from_sensor[:3, :3] = Quaternion(cs_record[\"rotation\"]).rotation_matrix\nvehicle_from_sensor[:3, 3] = cs_record[\"translation\"]</p>\n\n<p>ego_yaw = Quaternion(pose_record[\"rotation\"]).yaw_pitch_roll[0]\nrot_vehicle_flat_from_vehicle = np.dot(\n    Quaternion(scalar=np.cos(ego_yaw / 2), vector=[0, 0, np.sin(ego_yaw / 2)]).rotation_matrix,\n    Quaternion(pose_record[\"rotation\"]).inverse.rotation_matrix,\n)</p>\n\n<p>vehicle_flat_from_vehicle = np.eye(4)\nvehicle_flat_from_vehicle[:3, :3] = rot_vehicle_flat_from_vehicle</p>\n\n<h1>Show point cloud.</h1>\n\n<p>points = view_points(\n    pc.points[:3, :], np.dot(vehicle_flat_from_vehicle, vehicle_from_sensor), normalize=False\n)\n```</p>\n\n<p>First we compute <code>vehicle_from_sensor</code> which is used to switch the points to vehicle's frame of reference and that makes sense, but what is <code>vehicle_flat_from_vehicle</code> ? What does flat mean here? parallel to z-axis? Aren't the vehicles already parallel to z-axis?</p>",
  "messages": [
    {
      "id": "629950",
      "postDate": "09/19/2019 14:01:01",
      "content": "<p>SDK's <a href=\"https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/lyftdataset.py#L857\">render_sample_data</a> function converts lidar cloud points to vehicles frame of reference using following transformation matrices:</p>\n\n<p>```</p>\n\n<h1>Compute transformation matrices for lidar point cloud</h1>\n\n<p>cs_record = self.lyftd.get(\"calibrated_sensor\", sd_record[\"calibrated_sensor_token\"])\npose_record = self.lyftd.get(\"ego_pose\", sd_record[\"ego_pose_token\"])\nvehicle_from_sensor = np.eye(4)\nvehicle_from_sensor[:3, :3] = Quaternion(cs_record[\"rotation\"]).rotation_matrix\nvehicle_from_sensor[:3, 3] = cs_record[\"translation\"]</p>\n\n<p>ego_yaw = Quaternion(pose_record[\"rotation\"]).yaw_pitch_roll[0]\nrot_vehicle_flat_from_vehicle = np.dot(\n    Quaternion(scalar=np.cos(ego_yaw / 2), vector=[0, 0, np.sin(ego_yaw / 2)]).rotation_matrix,\n    Quaternion(pose_record[\"rotation\"]).inverse.rotation_matrix,\n)</p>\n\n<p>vehicle_flat_from_vehicle = np.eye(4)\nvehicle_flat_from_vehicle[:3, :3] = rot_vehicle_flat_from_vehicle</p>\n\n<h1>Show point cloud.</h1>\n\n<p>points = view_points(\n    pc.points[:3, :], np.dot(vehicle_flat_from_vehicle, vehicle_from_sensor), normalize=False\n)\n```</p>\n\n<p>First we compute <code>vehicle_from_sensor</code> which is used to switch the points to vehicle's frame of reference and that makes sense, but what is <code>vehicle_flat_from_vehicle</code> ? What does flat mean here? parallel to z-axis? Aren't the vehicles already parallel to z-axis?</p>",
      "rawMarkdown": "SDK's [render\\_sample\\_data](https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/lyftdataset.py#L857) function converts lidar cloud points to vehicles frame of reference using following transformation matrices:\n\n```\n\n# Compute transformation matrices for lidar point cloud\ncs_record = self.lyftd.get(\"calibrated_sensor\", sd_record[\"calibrated_sensor_token\"])\npose_record = self.lyftd.get(\"ego_pose\", sd_record[\"ego_pose_token\"])\nvehicle_from_sensor = np.eye(4)\nvehicle_from_sensor[:3, :3] = Quaternion(cs_record[\"rotation\"]).rotation_matrix\nvehicle_from_sensor[:3, 3] = cs_record[\"translation\"]\n\nego_yaw = Quaternion(pose_record[\"rotation\"]).yaw_pitch_roll[0]\nrot_vehicle_flat_from_vehicle = np.dot(\n    Quaternion(scalar=np.cos(ego_yaw / 2), vector=[0, 0, np.sin(ego_yaw / 2)]).rotation_matrix,\n    Quaternion(pose_record[\"rotation\"]).inverse.rotation_matrix,\n)\n\nvehicle_flat_from_vehicle = np.eye(4)\nvehicle_flat_from_vehicle[:3, :3] = rot_vehicle_flat_from_vehicle\n\n# Show point cloud.\npoints = view_points(\n    pc.points[:3, :], np.dot(vehicle_flat_from_vehicle, vehicle_from_sensor), normalize=False\n)\n```\n\nFirst we compute `vehicle_from_sensor` which is used to switch the points to vehicle's frame of reference and that makes sense, but what is `vehicle_flat_from_vehicle` ? What does flat mean here? parallel to z-axis? Aren't the vehicles already parallel to z-axis?",
      "votes": null
    },
    {
      "id": "630645",
      "postDate": "09/20/2019 14:38:16",
      "content": "<p>Hey <a href=\"/iglovikov\">@iglovikov</a> <a href=\"/gzuidhof\">@gzuidhof</a>, could you guys help me out with this question?</p>",
      "rawMarkdown": "Hey @iglovikov @gzuidhof, could you guys help me out with this question?",
      "votes": null
    },
    {
      "id": "640023",
      "postDate": "10/03/2019 18:53:49",
      "content": "<p>So it is done to get the lidar points in a frame parallel to the x-y plane (the ground). A flat vehicle means, that vehicle retains its yaw but no roll or pitch. </p>",
      "rawMarkdown": "So it is done to get the lidar points in a frame parallel to the x-y plane (the ground). A flat vehicle means, that vehicle retains its yaw but no roll or pitch.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 630645,
      "author_name": "rishabhiitbhu",
      "author_url": "",
      "post_date": "09/20/2019 14:38:16",
      "content": "<p>Hey <a href=\"/iglovikov\">@iglovikov</a> <a href=\"/gzuidhof\">@gzuidhof</a>, could you guys help me out with this question?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 640023,
      "author_name": "rishabhiitbhu",
      "author_url": "",
      "post_date": "10/03/2019 18:53:49",
      "content": "<p>So it is done to get the lidar points in a frame parallel to the x-y plane (the ground). A flat vehicle means, that vehicle retains its yaw but no roll or pitch. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "629950": "SDK's [render\\_sample\\_data](https://github.com/lyft/nuscenes-devkit/blob/master/lyft_dataset_sdk/lyftdataset.py#L857) function converts lidar cloud points to vehicles frame of reference using following transformation matrices:\n\n```\n\n# Compute transformation matrices for lidar point cloud\ncs_record = self.lyftd.get(\"calibrated_sensor\", sd_record[\"calibrated_sensor_token\"])\npose_record = self.lyftd.get(\"ego_pose\", sd_record[\"ego_pose_token\"])\nvehicle_from_sensor = np.eye(4)\nvehicle_from_sensor[:3, :3] = Quaternion(cs_record[\"rotation\"]).rotation_matrix\nvehicle_from_sensor[:3, 3] = cs_record[\"translation\"]\n\nego_yaw = Quaternion(pose_record[\"rotation\"]).yaw_pitch_roll[0]\nrot_vehicle_flat_from_vehicle = np.dot(\n    Quaternion(scalar=np.cos(ego_yaw / 2), vector=[0, 0, np.sin(ego_yaw / 2)]).rotation_matrix,\n    Quaternion(pose_record[\"rotation\"]).inverse.rotation_matrix,\n)\n\nvehicle_flat_from_vehicle = np.eye(4)\nvehicle_flat_from_vehicle[:3, :3] = rot_vehicle_flat_from_vehicle\n\n# Show point cloud.\npoints = view_points(\n    pc.points[:3, :], np.dot(vehicle_flat_from_vehicle, vehicle_from_sensor), normalize=False\n)\n```\n\nFirst we compute `vehicle_from_sensor` which is used to switch the points to vehicle's frame of reference and that makes sense, but what is `vehicle_flat_from_vehicle` ? What does flat mean here? parallel to z-axis? Aren't the vehicles already parallel to z-axis?",
    "630645": "Hey @iglovikov @gzuidhof, could you guys help me out with this question?",
    "640023": "So it is done to get the lidar points in a frame parallel to the x-y plane (the ground). A flat vehicle means, that vehicle retains its yaw but no roll or pitch."
  },
  "source": "meta"
}