![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3747125%2Fe6a0032a62607a0b08438f37401f937b%2FUnknown-2.png?generation=1573823001326042&alt=media\")
Tried extracting all points inside a bounding box and projecting it onto the camera image. There seems to be an offset between the two. The devkit method to project the bounding box to image also seems to show the offset.
\n\nDoes the offset really exist? Or is there a problem in the transformations?
\n\nProjection of points inside bounding box to image (my method)\n
Projection of bounding box to image (lyft devkit method)\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3747125%2F8d609c14d33cfe651208f7ffaf6fe6b1%2FUnknown-3.png?generation=1573823042452450&alt=media\")
My notebook is at Rending lidar points inside a bounding box
", "messages": [ { "id": "673760", "postDate": "11/15/2019 13:05:35", "content": "Tried extracting all points inside a bounding box and projecting it onto the camera image. There seems to be an offset between the two. The devkit method to project the bounding box to image also seems to show the offset.
\n\nDoes the offset really exist? Or is there a problem in the transformations?
\n\nProjection of points inside bounding box to image (my method)\n
Projection of bounding box to image (lyft devkit method)\n
My notebook is at Rending lidar points inside a bounding box
", "rawMarkdown": "Tried extracting all points inside a bounding box and projecting it onto the camera image. There seems to be an offset between the two. The devkit method to project the bounding box to image also seems to show the offset. \n\nDoes the offset really exist? Or is there a problem in the transformations?\n\nProjection of points inside bounding box to image (my method)\n\n \n\nProjection of bounding box to image (lyft devkit method)\n\n\n\n\nMy notebook is at [Rending lidar points inside a bounding box](https://www.kaggle.com/mpdroid/rendering-lidar-points-inside-a-bounding-box)", "votes": null }, { "id": "673781", "postDate": "11/15/2019 13:28:58", "content": "Yes: https://github.com/lyft/nuscenes-devkit/issues/73. I guess you can fix it yourself using the sensors calibration data.
", "rawMarkdown": "Yes: https://github.com/lyft/nuscenes-devkit/issues/73. I guess you can fix it yourself using the sensors calibration data.", "votes": null }, { "id": "674285", "postDate": "11/16/2019 06:55:53", "content": "Yes, they are not perfectly synchronized: https://www.kaggle.com/c/3d-object-detection-for-autonomous-vehicles/discussion/112439#latest-670426
\n\nFor parked cars and non-moving objects, I've seen very good alignment by simply projecting the lidar / cuboid points into the world frame, then projecting back into the camera frame. Take a look at map_pointcloud_to_image():\nhttps://github.com/nutonomy/nuscenes-devkit/blob/f3594b967cbf42396da5c6cb08bd714437b53111/python-sdk/nuscenes/nuscenes.py#L532
For moving cars, especially those with delta-v of 30mph or more, you'll still get somewhat poor alignment with the camera because (as shown above), there's a good deal of time gap between camera and lidar. It's not easy to \"correct\" the lidar cloud, but for the cuboids it helps to interpolate labels from before and after the camera timestamp using the approach outlined in get_boxes():\nhttps://github.com/nutonomy/nuscenes-devkit/blob/f3594b967cbf42396da5c6cb08bd714437b53111/python-sdk/nuscenes/nuscenes.py#L274
Note that, in the Lyft dataset, all frames are keyframes, so sadly the code above won't take advantage of the interpolation algo cited above (not even the Lyft fork). You might want to just hack a copy of get_boxes().
Yes: https://github.com/lyft/nuscenes-devkit/issues/73. I guess you can fix it yourself using the sensors calibration data.
", "votes": null, "replies": [] }, { "id": 674285, "author_name": "oarphme", "author_url": "", "post_date": "11/16/2019 06:55:53", "content": "Yes, they are not perfectly synchronized: https://www.kaggle.com/c/3d-object-detection-for-autonomous-vehicles/discussion/112439#latest-670426
\n\nFor parked cars and non-moving objects, I've seen very good alignment by simply projecting the lidar / cuboid points into the world frame, then projecting back into the camera frame. Take a look at map_pointcloud_to_image():\nhttps://github.com/nutonomy/nuscenes-devkit/blob/f3594b967cbf42396da5c6cb08bd714437b53111/python-sdk/nuscenes/nuscenes.py#L532
For moving cars, especially those with delta-v of 30mph or more, you'll still get somewhat poor alignment with the camera because (as shown above), there's a good deal of time gap between camera and lidar. It's not easy to \"correct\" the lidar cloud, but for the cuboids it helps to interpolate labels from before and after the camera timestamp using the approach outlined in get_boxes():\nhttps://github.com/nutonomy/nuscenes-devkit/blob/f3594b967cbf42396da5c6cb08bd714437b53111/python-sdk/nuscenes/nuscenes.py#L274
Note that, in the Lyft dataset, all frames are keyframes, so sadly the code above won't take advantage of the interpolation algo cited above (not even the Lyft fork). You might want to just hack a copy of get_boxes().