As mentioned by @stalkermustang @nywenjing in this post, there are multiple factors which need to be considered while convert Lyft dataset for PointRCNN. I am creating this thread for people who can tell us how can we properly convert to PointRCNN KITTI.
\n\nIs converting to KITTI using this great kernel not enough https://www.kaggle.com/stalkermustang/converting-lyft-dataset-to-kitty-format ?
\n\nAny help would be appreciated.
", "messages": [ { "id": "665571", "postDate": "11/05/2019 05:57:53", "content": "As mentioned by @stalkermustang @nywenjing in this post, there are multiple factors which need to be considered while convert Lyft dataset for PointRCNN. I am creating this thread for people who can tell us how can we properly convert to PointRCNN KITTI.
\n\nIs converting to KITTI using this great kernel not enough https://www.kaggle.com/stalkermustang/converting-lyft-dataset-to-kitty-format ?
\n\nAny help would be appreciated.
", "rawMarkdown": "As mentioned by @stalkermustang @nywenjing in this post, there are multiple factors which need to be considered while convert Lyft dataset for PointRCNN. I am creating this thread for people who can tell us how can we properly convert to PointRCNN KITTI.\n\nIs converting to KITTI using this great kernel not enough https://www.kaggle.com/stalkermustang/converting-lyft-dataset-to-kitty-format ?\n\nAny help would be appreciated.", "votes": null }, { "id": "665583", "postDate": "11/05/2019 06:25:39", "content": "careful to include this recent patch: https://github.com/lyft/nuscenes-devkit/pull/65/files
\n\nI'm less familiar with the Lyft fork, but in NuScenes they'll interpolate cuboids to a target timestamp (e.g. camera or lidar). Interpolation happens in this code, which unfortunately won't do much for Lyft b/c all samples are marked as keyframes despite there being ~100ms drift between some sensors and labels in Lyft: https://github.com/lyft/nuscenes-devkit/blob/80fc3edf88fbf3cfa779eb7915e338c6a343acfd/lyft_dataset_sdk/lyftdataset.py#L318
\n\nIn my own usage, without that interpolation (which I'm doing in my own code), I've seen a pretty big discrepancy (perhaps >0.5 IOU?) for some fast-moving cars. But it looks like the Kitti utility doesn't even use get_boxes(): https://github.com/megaserg/nuscenes-devkit/blob/0ce6855138689d38626bc9cebb61b53d11928bd9/lyft_dataset_sdk/utils/export_kitti.py#L234
\n\nYou might want to re-viz the data you're feeding into your network to ensure the boxes are aligned properly. If they're not, try interpolating labels to the timestamp of the target sensor. That may mean doing separate interpolations for lidar and for each camera.
\n\nLarger question: what are the timestamps of the private test labels ? Earlier, I thought I saw that training labels were synchronized with LIDAR_TOP, but after poking around some more, there might be a 50-100ms difference. If your network has perfect accuracy but is actually predicting boxes for timestamps that don't match the private labels, then you could see some major misses (again, >0.5 IOU for fast cars).
", "rawMarkdown": "careful to include this recent patch: https://github.com/lyft/nuscenes-devkit/pull/65/files \n\nI'm less familiar with the Lyft fork, but in NuScenes they'll interpolate cuboids to a target timestamp (e.g. camera or lidar). Interpolation happens in this code, which unfortunately won't do much for Lyft b/c all samples are marked as keyframes despite there being ~100ms drift between some sensors and labels in Lyft: https://github.com/lyft/nuscenes-devkit/blob/80fc3edf88fbf3cfa779eb7915e338c6a343acfd/lyft_dataset_sdk/lyftdataset.py#L318\n\nIn my own usage, *without* that interpolation (which I'm doing in my own code), I've seen a pretty big discrepancy (perhaps >0.5 IOU?) for some fast-moving cars. But it looks like the Kitti utility doesn't even use get_boxes(): https://github.com/megaserg/nuscenes-devkit/blob/0ce6855138689d38626bc9cebb61b53d11928bd9/lyft_dataset_sdk/utils/export_kitti.py#L234 \n\nYou might want to re-viz the data you're feeding into your network to ensure the boxes are aligned properly. If they're not, try interpolating labels to the timestamp of the target sensor. That may mean doing separate interpolations for lidar and for each camera.\n\nLarger question: what are the timestamps of the *private test labels* ? Earlier, I thought I saw that training labels were synchronized with LIDAR_TOP, but after poking around some more, there might be a 50-100ms difference. If your network has perfect accuracy but is actually predicting boxes for timestamps that don't match the private labels, then you could see some major misses (again, >0.5 IOU for fast cars).", "votes": null } ], "comments": [ { "id": 665583, "author_name": "oarphme", "author_url": "", "post_date": "11/05/2019 06:25:39", "content": "careful to include this recent patch: https://github.com/lyft/nuscenes-devkit/pull/65/files
\n\nI'm less familiar with the Lyft fork, but in NuScenes they'll interpolate cuboids to a target timestamp (e.g. camera or lidar). Interpolation happens in this code, which unfortunately won't do much for Lyft b/c all samples are marked as keyframes despite there being ~100ms drift between some sensors and labels in Lyft: https://github.com/lyft/nuscenes-devkit/blob/80fc3edf88fbf3cfa779eb7915e338c6a343acfd/lyft_dataset_sdk/lyftdataset.py#L318
\n\nIn my own usage, without that interpolation (which I'm doing in my own code), I've seen a pretty big discrepancy (perhaps >0.5 IOU?) for some fast-moving cars. But it looks like the Kitti utility doesn't even use get_boxes(): https://github.com/megaserg/nuscenes-devkit/blob/0ce6855138689d38626bc9cebb61b53d11928bd9/lyft_dataset_sdk/utils/export_kitti.py#L234
\n\nYou might want to re-viz the data you're feeding into your network to ensure the boxes are aligned properly. If they're not, try interpolating labels to the timestamp of the target sensor. That may mean doing separate interpolations for lidar and for each camera.
\n\nLarger question: what are the timestamps of the private test labels ? Earlier, I thought I saw that training labels were synchronized with LIDAR_TOP, but after poking around some more, there might be a 50-100ms difference. If your network has perfect accuracy but is actually predicting boxes for timestamps that don't match the private labels, then you could see some major misses (again, >0.5 IOU for fast cars).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "665571": "As mentioned by @stalkermustang @nywenjing in this post, there are multiple factors which need to be considered while convert Lyft dataset for PointRCNN. I am creating this thread for people who can tell us how can we properly convert to PointRCNN KITTI.\n\nIs converting to KITTI using this great kernel not enough https://www.kaggle.com/stalkermustang/converting-lyft-dataset-to-kitty-format ?\n\nAny help would be appreciated.", "665583": "careful to include this recent patch: https://github.com/lyft/nuscenes-devkit/pull/65/files \n\nI'm less familiar with the Lyft fork, but in NuScenes they'll interpolate cuboids to a target timestamp (e.g. camera or lidar). Interpolation happens in this code, which unfortunately won't do much for Lyft b/c all samples are marked as keyframes despite there being ~100ms drift between some sensors and labels in Lyft: https://github.com/lyft/nuscenes-devkit/blob/80fc3edf88fbf3cfa779eb7915e338c6a343acfd/lyft_dataset_sdk/lyftdataset.py#L318\n\nIn my own usage, *without* that interpolation (which I'm doing in my own code), I've seen a pretty big discrepancy (perhaps >0.5 IOU?) for some fast-moving cars. But it looks like the Kitti utility doesn't even use get_boxes(): https://github.com/megaserg/nuscenes-devkit/blob/0ce6855138689d38626bc9cebb61b53d11928bd9/lyft_dataset_sdk/utils/export_kitti.py#L234 \n\nYou might want to re-viz the data you're feeding into your network to ensure the boxes are aligned properly. If they're not, try interpolating labels to the timestamp of the target sensor. That may mean doing separate interpolations for lidar and for each camera.\n\nLarger question: what are the timestamps of the *private test labels* ? Earlier, I thought I saw that training labels were synchronized with LIDAR_TOP, but after poking around some more, there might be a 50-100ms difference. If your network has perfect accuracy but is actually predicting boxes for timestamps that don't match the private labels, then you could see some major misses (again, >0.5 IOU for fast cars)." }, "source": "meta" }