Until recently the only dataset that people used for 3D detections was KITTI and nearly all existing research and code that was developed by the research community uses KITTI format.
\n\nThe conversion script from the nuscenes SDK did not work on the Lyft dataset. I
\n\nI would like to thank @stalkermustang who figured out how to extend it and wrote the code for this conversion. He shared it in his fork of the SDK https://github.com/stalkermustang/nuscenes-devkit
\n\nand prepared a kernel with an example.
\n\nHis code in the process of the merge to the main Lyft SDK repo https://github.com/lyft/nuscenes-devkit/pull/52
\n\nI hope that it will be soon available as part of the Lyft SDK. Again. Big thanks to @stalkermustang !
\n\nI hope his work will help the participants to reuse the existing research code for 3D object detection and help with this competition!
", "messages": [ { "id": "647427", "postDate": "10/12/2019 15:52:02", "content": "Until recently the only dataset that people used for 3D detections was KITTI and nearly all existing research and code that was developed by the research community uses KITTI format.
\n\nThe conversion script from the nuscenes SDK did not work on the Lyft dataset. I
\n\nI would like to thank @stalkermustang who figured out how to extend it and wrote the code for this conversion. He shared it in his fork of the SDK https://github.com/stalkermustang/nuscenes-devkit
\n\nand prepared a kernel with an example.
\n\nHis code in the process of the merge to the main Lyft SDK repo https://github.com/lyft/nuscenes-devkit/pull/52
\n\nI hope that it will be soon available as part of the Lyft SDK. Again. Big thanks to @stalkermustang !
\n\nI hope his work will help the participants to reuse the existing research code for 3D object detection and help with this competition!
", "rawMarkdown": "Until recently the only dataset that people used for 3D detections was KITTI and nearly all existing research and code that was developed by the research community uses KITTI format.\n\nThe conversion script from the nuscenes SDK did not work on the Lyft dataset. I\n\n I would like to thank @stalkermustang who figured out how to extend it and wrote the code for this conversion. He shared it in his fork of the SDK https://github.com/stalkermustang/nuscenes-devkit\n\nand prepared a [kernel](https://www.kaggle.com/stalkermustang/converting-lyft-dataset-to-kitty-format) with an example.\n\nHis code in the process of the merge to the main Lyft SDK repo https://github.com/lyft/nuscenes-devkit/pull/52\n\nI hope that it will be soon available as part of the Lyft SDK. Again. Big thanks to @stalkermustang !\n\nI hope his work will help the participants to reuse the existing research code for 3D object detection and help with this competition!", "votes": null }, { "id": "649627", "postDate": "10/15/2019 15:31:41", "content": "Hi, I saw that the KITTI converter had been added to the SDK so I update it. But when I run the converting command, it always stops at 24% of the work and displays \"ValueError: cannot reshape array of size 265728 into shape (5)\". What may be the problem I'm currently facing? Thanks.
", "rawMarkdown": "Hi, I saw that the KITTI converter had been added to the SDK so I update it. But when I run the converting command, it always stops at 24% of the work and displays \"ValueError: cannot reshape array of size 265728 into shape (5)\". What may be the problem I'm currently facing? Thanks.", "votes": null }, { "id": "649874", "postDate": "10/15/2019 21:13:36", "content": "Hi @iglovikov ,
\n\nMost of the code, that @stalkermustang contributed comes from the original nuScenes dataset repo. (see here)
\n\nYet @stalkermustang got rid of the main author reference in the orignal code and the limitations of the script, that I think participants should know:
\n\n\n\n\nThis script converts nuScenes data to KITTI format and KITTI results to nuScenes.\n It is used for compatibility with software that uses KITTI-style annotations.\n We do not encourage this, as:\n - KITTI has only front-facing cameras, whereas nuScenes has a 360 degree horizontal fov.\n - KITTI has no radar data.\n - The nuScenes database format is more modular.\n - KITTI fields like occluded and truncated cannot be exactly reproduced from nuScenes data.\n - KITTI has different categories.\n Limitations:\n - We don't specify the KITTI imu_to_velo_kitti projection in this code base.\n - We map nuScenes categories to nuScenes detection categories, rather than KITTI categories.\n - Attributes are not part of KITTI and therefore set to '' in the nuScenes result format.\n - Velocities are not part of KITTI and therefore set to 0 in the nuScenes result format.\n - This script uses the
\ntrainandvalsplits of nuScenes, whereas standard KITTI hastrainingandtestingsplits.
I think the author's reference (especially under Apache License) and the disclaimers should be added back to the script.
", "rawMarkdown": "Hi @iglovikov , \n\nMost of the code, that @stalkermustang contributed comes from the original nuScenes dataset repo. (see [here](https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/scripts/export_kitti.py))\n\nYet @stalkermustang got rid of the main author reference in the orignal code and the limitations of the script, that I think participants should know:\n\n> This script converts nuScenes data to KITTI format and KITTI results to nuScenes.\nIt is used for compatibility with software that uses KITTI-style annotations.\nWe do not encourage this, as:\n- KITTI has only front-facing cameras, whereas nuScenes has a 360 degree horizontal fov.\n- KITTI has no radar data.\n- The nuScenes database format is more modular.\n- KITTI fields like occluded and truncated cannot be exactly reproduced from nuScenes data.\n- KITTI has different categories.\nLimitations:\n- We don't specify the KITTI imu_to_velo_kitti projection in this code base.\n- We map nuScenes categories to nuScenes detection categories, rather than KITTI categories.\n- Attributes are not part of KITTI and therefore set to '' in the nuScenes result format.\n- Velocities are not part of KITTI and therefore set to 0 in the nuScenes result format.\n- This script uses the `train` and `val` splits of nuScenes, whereas standard KITTI has `training` and `testing` splits.\n\nI think the author's reference (especially under Apache License) and the disclaimers should be added back to the script.", "votes": null }, { "id": "649990", "postDate": "10/16/2019 01:40:53", "content": "I met exactly same error XD
", "rawMarkdown": "I met exactly same error XD", "votes": null }, { "id": "650538", "postDate": "10/16/2019 13:39:47", "content": "You are absolutely right. Thank you!
\n\nI will add a header to the files with original authors.
\nI will also add to the header comments about the limitations of the current mapping script.
The license for all the code in the Lyft SDK has is exactly the same as the original Nuscenes SDK.
", "rawMarkdown": "You are absolutely right. Thank you!\n\nI will add a header to the files with original authors. \nI will also add to the header comments about the limitations of the current mapping script.\n\nThe license for all the code in the Lyft SDK has is exactly the same as the original Nuscenes SDK.", "votes": null }, { "id": "650885", "postDate": "10/16/2019 19:46:25", "content": "This is the problem of kaggle dataset. One binary file with lidar data was broken.
", "rawMarkdown": "This is the problem of kaggle dataset. One binary file with lidar data was broken.", "votes": null }, { "id": "651089", "postDate": "10/17/2019 03:12:47", "content": "Maybe try to catch the exception.
", "rawMarkdown": "Maybe try to catch the exception.", "votes": null }, { "id": "651091", "postDate": "10/17/2019 03:21:00", "content": "Very good point. command. No need to worry about some the broken lidar dataset.
", "rawMarkdown": "Very good point. command. No need to worry about some the broken lidar dataset.", "votes": null }, { "id": "651158", "postDate": "10/17/2019 06:03:00", "content": "See this: https://www.kaggle.com/c/3d-object-detection-for-autonomous-vehicles/discussion/110000#635230
", "rawMarkdown": "See this: https://www.kaggle.com/c/3d-object-detection-for-autonomous-vehicles/discussion/110000#635230", "votes": null }, { "id": "651196", "postDate": "10/17/2019 07:03:10", "content": "Thank you for the solution link. It really helps a lot.
", "rawMarkdown": "Thank you for the solution link. It really helps a lot.", "votes": null } ], "comments": [ { "id": 649627, "author_name": "henry0314", "author_url": "", "post_date": "10/15/2019 15:31:41", "content": "Hi, I saw that the KITTI converter had been added to the SDK so I update it. But when I run the converting command, it always stops at 24% of the work and displays \"ValueError: cannot reshape array of size 265728 into shape (5)\". What may be the problem I'm currently facing? Thanks.
", "votes": null, "replies": [ { "id": 649990, "author_name": "iamctr", "author_url": "", "post_date": "10/16/2019 01:40:53", "content": "I met exactly same error XD
", "votes": null, "replies": [] }, { "id": 650885, "author_name": "stalkermustang", "author_url": "", "post_date": "10/16/2019 19:46:25", "content": "This is the problem of kaggle dataset. One binary file with lidar data was broken.
", "votes": null, "replies": [] }, { "id": 651089, "author_name": "jionie", "author_url": "", "post_date": "10/17/2019 03:12:47", "content": "Maybe try to catch the exception.
", "votes": null, "replies": [] }, { "id": 651091, "author_name": "iamctr", "author_url": "", "post_date": "10/17/2019 03:21:00", "content": "Very good point. command. No need to worry about some the broken lidar dataset.
", "votes": null, "replies": [] }, { "id": 651158, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "10/17/2019 06:03:00", "content": "See this: https://www.kaggle.com/c/3d-object-detection-for-autonomous-vehicles/discussion/110000#635230
", "votes": null, "replies": [] }, { "id": 651196, "author_name": "henry0314", "author_url": "", "post_date": "10/17/2019 07:03:10", "content": "Thank you for the solution link. It really helps a lot.
", "votes": null, "replies": [] } ] }, { "id": 649874, "author_name": "michallys", "author_url": "", "post_date": "10/15/2019 21:13:36", "content": "Hi @iglovikov ,
\n\nMost of the code, that @stalkermustang contributed comes from the original nuScenes dataset repo. (see here)
\n\nYet @stalkermustang got rid of the main author reference in the orignal code and the limitations of the script, that I think participants should know:
\n\n\n\n\nThis script converts nuScenes data to KITTI format and KITTI results to nuScenes.\n It is used for compatibility with software that uses KITTI-style annotations.\n We do not encourage this, as:\n - KITTI has only front-facing cameras, whereas nuScenes has a 360 degree horizontal fov.\n - KITTI has no radar data.\n - The nuScenes database format is more modular.\n - KITTI fields like occluded and truncated cannot be exactly reproduced from nuScenes data.\n - KITTI has different categories.\n Limitations:\n - We don't specify the KITTI imu_to_velo_kitti projection in this code base.\n - We map nuScenes categories to nuScenes detection categories, rather than KITTI categories.\n - Attributes are not part of KITTI and therefore set to '' in the nuScenes result format.\n - Velocities are not part of KITTI and therefore set to 0 in the nuScenes result format.\n - This script uses the
\ntrainandvalsplits of nuScenes, whereas standard KITTI hastrainingandtestingsplits.
I think the author's reference (especially under Apache License) and the disclaimers should be added back to the script.
", "votes": null, "replies": [ { "id": 650538, "author_name": "iglovikov", "author_url": "", "post_date": "10/16/2019 13:39:47", "content": "You are absolutely right. Thank you!
\n\nI will add a header to the files with original authors.
\nI will also add to the header comments about the limitations of the current mapping script.
The license for all the code in the Lyft SDK has is exactly the same as the original Nuscenes SDK.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "647427": "Until recently the only dataset that people used for 3D detections was KITTI and nearly all existing research and code that was developed by the research community uses KITTI format.\n\nThe conversion script from the nuscenes SDK did not work on the Lyft dataset. I\n\n I would like to thank @stalkermustang who figured out how to extend it and wrote the code for this conversion. He shared it in his fork of the SDK https://github.com/stalkermustang/nuscenes-devkit\n\nand prepared a [kernel](https://www.kaggle.com/stalkermustang/converting-lyft-dataset-to-kitty-format) with an example.\n\nHis code in the process of the merge to the main Lyft SDK repo https://github.com/lyft/nuscenes-devkit/pull/52\n\nI hope that it will be soon available as part of the Lyft SDK. Again. Big thanks to @stalkermustang !\n\nI hope his work will help the participants to reuse the existing research code for 3D object detection and help with this competition!", "649627": "Hi, I saw that the KITTI converter had been added to the SDK so I update it. But when I run the converting command, it always stops at 24% of the work and displays \"ValueError: cannot reshape array of size 265728 into shape (5)\". What may be the problem I'm currently facing? Thanks.", "649874": "Hi @iglovikov , \n\nMost of the code, that @stalkermustang contributed comes from the original nuScenes dataset repo. (see [here](https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/scripts/export_kitti.py))\n\nYet @stalkermustang got rid of the main author reference in the orignal code and the limitations of the script, that I think participants should know:\n\n> This script converts nuScenes data to KITTI format and KITTI results to nuScenes.\nIt is used for compatibility with software that uses KITTI-style annotations.\nWe do not encourage this, as:\n- KITTI has only front-facing cameras, whereas nuScenes has a 360 degree horizontal fov.\n- KITTI has no radar data.\n- The nuScenes database format is more modular.\n- KITTI fields like occluded and truncated cannot be exactly reproduced from nuScenes data.\n- KITTI has different categories.\nLimitations:\n- We don't specify the KITTI imu_to_velo_kitti projection in this code base.\n- We map nuScenes categories to nuScenes detection categories, rather than KITTI categories.\n- Attributes are not part of KITTI and therefore set to '' in the nuScenes result format.\n- Velocities are not part of KITTI and therefore set to 0 in the nuScenes result format.\n- This script uses the `train` and `val` splits of nuScenes, whereas standard KITTI has `training` and `testing` splits.\n\nI think the author's reference (especially under Apache License) and the disclaimers should be added back to the script.", "649990": "I met exactly same error XD", "650538": "You are absolutely right. Thank you!\n\nI will add a header to the files with original authors. \nI will also add to the header comments about the limitations of the current mapping script.\n\nThe license for all the code in the Lyft SDK has is exactly the same as the original Nuscenes SDK.", "650885": "This is the problem of kaggle dataset. One binary file with lidar data was broken.", "651089": "Maybe try to catch the exception.", "651091": "Very good point. command. No need to worry about some the broken lidar dataset.", "651158": "See this: https://www.kaggle.com/c/3d-object-detection-for-autonomous-vehicles/discussion/110000#635230", "651196": "Thank you for the solution link. It really helps a lot." }, "source": "meta" }