![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2Fc968cd762607c3e881c6576f5a727685%2FScreen%20Shot%202019-10-02%20at%2017.07.09.png?generation=1570061474426426&alt=media\")
Is anybody else seeing some wonky camera-lidar calibration for some examples? Overall looks good, but in these cases it looks off by quite a bit:
\n\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2F2f9c77ac8fbb263b0873846966d5ec34%2FScreen%20Shot%202019-10-02%20at%2017.06.51.png?generation=1570061475384157&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2F81b2a359858efbbf67411bdbeb43845d%2FScreen%20Shot%202019-10-02%20at%2017.06.10.png?generation=1570061476030549&alt=media\")
Perhaps worth doing an EDA on test set to see how good calibration is there.
", "messages": [ { "id": "639228", "postDate": "10/03/2019 00:11:41", "content": "Is anybody else seeing some wonky camera-lidar calibration for some examples? Overall looks good, but in these cases it looks off by quite a bit:
\n\n
\n
Perhaps worth doing an EDA on test set to see how good calibration is there.
", "rawMarkdown": "Is anybody else seeing some wonky camera-lidar calibration for some examples? Overall looks good, but in these cases it looks off by quite a bit: \n\n\n\n \n\n\nPerhaps worth doing an EDA on test set to see how good calibration is there.", "votes": null }, { "id": "639230", "postDate": "10/03/2019 00:15:05", "content": "I also see that it appears cars in parking lots are not labeled, and in a couple other cases there appear to be missing labels? Some noise is expected, but I worry a bit about the test set, because one could produce a ton of \"false positives\" by correctly predicting cars in parking lots (if the test set has no labels for them).
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2F5d3365bc547985bd66fde1fe2c6f9094%2FScreen%20Shot%202019-10-02%20at%2017.13.40.png?generation=1570061748177542&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2F77edb8aa899c4855e0b303201dd622b0%2FScreen%20Shot%202019-10-02%20at%2017.12.33.png?generation=1570061750407075&alt=media\")
last observation: the lidar clouds are very dense! as dense as Waymo in the middle region. Those plots also only show top lidar, not including the sides (if available).
", "rawMarkdown": "last observation: the lidar clouds are very dense! as dense as Waymo in the middle region. Those plots also only show top lidar, not including the sides (if available).", "votes": null }, { "id": "639365", "postDate": "10/03/2019 05:54:54", "content": "The pix I posted earlier might not be keyframes, and only have one lidar. Here are some definite keyframes and with all three lidars. Most examples I see are pretty good, but these are some questionable ones. In the cars example, it looks like the footprint IoU of cars in camera and labels (which are probably in lidar) could be less than 0.5 O_O
\n\nI have not tried correcting the clouds for egomotion but as these are keyframes I would think such correction could only be small? At 30mph, if the camera-lidar sync is off by 10ms, then that's 13cm or so of shift. The cars example looks a little more than 13cm, and also it looks like the lidars don't agree perfectly either.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2F974721a239fe4bc2c52f2b946629b54e%2FScreen%20Shot%202019-10-02%20at%2022.41.39.png?generation=1570081549185623&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2Fa160904d6ee1a93a7930b0f26a48b843%2FScreen%20Shot%202019-10-02%20at%2022.42.07.png?generation=1570081552178031&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2F0adb761961864b24cc7e9b1f8c887b54%2FScreen%20Shot%202019-10-02%20at%2022.43.53.png?generation=1570081558556385&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3779103%2Fbd32d05a6a9bdb72cfdbc165d841da31%2FScreen%20Shot%202019-10-02%20at%2022.43.19.png?generation=1570081558817156&alt=media\")
Even I've faced this misplaced-boxes issue, share me the sample token for your case.
", "rawMarkdown": "Even I've faced this misplaced-boxes issue, share me the sample token for your case.", "votes": null }, { "id": "639877", "postDate": "10/03/2019 17:01:07", "content": "definetly something to worry about, yes. It's all model+human labeled. But errors in such a large dataset are pretty common
", "rawMarkdown": "definetly something to worry about, yes. It's all model+human labeled. But errors in such a large dataset are pretty common", "votes": null }, { "id": "641924", "postDate": "10/05/2019 10:18:55", "content": "ok thanks for the review. just curious if other people are seeing this and if there's some bug in the dataset or it's expected.
", "rawMarkdown": "ok thanks for the review. just curious if other people are seeing this and if there's some bug in the dataset or it's expected.", "votes": null } ], "comments": [ { "id": 639230, "author_name": "oarphme", "author_url": "", "post_date": "10/03/2019 00:15:05", "content": "I also see that it appears cars in parking lots are not labeled, and in a couple other cases there appear to be missing labels? Some noise is expected, but I worry a bit about the test set, because one could produce a ton of \"false positives\" by correctly predicting cars in parking lots (if the test set has no labels for them).
\n\n\n
definetly something to worry about, yes. It's all model+human labeled. But errors in such a large dataset are pretty common
", "votes": null, "replies": [] } ] }, { "id": 639231, "author_name": "oarphme", "author_url": "", "post_date": "10/03/2019 00:16:51", "content": "last observation: the lidar clouds are very dense! as dense as Waymo in the middle region. Those plots also only show top lidar, not including the sides (if available).
", "votes": null, "replies": [] }, { "id": 639365, "author_name": "oarphme", "author_url": "", "post_date": "10/03/2019 05:54:54", "content": "The pix I posted earlier might not be keyframes, and only have one lidar. Here are some definite keyframes and with all three lidars. Most examples I see are pretty good, but these are some questionable ones. In the cars example, it looks like the footprint IoU of cars in camera and labels (which are probably in lidar) could be less than 0.5 O_O
\n\nI have not tried correcting the clouds for egomotion but as these are keyframes I would think such correction could only be small? At 30mph, if the camera-lidar sync is off by 10ms, then that's 13cm or so of shift. The cars example looks a little more than 13cm, and also it looks like the lidars don't agree perfectly either.
\n\n\n\n\n
Even I've faced this misplaced-boxes issue, share me the sample token for your case.
", "votes": null, "replies": [] }, { "id": 641924, "author_name": "oarphme", "author_url": "", "post_date": "10/05/2019 10:18:55", "content": "ok thanks for the review. just curious if other people are seeing this and if there's some bug in the dataset or it's expected.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "639228": "Is anybody else seeing some wonky camera-lidar calibration for some examples? Overall looks good, but in these cases it looks off by quite a bit: \n\n\n\n \n\n\nPerhaps worth doing an EDA on test set to see how good calibration is there.", "639230": "I also see that it appears cars in parking lots are not labeled, and in a couple other cases there appear to be missing labels? Some noise is expected, but I worry a bit about the test set, because one could produce a ton of \"false positives\" by correctly predicting cars in parking lots (if the test set has no labels for them).\n\n\n", "639231": "last observation: the lidar clouds are very dense! as dense as Waymo in the middle region. Those plots also only show top lidar, not including the sides (if available).", "639365": "The pix I posted earlier might not be keyframes, and only have one lidar. Here are some definite keyframes and with all three lidars. Most examples I see are pretty good, but these are some questionable ones. In the cars example, it looks like the footprint IoU of cars in camera and labels (which are probably in lidar) could be less than 0.5 O_O \n\nI have not tried correcting the clouds for egomotion but as these are keyframes I would think such correction could only be small? At 30mph, if the camera-lidar sync is off by 10ms, then that's 13cm or so of shift. The cars example looks a little more than 13cm, and also it looks like the lidars don't agree perfectly either. \n\n\n\n\n", "639418": "Even I've faced this misplaced-boxes issue, share me the sample token for your case.", "639877": "definetly something to worry about, yes. It's all model+human labeled. But errors in such a large dataset are pretty common", "641924": "ok thanks for the review. just curious if other people are seeing this and if there's some bug in the dataset or it's expected." }, "source": "meta" }