![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F19390%2Ffe58d6611a6c408fe6c876047e4f2638%2Fgt-viz.jpg?generation=1572206919257859&alt=media\")
EDIT: I think you'd be better of using https://www.kaggle.com/ebouteillon/augmented-reality by @ebouteillon , see also discussion regarding correctness below.
\n\nWith the help of https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance I rendered 3d car models on top of the image, this is what it looks like:
\n\n
Making it a properly runnable kaggle notebook will take more effort, if anyone is willing to do that I would be super happy to upvote this :) I can share how I did this here: https://gist.github.com/lopuhin/1d87dd021ea456dcd443eb9a84983e02#file-gt-viz-ipynb
\n\nutils and car_models come from https://github.com/ApolloScapeAuto/dataset-api/blob/master/car_instance/car_models.py and https://github.com/ApolloScapeAuto/dataset-api/blob/master/utils/utils.py
\n\nrender_esl is from https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance/renderer (it's linux-only), with some minor python 3 fixes: https://gist.github.com/lopuhin/1d87dd021ea456dcd443eb9a84983e02#file-dataset-api-diff (I installed it with python setup.py develop at the end). Note that image size should be a multiple of 4, else the renderer will crash (it's not handled properly in the notebook, see dataset-api repo for a possible way to handle this, not sure I like it though).
EDIT: I think you'd be better of using https://www.kaggle.com/ebouteillon/augmented-reality by @ebouteillon , see also discussion regarding correctness below.
\n\nWith the help of https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance I rendered 3d car models on top of the image, this is what it looks like:
\n\n
Making it a properly runnable kaggle notebook will take more effort, if anyone is willing to do that I would be super happy to upvote this :) I can share how I did this here: https://gist.github.com/lopuhin/1d87dd021ea456dcd443eb9a84983e02#file-gt-viz-ipynb
\n\nutils and car_models come from https://github.com/ApolloScapeAuto/dataset-api/blob/master/car_instance/car_models.py and https://github.com/ApolloScapeAuto/dataset-api/blob/master/utils/utils.py
\n\nrender_esl is from https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance/renderer (it's linux-only), with some minor python 3 fixes: https://gist.github.com/lopuhin/1d87dd021ea456dcd443eb9a84983e02#file-dataset-api-diff (I installed it with python setup.py develop at the end). Note that image size should be a multiple of 4, else the renderer will crash (it's not handled properly in the notebook, see dataset-api repo for a possible way to handle this, not sure I like it though).
@ebouteillon btw I compared results from our approaches and they look different in some cases (and both look strange). If we check the car on the left on ID_0e6f0cc36:
your notebook:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F19390%2F5066f28ad1688a2cd8c5fcf7d03548c0%2FScreenshot%202019-10-28%20at%2018.54.00.png?generation=1572278750372834&alt=media\")
my notebook:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F19390%2F01c92677897224e8739eca81a2c96d65%2FScreenshot%202019-10-28%20at%2018.55.54.png?generation=1572278772994078&alt=media\")
But in most cases they agree quite well, and don't agree that well with what we can see on the images, that could be either annotation quality, or something we're still missing.
", "rawMarkdown": "@ebouteillon btw I compared results from our approaches and they look different in some cases (and both look strange). If we check the car on the left on ``ID_0e6f0cc36``:\n\nyour notebook:\n\n\n\n\nmy notebook:\n\n\n\nBut in most cases they agree quite well, and don't agree that well with what we can see on the images, that could be either annotation quality, or something we're still missing.", "votes": 1, "replies": [ { "id": 660076, "postDate": "2019-10-28T16:32:20.650Z", "content": "Upon more checking, it seems that this is an annotation (or ground-truth-generation) error, but from the look of this error it seems that my approach is not correct here, and I'll stick to https://www.kaggle.com/ebouteillon/augmented-reality
\n\nEDIT: yeah on ID_d10a385ea on the left it's clear that rendering is not correct with my approach.
", "rawMarkdown": "Upon more checking, it seems that this is an annotation (or ground-truth-generation) error, but from the look of this error it seems that my approach is not correct here, and I'll stick to https://www.kaggle.com/ebouteillon/augmented-reality\n\nEDIT: yeah on ID_d10a385ea on the left it's clear that rendering is not correct with my approach.", "votes": 1 }, { "id": 660933, "postDate": "2019-10-29T19:29:34.710Z", "content": "Thanks for sharing your method. The real hero is Li Linye ( @zstusnoopy ) for sharing the right formula to project 3D points onto the image. My notebook is simply standing upon his sharing.
", "rawMarkdown": "Thanks for sharing your method. The real hero is Li Linye ( @zstusnoopy ) for sharing the right formula to project 3D points onto the image. My notebook is simply standing upon his sharing.", "votes": 1 } ] }, { "id": 659590, "postDate": "2019-10-28T00:35:40.387Z", "content": "Hi @lopuhin , you may be interested in my augmented reality notebook then. 😄
", "rawMarkdown": "Hi @lopuhin , you may be interested in my [augmented reality](https://www.kaggle.com/ebouteillon/augmented-reality) notebook then. 😄 ", "votes": 1, "replies": [ { "id": 659720, "postDate": "2019-10-28T06:46:06.343Z", "content": "Oh yes, that's exactly what I needed myself, thank you :) And it's nice that it does not require any extra renderer with crazy dependencies.
", "rawMarkdown": "Oh yes, that's exactly what I needed myself, thank you :) And it's nice that it does not require any extra renderer with crazy dependencies.", "votes": 1 }, { "id": 659751, "postDate": "2019-10-28T08:16:12.787Z", "content": "you're welcomed!
", "rawMarkdown": "you're welcomed!" } ] }, { "id": 659827, "postDate": "2019-10-28T10:04:42.110Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 660066, "author_name": "Konstantin Lopukhin", "author_url": "", "post_date": "2019-10-28T16:08:37.380000", "content": "@ebouteillon btw I compared results from our approaches and they look different in some cases (and both look strange). If we check the car on the left on ID_0e6f0cc36:
your notebook:
\n\n
my notebook:
\n\n
But in most cases they agree quite well, and don't agree that well with what we can see on the images, that could be either annotation quality, or something we're still missing.
", "votes": 1, "replies": [ { "id": 660076, "author_name": "Konstantin Lopukhin", "author_url": "", "post_date": "2019-10-28T16:32:20.650000", "content": "Upon more checking, it seems that this is an annotation (or ground-truth-generation) error, but from the look of this error it seems that my approach is not correct here, and I'll stick to https://www.kaggle.com/ebouteillon/augmented-reality
\n\nEDIT: yeah on ID_d10a385ea on the left it's clear that rendering is not correct with my approach.
", "votes": 1, "replies": [] }, { "id": 660933, "author_name": "Eric Bouteillon", "author_url": "", "post_date": "2019-10-29T19:29:34.710000", "content": "Thanks for sharing your method. The real hero is Li Linye ( @zstusnoopy ) for sharing the right formula to project 3D points onto the image. My notebook is simply standing upon his sharing.
", "votes": 1, "replies": [] } ] }, { "id": 659590, "author_name": "Eric Bouteillon", "author_url": "", "post_date": "2019-10-28T00:35:40.387000", "content": "Hi @lopuhin , you may be interested in my augmented reality notebook then. 😄
", "votes": 1, "replies": [ { "id": 659720, "author_name": "Konstantin Lopukhin", "author_url": "", "post_date": "2019-10-28T06:46:06.343000", "content": "Oh yes, that's exactly what I needed myself, thank you :) And it's nice that it does not require any extra renderer with crazy dependencies.
", "votes": 1, "replies": [] }, { "id": 659751, "author_name": "Eric Bouteillon", "author_url": "", "post_date": "2019-10-28T08:16:12.787000", "content": "you're welcomed!
", "votes": 0, "replies": [] } ] }, { "id": 659827, "author_name": "", "author_url": "", "post_date": "2019-10-28T10:04:42.110000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "659511": "EDIT: I think you'd be better of using https://www.kaggle.com/ebouteillon/augmented-reality by @ebouteillon , see also discussion regarding correctness below.\n\nWith the help of https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance I rendered 3d car models on top of the image, this is what it looks like:\n\n\n\n\nMaking it a properly runnable kaggle notebook will take more effort, if anyone is willing to do that I would be super happy to upvote this :) I can share how I did this here: https://gist.github.com/lopuhin/1d87dd021ea456dcd443eb9a84983e02#file-gt-viz-ipynb\n\nutils and car_models come from https://github.com/ApolloScapeAuto/dataset-api/blob/master/car_instance/car_models.py and https://github.com/ApolloScapeAuto/dataset-api/blob/master/utils/utils.py\n\nrender_esl is from https://github.com/ApolloScapeAuto/dataset-api/tree/master/car_instance/renderer (it's linux-only), with some minor python 3 fixes: https://gist.github.com/lopuhin/1d87dd021ea456dcd443eb9a84983e02#file-dataset-api-diff (I installed it with ``python setup.py develop`` at the end). Note that image size should be a multiple of 4, else the renderer will crash (it's not handled properly in the notebook, see dataset-api repo for a possible way to handle this, not sure I like it though).", "660066": "@ebouteillon btw I compared results from our approaches and they look different in some cases (and both look strange). If we check the car on the left on ``ID_0e6f0cc36``:\n\nyour notebook:\n\n\n\n\nmy notebook:\n\n\n\nBut in most cases they agree quite well, and don't agree that well with what we can see on the images, that could be either annotation quality, or something we're still missing.", "659590": "Hi @lopuhin , you may be interested in my [augmented reality](https://www.kaggle.com/ebouteillon/augmented-reality) notebook then. 😄 ", "659827": "" } }