![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F344232%2F121b3135ae4e1634e2236dd55ef34c09%2FDD12BE13-227D-47C0-9491-78C700EAEC45.jpeg?generation=1599044663468657&alt=media\")
\nAnd this
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F344232%2F085476b16c394f1d4f447b1f38129006%2FF20A312D-4FCB-420E-829A-49C7D75ECD08.jpeg?generation=1599044697650371&alt=media\")
This competition is a great opportunity for me to understand what is the current status & what kind of problem is focused now to realize autonomous vehicles. Thank hosts for organizing this interesting competition.
\nI read the paper One Thousand and One Hours: Self-driving Motion Prediction Dataset, and surprised that perception module is already almost done and this competition dataset is made on top this perception module.
\nAlso I wondered how to predict other vehicle motions.
\nFor human, we predict others' motion by looking with our eye, from subjective point of view. We check which direction the pedestrian is facing and check the posture (walking, running etc). We also sometimes use eye contact when pedestrian cross the crosswalk.
\nHowever in this competition setting, we only use BEV (Birds-eye-view) picture with the history motion. We do not use any raw image information of the real world of current time.
\nWell I'm just interested why the community moving towards this approach. Is there any advantage (camera is not necessary in the car to reduce cost etc)?
", "messages": [ { "id": "994760", "postDate": "09/01/2020 23:22:22", "content": "This competition is a great opportunity for me to understand what is the current status & what kind of problem is focused now to realize autonomous vehicles. Thank hosts for organizing this interesting competition.
\nI read the paper One Thousand and One Hours: Self-driving Motion Prediction Dataset, and surprised that perception module is already almost done and this competition dataset is made on top this perception module.
\nAlso I wondered how to predict other vehicle motions.
\nFor human, we predict others' motion by looking with our eye, from subjective point of view. We check which direction the pedestrian is facing and check the posture (walking, running etc). We also sometimes use eye contact when pedestrian cross the crosswalk.
\nHowever in this competition setting, we only use BEV (Birds-eye-view) picture with the history motion. We do not use any raw image information of the real world of current time.
\nWell I'm just interested why the community moving towards this approach. Is there any advantage (camera is not necessary in the car to reduce cost etc)?
", "rawMarkdown": "This competition is a great opportunity for me to understand what is the current status & what kind of problem is focused now to realize autonomous vehicles. Thank hosts for organizing this interesting competition.\n\nI read the paper [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://arxiv.org/abs/2006.14480), and surprised that perception module is already almost done and this competition dataset is made on top this perception module.\n\nAlso I wondered how to predict other vehicle motions.\n\nFor human, we predict others' motion by looking with our eye, **from subjective point of view**. We check which direction the pedestrian is facing and check the **posture** (walking, running etc). We also sometimes use eye contact when pedestrian cross the crosswalk.\n\nHowever in this competition setting, we only use **BEV (Birds-eye-view)** picture with the history motion. We **do not use any raw image information of the real world of current time**.\n\nWell I'm just interested why the community moving towards this approach. Is there any advantage (camera is not necessary in the car to reduce cost etc)?", "votes": null }, { "id": "994767", "postDate": "09/01/2020 23:38:01", "content": "Hi @corochann Small bit of information that i can give you for you to think more on this and go forward is, maps, Camera, lidar, radar and ultrasonic are the sensors used in cars to perceive its environment. each of which has its advantage. Birds eye view is a constructed image from these sensors. Birds eye view is not coming from any other sensors. Though drones are employed these days because birds eye view motion prediction is most predominant method cooperative dynamic systems in real world like cars. Motion planning that comes after motion prediction is complex task and long term prediction becomes more important. Camera is necessary and is the most important sensor for AVs, lidar is the most expensive sensor. there are also methods that involves perception without lidar because of cost reason.
\nImportant thing to know is BEV is used for learning and validation.
", "rawMarkdown": "Hi @corochann Small bit of information that i can give you for you to think more on this and go forward is, maps, Camera, lidar, radar and ultrasonic are the sensors used in cars to perceive its environment. each of which has its advantage. Birds eye view is a constructed image from these sensors. Birds eye view is not coming from any other sensors. Though drones are employed these days because birds eye view motion prediction is most predominant method cooperative dynamic systems in real world like cars. Motion planning that comes after motion prediction is complex task and long term prediction becomes more important. Camera is necessary and is the most important sensor for AVs, lidar is the most expensive sensor. there are also methods that involves perception without lidar because of cost reason. \n\nImportant thing to know is BEV is used for learning and validation.", "votes": null }, { "id": "995118", "postDate": "09/02/2020 07:17:26", "content": "Thank you for reply and information.
\n\n\nImportant thing to know is BEV is used for learning and validation.
\n
The prediction model built in this competition is useful for motion planning simulator to estimate its performance?
", "rawMarkdown": "Thank you for reply and information.\n\n> Important thing to know is BEV is used for learning and validation.\n\nThe prediction model built in this competition is useful for motion planning simulator to estimate its performance?", "votes": null }, { "id": "995324", "postDate": "09/02/2020 11:03:42", "content": "check out comma AI . . .
", "rawMarkdown": "check out comma AI . . .", "votes": null }, { "id": "995325", "postDate": "09/02/2020 11:05:14", "content": "Yes I think it can be! These two pictures will explain more.
\nAnd this
\n
Hi @corochann, I can probably add a little bit of context here as an \"insider\" :)
\nI can think of mainly two reasons for BEV:
\nHowever, this is obviously not the only way forward. End-to-end approaches also exist (see wayve as an example). In that case, perception and prediction are jointly learnt.
\nNote that regardless of your choice, you will still need raw sensors on the car in the end as the perception part is still there.
\nHope this sheds some lights on the overall approach :)
", "rawMarkdown": "Hi @corochann, I can probably add a little bit of context here as an \"insider\" :)\n\nI can think of mainly two reasons for BEV:\n- when using a CNN approach that goes from a BEV to future predictions the two spaces are already aligned. This helps the network because it doesn't have to learn an additional mapping. We're reasoning in the same space (differently from using cameras or LIDAR);\n- By using the outputs of perception to build the BEV we decouple the first part of the stack (perception) from the second one (prediction and planning). The prediction part can be trained using relatively light inputs and networks, as the heavy work with LIDAR and camera (which require massive architecture and time) has already been done in the perception stack.\n\nHowever, this is obviously not the only way forward. End-to-end approaches also exist (see [wayve](https://wayve.ai/) as an example). In that case, perception and prediction are jointly learnt.\n\nNote that regardless of your choice, you will still need raw sensors on the car in the end as the perception part is still there.\n\nHope this sheds some lights on the overall approach :)", "votes": null }, { "id": "995870", "postDate": "09/02/2020 22:21:43", "content": "Thank you @lucabergamini, I'm glad to get a \"internal\" comment.
\n\n\nBy using the outputs of perception to build the BEV we decouple the first part of the stack (perception) from the second one (prediction and planning).
\n
Since raw image information is not used BEV, there may be a benefit to avoid overfiting. The prediction module with BEV may be robust to adversarial attack since only position/direction/extent information is used and other too much detail is not used as input.
\nI understand that several approach exist now, I'm interested what approach is adopted for first level-5 autonomous vehicle.
", "rawMarkdown": "Thank you @lucabergamini, I'm glad to get a \"internal\" comment.\n\n> By using the outputs of perception to build the BEV we decouple the first part of the stack (perception) from the second one (prediction and planning).\n\nSince raw image information is not used BEV, there may be a benefit to avoid overfiting. The prediction module with BEV may be robust to adversarial attack since only position/direction/extent information is used and other too much detail is not used as input.\n\nI understand that several approach exist now, I'm interested what approach is adopted for first level-5 autonomous vehicle.", "votes": null }, { "id": "995874", "postDate": "09/02/2020 22:30:28", "content": "Thanks for info! There are many hints in published document. Is this paper open? Could you paste the link if possible?
", "rawMarkdown": "Thanks for info! There are many hints in published document. Is this paper open? Could you paste the link if possible?", "votes": null }, { "id": "995876", "postDate": "09/02/2020 22:35:19", "content": "I just checked some modeling method introduced above (& python code):
\nparticle model:
\n\nkinematic bicycle model:
\nThis one?
\n\nSo does it predict future only from subjective view of raw image?
", "rawMarkdown": "This one?\n - https://comma.ai/\n - https://github.com/commaai/openpilot\n\nSo does it predict future only from subjective view of raw image?", "votes": null } ], "comments": [ { "id": 994767, "author_name": "deepakrajpurushothaman", "author_url": "", "post_date": "09/01/2020 23:38:01", "content": "Hi @corochann Small bit of information that i can give you for you to think more on this and go forward is, maps, Camera, lidar, radar and ultrasonic are the sensors used in cars to perceive its environment. each of which has its advantage. Birds eye view is a constructed image from these sensors. Birds eye view is not coming from any other sensors. Though drones are employed these days because birds eye view motion prediction is most predominant method cooperative dynamic systems in real world like cars. Motion planning that comes after motion prediction is complex task and long term prediction becomes more important. Camera is necessary and is the most important sensor for AVs, lidar is the most expensive sensor. there are also methods that involves perception without lidar because of cost reason.
\nImportant thing to know is BEV is used for learning and validation.
", "votes": null, "replies": [ { "id": 995118, "author_name": "corochann", "author_url": "", "post_date": "09/02/2020 07:17:26", "content": "Thank you for reply and information.
\n\n\nImportant thing to know is BEV is used for learning and validation.
\n
The prediction model built in this competition is useful for motion planning simulator to estimate its performance?
", "votes": null, "replies": [] }, { "id": 995325, "author_name": "deepakrajpurushothaman", "author_url": "", "post_date": "09/02/2020 11:05:14", "content": "Yes I think it can be! These two pictures will explain more.
\nAnd this
\n
Thanks for info! There are many hints in published document. Is this paper open? Could you paste the link if possible?
", "votes": null, "replies": [] }, { "id": 995876, "author_name": "corochann", "author_url": "", "post_date": "09/02/2020 22:35:19", "content": "I just checked some modeling method introduced above (& python code):
\nparticle model:
\n\nkinematic bicycle model:
\nHi @corochann, I can probably add a little bit of context here as an \"insider\" :)
\nI can think of mainly two reasons for BEV:
\nHowever, this is obviously not the only way forward. End-to-end approaches also exist (see wayve as an example). In that case, perception and prediction are jointly learnt.
\nNote that regardless of your choice, you will still need raw sensors on the car in the end as the perception part is still there.
\nHope this sheds some lights on the overall approach :)
", "votes": null, "replies": [ { "id": 995870, "author_name": "corochann", "author_url": "", "post_date": "09/02/2020 22:21:43", "content": "Thank you @lucabergamini, I'm glad to get a \"internal\" comment.
\n\n\nBy using the outputs of perception to build the BEV we decouple the first part of the stack (perception) from the second one (prediction and planning).
\n
Since raw image information is not used BEV, there may be a benefit to avoid overfiting. The prediction module with BEV may be robust to adversarial attack since only position/direction/extent information is used and other too much detail is not used as input.
\nI understand that several approach exist now, I'm interested what approach is adopted for first level-5 autonomous vehicle.
", "votes": null, "replies": [] } ] }, { "id": 995324, "author_name": "preethamrakshithp", "author_url": "", "post_date": "09/02/2020 11:03:42", "content": "check out comma AI . . .
", "votes": null, "replies": [ { "id": 995877, "author_name": "corochann", "author_url": "", "post_date": "09/02/2020 22:38:12", "content": "This one?
\n\nSo does it predict future only from subjective view of raw image?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "994760": "This competition is a great opportunity for me to understand what is the current status & what kind of problem is focused now to realize autonomous vehicles. Thank hosts for organizing this interesting competition.\n\nI read the paper [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://arxiv.org/abs/2006.14480), and surprised that perception module is already almost done and this competition dataset is made on top this perception module.\n\nAlso I wondered how to predict other vehicle motions.\n\nFor human, we predict others' motion by looking with our eye, **from subjective point of view**. We check which direction the pedestrian is facing and check the **posture** (walking, running etc). We also sometimes use eye contact when pedestrian cross the crosswalk.\n\nHowever in this competition setting, we only use **BEV (Birds-eye-view)** picture with the history motion. We **do not use any raw image information of the real world of current time**.\n\nWell I'm just interested why the community moving towards this approach. Is there any advantage (camera is not necessary in the car to reduce cost etc)?", "994767": "Hi @corochann Small bit of information that i can give you for you to think more on this and go forward is, maps, Camera, lidar, radar and ultrasonic are the sensors used in cars to perceive its environment. each of which has its advantage. Birds eye view is a constructed image from these sensors. Birds eye view is not coming from any other sensors. Though drones are employed these days because birds eye view motion prediction is most predominant method cooperative dynamic systems in real world like cars. Motion planning that comes after motion prediction is complex task and long term prediction becomes more important. Camera is necessary and is the most important sensor for AVs, lidar is the most expensive sensor. there are also methods that involves perception without lidar because of cost reason. \n\nImportant thing to know is BEV is used for learning and validation.", "995118": "Thank you for reply and information.\n\n> Important thing to know is BEV is used for learning and validation.\n\nThe prediction model built in this competition is useful for motion planning simulator to estimate its performance?", "995324": "check out comma AI . . .", "995325": "Yes I think it can be! These two pictures will explain more. \nAnd this \n", "995341": "Hi @corochann, I can probably add a little bit of context here as an \"insider\" :)\n\nI can think of mainly two reasons for BEV:\n- when using a CNN approach that goes from a BEV to future predictions the two spaces are already aligned. This helps the network because it doesn't have to learn an additional mapping. We're reasoning in the same space (differently from using cameras or LIDAR);\n- By using the outputs of perception to build the BEV we decouple the first part of the stack (perception) from the second one (prediction and planning). The prediction part can be trained using relatively light inputs and networks, as the heavy work with LIDAR and camera (which require massive architecture and time) has already been done in the perception stack.\n\nHowever, this is obviously not the only way forward. End-to-end approaches also exist (see [wayve](https://wayve.ai/) as an example). In that case, perception and prediction are jointly learnt.\n\nNote that regardless of your choice, you will still need raw sensors on the car in the end as the perception part is still there.\n\nHope this sheds some lights on the overall approach :)", "995870": "Thank you @lucabergamini, I'm glad to get a \"internal\" comment.\n\n> By using the outputs of perception to build the BEV we decouple the first part of the stack (perception) from the second one (prediction and planning).\n\nSince raw image information is not used BEV, there may be a benefit to avoid overfiting. The prediction module with BEV may be robust to adversarial attack since only position/direction/extent information is used and other too much detail is not used as input.\n\nI understand that several approach exist now, I'm interested what approach is adopted for first level-5 autonomous vehicle.", "995874": "Thanks for info! There are many hints in published document. Is this paper open? Could you paste the link if possible?", "995876": "I just checked some modeling method introduced above (& python code):\n\nparticle model:\n - https://github.com/simon-r/PyParticles\n\nkinematic bicycle model:\n - https://www.coursera.org/lecture/intro-self-driving-cars/lesson-2-the-kinematic-bicycle-model-Bi8yE\n - https://medium.com/@dingyan7361/simple-understanding-of-kinematic-bicycle-model-81cac6420357\n - https://github.com/Derekabc/PathTrackingBicycle\n - https://github.com/caiofis/Bicycle-Model", "995877": "This one?\n - https://comma.ai/\n - https://github.com/commaai/openpilot\n\nSo does it predict future only from subjective view of raw image?" }, "source": "meta" }