This post is meant to be both a question as well as a feedback to the organizers. I really do appreciate posting this exciting problem, sharing the data and the following threads from the organizer:
\nBecause of that I wanted to join this competition also at a late stage for the purpose of learning. However, after spending ~3-4 hours reading the links above and some example notebooks, I still have trouble understanding the data and the exact problem setting. Because of that, I will now probably focus on other competitions… Anyhow, those are my current most important questions:
\nIn the baseline model as well as in the notebook \"pytorch-baseline-train\" by @pestipeti, the model input is an image with 25 \"channels\". What does each channel represent, in particular the last three? I believed it to be a boolean mask, but if I'm not mistaken it's somehow encoded uint8 values? Github only states that \"image\" in EgoDataset represents \"The BEV raster as a multi-channel tensor\". Since this seems to be the most important input, I find it strange to have such poor documentation, or am I missing something?
In the Q&A, it is stated that \"you will have to predict multiple agents in a given frame\". Yet, the baseline model only seems to output a 50x2 vector representing the trajectory of ONE vehicle?!
Each item of the EgoDataset contains not only images, but also e.g. the history positions and yaws for the egovehicle. Can those be used as well? Or more generally: Which of the keys are meant to be known during test time? (On a similar note, the \"test.zarr\" also seems to contain non-empty values for the target positions, which I though we were meant to predict?)
Again, thank you very much for hosting this event. Personally I just feel that the entrance hurdle seems quite big. Or I missed some crucial information pages…
", "messages": [ { "id": "1066312", "postDate": "11/01/2020 15:52:12", "content": "This post is meant to be both a question as well as a feedback to the organizers. I really do appreciate posting this exciting problem, sharing the data and the following threads from the organizer:
\nBecause of that I wanted to join this competition also at a late stage for the purpose of learning. However, after spending ~3-4 hours reading the links above and some example notebooks, I still have trouble understanding the data and the exact problem setting. Because of that, I will now probably focus on other competitions… Anyhow, those are my current most important questions:
\nIn the baseline model as well as in the notebook \"pytorch-baseline-train\" by @pestipeti, the model input is an image with 25 \"channels\". What does each channel represent, in particular the last three? I believed it to be a boolean mask, but if I'm not mistaken it's somehow encoded uint8 values? Github only states that \"image\" in EgoDataset represents \"The BEV raster as a multi-channel tensor\". Since this seems to be the most important input, I find it strange to have such poor documentation, or am I missing something?
In the Q&A, it is stated that \"you will have to predict multiple agents in a given frame\". Yet, the baseline model only seems to output a 50x2 vector representing the trajectory of ONE vehicle?!
Each item of the EgoDataset contains not only images, but also e.g. the history positions and yaws for the egovehicle. Can those be used as well? Or more generally: Which of the keys are meant to be known during test time? (On a similar note, the \"test.zarr\" also seems to contain non-empty values for the target positions, which I though we were meant to predict?)
Again, thank you very much for hosting this event. Personally I just feel that the entrance hurdle seems quite big. Or I missed some crucial information pages…
", "rawMarkdown": "This post is meant to be both a question as well as a feedback to the organizers. I really do appreciate posting this exciting problem, sharing the data and the following threads from the organizer:\n- https://github.com/lyft/l5kit/blob/master/data_format.md#2020-lyft-competition-dataset-format\n- https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/179246\n- https://www.kaggle.com/lucabergamini/lyft-baseline-09-02\n- https://github.com/lyft/l5kit/blob/master/data_format.md#2020-lyft-competition-dataset-format\n\nBecause of that I wanted to join this competition also at a late stage for the purpose of learning. However, after spending ~3-4 hours reading the links above and some example notebooks, I still have trouble understanding the data and the exact problem setting. Because of that, I will now probably focus on other competitions... Anyhow, those are my current most important questions:\n\n- In the baseline model as well as in the notebook \"pytorch-baseline-train\" by @pestipeti, the model input is an image with 25 \"channels\". What does each channel represent, in particular the last three? I believed it to be a boolean mask, but if I'm not mistaken it's somehow encoded uint8 values? Github only states that \"image\" in EgoDataset represents \"The BEV raster as a multi-channel tensor\". Since this seems to be the most important input, I find it strange to have such poor documentation, or am I missing something?\n\n- In the Q&A, it is stated that \"you will have to predict multiple agents in a given frame\". Yet, the baseline model only seems to output a 50x2 vector representing the trajectory of ONE vehicle?!\n\n- Each item of the EgoDataset contains not only images, but also e.g. the history positions and yaws for the egovehicle. Can those be used as well? Or more generally: Which of the keys are meant to be known during test time? (On a similar note, the \"test.zarr\" also seems to contain non-empty values for the target positions, which I though we were meant to predict?)\n\nAgain, thank you very much for hosting this event. Personally I just feel that the entrance hurdle seems quite big. Or I missed some crucial information pages...", "votes": null }, { "id": "1066356", "postDate": "11/01/2020 16:59:54", "content": "That image is really helpful @zaharch, thank you.
\nIt looks like there is a lot of redundant info - especially for the \"agent under consideration\".
\nDo you think we would lose much info by setting history_step_size =2, effectively making it 15 channels ?
Probably not much loss, the frames are separated by only 0.1s. But I haven't tried it, so can't say for sure.
", "rawMarkdown": "Probably not much loss, the frames are separated by only 0.1s. But I haven't tried it, so can't say for sure.", "votes": null } ], "comments": [ { "id": 1066356, "author_name": "zaharch", "author_url": "", "post_date": "11/01/2020 16:59:54", "content": "That image is really helpful @zaharch, thank you.
\nIt looks like there is a lot of redundant info - especially for the \"agent under consideration\".
\nDo you think we would lose much info by setting history_step_size =2, effectively making it 15 channels ?
Probably not much loss, the frames are separated by only 0.1s. But I haven't tried it, so can't say for sure.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1066312": "This post is meant to be both a question as well as a feedback to the organizers. I really do appreciate posting this exciting problem, sharing the data and the following threads from the organizer:\n- https://github.com/lyft/l5kit/blob/master/data_format.md#2020-lyft-competition-dataset-format\n- https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/179246\n- https://www.kaggle.com/lucabergamini/lyft-baseline-09-02\n- https://github.com/lyft/l5kit/blob/master/data_format.md#2020-lyft-competition-dataset-format\n\nBecause of that I wanted to join this competition also at a late stage for the purpose of learning. However, after spending ~3-4 hours reading the links above and some example notebooks, I still have trouble understanding the data and the exact problem setting. Because of that, I will now probably focus on other competitions... Anyhow, those are my current most important questions:\n\n- In the baseline model as well as in the notebook \"pytorch-baseline-train\" by @pestipeti, the model input is an image with 25 \"channels\". What does each channel represent, in particular the last three? I believed it to be a boolean mask, but if I'm not mistaken it's somehow encoded uint8 values? Github only states that \"image\" in EgoDataset represents \"The BEV raster as a multi-channel tensor\". Since this seems to be the most important input, I find it strange to have such poor documentation, or am I missing something?\n\n- In the Q&A, it is stated that \"you will have to predict multiple agents in a given frame\". Yet, the baseline model only seems to output a 50x2 vector representing the trajectory of ONE vehicle?!\n\n- Each item of the EgoDataset contains not only images, but also e.g. the history positions and yaws for the egovehicle. Can those be used as well? Or more generally: Which of the keys are meant to be known during test time? (On a similar note, the \"test.zarr\" also seems to contain non-empty values for the target positions, which I though we were meant to predict?)\n\nAgain, thank you very much for hosting this event. Personally I just feel that the entrance hurdle seems quite big. Or I missed some crucial information pages...", "1066356": "1. Take a look at [this pic](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/187366#1030547) for an example of channels visualization.\n2. Yes, the prediction is for one agent at a time, but many of these agents share same frame. It can be about 5-10 predictions (different agents) that we need to make per frame. There are about 16k frames in the test set if I remember correctly, with about 70k agents.\n3. Target positions, target availability and target yaws are zero in the test set (a placeholder), target positions is what needs to be predicted. The rest you can use.", "1066484": "That image is really helpful @zaharch, thank you.\n\nIt looks like there is a lot of redundant info - especially for the \"agent under consideration\". \nDo you think we would lose much info by setting history_step_size =2, effectively making it 15 channels ?", "1066488": "Probably not much loss, the frames are separated by only 0.1s. But I haven't tried it, so can't say for sure." }, "source": "meta" }