In the evaluation: the translation distance is : \n[ 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]
\n\nIt is a normalised distance? I.e., if the car is 100 meters away, the prediction should be within a 100*[0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]
\n\nOtherwise, from my previous project experience, estimating the distance using RGB only within 0.1 meter prediction error is simply impossible.
\n\nI might have missed the clue if this topic is a duplication.
", "messages": [ { "id": "664810", "postDate": "11/04/2019 08:55:53", "content": "In the evaluation: the translation distance is : \n[ 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]
\n\nIt is a normalised distance? I.e., if the car is 100 meters away, the prediction should be within a 100*[0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]
\n\nOtherwise, from my previous project experience, estimating the distance using RGB only within 0.1 meter prediction error is simply impossible.
\n\nI might have missed the clue if this topic is a duplication.
", "rawMarkdown": "In the evaluation: the translation distance is : \n[ 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]\n\nIt is a normalised distance? I.e., if the car is 100 meters away, the prediction should be within a 100*[0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]\n\nOtherwise, from my previous project experience, estimating the distance using RGB only within 0.1 meter prediction error is simply impossible.\n\nI might have missed the clue if this topic is a duplication.", "votes": null }, { "id": "664847", "postDate": "11/04/2019 09:58:35", "content": "I don't think it is normalized. The host would have mentioned that on the evaluation description. On the Apolloscape website, the loosest threshold is 2.8 meters. I don't know why they changed.
", "rawMarkdown": "I don't think it is normalized. The host would have mentioned that on the evaluation description. On the Apolloscape website, the loosest threshold is 2.8 meters. I don't know why they changed.", "votes": null }, { "id": "664883", "postDate": "11/04/2019 10:49:10", "content": "I know, 2.8 meters is hard enough, I doubt without normalisation, the leader board can achieve 0.2.
", "rawMarkdown": "I know, 2.8 meters is hard enough, I doubt without normalisation, the leader board can achieve 0.2.", "votes": null }, { "id": "664968", "postDate": "11/04/2019 13:29:05", "content": "I posted about this too, doesn't make any sense. My current predictions give a distance around 1-1.5 and rarely get below .1. How are you supposed to predict within less than a centimeter away?
", "rawMarkdown": "I posted about this too, doesn't make any sense. My current predictions give a distance around 1-1.5 and rarely get below .1. How are you supposed to predict within less than a centimeter away?", "votes": null }, { "id": "665106", "postDate": "11/04/2019 16:24:05", "content": "Yet, you are the second on the leader board. I think there is a discrepancy of data evaluation description and the code for the evaluation. Hope the organisors can come out and clarify this issue. @greatgamedota
", "rawMarkdown": "Yet, you are the second on the leader board. I think there is a discrepancy of data evaluation description and the code for the evaluation. Hope the organisors can come out and clarify this issue. @greatgamedota", "votes": null }, { "id": "665125", "postDate": "11/04/2019 16:50:08", "content": "I think the thresholds are (sort of) reasonable. Think about it: 2.8 meters is roughly the width of a lane. I would not sit in a self-driving car with that large margin of error. \nI agree with your normalizing idea; it would be better. In some cases, a 1-pixel difference in the image could mean 10 meters.
\n\nOn the other hand, everyone has the same data/threshold/metric. We don't have to predict above 0.9 to win this competition. Only a little bit higher than the 2nd :)
", "rawMarkdown": "I think the thresholds are (sort of) reasonable. Think about it: 2.8 meters is roughly the width of a lane. I would not sit in a self-driving car with that large margin of error. \nI agree with your normalizing idea; it would be better. In some cases, a 1-pixel difference in the image could mean 10 meters.\n\nOn the other hand, everyone has the same data/threshold/metric. We don't have to predict above 0.9 to win this competition. Only a little bit higher than the 2nd :)", "votes": null }, { "id": "665133", "postDate": "11/04/2019 17:02:34", "content": "Self driving cars you sit in have a lot more than just one rgb image ;)
", "rawMarkdown": "Self driving cars you sit in have a lot more than just one rgb image ;)", "votes": null }, { "id": "665163", "postDate": "11/04/2019 17:24:08", "content": "Yes, of course, but it does not mean that they should loosen the thresholds of any sensors to get better results. IMO, any self-driving car developer/manufacturer, should squeeze every bit of information from every sensor; therefore, a strict threshold not just acceptable but a requirement.
", "rawMarkdown": "Yes, of course, but it does not mean that they should loosen the thresholds of any sensors to get better results. IMO, any self-driving car developer/manufacturer, should squeeze every bit of information from every sensor; therefore, a strict threshold not just acceptable but a requirement.", "votes": null }, { "id": "679823", "postDate": "11/23/2019 13:07:12", "content": "I listed other 3d detection competitions to highlight that the threshold is too strict as well: https://www.kaggle.com/c/pku-autonomous-driving/discussion/118677#latest-679813
", "rawMarkdown": "I listed other 3d detection competitions to highlight that the threshold is too strict as well: https://www.kaggle.com/c/pku-autonomous-driving/discussion/118677#latest-679813", "votes": null } ], "comments": [ { "id": 664847, "author_name": "pestipeti", "author_url": "", "post_date": "11/04/2019 09:58:35", "content": "I don't think it is normalized. The host would have mentioned that on the evaluation description. On the Apolloscape website, the loosest threshold is 2.8 meters. I don't know why they changed.
", "votes": null, "replies": [ { "id": 664883, "author_name": "stevenwudi", "author_url": "", "post_date": "11/04/2019 10:49:10", "content": "I know, 2.8 meters is hard enough, I doubt without normalisation, the leader board can achieve 0.2.
", "votes": null, "replies": [] }, { "id": 665125, "author_name": "pestipeti", "author_url": "", "post_date": "11/04/2019 16:50:08", "content": "I think the thresholds are (sort of) reasonable. Think about it: 2.8 meters is roughly the width of a lane. I would not sit in a self-driving car with that large margin of error. \nI agree with your normalizing idea; it would be better. In some cases, a 1-pixel difference in the image could mean 10 meters.
\n\nOn the other hand, everyone has the same data/threshold/metric. We don't have to predict above 0.9 to win this competition. Only a little bit higher than the 2nd :)
", "votes": null, "replies": [] }, { "id": 665133, "author_name": "greatgamedota", "author_url": "", "post_date": "11/04/2019 17:02:34", "content": "Self driving cars you sit in have a lot more than just one rgb image ;)
", "votes": null, "replies": [] }, { "id": 665163, "author_name": "pestipeti", "author_url": "", "post_date": "11/04/2019 17:24:08", "content": "Yes, of course, but it does not mean that they should loosen the thresholds of any sensors to get better results. IMO, any self-driving car developer/manufacturer, should squeeze every bit of information from every sensor; therefore, a strict threshold not just acceptable but a requirement.
", "votes": null, "replies": [] } ] }, { "id": 664968, "author_name": "greatgamedota", "author_url": "", "post_date": "11/04/2019 13:29:05", "content": "I posted about this too, doesn't make any sense. My current predictions give a distance around 1-1.5 and rarely get below .1. How are you supposed to predict within less than a centimeter away?
", "votes": null, "replies": [ { "id": 665106, "author_name": "stevenwudi", "author_url": "", "post_date": "11/04/2019 16:24:05", "content": "Yet, you are the second on the leader board. I think there is a discrepancy of data evaluation description and the code for the evaluation. Hope the organisors can come out and clarify this issue. @greatgamedota
", "votes": null, "replies": [] } ] }, { "id": 679823, "author_name": "kwea123", "author_url": "", "post_date": "11/23/2019 13:07:12", "content": "I listed other 3d detection competitions to highlight that the threshold is too strict as well: https://www.kaggle.com/c/pku-autonomous-driving/discussion/118677#latest-679813
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "664810": "In the evaluation: the translation distance is : \n[ 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]\n\nIt is a normalised distance? I.e., if the car is 100 meters away, the prediction should be within a 100*[0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01]\n\nOtherwise, from my previous project experience, estimating the distance using RGB only within 0.1 meter prediction error is simply impossible.\n\nI might have missed the clue if this topic is a duplication.", "664847": "I don't think it is normalized. The host would have mentioned that on the evaluation description. On the Apolloscape website, the loosest threshold is 2.8 meters. I don't know why they changed.", "664883": "I know, 2.8 meters is hard enough, I doubt without normalisation, the leader board can achieve 0.2.", "664968": "I posted about this too, doesn't make any sense. My current predictions give a distance around 1-1.5 and rarely get below .1. How are you supposed to predict within less than a centimeter away?", "665106": "Yet, you are the second on the leader board. I think there is a discrepancy of data evaluation description and the code for the evaluation. Hope the organisors can come out and clarify this issue. @greatgamedota", "665125": "I think the thresholds are (sort of) reasonable. Think about it: 2.8 meters is roughly the width of a lane. I would not sit in a self-driving car with that large margin of error. \nI agree with your normalizing idea; it would be better. In some cases, a 1-pixel difference in the image could mean 10 meters.\n\nOn the other hand, everyone has the same data/threshold/metric. We don't have to predict above 0.9 to win this competition. Only a little bit higher than the 2nd :)", "665133": "Self driving cars you sit in have a lot more than just one rgb image ;)", "665163": "Yes, of course, but it does not mean that they should loosen the thresholds of any sensors to get better results. IMO, any self-driving car developer/manufacturer, should squeeze every bit of information from every sensor; therefore, a strict threshold not just acceptable but a requirement.", "679823": "I listed other 3d detection competitions to highlight that the threshold is too strict as well: https://www.kaggle.com/c/pku-autonomous-driving/discussion/118677#latest-679813" }, "source": "meta" }