Among all 206 training (170) and test (36) traces, we found that 19 datasets have their ground truth not at centimeter level. Please use the datasets in the following folders with caution when training your models.
\nThe above traces were investigated using a high accuracy positioning algorithm and plotting the error distribution. In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect. In the rest of these traces, the other culprit we found is the wrong heading values reported from the GT receiver, which leads to an incorrect compensation of lever arm.
\nA few additional traces show a larger spread of error distribution than others. But we do not see sufficient evidence about the incorrectness of their GT. Possible reasons for this larger spread include: measurements of the specific device under test are noisier, and inaccuracies in the computations of position solutions, satellite positions, ionospheric, tropospheric corrections, etc.
\nThese datasets with ground truth issues were not caught before the competition launch, because our lever arm compensation process contains some manual steps that couldn't be programmatically checked. Also, the GT receiver's quality metrics do not indicate the bad quality of lever arms. Fortunately, these errors happen rarely and we confirmed that they do not affect the test datasets. More automated quality checks are added to prevent this from happening in the future.
", "messages": [ { "id": "1857248", "postDate": "07/16/2022 01:50:41", "content": "Among all 206 training (170) and test (36) traces, we found that 19 datasets have their ground truth not at centimeter level. Please use the datasets in the following folders with caution when training your models.
\nThe above traces were investigated using a high accuracy positioning algorithm and plotting the error distribution. In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect. In the rest of these traces, the other culprit we found is the wrong heading values reported from the GT receiver, which leads to an incorrect compensation of lever arm.
\nA few additional traces show a larger spread of error distribution than others. But we do not see sufficient evidence about the incorrectness of their GT. Possible reasons for this larger spread include: measurements of the specific device under test are noisier, and inaccuracies in the computations of position solutions, satellite positions, ionospheric, tropospheric corrections, etc.
\nThese datasets with ground truth issues were not caught before the competition launch, because our lever arm compensation process contains some manual steps that couldn't be programmatically checked. Also, the GT receiver's quality metrics do not indicate the bad quality of lever arms. Fortunately, these errors happen rarely and we confirmed that they do not affect the test datasets. More automated quality checks are added to prevent this from happening in the future.
", "rawMarkdown": "Among all 206 training (170) and test (36) traces, we found that 19 datasets have their ground truth not at centimeter level. Please use the datasets in the following folders with caution when training your models. \n\n1. training/2020-08-06-US-MTV-1/\n2. training/2020-08-06-US-MTV-2/\n3. training/2021-04-29-US-MTV-1/\n4. training/2021-04-29-US-MTV-2/\n5. training/2020-11-23-US-MTV-1/\n6. training/2021-03-16-US-MTV-2/\n7. training/2021-04-21-US-MTV-2/\n8. training/2020-12-10-US-SJC-1/\n9. training/2021-12-08-US-LAX-3/\n\nThe above traces were investigated using a high accuracy positioning algorithm and plotting the error distribution. In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect. In the rest of these traces, the other culprit we found is the wrong heading values reported from the GT receiver, which leads to an incorrect compensation of lever arm. \n\nA few additional traces show a larger spread of error distribution than others. But we do not see sufficient evidence about the incorrectness of their GT. Possible reasons for this larger spread include: measurements of the specific device under test are noisier, and inaccuracies in the computations of position solutions, satellite positions, ionospheric, tropospheric corrections, etc. \n\nThese datasets with ground truth issues were not caught before the competition launch, because our lever arm compensation process contains some manual steps that couldn't be programmatically checked. Also, the GT receiver's quality metrics do not indicate the bad quality of lever arms. Fortunately, these errors happen rarely and we confirmed that they do not affect the test datasets. More automated quality checks are added to prevent this from happening in the future.", "votes": null }, { "id": "1860795", "postDate": "07/18/2022 15:13:52", "content": "Thanks for this explanation. What is the accuracy of this high accuracy positioning algorithm?
", "rawMarkdown": "Thanks for this explanation. What is the accuracy of this high accuracy positioning algorithm?", "votes": null }, { "id": "1862045", "postDate": "07/19/2022 12:23:50", "content": "\n\nAccording to this paper, NovAtel SPAN, a GNSS/INS navigation system, is used to obtain ground truth. The accuracy depends on the grade of the IMU but should usually be a few centimeters to 10 centimeters (unless in a tunnel, etc.).
\n
By Taro : in this topic
", "rawMarkdown": "> According to this paper, [NovAtel SPAN](https://novatel.com/products/gnss-inertial-navigation-systems), a GNSS/INS navigation system, is used to obtain ground truth. The accuracy depends on the grade of the IMU but should usually be a few centimeters to 10 centimeters (unless in a tunnel, etc.).\n\nBy [Taro ](https://www.kaggle.com/taroz1461) : in [this topic](https://www.kaggle.com/competitions/smartphone-decimeter-2022/discussion/334964)", "votes": null }, { "id": "1862118", "postDate": "07/19/2022 13:19:31", "content": "\"In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect.\"
", "rawMarkdown": "\"In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect.\"", "votes": null } ], "comments": [ { "id": 1860795, "author_name": "yanbing06", "author_url": "", "post_date": "07/18/2022 15:13:52", "content": "Thanks for this explanation. What is the accuracy of this high accuracy positioning algorithm?
", "votes": null, "replies": [ { "id": 1862045, "author_name": "thomasdubail", "author_url": "", "post_date": "07/19/2022 12:23:50", "content": "\n\nAccording to this paper, NovAtel SPAN, a GNSS/INS navigation system, is used to obtain ground truth. The accuracy depends on the grade of the IMU but should usually be a few centimeters to 10 centimeters (unless in a tunnel, etc.).
\n
By Taro : in this topic
", "votes": null, "replies": [] }, { "id": 1862118, "author_name": "yanbing06", "author_url": "", "post_date": "07/19/2022 13:19:31", "content": "\"In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect.\"
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1857248": "Among all 206 training (170) and test (36) traces, we found that 19 datasets have their ground truth not at centimeter level. Please use the datasets in the following folders with caution when training your models. \n\n1. training/2020-08-06-US-MTV-1/\n2. training/2020-08-06-US-MTV-2/\n3. training/2021-04-29-US-MTV-1/\n4. training/2021-04-29-US-MTV-2/\n5. training/2020-11-23-US-MTV-1/\n6. training/2021-03-16-US-MTV-2/\n7. training/2021-04-21-US-MTV-2/\n8. training/2020-12-10-US-SJC-1/\n9. training/2021-12-08-US-LAX-3/\n\nThe above traces were investigated using a high accuracy positioning algorithm and plotting the error distribution. In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect. In the rest of these traces, the other culprit we found is the wrong heading values reported from the GT receiver, which leads to an incorrect compensation of lever arm. \n\nA few additional traces show a larger spread of error distribution than others. But we do not see sufficient evidence about the incorrectness of their GT. Possible reasons for this larger spread include: measurements of the specific device under test are noisier, and inaccuracies in the computations of position solutions, satellite positions, ionospheric, tropospheric corrections, etc. \n\nThese datasets with ground truth issues were not caught before the competition launch, because our lever arm compensation process contains some manual steps that couldn't be programmatically checked. Also, the GT receiver's quality metrics do not indicate the bad quality of lever arms. Fortunately, these errors happen rarely and we confirmed that they do not affect the test datasets. More automated quality checks are added to prevent this from happening in the future.", "1860795": "Thanks for this explanation. What is the accuracy of this high accuracy positioning algorithm?", "1862045": "> According to this paper, [NovAtel SPAN](https://novatel.com/products/gnss-inertial-navigation-systems), a GNSS/INS navigation system, is used to obtain ground truth. The accuracy depends on the grade of the IMU but should usually be a few centimeters to 10 centimeters (unless in a tunnel, etc.).\n\nBy [Taro ](https://www.kaggle.com/taroz1461) : in [this topic](https://www.kaggle.com/competitions/smartphone-decimeter-2022/discussion/334964)", "1862118": "\"In some of these traces, we suspect that the lever arm measurement from the ground truth (GT) receiver (NovAtel SPAN) to the phone is incorrect.\"" }, "source": "meta" }