Hello competition hosts and kagglers.
\nI've got a question about ground truth gps antenna coordinates to hosts @gymf123
\nI've found that shifting baseline location about 0.6m-0.8m improves both baseline train (about 10cm) and public test baseline solution (about 5cm) (published here).
\nIs this just a coincidence, or should we consider ground truth shift during training and submission?
Hello competition hosts and kagglers.
\nI've got a question about ground truth gps antenna coordinates to hosts @gymf123
\nI've found that shifting baseline location about 0.6m-0.8m improves both baseline train (about 10cm) and public test baseline solution (about 5cm) (published here).
\nIs this just a coincidence, or should we consider ground truth shift during training and submission?
I tried a similar process.I thought it would improve by 0.6m ~ 0.8m.
\nBut it did not result in a significant improvement in my score.(Only 0.004m)
I would like to know if ground truth is shifting too.
", "rawMarkdown": "I tried a similar process.I thought it would improve by 0.6m ~ 0.8m.\nBut it did not result in a significant improvement in my score.(Only 0.004m)\n\nI would like to know if ground truth is shifting too.", "votes": null }, { "id": "1332965", "postDate": "06/02/2021 12:06:00", "content": "I believe they have accounted for the difference between the ground truth antenna and the phones: in the paper (https://www.kaggle.com/google/android-smartphones-high-accuracy-datasets?select=ION+GNSS+2020+Android+Raw+GNSS+Measurement+Datasets+for+Precise+Positioning.pdf) on page 1929 it says:
\n\n\nThe ground truth file has compensated the lever arm offset and is referenced to the smartphone itself.
\n
They also reference that \"lever arm offset\" in the video as well.
\nEDIT:
\nThey also say (in the paper):
\n\n\nIn practice, we have compensated these lever arms. However, several centimeters of errors are inevitable because the lever arms are coarsely measured with regular rulers.
\n
So there may still be some error there (but it will be potentially different for each collection/phone)
", "rawMarkdown": "I believe they have accounted for the difference between the ground truth antenna and the phones: in the paper (https://www.kaggle.com/google/android-smartphones-high-accuracy-datasets?select=ION+GNSS+2020+Android+Raw+GNSS+Measurement+Datasets+for+Precise+Positioning.pdf) on page 1929 it says: \n\n> The ground truth file has compensated the lever arm offset and is referenced to the smartphone itself.\n\nThey also reference that \"lever arm offset\" in the video as well.\n\nEDIT:\n\nThey also say (in the paper):\n\n> In practice, we have compensated these lever arms. However, several centimeters of errors are inevitable because the lever arms are coarsely measured with regular rulers.\n\nSo there may still be some error there (but it will be potentially different for each collection/phone)", "votes": null }, { "id": "1332972", "postDate": "06/02/2021 12:10:47", "content": "At first glance, this should be mprovement of about 0.6-0.8
\nBut, there is a 95% error in the metric, so probably it won't affect large errors.
\nSo one would expect an improvement of about 0.3-0.4,
\nbut if we do experiments, one more time some measurements will be wrongly shifted.
\nOn the other hand, according to the document attached to the dataset, the antenna was placed about 80cm back, which is confirmed by the optimization on the train dataset.
I've also noticed (consistently) that the better quality the results are, the greater the improvement.
\nOn the training set (score of 5.28 improved to 5.18)
\nOn the public test set (score of 6.164 improved to 6.117 )
\nMy best score of 5.350 improved to 5.280
\nIdeal submission should be improved from 0.8 to 0 :)
That's what I thought at first.
\nBut in the training set, we get an improvement with more than 60 cm backward shift, so aiming high precision it is hard to talk about some error here
\nBaseline solution is unlikely to have such a shift either (imho)
hm, interesting! I'm not sure then…
", "rawMarkdown": "hm, interesting! I'm not sure then...", "votes": null }, { "id": "1333081", "postDate": "06/02/2021 13:29:59", "content": "Well, after a few tries, I think you're right after all - lever arm offset has been applied.
\nMaybe the position shift is related to the velocity of the receiver and that's why it works 'on average' - interesting clue.
\nRegardless, I'm curious about the hosts response.
Thanks for the detailed explanation. You are right.
\nbtw ,I tried running your process on my best score.
\nThe score got worse. Score : 5.028 → 5.979
\"The average distance\" you mentioned may have already been solved in my model.
\n(Or maybe my code is wrong. lol )
Hi Wojtek, could you please point me to the collection where you observed the ground truth shift?
", "rawMarkdown": "Hi Wojtek, could you please point me to the collection where you observed the ground truth shift?", "votes": null } ], "comments": [ { "id": 1332959, "author_name": "dehokanta", "author_url": "", "post_date": "06/02/2021 12:01:09", "content": "I tried a similar process.I thought it would improve by 0.6m ~ 0.8m.
\nBut it did not result in a significant improvement in my score.(Only 0.004m)
I would like to know if ground truth is shifting too.
", "votes": null, "replies": [ { "id": 1332972, "author_name": "wrrosa", "author_url": "", "post_date": "06/02/2021 12:10:47", "content": "At first glance, this should be mprovement of about 0.6-0.8
\nBut, there is a 95% error in the metric, so probably it won't affect large errors.
\nSo one would expect an improvement of about 0.3-0.4,
\nbut if we do experiments, one more time some measurements will be wrongly shifted.
\nOn the other hand, according to the document attached to the dataset, the antenna was placed about 80cm back, which is confirmed by the optimization on the train dataset.
I've also noticed (consistently) that the better quality the results are, the greater the improvement.
\nOn the training set (score of 5.28 improved to 5.18)
\nOn the public test set (score of 6.164 improved to 6.117 )
\nMy best score of 5.350 improved to 5.280
\nIdeal submission should be improved from 0.8 to 0 :)
Thanks for the detailed explanation. You are right.
\nbtw ,I tried running your process on my best score.
\nThe score got worse. Score : 5.028 → 5.979
\"The average distance\" you mentioned may have already been solved in my model.
\n(Or maybe my code is wrong. lol )
I believe they have accounted for the difference between the ground truth antenna and the phones: in the paper (https://www.kaggle.com/google/android-smartphones-high-accuracy-datasets?select=ION+GNSS+2020+Android+Raw+GNSS+Measurement+Datasets+for+Precise+Positioning.pdf) on page 1929 it says:
\n\n\nThe ground truth file has compensated the lever arm offset and is referenced to the smartphone itself.
\n
They also reference that \"lever arm offset\" in the video as well.
\nEDIT:
\nThey also say (in the paper):
\n\n\nIn practice, we have compensated these lever arms. However, several centimeters of errors are inevitable because the lever arms are coarsely measured with regular rulers.
\n
So there may still be some error there (but it will be potentially different for each collection/phone)
", "votes": null, "replies": [ { "id": 1332983, "author_name": "wrrosa", "author_url": "", "post_date": "06/02/2021 12:15:34", "content": "That's what I thought at first.
\nBut in the training set, we get an improvement with more than 60 cm backward shift, so aiming high precision it is hard to talk about some error here
\nBaseline solution is unlikely to have such a shift either (imho)
hm, interesting! I'm not sure then…
", "votes": null, "replies": [] }, { "id": 1333081, "author_name": "wrrosa", "author_url": "", "post_date": "06/02/2021 13:29:59", "content": "Well, after a few tries, I think you're right after all - lever arm offset has been applied.
\nMaybe the position shift is related to the velocity of the receiver and that's why it works 'on average' - interesting clue.
\nRegardless, I'm curious about the hosts response.
Hi Wojtek, could you please point me to the collection where you observed the ground truth shift?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1332723": "Hello competition hosts and kagglers.\nI've got a question about ground truth gps antenna coordinates to hosts @gymf123 \nI've found that shifting baseline location about 0.6m-0.8m improves both baseline train (about 10cm) and public test baseline solution (about 5cm) (published [here](https://www.kaggle.com/wrrosa/gsdc-position-shift)).\nIs this just a coincidence, or should we consider ground truth shift during training and submission?", "1332959": "I tried a similar process.I thought it would improve by 0.6m ~ 0.8m.\nBut it did not result in a significant improvement in my score.(Only 0.004m)\n\nI would like to know if ground truth is shifting too.", "1332965": "I believe they have accounted for the difference between the ground truth antenna and the phones: in the paper (https://www.kaggle.com/google/android-smartphones-high-accuracy-datasets?select=ION+GNSS+2020+Android+Raw+GNSS+Measurement+Datasets+for+Precise+Positioning.pdf) on page 1929 it says: \n\n> The ground truth file has compensated the lever arm offset and is referenced to the smartphone itself.\n\nThey also reference that \"lever arm offset\" in the video as well.\n\nEDIT:\n\nThey also say (in the paper):\n\n> In practice, we have compensated these lever arms. However, several centimeters of errors are inevitable because the lever arms are coarsely measured with regular rulers.\n\nSo there may still be some error there (but it will be potentially different for each collection/phone)", "1332972": "At first glance, this should be mprovement of about 0.6-0.8\nBut, there is a 95% error in the metric, so probably it won't affect large errors.\nSo one would expect an improvement of about 0.3-0.4,\nbut if we do experiments, one more time some measurements will be wrongly shifted. \nOn the other hand, according to the document attached to the dataset, the antenna was placed about 80cm back, which is confirmed by the optimization on the train dataset.\n\nI've also noticed (consistently) that the better quality the results are, the greater the improvement.\nOn the training set (score of 5.28 improved to 5.18)\nOn the public test set (score of 6.164 improved to 6.117 )\nMy best score of 5.350 improved to 5.280\nIdeal submission should be improved from 0.8 to 0 :)", "1332983": "That's what I thought at first.\nBut in the training set, we get an improvement with more than 60 cm backward shift, so aiming high precision it is hard to talk about *some error* here\nBaseline solution is unlikely to have such a shift either (imho)", "1332990": "hm, interesting! I'm not sure then...", "1333081": "Well, after a few tries, I think you're right after all - lever arm offset has been applied.\nMaybe the position shift is related to the velocity of the receiver and that's why it works 'on average' - interesting clue.\nRegardless, I'm curious about the hosts response.", "1333536": "Thanks for the detailed explanation. You are right.\n\nbtw ,I tried running your process on my best score.\nThe score got worse. Score : 5.028 → 5.979\n\n\"The average distance\" you mentioned may have already been solved in my model.\n(Or maybe my code is wrong. lol )", "1354545": "Hi Wojtek, could you please point me to the collection where you observed the ground truth shift?" }, "source": "meta" }