Thanks to the hosts for this competition!
\nI knew nothing really about GPS prior to this competition, so it was a lot of fun (though also often frustrating!) learning all about it.
\nOur team had three main parts to the solution:
\nMy primary focus was to improve #1 - the baseline lat/long points and a delta lat/long from one point to the next using the raw GPS information.
\nFirst, I'd like to say that there were a lot of pitfalls and dangers in the data! It was rather difficult (since I'm not a GPS expert at all) to really understand the data, and even after I understood it, the data was often a bit messy (with missing or skipped values), and it was tricky to really get everything working.
\nAlso, can I just say: converting between UTC and GPS time is no fun! 😂 Add in the rotation of the earth between the signal transmit and receive time, converting between ECEF, ENU, lat/long, and that all meant a lot of debugging sessions trying to figure out exactly where I went wrong…
\nHere are several things I did that worked, and some that didn't.
\nI used the same filtering criteria as the host's baseline guidelines: https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/238583
\nThat seemed to work fairly well to remove most pseudo ranges that were obviously bad
\nThe _derived files contained satellite x,y,z positions, but the contest also included satellite correction information from SwiftNav. I was able to apply the SwiftNav corrections to more accurate satellite x,y,z positions, but that ultimately didn't have much effect on the GPS baseline (because the given positions were already pretty close)
\nTo smooth the very jumpy pseudo ranges, I used AccumulatedDeltaRangeMeters to calculate a delta pseudo range for every satellite for every point. Then, I could calculate a smoothed, or average pseudo range for each point and use that.
\nThis was fairly effective, since the AccumulatedDeltaRangeMeters is much more accurate than the raw pseudo range measurement
\nWhenever the carrier phase cycled (which was often - especially in SJC), I used PseudorangeRateMetersPerSecond to approximate the AccumulatedDeltaRangeMeters between points. This worked well for a few seconds at a time, but more than 10 or so seconds became a problem, since PseudorangeRateMetersPerSecond error rates were higher than AccumulatedDeltaRangeMeters.
\nAlso using doppler, I was able to calculate a relative, or delta lat/long between each point by taking the difference between the measured pseudo ranges and the expected difference given by either the doppler or carrier phase.
\nThat delta lat/long measurement was later used in post processing.
\nI noticed that the GPS and GAL satellites were more accurate for most phones, especially when surrounded by tall buildings (I'm not 100% sure of the reason behind that).
\nTo account for that I tried a variety of things, from excluding other satellites entirely, to just multiplying the uncertainty by a factor of 2, 4, or 10. That seemed to work well for this dataset, but may not be valid for other geographies.
\nOnce we had some good post processed data points, I also increased the uncertainty for pseudo ranges that were far outside of that range.
\nI also assumed that pseudo ranges that were too \"long\" (compared to the current best known location) had a higher chance of being affected by multipath (because they could have reflected off of buildings or cars). (Not being a GPS expert, I actually don't know if this is correct at all :) but it was a guess).
\nTo try to account for that, I experimented with removing the top X% of pseudo ranges (when sorted by expected residual error), or increasing their uncertainty. Removing them actually improved downtown areas quite a bit.
\nI tried other off the wall ideas to reduce the impact of really bad pseudo ranges, some of which had a small effect, but none of which were very noticeable.
\nFor example: I tried taking both the square root and the log of the (measured - pseudo range) value in an attempt to reduce the large errors (before putting it into least squares). That actually DID work, but the result was a bit inconsistent between collections, and wasn't a very large effect on most.
\nI do think there is more work that could be done there though.
\nIf there was one thing I would have liked to have spent more time on, it would be differential or relative positioning based on known GPS reference points. I tried this several times - both with the given SwiftNav reference point and others, but I just couldn't get it to work.
\nPart of the problem is the gps receivers in the phones were just very noisy - it seemed that a lot (if not MOST) of the error came from receiver noise, which means both differential and relative positioning (including double difference and triple difference) would actually INCREASE that error (if I understand correctly).
\nAnother rather annoying thing was that the phone receivers were recording on a fraction of a second (e.g., 0.44) rather than right on the GPS second (like the reference receivers). That made it much more difficult to even attempt differential or relative positioning.
\nI was never able to get this working completely, and I think it's probably the biggest thing I could have done differently.
\nOne thing that I did get working but that didn't actually improve the baseline like I expected was doing relative GPS positioning with each phone to itself - that is, to other positions in that phone's path.
\nI was assuming (hoping) that the receiver error was approximately normally distributed around the correct answer, and thought that doing relative positioning with (for example), 10 seconds of other points around each point would even out that error.
\nI kind of got this working, but I could never get it to produce the results I thought it should… maybe if there's a new GPS contest I can figure out why that is :)
\nI really wanted to incorporate the IMU data into the pseudo range averaging (with a Kalman filter, or even just a regular weighted average) - but I just ran out of time.
\nI think this would have greatly reduced the receiver noise that I saw, which should mean a much better baseline solution.
\nI didn't find any magic bullets in the GPS baseline calculations
\nIt took a LONG time to really figure out all the data, and what I was doing
\nThe data was pretty noisy, and a bit difficult to work with - but I think a lot of that was because I've never worked with GPS data before.
\nI'd love to give it another try (now that I actually know what I'm doing) if there's a new competition sometime!
", "messages": [ { "id": 1449818, "postDate": "2021-08-05T01:01:37.413Z", "content": "Thanks to the hosts for this competition!
\nI knew nothing really about GPS prior to this competition, so it was a lot of fun (though also often frustrating!) learning all about it.
\nOur team had three main parts to the solution:
\nMy primary focus was to improve #1 - the baseline lat/long points and a delta lat/long from one point to the next using the raw GPS information.
\nFirst, I'd like to say that there were a lot of pitfalls and dangers in the data! It was rather difficult (since I'm not a GPS expert at all) to really understand the data, and even after I understood it, the data was often a bit messy (with missing or skipped values), and it was tricky to really get everything working.
\nAlso, can I just say: converting between UTC and GPS time is no fun! 😂 Add in the rotation of the earth between the signal transmit and receive time, converting between ECEF, ENU, lat/long, and that all meant a lot of debugging sessions trying to figure out exactly where I went wrong…
\nHere are several things I did that worked, and some that didn't.
\nI used the same filtering criteria as the host's baseline guidelines: https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/238583
\nThat seemed to work fairly well to remove most pseudo ranges that were obviously bad
\nThe _derived files contained satellite x,y,z positions, but the contest also included satellite correction information from SwiftNav. I was able to apply the SwiftNav corrections to more accurate satellite x,y,z positions, but that ultimately didn't have much effect on the GPS baseline (because the given positions were already pretty close)
\nTo smooth the very jumpy pseudo ranges, I used AccumulatedDeltaRangeMeters to calculate a delta pseudo range for every satellite for every point. Then, I could calculate a smoothed, or average pseudo range for each point and use that.
\nThis was fairly effective, since the AccumulatedDeltaRangeMeters is much more accurate than the raw pseudo range measurement
\nWhenever the carrier phase cycled (which was often - especially in SJC), I used PseudorangeRateMetersPerSecond to approximate the AccumulatedDeltaRangeMeters between points. This worked well for a few seconds at a time, but more than 10 or so seconds became a problem, since PseudorangeRateMetersPerSecond error rates were higher than AccumulatedDeltaRangeMeters.
\nAlso using doppler, I was able to calculate a relative, or delta lat/long between each point by taking the difference between the measured pseudo ranges and the expected difference given by either the doppler or carrier phase.
\nThat delta lat/long measurement was later used in post processing.
\nI noticed that the GPS and GAL satellites were more accurate for most phones, especially when surrounded by tall buildings (I'm not 100% sure of the reason behind that).
\nTo account for that I tried a variety of things, from excluding other satellites entirely, to just multiplying the uncertainty by a factor of 2, 4, or 10. That seemed to work well for this dataset, but may not be valid for other geographies.
\nOnce we had some good post processed data points, I also increased the uncertainty for pseudo ranges that were far outside of that range.
\nI also assumed that pseudo ranges that were too \"long\" (compared to the current best known location) had a higher chance of being affected by multipath (because they could have reflected off of buildings or cars). (Not being a GPS expert, I actually don't know if this is correct at all :) but it was a guess).
\nTo try to account for that, I experimented with removing the top X% of pseudo ranges (when sorted by expected residual error), or increasing their uncertainty. Removing them actually improved downtown areas quite a bit.
\nI tried other off the wall ideas to reduce the impact of really bad pseudo ranges, some of which had a small effect, but none of which were very noticeable.
\nFor example: I tried taking both the square root and the log of the (measured - pseudo range) value in an attempt to reduce the large errors (before putting it into least squares). That actually DID work, but the result was a bit inconsistent between collections, and wasn't a very large effect on most.
\nI do think there is more work that could be done there though.
\nIf there was one thing I would have liked to have spent more time on, it would be differential or relative positioning based on known GPS reference points. I tried this several times - both with the given SwiftNav reference point and others, but I just couldn't get it to work.
\nPart of the problem is the gps receivers in the phones were just very noisy - it seemed that a lot (if not MOST) of the error came from receiver noise, which means both differential and relative positioning (including double difference and triple difference) would actually INCREASE that error (if I understand correctly).
\nAnother rather annoying thing was that the phone receivers were recording on a fraction of a second (e.g., 0.44) rather than right on the GPS second (like the reference receivers). That made it much more difficult to even attempt differential or relative positioning.
\nI was never able to get this working completely, and I think it's probably the biggest thing I could have done differently.
\nOne thing that I did get working but that didn't actually improve the baseline like I expected was doing relative GPS positioning with each phone to itself - that is, to other positions in that phone's path.
\nI was assuming (hoping) that the receiver error was approximately normally distributed around the correct answer, and thought that doing relative positioning with (for example), 10 seconds of other points around each point would even out that error.
\nI kind of got this working, but I could never get it to produce the results I thought it should… maybe if there's a new GPS contest I can figure out why that is :)
\nI really wanted to incorporate the IMU data into the pseudo range averaging (with a Kalman filter, or even just a regular weighted average) - but I just ran out of time.
\nI think this would have greatly reduced the receiver noise that I saw, which should mean a much better baseline solution.
\nI didn't find any magic bullets in the GPS baseline calculations
\nIt took a LONG time to really figure out all the data, and what I was doing
\nThe data was pretty noisy, and a bit difficult to work with - but I think a lot of that was because I've never worked with GPS data before.
\nI'd love to give it another try (now that I actually know what I'm doing) if there's a new competition sometime!
", "rawMarkdown": "Thanks to the hosts for this competition!\n\nI knew nothing really about GPS prior to this competition, so it was a lot of fun (though also often frustrating!) learning all about it.\n\n\n\nOur team had three main parts to the solution:\n\n1. GPS baseline and relative position\n2. Post-processing\n3. ensembling / averaging\n\nMy primary focus was to improve #1 - the baseline lat/long points and a delta lat/long from one point to the next using the raw GPS information.\n\nFirst, I'd like to say that there were a lot of pitfalls and dangers in the data! It was rather difficult (since I'm not a GPS expert at all) to really understand the data, and even after I understood it, the data was often a bit messy (with missing or skipped values), and it was tricky to really get everything working.\n\nAlso, can I just say: converting between UTC and GPS time is no fun! 😂 Add in the rotation of the earth between the signal transmit and receive time, converting between ECEF, ENU, lat/long, and that all meant a lot of debugging sessions trying to figure out exactly where I went wrong... \n\n\nHere are several things I did that worked, and some that didn't.\n\n\n# Filter out obviously bad pseudo ranges\n\nI used the same filtering criteria as the host's baseline guidelines: https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/238583\n\nThat seemed to work fairly well to remove most pseudo ranges that were obviously bad\n\n\n\n# Improve Satellite x,y,z positions\n\nThe _derived files contained satellite x,y,z positions, but the contest also included satellite correction information from SwiftNav. I was able to apply the SwiftNav corrections to more accurate satellite x,y,z positions, but that ultimately didn't have much effect on the GPS baseline (because the given positions were already pretty close)\n\n\n# Using carrier phase / AccumulatedDeltaRangeMeters\n\nTo smooth the very jumpy pseudo ranges, I used AccumulatedDeltaRangeMeters to calculate a delta pseudo range for every satellite for every point. Then, I could calculate a smoothed, or average pseudo range for each point and use that.\n\nThis was fairly effective, since the AccumulatedDeltaRangeMeters is much more accurate than the raw pseudo range measurement\n\n\n# Using doppler / PseudorangeRateMetersPerSecond\n\nWhenever the carrier phase cycled (which was often - especially in SJC), I used PseudorangeRateMetersPerSecond to approximate the AccumulatedDeltaRangeMeters between points. This worked well for a few seconds at a time, but more than 10 or so seconds became a problem, since PseudorangeRateMetersPerSecond error rates were higher than AccumulatedDeltaRangeMeters.\n\nAlso using doppler, I was able to calculate a relative, or delta lat/long between each point by taking the difference between the measured pseudo ranges and the expected difference given by either the doppler or carrier phase.\n\nThat delta lat/long measurement was later used in post processing.\n\n\n# Adjusting the satellite uncertainty\n\nI noticed that the GPS and GAL satellites were more accurate for most phones, especially when surrounded by tall buildings (I'm not 100% sure of the reason behind that).\n\nTo account for that I tried a variety of things, from excluding other satellites entirely, to just multiplying the uncertainty by a factor of 2, 4, or 10. That seemed to work well for this dataset, but may not be valid for other geographies.\n\nOnce we had some good post processed data points, I also increased the uncertainty for pseudo ranges that were far outside of that range. \n\n\n# Reducing multipath\n\nI also assumed that pseudo ranges that were too \"long\" (compared to the current best known location) had a higher chance of being affected by multipath (because they could have reflected off of buildings or cars). (Not being a GPS expert, I actually don't know if this is correct at all :) but it was a guess).\n\nTo try to account for that, I experimented with removing the top X% of pseudo ranges (when sorted by expected residual error), or increasing their uncertainty. Removing them actually improved downtown areas quite a bit.\n\n\n# Other ways to reduce the impact of bad pseudo ranges\n\nI tried other off the wall ideas to reduce the impact of _really_ bad pseudo ranges, some of which had a small effect, but none of which were very noticeable. \n\nFor example: I tried taking both the square root and the log of the (measured - pseudo range) value in an attempt to reduce the large errors (before putting it into least squares). That actually DID work, but the result was a bit inconsistent between collections, and wasn't a very large effect on most. \n\nI do think there is more work that could be done there though.\n\n\n\n# Differential or relative positioning\n\nIf there was one thing I would have liked to have spent more time on, it would be differential or relative positioning based on known GPS reference points. I tried this several times - both with the given SwiftNav reference point and others, but I just couldn't get it to work.\n\nPart of the problem is the gps receivers in the phones were just very noisy - it seemed that a lot (if not MOST) of the error came from receiver noise, which means both differential and relative positioning (including double difference and triple difference) would actually INCREASE that error (if I understand correctly).\n\nAnother rather annoying thing was that the phone receivers were recording on a fraction of a second (e.g., 0.44) rather than right on the GPS second (like the reference receivers). That made it much more difficult to even attempt differential or relative positioning.\n\nI was never able to get this working completely, and I think it's probably the biggest thing I could have done differently.\n\n\n\n# \"self\" relative positioning\n\nOne thing that I did get working but that didn't actually improve the baseline like I expected was doing relative GPS positioning with each phone to itself - that is, to other positions in that phone's path.\n\nI was assuming (hoping) that the receiver error was approximately normally distributed around the correct answer, and thought that doing relative positioning with (for example), 10 seconds of other points around each point would even out that error.\n\nI _kind of_ got this working, but I could never get it to produce the results I thought it should... maybe if there's a new GPS contest I can figure out why that is :)\n\n\n# Incorporating IMU data\n\nI really wanted to incorporate the IMU data into the pseudo range averaging (with a Kalman filter, or even just a regular weighted average) - but I just ran out of time.\n\nI think this would have greatly reduced the receiver noise that I saw, which should mean a much better baseline solution.\n\n\n# Overall\n\nI didn't find any magic bullets in the GPS baseline calculations\n\nIt took a LONG time to really figure out all the data, and what I was doing\n\nThe data was pretty noisy, and a bit difficult to work with - but I think a lot of that was because I've never worked with GPS data before.\n\nI'd love to give it another try (now that I actually know what I'm doing) if there's a new competition sometime!\n", "votes": 33 } ], "comments": [], "raw_markdown_by_id": { "1449818": "Thanks to the hosts for this competition!\n\nI knew nothing really about GPS prior to this competition, so it was a lot of fun (though also often frustrating!) learning all about it.\n\n\n\nOur team had three main parts to the solution:\n\n1. GPS baseline and relative position\n2. Post-processing\n3. ensembling / averaging\n\nMy primary focus was to improve #1 - the baseline lat/long points and a delta lat/long from one point to the next using the raw GPS information.\n\nFirst, I'd like to say that there were a lot of pitfalls and dangers in the data! It was rather difficult (since I'm not a GPS expert at all) to really understand the data, and even after I understood it, the data was often a bit messy (with missing or skipped values), and it was tricky to really get everything working.\n\nAlso, can I just say: converting between UTC and GPS time is no fun! 😂 Add in the rotation of the earth between the signal transmit and receive time, converting between ECEF, ENU, lat/long, and that all meant a lot of debugging sessions trying to figure out exactly where I went wrong... \n\n\nHere are several things I did that worked, and some that didn't.\n\n\n# Filter out obviously bad pseudo ranges\n\nI used the same filtering criteria as the host's baseline guidelines: https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/238583\n\nThat seemed to work fairly well to remove most pseudo ranges that were obviously bad\n\n\n\n# Improve Satellite x,y,z positions\n\nThe _derived files contained satellite x,y,z positions, but the contest also included satellite correction information from SwiftNav. I was able to apply the SwiftNav corrections to more accurate satellite x,y,z positions, but that ultimately didn't have much effect on the GPS baseline (because the given positions were already pretty close)\n\n\n# Using carrier phase / AccumulatedDeltaRangeMeters\n\nTo smooth the very jumpy pseudo ranges, I used AccumulatedDeltaRangeMeters to calculate a delta pseudo range for every satellite for every point. Then, I could calculate a smoothed, or average pseudo range for each point and use that.\n\nThis was fairly effective, since the AccumulatedDeltaRangeMeters is much more accurate than the raw pseudo range measurement\n\n\n# Using doppler / PseudorangeRateMetersPerSecond\n\nWhenever the carrier phase cycled (which was often - especially in SJC), I used PseudorangeRateMetersPerSecond to approximate the AccumulatedDeltaRangeMeters between points. This worked well for a few seconds at a time, but more than 10 or so seconds became a problem, since PseudorangeRateMetersPerSecond error rates were higher than AccumulatedDeltaRangeMeters.\n\nAlso using doppler, I was able to calculate a relative, or delta lat/long between each point by taking the difference between the measured pseudo ranges and the expected difference given by either the doppler or carrier phase.\n\nThat delta lat/long measurement was later used in post processing.\n\n\n# Adjusting the satellite uncertainty\n\nI noticed that the GPS and GAL satellites were more accurate for most phones, especially when surrounded by tall buildings (I'm not 100% sure of the reason behind that).\n\nTo account for that I tried a variety of things, from excluding other satellites entirely, to just multiplying the uncertainty by a factor of 2, 4, or 10. That seemed to work well for this dataset, but may not be valid for other geographies.\n\nOnce we had some good post processed data points, I also increased the uncertainty for pseudo ranges that were far outside of that range. \n\n\n# Reducing multipath\n\nI also assumed that pseudo ranges that were too \"long\" (compared to the current best known location) had a higher chance of being affected by multipath (because they could have reflected off of buildings or cars). (Not being a GPS expert, I actually don't know if this is correct at all :) but it was a guess).\n\nTo try to account for that, I experimented with removing the top X% of pseudo ranges (when sorted by expected residual error), or increasing their uncertainty. Removing them actually improved downtown areas quite a bit.\n\n\n# Other ways to reduce the impact of bad pseudo ranges\n\nI tried other off the wall ideas to reduce the impact of _really_ bad pseudo ranges, some of which had a small effect, but none of which were very noticeable. \n\nFor example: I tried taking both the square root and the log of the (measured - pseudo range) value in an attempt to reduce the large errors (before putting it into least squares). That actually DID work, but the result was a bit inconsistent between collections, and wasn't a very large effect on most. \n\nI do think there is more work that could be done there though.\n\n\n\n# Differential or relative positioning\n\nIf there was one thing I would have liked to have spent more time on, it would be differential or relative positioning based on known GPS reference points. I tried this several times - both with the given SwiftNav reference point and others, but I just couldn't get it to work.\n\nPart of the problem is the gps receivers in the phones were just very noisy - it seemed that a lot (if not MOST) of the error came from receiver noise, which means both differential and relative positioning (including double difference and triple difference) would actually INCREASE that error (if I understand correctly).\n\nAnother rather annoying thing was that the phone receivers were recording on a fraction of a second (e.g., 0.44) rather than right on the GPS second (like the reference receivers). That made it much more difficult to even attempt differential or relative positioning.\n\nI was never able to get this working completely, and I think it's probably the biggest thing I could have done differently.\n\n\n\n# \"self\" relative positioning\n\nOne thing that I did get working but that didn't actually improve the baseline like I expected was doing relative GPS positioning with each phone to itself - that is, to other positions in that phone's path.\n\nI was assuming (hoping) that the receiver error was approximately normally distributed around the correct answer, and thought that doing relative positioning with (for example), 10 seconds of other points around each point would even out that error.\n\nI _kind of_ got this working, but I could never get it to produce the results I thought it should... maybe if there's a new GPS contest I can figure out why that is :)\n\n\n# Incorporating IMU data\n\nI really wanted to incorporate the IMU data into the pseudo range averaging (with a Kalman filter, or even just a regular weighted average) - but I just ran out of time.\n\nI think this would have greatly reduced the receiver noise that I saw, which should mean a much better baseline solution.\n\n\n# Overall\n\nI didn't find any magic bullets in the GPS baseline calculations\n\nIt took a LONG time to really figure out all the data, and what I was doing\n\nThe data was pretty noisy, and a bit difficult to work with - but I think a lot of that was because I've never worked with GPS data before.\n\nI'd love to give it another try (now that I actually know what I'm doing) if there's a new competition sometime!\n" } }