@junkoda has a great notebook that's trying to understand & reproduce how Google has calculated the baseline ECEF positions that are provided to us. This seems important for really getting to grips with the data. Therefore, I'd like to follow up on that from 2 angles.
\nFirstly: @mohammedkhider / @gymf123 - Please can you provide a code sample that reproduces the baseline from the raw data? Failing that, please can you provide a detailed description (down to the raw maths level) of how the baseline is created from the raw data?
\nSecondly: @junkoda - I've seen (on an Android page somewhere that I sadly now can't find again) that there's at least 1 WLS solver that excludes (a) unhealthy satellites and (b) satellites that are less than 5 degrees above the horizon. I wonder if they've done the same here? Perhaps that explains the difference between the baseline results and your notebook.
", "messages": [ { "id": "1793427", "postDate": "05/17/2022 21:53:58", "content": "@junkoda has a great notebook that's trying to understand & reproduce how Google has calculated the baseline ECEF positions that are provided to us. This seems important for really getting to grips with the data. Therefore, I'd like to follow up on that from 2 angles.
\nFirstly: @mohammedkhider / @gymf123 - Please can you provide a code sample that reproduces the baseline from the raw data? Failing that, please can you provide a detailed description (down to the raw maths level) of how the baseline is created from the raw data?
\nSecondly: @junkoda - I've seen (on an Android page somewhere that I sadly now can't find again) that there's at least 1 WLS solver that excludes (a) unhealthy satellites and (b) satellites that are less than 5 degrees above the horizon. I wonder if they've done the same here? Perhaps that explains the difference between the baseline results and your notebook.
", "rawMarkdown": "junkoda has [a great notebook](https://www.kaggle.com/code/junkoda/deriving-baseline-wls-positions-in-progress/notebook) that's trying to understand & reproduce how Google has calculated the baseline ECEF positions that are provided to us. This seems important for really getting to grips with the data. Therefore, I'd like to follow up on that from 2 angles.\n\n**Firstly**: @mohammedkhider / @gymf123 - Please can you provide a code sample that reproduces the baseline from the raw data? Failing that, please can you provide a detailed description (down to the raw maths level) of how the baseline is created from the raw data?\n\n**Secondly**: @junkoda - I've seen (on an Android page somewhere that I sadly now can't find again) that there's at least 1 WLS solver that excludes (a) unhealthy satellites and (b) satellites that are less than 5 degrees above the horizon. I wonder if they've done the same here? Perhaps that explains the difference between the baseline results and your notebook.", "votes": null }, { "id": "1793447", "postDate": "05/17/2022 22:17:46", "content": "Following up on the second point and looking at the specific example you use in your notebook…
\nYou can get the residuals (i.e. per-satellite pseudorange differences from the final position) from the WLS optimizer output (using opt.fun). You'll see that most are <1m off but there's one satellite (with Svid 21) that has a residual of >7m. Excluding this single satellite and re-running the optimizer brings the final error down to 0.35m. I appreciate that still isn't 0, and my choice of excluding this satellite was arbitrary. However, if there's a principled way to know which satellites they've excluded then perhaps we'll be able to reproduce exactly.
Edit: Sure enough, satellite 21 is much lower in the sky. The data we're provided says 6.8 degrees. Sadly this doesn't quite match with elevation calculated (from the provided reference position) using this post - which gives 6.6 degrees. But that's above the 5 degree cut-off I read about. Also, satellite 21 already has a high uncertainty (and therefore low weight) so perhaps this is all a red herring. Shame that there's another calculation I can't reproduce though.
\nI've tried re-running WLS after excluding all satellites below a certain elevation threshold and all satellites below an uncertainty threshold (trying different values of the thresholds). However, I can't reproduce the provided baseline with either of these.
", "rawMarkdown": "Following up on the second point and looking at the specific example you use in your notebook...\n\nYou can get the residuals (i.e. per-satellite pseudorange differences from the final position) from the WLS optimizer output (using `opt.fun`). You'll see that most are <1m off but there's one satellite (with `Svid` 21) that has a residual of >7m. Excluding this single satellite and re-running the optimizer brings the final error down to 0.35m. I appreciate that still isn't 0, and my choice of excluding this satellite was arbitrary. However, if there's a principled way to know which satellites they've excluded then perhaps we'll be able to reproduce exactly.\n\n---\n\nEdit: Sure enough, satellite 21 is much lower in the sky. The data we're provided says 6.8 degrees. Sadly this doesn't quite match with elevation calculated (from the provided reference position) using [this post](https://gis.stackexchange.com/questions/58923/calculating-view-angle) - which gives 6.6 degrees. But that's above the 5 degree cut-off I read about. Also, satellite 21 already has a high uncertainty (and therefore low weight) so perhaps this is all a red herring. Shame that there's another calculation I can't reproduce though.\n\nI've tried re-running WLS after excluding all satellites below a certain elevation threshold and all satellites below an uncertainty threshold (trying different values of the thresholds). However, I can't reproduce the provided baseline with either of these.", "votes": null }, { "id": "1794534", "postDate": "05/18/2022 22:53:59", "content": "Thanks for mentioning my notebook, working on understanding WLS, and checking the satellite elevation hypothesis!
\nI realized that many criteria for invalid measurements in last year's WLS method are the same as the condition for supplemental/rinex.o in the data section. E.g., some data in gnss have low signal to noise: \"CN0 is less than 20 dB-Hz\". I tried to select the same data as rinex.o by finding missing pseudo ranges in this file, but haven't improved the match in position so far. (Also, there is no \"invalid data\" in the first epoch of the first drive.)
\nSelection based on CarrierFrequencyHz improves the match, but not by orders of magnitude
\nfor all epochs in the first drive.
maybe these are useful:
\nhttps://webthesis.biblio.polito.it/11702/1/tesi.pdf
\nsection 2.2 and section 4 tries to explain google code:
\nhttps://github.com/google/gps-measurement-tools/blob/master/GNSSLogger/pseudorange/src/main/java/com/google/location/lbs/gnss/gps/pseudorange/UserPositionVelocityWeightedLeastSquare.java
related:
\nhttps://insidegnss.com/gnss-analysis-tools-from-google/
RawPseudorangeMeters all NaN
\nThere are only 33 NaNs in the entire WlsPositionXEcefMeters in the training set (7 in test), but
\nthere are a lot more epochs that all RawPseudorangeMeters in an epoch (timestep) are NaN. In such cases, I confirmed that WlsPositionXEcefMeters are filled by previous finite values.
Epochs with NaN-only pseudo ranges / total number of epochs
\n787 / 295633 (train)\n104 / 66097 (test)\nThere are a lot of NaNs in RawPseudorangeMeters, but epochs with all NaN pseudo ranges are minor and not a serious problem unless you submit NaN as a prediction.
\nThe filled values are not exactly equal but differ by machine precision ~ 1e-10, which indicates that they are not substituted but recalculated with identical data in a non-deterministic way (e.g., multi-core parallel)
", "rawMarkdown": "**RawPseudorangeMeters all NaN**\n\nThere are only 33 NaNs in the entire `WlsPositionXEcefMeters` in the training set (7 in test), but\nthere are a lot more epochs that all `RawPseudorangeMeters` in an epoch (timestep) are NaN. In such cases, I confirmed that `WlsPositionXEcefMeters` are filled by previous finite values.\n\nEpochs with NaN-only pseudo ranges / total number of epochs\n\n```\n787 / 295633 (train)\n104 / 66097 (test)\n```\n\nThere are a lot of NaNs in RawPseudorangeMeters, but epochs with all NaN pseudo ranges are minor and not a serious problem unless you submit NaN as a prediction.\n\nThe filled values are not exactly equal but differ by machine precision ~ 1e-10, which indicates that they are not substituted but recalculated with identical data in a non-deterministic way (e.g., multi-core parallel)", "votes": null } ], "comments": [ { "id": 1793447, "author_name": "andrewrrose", "author_url": "", "post_date": "05/17/2022 22:17:46", "content": "Following up on the second point and looking at the specific example you use in your notebook…
\nYou can get the residuals (i.e. per-satellite pseudorange differences from the final position) from the WLS optimizer output (using opt.fun). You'll see that most are <1m off but there's one satellite (with Svid 21) that has a residual of >7m. Excluding this single satellite and re-running the optimizer brings the final error down to 0.35m. I appreciate that still isn't 0, and my choice of excluding this satellite was arbitrary. However, if there's a principled way to know which satellites they've excluded then perhaps we'll be able to reproduce exactly.
Edit: Sure enough, satellite 21 is much lower in the sky. The data we're provided says 6.8 degrees. Sadly this doesn't quite match with elevation calculated (from the provided reference position) using this post - which gives 6.6 degrees. But that's above the 5 degree cut-off I read about. Also, satellite 21 already has a high uncertainty (and therefore low weight) so perhaps this is all a red herring. Shame that there's another calculation I can't reproduce though.
\nI've tried re-running WLS after excluding all satellites below a certain elevation threshold and all satellites below an uncertainty threshold (trying different values of the thresholds). However, I can't reproduce the provided baseline with either of these.
", "votes": null, "replies": [] }, { "id": 1794534, "author_name": "junkoda", "author_url": "", "post_date": "05/18/2022 22:53:59", "content": "Thanks for mentioning my notebook, working on understanding WLS, and checking the satellite elevation hypothesis!
\nI realized that many criteria for invalid measurements in last year's WLS method are the same as the condition for supplemental/rinex.o in the data section. E.g., some data in gnss have low signal to noise: \"CN0 is less than 20 dB-Hz\". I tried to select the same data as rinex.o by finding missing pseudo ranges in this file, but haven't improved the match in position so far. (Also, there is no \"invalid data\" in the first epoch of the first drive.)
\nSelection based on CarrierFrequencyHz improves the match, but not by orders of magnitude
\nfor all epochs in the first drive.
maybe these are useful:
\nhttps://webthesis.biblio.polito.it/11702/1/tesi.pdf
\nsection 2.2 and section 4 tries to explain google code:
\nhttps://github.com/google/gps-measurement-tools/blob/master/GNSSLogger/pseudorange/src/main/java/com/google/location/lbs/gnss/gps/pseudorange/UserPositionVelocityWeightedLeastSquare.java
related:
\nhttps://insidegnss.com/gnss-analysis-tools-from-google/
RawPseudorangeMeters all NaN
\nThere are only 33 NaNs in the entire WlsPositionXEcefMeters in the training set (7 in test), but
\nthere are a lot more epochs that all RawPseudorangeMeters in an epoch (timestep) are NaN. In such cases, I confirmed that WlsPositionXEcefMeters are filled by previous finite values.
Epochs with NaN-only pseudo ranges / total number of epochs
\n787 / 295633 (train)\n104 / 66097 (test)\nThere are a lot of NaNs in RawPseudorangeMeters, but epochs with all NaN pseudo ranges are minor and not a serious problem unless you submit NaN as a prediction.
\nThe filled values are not exactly equal but differ by machine precision ~ 1e-10, which indicates that they are not substituted but recalculated with identical data in a non-deterministic way (e.g., multi-core parallel)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1793427": "junkoda has [a great notebook](https://www.kaggle.com/code/junkoda/deriving-baseline-wls-positions-in-progress/notebook) that's trying to understand & reproduce how Google has calculated the baseline ECEF positions that are provided to us. This seems important for really getting to grips with the data. Therefore, I'd like to follow up on that from 2 angles.\n\n**Firstly**: @mohammedkhider / @gymf123 - Please can you provide a code sample that reproduces the baseline from the raw data? Failing that, please can you provide a detailed description (down to the raw maths level) of how the baseline is created from the raw data?\n\n**Secondly**: @junkoda - I've seen (on an Android page somewhere that I sadly now can't find again) that there's at least 1 WLS solver that excludes (a) unhealthy satellites and (b) satellites that are less than 5 degrees above the horizon. I wonder if they've done the same here? Perhaps that explains the difference between the baseline results and your notebook.", "1793447": "Following up on the second point and looking at the specific example you use in your notebook...\n\nYou can get the residuals (i.e. per-satellite pseudorange differences from the final position) from the WLS optimizer output (using `opt.fun`). You'll see that most are <1m off but there's one satellite (with `Svid` 21) that has a residual of >7m. Excluding this single satellite and re-running the optimizer brings the final error down to 0.35m. I appreciate that still isn't 0, and my choice of excluding this satellite was arbitrary. However, if there's a principled way to know which satellites they've excluded then perhaps we'll be able to reproduce exactly.\n\n---\n\nEdit: Sure enough, satellite 21 is much lower in the sky. The data we're provided says 6.8 degrees. Sadly this doesn't quite match with elevation calculated (from the provided reference position) using [this post](https://gis.stackexchange.com/questions/58923/calculating-view-angle) - which gives 6.6 degrees. But that's above the 5 degree cut-off I read about. Also, satellite 21 already has a high uncertainty (and therefore low weight) so perhaps this is all a red herring. Shame that there's another calculation I can't reproduce though.\n\nI've tried re-running WLS after excluding all satellites below a certain elevation threshold and all satellites below an uncertainty threshold (trying different values of the thresholds). However, I can't reproduce the provided baseline with either of these.", "1794534": "Thanks for mentioning my notebook, working on understanding WLS, and checking the satellite elevation hypothesis!\n\nI realized that many criteria for invalid measurements in [last year's WLS method](https://www.kaggle.com/c/google-smartphone-decimeter-challenge/discussion/238583) are the same as the condition for supplemental/rinex.o in the data section. E.g., some data in gnss have low signal to noise: \"CN0 is less than 20 dB-Hz\". I tried to select the same data as rinex.o by finding missing pseudo ranges in this file, but haven't improved the match in position so far. (Also, there is no \"invalid data\" in the first epoch of the first drive.)\n\nSelection based on `CarrierFrequencyHz` improves the match, but not by orders of magnitude \nfor all epochs in the first drive.", "1803936": "maybe these are useful:\n\nhttps://webthesis.biblio.polito.it/11702/1/tesi.pdf\n\nsection 2.2 and section 4 tries to explain google code:\nhttps://github.com/google/gps-measurement-tools/blob/master/GNSSLogger/pseudorange/src/main/java/com/google/location/lbs/gnss/gps/pseudorange/UserPositionVelocityWeightedLeastSquare.java\n\n\nrelated:\nhttps://insidegnss.com/gnss-analysis-tools-from-google/", "1805363": "**RawPseudorangeMeters all NaN**\n\nThere are only 33 NaNs in the entire `WlsPositionXEcefMeters` in the training set (7 in test), but\nthere are a lot more epochs that all `RawPseudorangeMeters` in an epoch (timestep) are NaN. In such cases, I confirmed that `WlsPositionXEcefMeters` are filled by previous finite values.\n\nEpochs with NaN-only pseudo ranges / total number of epochs\n\n```\n787 / 295633 (train)\n104 / 66097 (test)\n```\n\nThere are a lot of NaNs in RawPseudorangeMeters, but epochs with all NaN pseudo ranges are minor and not a serious problem unless you submit NaN as a prediction.\n\nThe filled values are not exactly equal but differ by machine precision ~ 1e-10, which indicates that they are not substituted but recalculated with identical data in a non-deterministic way (e.g., multi-core parallel)" }, "source": "meta" }