{ "id": 332127, "title": "Understanding carrier phase cycle slips - Duty cycling", "url": "/competitions/smartphone-decimeter-2022/discussion/332127", "author_name": "Torey Hilbert", "post_date": "2022-06-20T14:08:07.645000", "votes": 10, "comment_count": 0, "views": 0, "content": "

Hello! Let me preface this by saying that I'm very new to GNSS so take anything I say with a grain of salt - but hopefully this will be helpful for other beginners like me!

\n

If you glance around online, you'll probably notice a lot of people outside of this competition talking about centimeter level precision for GNSS stuff on Android devices, and you might be wondering why people in this competition haven't gotten to below a single meter precision yet? You might have also observed that all of our measurements are exactly one second apart, which seems kind of far. The answer to both of these questions is duty cycling.

\n

Duty cycling, discussed here, is when the phone doesn't run its receivers continuously, but starts them and stops it every second. This is done to save battery, but it comes at a cost. On the one hand, it means we only are receiving information once a second; but more critically, it means that in 30-70% of our measurements, a so-called \"carrier phase cycle slip\" (or, if not, a cycle reset) is happening - see this site for details. The methods that are hitting centimeter level precision involve keeping a lock on the carrier signal and using the very precise carrier phase information - but naturally for a variety of reasons this information will \"slip\" and it'll take some time to calibrate. But when duty cycling is happening, these slips become often enough to severely limit how effective this method is. (Note: Carrier phase data is still useful for this competition, and you can access it with the AccumulatedDeltaRangeMeters variables)

\n

As of right now, duty cycling is the default behavior for Android devices, and it requires developer tools to be enabled to turn it off. As such, the data for this competition reflects the typical use case, even if more precise methods are potentially available on the devices.

\n

Here is some code to reproduce the above summary statistics, modeled after this notebook:

\n
import numpy as np\nimport pandas as pd\nimport glob as gl\n\nbase_path = \"/kaggle/input/smartphone-decimeter-2022\"\n\nfor i, dirname in enumerate(sorted(gl.glob(f\"{base_path}/test/*/*/\"))):\n    drive, phone = dirname.split('/')[-3:-1]\n    tripID = f\"{drive}/{phone}\"\n    print(\"\\nReading\", i, tripID)\n\n    gnss_df = pd.read_csv(f\"{dirname}/device_gnss.csv\")\n\n    gnss_df[\"adr_valid\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**0) != 0\n    gnss_df[\"adr_reset\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**1) != 0\n    gnss_df[\"adr_slip\"]  = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**2) != 0\n\n    print(\"Slip rate\", (~gnss_df[\"adr_valid\"] | gnss_df[\"adr_reset\"] | gnss_df[\"adr_slip\"]).mean())\n\n    gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().value_counts()\n\n    print(\"Number of sub-second samples\",\n        (\n            gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().unique() < 1e9\n        ).sum()\n    )\n
\n

While this probably wasn't new info to most of the people in this competition, it was helpful for me so I figured I'd share it too - have a great day!

", "messages": [ { "id": 1826642, "postDate": "2022-06-20T14:08:07.647Z", "content": "

Hello! Let me preface this by saying that I'm very new to GNSS so take anything I say with a grain of salt - but hopefully this will be helpful for other beginners like me!

\n

If you glance around online, you'll probably notice a lot of people outside of this competition talking about centimeter level precision for GNSS stuff on Android devices, and you might be wondering why people in this competition haven't gotten to below a single meter precision yet? You might have also observed that all of our measurements are exactly one second apart, which seems kind of far. The answer to both of these questions is duty cycling.

\n

Duty cycling, discussed here, is when the phone doesn't run its receivers continuously, but starts them and stops it every second. This is done to save battery, but it comes at a cost. On the one hand, it means we only are receiving information once a second; but more critically, it means that in 30-70% of our measurements, a so-called \"carrier phase cycle slip\" (or, if not, a cycle reset) is happening - see this site for details. The methods that are hitting centimeter level precision involve keeping a lock on the carrier signal and using the very precise carrier phase information - but naturally for a variety of reasons this information will \"slip\" and it'll take some time to calibrate. But when duty cycling is happening, these slips become often enough to severely limit how effective this method is. (Note: Carrier phase data is still useful for this competition, and you can access it with the AccumulatedDeltaRangeMeters variables)

\n

As of right now, duty cycling is the default behavior for Android devices, and it requires developer tools to be enabled to turn it off. As such, the data for this competition reflects the typical use case, even if more precise methods are potentially available on the devices.

\n

Here is some code to reproduce the above summary statistics, modeled after this notebook:

\n
import numpy as np\nimport pandas as pd\nimport glob as gl\n\nbase_path = \"/kaggle/input/smartphone-decimeter-2022\"\n\nfor i, dirname in enumerate(sorted(gl.glob(f\"{base_path}/test/*/*/\"))):\n    drive, phone = dirname.split('/')[-3:-1]\n    tripID = f\"{drive}/{phone}\"\n    print(\"\\nReading\", i, tripID)\n\n    gnss_df = pd.read_csv(f\"{dirname}/device_gnss.csv\")\n\n    gnss_df[\"adr_valid\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**0) != 0\n    gnss_df[\"adr_reset\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**1) != 0\n    gnss_df[\"adr_slip\"]  = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**2) != 0\n\n    print(\"Slip rate\", (~gnss_df[\"adr_valid\"] | gnss_df[\"adr_reset\"] | gnss_df[\"adr_slip\"]).mean())\n\n    gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().value_counts()\n\n    print(\"Number of sub-second samples\",\n        (\n            gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().unique() < 1e9\n        ).sum()\n    )\n
\n

While this probably wasn't new info to most of the people in this competition, it was helpful for me so I figured I'd share it too - have a great day!

", "rawMarkdown": "Hello! Let me preface this by saying that I'm very new to GNSS so take anything I say with a grain of salt - but hopefully this will be helpful for other beginners like me!\n\nIf you glance around online, you'll probably notice a lot of people outside of this competition talking about centimeter level precision for GNSS stuff on Android devices, and you might be wondering why people in this competition haven't gotten to below a single meter precision yet? You might have also observed that all of our measurements are exactly one second apart, which seems kind of far. The answer to both of these questions is duty cycling.\n\nDuty cycling, discussed [here](https://youtu.be/vywGgSrGODU?t=1758), is when the phone doesn't run its receivers continuously, but starts them and stops it every second. This is done to save battery, but it comes at a cost. On the one hand, it means we only are receiving information once a second; but more critically, it means that in 30-70% of our measurements, a so-called \"carrier phase cycle slip\" (or, if not, a cycle reset) is happening - see [this site](https://gssc.esa.int/navipedia/index.php/GNSS_Basic_Observables#Carrier_phase) for details. The methods that are hitting centimeter level precision involve keeping a lock on the carrier signal and using the very precise carrier phase information - but naturally for a variety of reasons this information will \"slip\" and it'll take some time to calibrate. But when duty cycling is happening, these slips become often enough to severely limit how effective this method is. (Note: Carrier phase data is still useful for this competition, and you can access it with the `AccumulatedDeltaRangeMeters` variables)\n\nAs of right now, duty cycling is the default behavior for Android devices, and it requires developer tools to be enabled to turn it off. As such, the data for this competition reflects the typical use case, even if more precise methods are potentially available on the devices.\n\nHere is some code to reproduce the above summary statistics, modeled after [this notebook](https://www.kaggle.com/code/taroz1461/carrier-smoothing-robust-wls-kalman-smoother):\n```python\nimport numpy as np\nimport pandas as pd\nimport glob as gl\n\nbase_path = \"/kaggle/input/smartphone-decimeter-2022\"\n\nfor i, dirname in enumerate(sorted(gl.glob(f\"{base_path}/test/*/*/\"))):\n drive, phone = dirname.split('/')[-3:-1]\n tripID = f\"{drive}/{phone}\"\n print(\"\\nReading\", i, tripID)\n\n gnss_df = pd.read_csv(f\"{dirname}/device_gnss.csv\")\n \n gnss_df[\"adr_valid\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**0) != 0\n gnss_df[\"adr_reset\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**1) != 0\n gnss_df[\"adr_slip\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**2) != 0\n\n print(\"Slip rate\", (~gnss_df[\"adr_valid\"] | gnss_df[\"adr_reset\"] | gnss_df[\"adr_slip\"]).mean())\n \n gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().value_counts()\n \n print(\"Number of sub-second samples\",\n (\n gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().unique() < 1e9\n ).sum()\n )\n```\n\nWhile this probably wasn't new info to most of the people in this competition, it was helpful for me so I figured I'd share it too - have a great day!", "votes": 10 } ], "comments": [], "raw_markdown_by_id": { "1826642": "Hello! Let me preface this by saying that I'm very new to GNSS so take anything I say with a grain of salt - but hopefully this will be helpful for other beginners like me!\n\nIf you glance around online, you'll probably notice a lot of people outside of this competition talking about centimeter level precision for GNSS stuff on Android devices, and you might be wondering why people in this competition haven't gotten to below a single meter precision yet? You might have also observed that all of our measurements are exactly one second apart, which seems kind of far. The answer to both of these questions is duty cycling.\n\nDuty cycling, discussed [here](https://youtu.be/vywGgSrGODU?t=1758), is when the phone doesn't run its receivers continuously, but starts them and stops it every second. This is done to save battery, but it comes at a cost. On the one hand, it means we only are receiving information once a second; but more critically, it means that in 30-70% of our measurements, a so-called \"carrier phase cycle slip\" (or, if not, a cycle reset) is happening - see [this site](https://gssc.esa.int/navipedia/index.php/GNSS_Basic_Observables#Carrier_phase) for details. The methods that are hitting centimeter level precision involve keeping a lock on the carrier signal and using the very precise carrier phase information - but naturally for a variety of reasons this information will \"slip\" and it'll take some time to calibrate. But when duty cycling is happening, these slips become often enough to severely limit how effective this method is. (Note: Carrier phase data is still useful for this competition, and you can access it with the `AccumulatedDeltaRangeMeters` variables)\n\nAs of right now, duty cycling is the default behavior for Android devices, and it requires developer tools to be enabled to turn it off. As such, the data for this competition reflects the typical use case, even if more precise methods are potentially available on the devices.\n\nHere is some code to reproduce the above summary statistics, modeled after [this notebook](https://www.kaggle.com/code/taroz1461/carrier-smoothing-robust-wls-kalman-smoother):\n```python\nimport numpy as np\nimport pandas as pd\nimport glob as gl\n\nbase_path = \"/kaggle/input/smartphone-decimeter-2022\"\n\nfor i, dirname in enumerate(sorted(gl.glob(f\"{base_path}/test/*/*/\"))):\n drive, phone = dirname.split('/')[-3:-1]\n tripID = f\"{drive}/{phone}\"\n print(\"\\nReading\", i, tripID)\n\n gnss_df = pd.read_csv(f\"{dirname}/device_gnss.csv\")\n \n gnss_df[\"adr_valid\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**0) != 0\n gnss_df[\"adr_reset\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**1) != 0\n gnss_df[\"adr_slip\"] = (gnss_df[\"AccumulatedDeltaRangeState\"] & 2**2) != 0\n\n print(\"Slip rate\", (~gnss_df[\"adr_valid\"] | gnss_df[\"adr_reset\"] | gnss_df[\"adr_slip\"]).mean())\n \n gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().value_counts()\n \n print(\"Number of sub-second samples\",\n (\n gnss_df.groupby([\"Svid\", \"SignalType\"])[\"TimeNanos\"].diff().unique() < 1e9\n ).sum()\n )\n```\n\nWhile this probably wasn't new info to most of the people in this competition, it was helpful for me so I figured I'd share it too - have a great day!" } }