{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Introduction:**\n\nThis notebook is really an addendum to my previous notebook (Getting Started with RTKLIB).  It includes the additional steps required to run GNSS PPK solutions for the raw data sets using rtklib-py, a python version of RTKLIB, instead of the original C version.  Like the previous notebook, it is not intended to be run here, but downloaded to a local computer and run there.  I ran this code in Windows, but with minor modificatons it should run in Linux as well.\n\nThis code is primarily intended for those planning to develop or modify alogrithms internal to the PPK solutions and not just running the code as-is or with just configuration changes. For those users, the C code will run much faster.  However, the python version provides a friendlier development platform.  When development is complete, the modified python code can either be run on the complete data set on a faster PC with a little patience, or the completed changes can be fairly easily ported back into the C code since two code sets are very closely aligned.  This alignment includes file names, function names, variable names, and comments.  The code does not align on a line by line basis because of extensive use of Numpy in the python code, but structurally it is very similar.\n\nWhen following these intructions, it will be easier if you use the folder structure as described in the previous notebook.\n","metadata":{}},{"cell_type":"markdown","source":"**Step 1: Retrieve base observation and satellite navigation files**\n\nThis step is identical as step 1 for the RTKLIB code, so please follow the directions in the previous notebook for this step.","metadata":{}},{"cell_type":"markdown","source":"**Step 2: Converting base observation files from rinex2 to rinex3 format**\n\nRtklib-py is only a subset of RTKLIB and not all features are supported.  We will need to use RTKLIB to convert the base station observations from rinex2 format which rtklib-py does not support, to rinex3 format, which it does.\n\nThe following code will do that.  In addition, this code filters out all the signal types in the original files that are not necessary for this exercise.\n\nBefore running the code you will need to retrieve the RTKLIB convbin executable and place it into the GSDC_2022/rtklib folder.  Follow the instructions in the previous notebook to do this.","metadata":{}},{"cell_type":"code","source":"\"\"\" \nrinex2_to_rinex3.py - Convert rinex 2 obs files to rinex 3\n\"\"\"\n\nimport os\nfrom os.path import join\nimport subprocess\n\n# specify input folder and files\ndatadir = r'C:\\gps\\GSDC_2022\\data\\test'\nobsfiles = '*0.2*o' # base obs files with wild cards\nsigmask = 'GL1C,GL5X,RL1C,EL1X,EL5X'\n\n# Setup for RTKLIB \nbinpath  = r'C:\\gps\\GSDC_2022\\rtklib\\convbin'\n\n\n# get list of data sets in data path\ndatasets = os.listdir(datadir)\n    \nfor dataset in datasets:\n    try:\n        os.chdir(join(datadir, dataset))\n    except:\n        continue\n    print(dataset)\n    rtkcmd = '%s %s -r rinex -mask %s -od -os -o base.obs' % \\\n        (binpath, obsfiles, sigmask)\n    subprocess.run(rtkcmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Step 3: Download the android-rinex and rklib-py libraries and create rtklib-py configuration file:**\n\nDownload the android-rinex and rtklib-py libraries using the code below.  Place them into the GSDC_2022/python/android-rinex folder and the GSDC_2022/python/rtklib-py folder.\n\nYou will also need a configuration file for the solution. Copy the config file below to the GSDC_2022/config folder with a file name of ppk_phone_0510.py","metadata":{}},{"cell_type":"code","source":"!git clone https://github.com/rtklibexplorer/android_rinex.git\n!git clone https://github.com/rtklibexplorer/rtklib-py.git","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ppk_phone_0510.py - config file for rtklib-py PPK solution\n\nfrom rtkcmn import uGNSS, rSIG\n\n# ----------- PPK options -------------------------------\nnf = 2                   # num frequencies ( 1 or 2)\npmode = 'kinematic'      # static, kinematic\nfiltertype = 'combined_noreset'  # forward, backward, combined, combined_noreset\nuse_sing_pos = False     # run initial single precision sol each epoch, not\n                         # necessary unless receiever clock errors are large\nelmin = 15               # minimum elevation for float solution (degrees)\ncnr_min = 24             # min signal strength (dB-Hz)\nexcsats = []             # excluded sats\n\nmaxinno = 1              # outlier threshold for phase (m)\nmaxage = 30              # mag age of differential\nmaxout = 4               # maximum outage [epoch]\nthresdop = 5             # cycle slip detection by doppler method\nthresslip = 0.10         # cycle slip detection by geom-free LC\ninterp_base = False      # interpolate base observations\n\n# ------------  Kalman Filter Statistics ------------------------\neratio = [300, 100]    # ratio between psuedorange noise and carrier phase noise for L1, L5\nefact = {uGNSS.GPS: 1.0, uGNSS.GLO: 1.5, uGNSS.GAL: 1.0} # relative weighting of each constellation\nerr = [0, 0.003, 0.003, 0.0, 0, 5e-12]  # err sigmas [-, base, el, rcvstd, bl, satclk]\n\naccelh = 3               # horiz accel noise sigma (m/sec2)\naccelv = 1               # vert accel noise sigma (m/sec2)\nprnbias = 1e-2           # Carrier phase bias sigma ( cycles)\nsig_p0 = 30.0            # initial pos sigma (m)\nsig_v0 = 10.0            # initial vel/acc sigma (m/sec)\nsig_n0 = 30.0            # inital bias sigma (m)\n\n#  -------------Ambiguity resolution options ----------------\narmode = 0               # 0:off, 1:continuos,3:fix-and-hold\nthresar1 = 0.05           # max pos variation for AR (and accel update in kalman filter)\n# the rest of these aren't used since AR is disabled\nminlock = 0              # min consecutive fix samples to include sat in AR \nthresar = 3              # AR threshold\nglo_hwbias = 0.0         # GLONASS HW bias\nelmaskar = 15            # elevation mask for AR\nvar_holdamb = 0.1        # Hold ambiguity variance (m)\nminfix = 20              # min fix samples to set hold\nminfixsats = 4           # min sat pairs to test for fix\nminholdsats = 5          # min sat pairs to test for hold\nmindropsats = 10         # min sat pairs to drop sats from AR\n\n# -----------  Single precision parameters ----------------------------\nsing_p0 = 100            # initial pos sigma\nsing_v0 = 10             # initial vel/acc sigma\nsing_elmin = 10          # minimum elevation (degrees)\n\n# -------------  Base and Rover positions ------------------\n# base position, set to zeros to use rinex header pos\n#rb = [-2703115.9211, -4291767.2078, 3854247.9065] # SLAC\nrb = [0, 0, 0]  # will set based on base file name\n\n# initial rover position/velocity for alignment to RTKLIB\n#rr_f = [-2694556.682853,  -4296492.691977,   3854819.048563, # forwards\n#        -0.009033 ,       0.042808,         -0.027949] \n#rr_b = [-2709836.020965, -4269097.509128, 3874376.664473, # backwards\n#      0.018097,      0.051379,     -0.027851 ]\n# Set to zero to use standard precision computed starting position\nrr_f = rr_b  = [0, 0, 0]\n\n\n# ----------- Configure observation signals ----------------\ngnss_t = [uGNSS.GPS, uGNSS.GLO, uGNSS.GAL]\n\nsig_tbl = {'1C': rSIG.L1C, '1X': rSIG.L1X, '1W': rSIG.L1W,\n           '2W': rSIG.L2W, '2L': rSIG.L2L, '2X': rSIG.L2X,\n           '5Q': rSIG.L5Q, '5X': rSIG.L5X, '7Q': rSIG.L7Q,\n           '7X': rSIG.L7X}\nskip_sig_tbl = {uGNSS.GPS: [],   # skip these obs\n                uGNSS.GLO: [],\n                uGNSS.GAL: [],\n                uGNSS.QZS: []}\n\n# set these from table below\nfreq_ix0 = {uGNSS.GPS: 0, uGNSS.GLO: 4, uGNSS.GAL: 0} # L1\nfreq_ix1 = {uGNSS.GPS: 2, uGNSS.GLO: 5, uGNSS.GAL: 2} # L5\n\n# ---------- Frequencies currently supported-------------\nfreq = [1.57542e9,   # L1/E1\n        1.22760e9,   # L2\n        1.17645e9,   # L5/E5a/B2a\n        1.20714e9,   # E5b\n        1.60200E9,   # G1\n        1.24600E9]   # G2\ndfreq_glo = [0.56250E6, 0.43750E6]\n\n\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Step 4: Convert raw files and run PPK solutions**\n\nFollow the instructions and use the code from step 4 in the previous notebook after replacing the following set of parameter definitions in the previous code.  This will run the python version of RTKLIB instead of the C version and to use the rinex3 base files instead of the rinex2 versions.\n\nIn the main code at the bottom of the file, the execution can be set up as either sequential or multiprocessing by commenting or uncommenting the appropriate lines, both for the file conversion, and for running the solutions. It is easier to debug when run sequentially but is much slower. I recommend running each step sequentially until you are convinced it's working, then switch it to multiprocessing.  \n\nHowever, there appears to be a path issue when running in multiprocessing mode so for now you will either need to set your PYTHONPATH to include the android_rinex/src and rtklib-py/src folders or move the files in those folders to the python folder.","metadata":{}},{"cell_type":"code","source":"# Set solution choices\nENABLE_PY = True        # Use RTKLIB-PY to generate solutions \nENABLE_RTKLIB = False   # Use RTKLIB to generate solutions\n\n#basefiles = '../*0.2*o' # rinex2, use this for rtklib only\nbasefiles = '../base.obs' # rinex3, use this for python only","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Step 5: Combine PPK solutions into a single .csv file**\n\nFollow the instructions for step 5 in the previous notebook after replacing the following set of parameter definitions in the previous code.","metadata":{}},{"cell_type":"code","source":"########### Input parameters ###############################\n\nDATA_SET = 'test'\nSOL_TAG = '_py0510'\ndatapath = r'C:\\gps\\GSDC_2022\\data'\nhdrlen = 1    # 25 for RTKLIB, 1 for rtklib-py","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Step 6: Filtering out RTKLIB solutions with hardware clock discontinuites**\n\nFollow the instructions for step 6 in the previous notebook","metadata":{}},{"cell_type":"markdown","source":"**Step 7: Submit csv file to Kaggle**\n\nYou can now submit the modified csv file to Kaggle. I have not done this yet, but based on the results from the training set and from last year's results, I would expect the score to be just slightly better than the RTKLIB results.\n\nSee the previous notebook for final thoughts and suggestions on improving the results.","metadata":{}}]}