{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Google Smartphone Decimeter Challenge by Mean(Interpolation)\n\nSolution is based on referring to various notebooks posted at Kaggle, along with my changes. Seeing this as a learning experience.\n\n## Notes\n\n If you’re outside, with open sky, the GPS accuracy from your phone is about five meters, and that’s been constant for a while. With raw GNSS measurements from the phones, this can now improve dramatically.\n \n### The GNSS problem description\nhttps://github.com/commaai/laika\n\nGNSS satellites orbit the earth broadcasting signals that allow the receiver to determine the distance to each satellite. These satellites have known orbits and so their positions are known. This makes determining the receiver's position a basic 3-dimensional trilateration problem. In practice observed distances to each satellite will be measured with some offset that is caused by the receiver's clock error. This offset also needs to be determined, making it a 4-dimensional trilateration problem. \n\n\"GNSS\"\n\nSince this problem is generally overdetermined (more than 4 satellites to solve the 4d problem) there is a variety of methods to compute a position estimate from the measurements. One can use a basic weighted least squares solver for experimental purposes. This is far from optimal due to the dynamic nature of the system, this makes a Bayesian estimator like a Kalman filter the preferred estimator.\n\nHowever, the above description is over-simplified. Getting accurate distance estimates to satellites and the satellite's position from the receiver observations is not trivial. This is what we call processing of the GNSS observables and it is this procedure laika is designed to make easy.\n \n ### How positioning works?\n - Send a burst of these transactions and, as a consequence, the system can calculate ranging statistics, such as the mean and the variance.\n \n \"Wifi\n \n Wi-Fi RTT principles, basic concept. Image by Frank van Diggelen, Roy Want and Wei Wang\n \n - Take these ranges of separate access points; if those ranges were accurate, they would define four circles that would intersect at a single point. In practice, because of error in each range, a maximum likelihood position is calculated using a least squares multilateration algorithm.\n \n - Further refine this position by repeating the process, particularly as the phone moves, and then calculate trajectory using filtering techniques, such as Kalman filtering, to optimize the estimate.\n \n \"Workflow\"\n \n Wi-Fi Workflow. Image by: Frank van Diggelen, Roy Want and Wei Wang.\n\n \n \n### Steps for the solution from Sohier Dane\n- Smoothing out the baseline estimates\n- Integrating readings from other phone instruments, like the accelerometer.\n- Satellite triangulation using the *derived.csv files.\n- Building triangulations directly from the raw gnss logs. \n- Incorporating external data for controls like satellite readings from base stations in the area.\n\n### References: \n- https://www.kaggle.com/t88take/gsdc-phones-mean-prediction\n- Discussions topics from Sohier Dane\n- Google I/O https://www.gpsworld.com/how-to-achieve-1-meter-accuracy-in-android/\n- GPS Survey Workshop video https://www.youtube.com/watch?v=vOJ3u7Zd_i0\n- Hardware: Centimeter Positioning with a Smartphone-Quality GNSS Antenna https://www.youtube.com/watch?v=rCOvklUB5vQ","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nfrom scipy.interpolate import interp1d\nimport matplotlib.pyplot as plt\nfrom matplotlib_venn import venn2, venn2_circles\nimport seaborn as sns\nfrom tqdm.notebook import tqdm\nimport pathlib\nimport plotly\nimport plotly.express as px","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-07-17T02:45:33.719176Z","iopub.execute_input":"2021-07-17T02:45:33.719476Z","iopub.status.idle":"2021-07-17T02:45:36.131819Z","shell.execute_reply.started":"2021-07-17T02:45:33.719409Z","shell.execute_reply":"2021-07-17T02:45:36.130967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def calc_haversine(lat1, lon1, lat2, lon2):\n \"\"\"Calculates the great circle distance between two points\n on the earth. Inputs are array-like and specified in decimal degrees.\n \"\"\"\n RADIUS = 6_367_000\n lat1, lon1, lat2, lon2 = map(np.radians, [lat1, lon1, lat2, lon2])\n dlat = lat2 - lat1\n dlon = lon2 - lon1\n a = np.sin(dlat/2)**2 + \\\n np.cos(lat1) * np.cos(lat2) * np.sin(dlon/2)**2\n dist = 2 * RADIUS * np.arcsin(a**0.5)\n return dist","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:36.13316Z","iopub.execute_input":"2021-07-17T02:45:36.133447Z","iopub.status.idle":"2021-07-17T02:45:36.140024Z","shell.execute_reply.started":"2021-07-17T02:45:36.133409Z","shell.execute_reply":"2021-07-17T02:45:36.139169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def visualize_trafic(df, center, zoom=9):\n fig = px.scatter_mapbox(df,\n \n # Here, plotly gets, (x,y) coordinates\n lat=\"latDeg\",\n lon=\"lngDeg\",\n \n #Here, plotly detects color of series\n color=\"phoneName\",\n labels=\"phoneName\",\n \n zoom=zoom,\n center=center,\n height=600,\n width=800)\n fig.update_layout(mapbox_style='stamen-terrain')\n fig.update_layout(margin={\"r\": 0, \"t\": 0, \"l\": 0, \"b\": 0})\n fig.update_layout(title_text=\"GPS trafic\")\n fig.show()\n \ndef visualize_collection(df, collection):\n target_df = df[df['collectionName']==collection].copy()\n lat_center = target_df['latDeg'].mean()\n lng_center = target_df['lngDeg'].mean()\n center = {\"lat\":lat_center, \"lon\":lng_center}\n \n visualize_trafic(target_df, center)","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:36.14171Z","iopub.execute_input":"2021-07-17T02:45:36.141982Z","iopub.status.idle":"2021-07-17T02:45:36.157789Z","shell.execute_reply.started":"2021-07-17T02:45:36.14195Z","shell.execute_reply":"2021-07-17T02:45:36.156592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# directory setting\nINPUT = '../input/google-smartphone-decimeter-challenge'","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:36.160142Z","iopub.execute_input":"2021-07-17T02:45:36.160411Z","iopub.status.idle":"2021-07-17T02:45:36.17338Z","shell.execute_reply.started":"2021-07-17T02:45:36.160382Z","shell.execute_reply":"2021-07-17T02:45:36.172637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"base_train = pd.read_csv(INPUT + '/' + 'baseline_locations_train.csv')\nbase_test = pd.read_csv(INPUT + '/' + 'baseline_locations_test.csv')\nsample_sub = pd.read_csv(INPUT + '/' + 'sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:36.174392Z","iopub.execute_input":"2021-07-17T02:45:36.174596Z","iopub.status.idle":"2021-07-17T02:45:36.89806Z","shell.execute_reply.started":"2021-07-17T02:45:36.174575Z","shell.execute_reply":"2021-07-17T02:45:36.896745Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ground_truth\np = pathlib.Path(INPUT)\ngt_files = list(p.glob('train/*/*/ground_truth.csv'))\nprint('ground_truth.csv count : ', len(gt_files))\n\ngts = []\nfor gt_file in tqdm(gt_files):\n gts.append(pd.read_csv(gt_file))\nground_truth = pd.concat(gts)\n\ndisplay(ground_truth.head())","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:36.899328Z","iopub.execute_input":"2021-07-17T02:45:36.899621Z","iopub.status.idle":"2021-07-17T02:45:37.877162Z","shell.execute_reply.started":"2021-07-17T02:45:36.899592Z","shell.execute_reply":"2021-07-17T02:45:37.876581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_distance_diff(df):\n df['latDeg_prev'] = df['latDeg'].shift(1)\n df['latDeg_next'] = df['latDeg'].shift(-1)\n df['lngDeg_prev'] = df['lngDeg'].shift(1)\n df['lngDeg_next'] = df['lngDeg'].shift(-1)\n df['phone_prev'] = df['phone'].shift(1)\n df['phone_next'] = df['phone'].shift(-1)\n \n df['dist_prev'] = calc_haversine(df['latDeg'], df['lngDeg'], df['latDeg_prev'], df['lngDeg_prev'])\n df['dist_next'] = calc_haversine(df['latDeg'], df['lngDeg'], df['latDeg_next'], df['lngDeg_next'])\n \n df.loc[df['phone']!=df['phone_prev'], ['latDeg_prev', 'lngDeg_prev', 'dist_prev']] = np.nan\n df.loc[df['phone']!=df['phone_next'], ['latDeg_next', 'lngDeg_next', 'dist_next']] = np.nan\n \n return df","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:37.878227Z","iopub.execute_input":"2021-07-17T02:45:37.878616Z","iopub.status.idle":"2021-07-17T02:45:37.886675Z","shell.execute_reply.started":"2021-07-17T02:45:37.878586Z","shell.execute_reply":"2021-07-17T02:45:37.885645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# reject outlier\ntrain_ro = add_distance_diff(base_train)\nth = 50\ntrain_ro.loc[((train_ro['dist_prev'] > th) & (train_ro['dist_next'] > th)), ['latDeg', 'lngDeg']] = np.nan","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:37.889284Z","iopub.execute_input":"2021-07-17T02:45:37.88957Z","iopub.status.idle":"2021-07-17T02:45:38.020378Z","shell.execute_reply.started":"2021-07-17T02:45:37.889537Z","shell.execute_reply":"2021-07-17T02:45:38.019838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install simdkalman","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:38.021389Z","iopub.execute_input":"2021-07-17T02:45:38.021673Z","iopub.status.idle":"2021-07-17T02:45:47.091818Z","shell.execute_reply.started":"2021-07-17T02:45:38.02165Z","shell.execute_reply":"2021-07-17T02:45:47.090575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import simdkalman","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:47.093997Z","iopub.execute_input":"2021-07-17T02:45:47.094279Z","iopub.status.idle":"2021-07-17T02:45:47.103857Z","shell.execute_reply.started":"2021-07-17T02:45:47.09425Z","shell.execute_reply":"2021-07-17T02:45:47.102288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"T = 1.0\nstate_transition = np.array([[1, 0, T, 0, 0.5 * T ** 2, 0], [0, 1, 0, T, 0, 0.5 * T ** 2], [0, 0, 1, 0, T, 0],\n [0, 0, 0, 1, 0, T], [0, 0, 0, 0, 1, 0], [0, 0, 0, 0, 0, 1]])\nprocess_noise = np.diag([1e-5, 1e-5, 5e-6, 5e-6, 1e-6, 1e-6]) + np.ones((6, 6)) * 1e-9\nobservation_model = np.array([[1, 0, 0, 0, 0, 0], [0, 1, 0, 0, 0, 0]])\nobservation_noise = np.diag([5e-5, 5e-5]) + np.ones((2, 2)) * 1e-9\n\nkf = simdkalman.KalmanFilter(\n state_transition = state_transition,\n process_noise = process_noise,\n observation_model = observation_model,\n observation_noise = observation_noise)\n\ndef apply_kf_smoothing(df, kf_=kf):\n unique_paths = df[['collectionName', 'phoneName']].drop_duplicates().to_numpy()\n for collection, phone in unique_paths:\n cond = np.logical_and(df['collectionName'] == collection, df['phoneName'] == phone)\n data = df[cond][['latDeg', 'lngDeg']].to_numpy()\n data = data.reshape(1, len(data), 2)\n smoothed = kf_.smooth(data)\n df.loc[cond, 'latDeg'] = smoothed.states.mean[0, :, 0]\n df.loc[cond, 'lngDeg'] = smoothed.states.mean[0, :, 1]\n return df","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:47.105699Z","iopub.execute_input":"2021-07-17T02:45:47.106145Z","iopub.status.idle":"2021-07-17T02:45:47.12229Z","shell.execute_reply.started":"2021-07-17T02:45:47.106105Z","shell.execute_reply":"2021-07-17T02:45:47.121007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cols = ['collectionName', 'phoneName', 'millisSinceGpsEpoch', 'latDeg', 'lngDeg']\ntrain_ro_kf = apply_kf_smoothing(train_ro[cols])","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:45:47.123499Z","iopub.execute_input":"2021-07-17T02:45:47.123827Z","iopub.status.idle":"2021-07-17T02:46:30.71124Z","shell.execute_reply.started":"2021-07-17T02:45:47.1238Z","shell.execute_reply":"2021-07-17T02:46:30.710433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def make_lerp_data(df):\n '''\n Generate interpolated lat,lng values for different phone times in the same collection.\n '''\n org_columns = df.columns\n \n # Generate a combination of time x collection x phone and combine it with the original data (generate records to be interpolated)\n time_list = df[['collectionName', 'millisSinceGpsEpoch']].drop_duplicates()\n phone_list =df[['collectionName', 'phoneName']].drop_duplicates()\n tmp = time_list.merge(phone_list, on='collectionName', how='outer')\n \n lerp_df = tmp.merge(df, on=['collectionName', 'millisSinceGpsEpoch', 'phoneName'], how='left')\n lerp_df['phone'] = lerp_df['collectionName'] + '_' + lerp_df['phoneName']\n lerp_df = lerp_df.sort_values(['phone', 'millisSinceGpsEpoch'])\n \n # linear interpolation\n lerp_df['latDeg_prev'] = lerp_df['latDeg'].shift(1)\n lerp_df['latDeg_next'] = lerp_df['latDeg'].shift(-1)\n lerp_df['lngDeg_prev'] = lerp_df['lngDeg'].shift(1)\n lerp_df['lngDeg_next'] = lerp_df['lngDeg'].shift(-1)\n lerp_df['phone_prev'] = lerp_df['phone'].shift(1)\n lerp_df['phone_next'] = lerp_df['phone'].shift(-1)\n lerp_df['time_prev'] = lerp_df['millisSinceGpsEpoch'].shift(1)\n lerp_df['time_next'] = lerp_df['millisSinceGpsEpoch'].shift(-1)\n # Leave only records to be interpolated\n lerp_df = lerp_df[(lerp_df['latDeg'].isnull())&(lerp_df['phone']==lerp_df['phone_prev'])&(lerp_df['phone']==lerp_df['phone_next'])].copy()\n \n # calc lerp\n# lerp_df_copy = lerp_df.copy()\n# lerp_df_copy = lerp_df_copy.drop_duplicates(subset = [\"millisSinceGpsEpoch\"])\n\n# x = lerp_df_copy['millisSinceGpsEpoch']\n \n# y_lat = lerp_df_copy['latDeg']\n# y_lng = lerp_df_copy['lngDeg']\n# interpolation_function_lat = interp1d(x, y_lat, kind='cubic')\n# interpolation_function_lng = interp1d(x, y_lng, kind='cubic')\n# lerp_df['latDeg'] = interpolation_function_lat(lerp_df[\"millisSinceGpsEpoch\"])\n# lerp_df['lngDeg'] = interpolation_function_lng(lerp_df[\"millisSinceGpsEpoch\"])\n latDeg_next = lerp_df['latDeg_next']\n latDeg_prev = lerp_df['latDeg_prev']\n \n lngDeg_prev = lerp_df['lngDeg_prev']\n lngDeg_next = lerp_df['lngDeg_next']\n \n millisSinceGpsEpoch = lerp_df['millisSinceGpsEpoch']\n time_prev = lerp_df['time_prev']\n time_next = lerp_df['time_next']\n \n time_ratio = (millisSinceGpsEpoch - time_prev) / (time_next - time_prev)\n# time_ratio_cosine = (1 - np.cos(time_ratio * np.pi))/2.0\n \n# lerp_df['latDeg'] = latDeg_prev + ((latDeg_next - latDeg_prev) * time_ratio_cosine) \n# lerp_df['lngDeg'] = lngDeg_prev + ((lngDeg_next - lngDeg_prev) * time_ratio_cosine) \n\n lerp_df['latDeg'] = latDeg_prev + ((latDeg_next - latDeg_prev) * time_ratio) \n lerp_df['lngDeg'] = lngDeg_prev + ((lngDeg_next - lngDeg_prev) * time_ratio) \n \n# lerp_df['latDeg'] = lerp_df['latDeg_prev'] + ((lerp_df['latDeg_next'] - lerp_df['latDeg_prev']) * ((lerp_df['millisSinceGpsEpoch'] - lerp_df['time_prev']) / (lerp_df['time_next'] - lerp_df['time_prev']))) \n# lerp_df['lngDeg'] = lerp_df['lngDeg_prev'] + ((lerp_df['lngDeg_next'] - lerp_df['lngDeg_prev']) * ((lerp_df['millisSinceGpsEpoch'] - lerp_df['time_prev']) / (lerp_df['time_next'] - lerp_df['time_prev']))) \n \n # Leave only the data that has a complete set of previous and next data.\n lerp_df = lerp_df[~lerp_df['latDeg'].isnull()]\n \n return lerp_df[org_columns]","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:30.712494Z","iopub.execute_input":"2021-07-17T02:46:30.713113Z","iopub.status.idle":"2021-07-17T02:46:30.729806Z","shell.execute_reply.started":"2021-07-17T02:46:30.713066Z","shell.execute_reply":"2021-07-17T02:46:30.729038Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def calc_mean_pred(df, lerp_df):\n '''\n Make a prediction based on the average of the predictions of phones in the same collection.\n '''\n add_lerp = pd.concat([df, lerp_df])\n mean_pred_result = add_lerp.groupby(['collectionName', 'millisSinceGpsEpoch'])[['latDeg', 'lngDeg']].mean().reset_index()\n mean_pred_df = df[['collectionName', 'phoneName', 'millisSinceGpsEpoch']].copy()\n mean_pred_df = mean_pred_df.merge(mean_pred_result[['collectionName', 'millisSinceGpsEpoch', 'latDeg', 'lngDeg']], on=['collectionName', 'millisSinceGpsEpoch'], how='left')\n return mean_pred_df","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:30.731283Z","iopub.execute_input":"2021-07-17T02:46:30.731883Z","iopub.status.idle":"2021-07-17T02:46:30.746753Z","shell.execute_reply.started":"2021-07-17T02:46:30.731843Z","shell.execute_reply":"2021-07-17T02:46:30.745875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_lerp = make_lerp_data(train_ro_kf)\ntrain_mean_pred = calc_mean_pred(train_ro_kf, train_lerp)","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:30.748093Z","iopub.execute_input":"2021-07-17T02:46:30.748721Z","iopub.status.idle":"2021-07-17T02:46:31.833378Z","shell.execute_reply.started":"2021-07-17T02:46:30.748684Z","shell.execute_reply":"2021-07-17T02:46:31.832554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tmp1 = train_ro_kf.copy()\ntmp2 = train_mean_pred.copy()\ntmp2['phoneName'] = tmp2['phoneName'] + '_MEAN'\ntmp3 = ground_truth.copy()\ntmp3['phoneName'] = tmp3['phoneName'] + '_GT'\ntmp = pd.concat([tmp1, tmp2, tmp3])\nvisualize_collection(tmp, '2020-05-14-US-MTV-1')","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:31.834445Z","iopub.execute_input":"2021-07-17T02:46:31.834723Z","iopub.status.idle":"2021-07-17T02:46:33.020985Z","shell.execute_reply.started":"2021-07-17T02:46:31.83469Z","shell.execute_reply":"2021-07-17T02:46:33.019815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def percentile50(x):\n return np.percentile(x, 50)\ndef percentile95(x):\n return np.percentile(x, 95)","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:33.022229Z","iopub.execute_input":"2021-07-17T02:46:33.022471Z","iopub.status.idle":"2021-07-17T02:46:33.025958Z","shell.execute_reply.started":"2021-07-17T02:46:33.022447Z","shell.execute_reply":"2021-07-17T02:46:33.02522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_train_score(df, gt):\n gt = gt.rename(columns={'latDeg':'latDeg_gt', 'lngDeg':'lngDeg_gt'})\n df = df.merge(gt, on=['collectionName', 'phoneName', 'millisSinceGpsEpoch'], how='inner')\n # calc_distance_error\n df['err'] = calc_haversine(df['latDeg_gt'], df['lngDeg_gt'], df['latDeg'], df['lngDeg'])\n # calc_evaluate_score\n df['phone'] = df['collectionName'] + '_' + df['phoneName']\n res = df.groupby('phone')['err'].agg([percentile50, percentile95])\n res['p50_p90_mean'] = (res['percentile50'] + res['percentile95']) / 2 \n score = res['p50_p90_mean'].mean()\n return score","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:33.026776Z","iopub.execute_input":"2021-07-17T02:46:33.027014Z","iopub.status.idle":"2021-07-17T02:46:33.042582Z","shell.execute_reply.started":"2021-07-17T02:46:33.026991Z","shell.execute_reply":"2021-07-17T02:46:33.041478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('kf + reject_outlier : ', get_train_score(train_ro_kf, ground_truth))\nprint('+ phones_mean_pred : ', get_train_score(train_mean_pred, ground_truth))","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:46:33.044221Z","iopub.execute_input":"2021-07-17T02:46:33.044582Z","iopub.status.idle":"2021-07-17T02:46:33.518612Z","shell.execute_reply.started":"2021-07-17T02:46:33.044553Z","shell.execute_reply":"2021-07-17T02:46:33.517814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"base_test = add_distance_diff(base_test)\nth = 100000\nbase_test.loc[((base_test['dist_prev'] > th) & (base_test['dist_next'] > th)), ['latDeg', 'lngDeg']] = np.nan\n\ntest_kf = apply_kf_smoothing(base_test)\n\ntest_lerp = make_lerp_data(test_kf)\ntest_mean_pred = calc_mean_pred(test_kf, test_lerp)\n\nsample_sub['latDeg'] = test_mean_pred['latDeg']\nsample_sub['lngDeg'] = test_mean_pred['lngDeg']\nsample_sub.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2021-07-17T02:51:40.916408Z","iopub.execute_input":"2021-07-17T02:51:40.916745Z","iopub.status.idle":"2021-07-17T02:52:05.306048Z","shell.execute_reply.started":"2021-07-17T02:51:40.916717Z","shell.execute_reply":"2021-07-17T02:52:05.305392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}