{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

EDA+Google Smartphone Decimeter

\n\n### What is the EDA\n\nExploratory Data Analysis refers to the critical process of performing initial investigations on data so as to discover patterns,to spot anomalies,to test hypothesis and to check assumptions with the help of summary statistics and graphical representations.\n\n\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"\n

Google Smartphone Decimeter Challenge

\n\n### Improve high precision GNSS positioning and navigation accuracy on smartphones\n\n\n\n\n\n\n\nGlobal Navigation Satellite System (GNSS) provides raw signals, which the GPS chipset uses to compute a position. Current mobile phones only offer 3-5 meters of positioning accuracy. While useful in many cases, it can create a “jumpy” experience. For many use cases the results are not fine nor stable enough to be reliable.\n\nThis competition, hosted by the Android GPS team, is being presented at the ION GNSS+ 2021 Conference. They seek to advance research in smartphone GNSS positioning accuracy and help people better navigate the world around them.\n\n\n### Data Description\n\n[train]/[drive_id]/[phone_name]/ground_truth.csv - Only provided for the training set. Reference locations at expected timestamps.\n\n\n[train/test]/[drive_id]/[phone_name]/supplemental/[phone_name][.20o/.21o/.nmea] - Equivalent data to the gnss logs in other formats used by the GPS community.\n\n\nbaseline_locations_[train/test].csv - Estimated coordinates generated using a simple approach.\n\n#### Dataset Link \n\n##### [Here](https://www.kaggle.com/c/google-smartphone-decimeter-challenge/data)","metadata":{}},{"cell_type":"code","source":"!pip install dataprep by","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-06-07T19:03:22.147431Z","iopub.execute_input":"2021-06-07T19:03:22.147813Z","iopub.status.idle":"2021-06-07T19:03:29.415290Z","shell.execute_reply.started":"2021-06-07T19:03:22.147782Z","shell.execute_reply":"2021-06-07T19:03:29.414385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pip install autoviz","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-07T19:03:29.416660Z","iopub.execute_input":"2021-06-07T19:03:29.416946Z","iopub.status.idle":"2021-06-07T19:03:36.339663Z","shell.execute_reply.started":"2021-06-07T19:03:29.416914Z","shell.execute_reply":"2021-06-07T19:03:36.338453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install xlrd","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-07T19:03:36.342250Z","iopub.execute_input":"2021-06-07T19:03:36.342685Z","iopub.status.idle":"2021-06-07T19:03:42.928953Z","shell.execute_reply.started":"2021-06-07T19:03:36.342633Z","shell.execute_reply":"2021-06-07T19:03:42.927749Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom collections import Counter\nimport seaborn as sns\nfrom pathlib import Path\nimport numpy as np\nfrom tqdm.notebook import tqdm\nfrom dataprep.eda import *\nfrom dataprep.eda import plot\nfrom dataprep.eda import plot_diff\nfrom dataprep.eda import plot_correlation\nfrom dataprep.eda import plot_missing\nimport plotly.express as px\nimport plotly.figure_factory as ff\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nPH = \"../input/google-smartphone-decimeter-challenge/\"","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:42.930864Z","iopub.execute_input":"2021-06-07T19:03:42.931156Z","iopub.status.idle":"2021-06-07T19:03:42.940715Z","shell.execute_reply.started":"2021-06-07T19:03:42.931125Z","shell.execute_reply":"2021-06-07T19:03:42.939632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv(PH+'baseline_locations_train.csv')\ntest = pd.read_csv(PH+'baseline_locations_test.csv')\nsubmission = pd.read_csv(PH+'sample_submission.csv')\nsubmission = submission.assign(latDeg = test.latDeg.round(6),lngDeg = test.lngDeg.round(6))\nsubmission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:42.941979Z","iopub.execute_input":"2021-06-07T19:03:42.942364Z","iopub.status.idle":"2021-06-07T19:03:43.852296Z","shell.execute_reply.started":"2021-06-07T19:03:42.942334Z","shell.execute_reply":"2021-06-07T19:03:43.851300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:43.853526Z","iopub.execute_input":"2021-06-07T19:03:43.853813Z","iopub.status.idle":"2021-06-07T19:03:43.870169Z","shell.execute_reply.started":"2021-06-07T19:03:43.853785Z","shell.execute_reply":"2021-06-07T19:03:43.869067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:43.871759Z","iopub.execute_input":"2021-06-07T19:03:43.872188Z","iopub.status.idle":"2021-06-07T19:03:43.892144Z","shell.execute_reply.started":"2021-06-07T19:03:43.872144Z","shell.execute_reply":"2021-06-07T19:03:43.890979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:43.895529Z","iopub.execute_input":"2021-06-07T19:03:43.895976Z","iopub.status.idle":"2021-06-07T19:03:43.912831Z","shell.execute_reply.started":"2021-06-07T19:03:43.895928Z","shell.execute_reply":"2021-06-07T19:03:43.911710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.info()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:43.914672Z","iopub.execute_input":"2021-06-07T19:03:43.914955Z","iopub.status.idle":"2021-06-07T19:03:43.980394Z","shell.execute_reply.started":"2021-06-07T19:03:43.914928Z","shell.execute_reply":"2021-06-07T19:03:43.979112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.info()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:43.981734Z","iopub.execute_input":"2021-06-07T19:03:43.982057Z","iopub.status.idle":"2021-06-07T19:03:44.022946Z","shell.execute_reply.started":"2021-06-07T19:03:43.982021Z","shell.execute_reply":"2021-06-07T19:03:44.021928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.info()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:44.024282Z","iopub.execute_input":"2021-06-07T19:03:44.024566Z","iopub.status.idle":"2021-06-07T19:03:44.048868Z","shell.execute_reply.started":"2021-06-07T19:03:44.024539Z","shell.execute_reply":"2021-06-07T19:03:44.047742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:44.050143Z","iopub.execute_input":"2021-06-07T19:03:44.050490Z","iopub.status.idle":"2021-06-07T19:03:44.072605Z","shell.execute_reply.started":"2021-06-07T19:03:44.050458Z","shell.execute_reply":"2021-06-07T19:03:44.071391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_diff([train, test])","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:44.074027Z","iopub.execute_input":"2021-06-07T19:03:44.074437Z","iopub.status.idle":"2021-06-07T19:03:48.316901Z","shell.execute_reply.started":"2021-06-07T19:03:44.074405Z","shell.execute_reply":"2021-06-07T19:03:48.315983Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_correlation(train)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:48.318052Z","iopub.execute_input":"2021-06-07T19:03:48.318497Z","iopub.status.idle":"2021-06-07T19:03:49.531157Z","shell.execute_reply.started":"2021-06-07T19:03:48.318459Z","shell.execute_reply":"2021-06-07T19:03:49.530384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(train, \"phoneName\")","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:49.532176Z","iopub.execute_input":"2021-06-07T19:03:49.532583Z","iopub.status.idle":"2021-06-07T19:03:50.559090Z","shell.execute_reply.started":"2021-06-07T19:03:49.532553Z","shell.execute_reply":"2021-06-07T19:03:50.558100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas_profiling as pp\npp.ProfileReport(test)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:03:50.560412Z","iopub.execute_input":"2021-06-07T19:03:50.560710Z","iopub.status.idle":"2021-06-07T19:04:09.087988Z","shell.execute_reply.started":"2021-06-07T19:03:50.560681Z","shell.execute_reply":"2021-06-07T19:04:09.087035Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"create_report(train)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:09.089474Z","iopub.execute_input":"2021-06-07T19:04:09.089764Z","iopub.status.idle":"2021-06-07T19:04:15.651284Z","shell.execute_reply.started":"2021-06-07T19:04:09.089736Z","shell.execute_reply":"2021-06-07T19:04:15.650338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"create_report(test)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:15.652404Z","iopub.execute_input":"2021-06-07T19:04:15.652831Z","iopub.status.idle":"2021-06-07T19:04:21.510060Z","shell.execute_reply.started":"2021-06-07T19:04:15.652801Z","shell.execute_reply":"2021-06-07T19:04:21.508881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:21.511384Z","iopub.execute_input":"2021-06-07T19:04:21.511667Z","iopub.status.idle":"2021-06-07T19:04:21.533526Z","shell.execute_reply.started":"2021-06-07T19:04:21.511639Z","shell.execute_reply":"2021-06-07T19:04:21.532528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"datapath = Path(\"../input/google-smartphone-decimeter-challenge\")\n\ntruths = (datapath / 'train').rglob('ground_truth.csv')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:21.534960Z","iopub.execute_input":"2021-06-07T19:04:21.535300Z","iopub.status.idle":"2021-06-07T19:04:21.540444Z","shell.execute_reply.started":"2021-06-07T19:04:21.535269Z","shell.execute_reply":"2021-06-07T19:04:21.539330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_list = []\ncols = ['collectionName', 'phoneName', 'millisSinceGpsEpoch', 'latDeg','lngDeg']\n\nfor t in tqdm(truths, total=73):\n df_phone = pd.read_csv(t, usecols=cols) \n df_list.append(df_phone)\ndf_truth = pd.concat(df_list, ignore_index=True)\n\ndf_basepreds = pd.read_csv('../input/google-smartphone-decimeter-challenge/baseline_locations_train.csv', usecols=cols)\ndf_all = df_truth.merge(df_basepreds, how='inner', on=cols[:3], suffixes=('_truth', '_basepred'))\ndf_all[:100]","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:21.542124Z","iopub.execute_input":"2021-06-07T19:04:21.542490Z","iopub.status.idle":"2021-06-07T19:04:22.559029Z","shell.execute_reply.started":"2021-06-07T19:04:21.542446Z","shell.execute_reply":"2021-06-07T19:04:22.557924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# https://plotly.com/python/mapbox-layers/\nimport plotly.express as px\ndef MAP(df_all, center):\n fig = px.scatter_mapbox(df_all,lat=\"latDeg\", lon=\"lngDeg\", color=\"phoneName\",labels=\"phoneName\",zoom=9,center=center,height=500, width=600)\n fig.update_layout(mapbox_style=\"open-street-map\")\n fig.update_layout(margin={\"r\":0,\"t\":0,\"l\":0,\"b\":0})\n fig.show()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.560621Z","iopub.execute_input":"2021-06-07T19:04:22.561301Z","iopub.status.idle":"2021-06-07T19:04:22.570615Z","shell.execute_reply.started":"2021-06-07T19:04:22.561242Z","shell.execute_reply":"2021-06-07T19:04:22.569345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Pixel4 = pd.read_csv(PH + \"train/2020-05-14-US-MTV-1/Pixel4/ground_truth.csv\")\n\ncenter = {\"lat\":37.428281, \"lon\":-122.072541}\nMAP(Pixel4, center)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.574999Z","iopub.execute_input":"2021-06-07T19:04:22.575360Z","iopub.status.idle":"2021-06-07T19:04:22.662672Z","shell.execute_reply.started":"2021-06-07T19:04:22.575327Z","shell.execute_reply":"2021-06-07T19:04:22.661549Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Pixel4XLModded = pd.read_csv(PH + \"train/2020-05-14-US-MTV-1/Pixel4XLModded/ground_truth.csv\")\n\ncenter = {\"lat\":37.6901944483, \"lon\":-122.3923280950}\nMAP(Pixel4XLModded, center)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.664723Z","iopub.execute_input":"2021-06-07T19:04:22.665049Z","iopub.status.idle":"2021-06-07T19:04:22.745703Z","shell.execute_reply.started":"2021-06-07T19:04:22.665018Z","shell.execute_reply":"2021-06-07T19:04:22.744428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Mi8 = pd.read_csv(PH + \"train/2020-07-17-US-MTV-1/Mi8/ground_truth.csv\")\n\ncenter = {\"lat\":37.4282808217, \"lon\":-122.0725408417}\nMAP(Mi8, center)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.747128Z","iopub.execute_input":"2021-06-07T19:04:22.747466Z","iopub.status.idle":"2021-06-07T19:04:22.828268Z","shell.execute_reply.started":"2021-06-07T19:04:22.747434Z","shell.execute_reply":"2021-06-07T19:04:22.827306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Pixel5 = pd.read_csv(PH + \"train/2021-01-04-US-RWC-2/Pixel5/ground_truth.csv\")\n\ncenter = {\"lat\":37.4443845150, \"lon\":-122.2326749300}\nMAP(Pixel5, center)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.829504Z","iopub.execute_input":"2021-06-07T19:04:22.829798Z","iopub.status.idle":"2021-06-07T19:04:22.908631Z","shell.execute_reply.started":"2021-06-07T19:04:22.829768Z","shell.execute_reply":"2021-06-07T19:04:22.907635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"SamsungS20Ultra = pd.read_csv(PH + \"train/2021-03-10-US-SVL-1/SamsungS20Ultra/ground_truth.csv\")\n\ncenter = {\"lat\":37.3713148617, \"lon\":-122.04798264007}\nMAP(SamsungS20Ultra, center)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.909943Z","iopub.execute_input":"2021-06-07T19:04:22.910269Z","iopub.status.idle":"2021-06-07T19:04:22.987554Z","shell.execute_reply.started":"2021-06-07T19:04:22.910237Z","shell.execute_reply":"2021-06-07T19:04:22.986403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"All = pd.concat([Pixel4, Pixel4XLModded, Mi8,Pixel5,SamsungS20Ultra])\nMAP(All, center)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:22.988886Z","iopub.execute_input":"2021-06-07T19:04:22.989212Z","iopub.status.idle":"2021-06-07T19:04:23.469651Z","shell.execute_reply.started":"2021-06-07T19:04:22.989168Z","shell.execute_reply":"2021-06-07T19:04:23.468721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all.info()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:23.470987Z","iopub.execute_input":"2021-06-07T19:04:23.471273Z","iopub.status.idle":"2021-06-07T19:04:23.511358Z","shell.execute_reply.started":"2021-06-07T19:04:23.471246Z","shell.execute_reply":"2021-06-07T19:04:23.510239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all.isna()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:23.512787Z","iopub.execute_input":"2021-06-07T19:04:23.513220Z","iopub.status.idle":"2021-06-07T19:04:23.558260Z","shell.execute_reply.started":"2021-06-07T19:04:23.513171Z","shell.execute_reply":"2021-06-07T19:04:23.556988Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:23.559555Z","iopub.execute_input":"2021-06-07T19:04:23.559825Z","iopub.status.idle":"2021-06-07T19:04:23.594760Z","shell.execute_reply.started":"2021-06-07T19:04:23.559799Z","shell.execute_reply":"2021-06-07T19:04:23.593931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Chack if we have miss data\nplot_missing(df_all)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:23.596022Z","iopub.execute_input":"2021-06-07T19:04:23.596739Z","iopub.status.idle":"2021-06-07T19:04:24.793916Z","shell.execute_reply.started":"2021-06-07T19:04:23.596700Z","shell.execute_reply":"2021-06-07T19:04:24.793130Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:24.794907Z","iopub.execute_input":"2021-06-07T19:04:24.795290Z","iopub.status.idle":"2021-06-07T19:04:24.814581Z","shell.execute_reply.started":"2021-06-07T19:04:24.795261Z","shell.execute_reply":"2021-06-07T19:04:24.813788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from autoviz.AutoViz_Class import AutoViz_Class\nAV = AutoViz_Class()\ntarget='phoneName'\ndf = AV.AutoViz(filename=\"\",sep=',', depVar=target, dfte=df_all, header=0, verbose=2, \n lowess=False, chart_format='svg', max_rows_analyzed=150000, max_cols_analyzed=30)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:04:24.815656Z","iopub.execute_input":"2021-06-07T19:04:24.816085Z","iopub.status.idle":"2021-06-07T19:06:36.856449Z","shell.execute_reply.started":"2021-06-07T19:04:24.816052Z","shell.execute_reply":"2021-06-07T19:06:36.855413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas_profiling as pp\npp.ProfileReport(df_all)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:36.857639Z","iopub.execute_input":"2021-06-07T19:06:36.857914Z","iopub.status.idle":"2021-06-07T19:06:54.002056Z","shell.execute_reply.started":"2021-06-07T19:06:36.857887Z","shell.execute_reply":"2021-06-07T19:06:54.000751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(df_all)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:54.003421Z","iopub.execute_input":"2021-06-07T19:06:54.003715Z","iopub.status.idle":"2021-06-07T19:06:56.138070Z","shell.execute_reply.started":"2021-06-07T19:06:54.003686Z","shell.execute_reply":"2021-06-07T19:06:56.137334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(df_all, 'lngDeg_basepred')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:56.139240Z","iopub.execute_input":"2021-06-07T19:06:56.139713Z","iopub.status.idle":"2021-06-07T19:06:56.795758Z","shell.execute_reply.started":"2021-06-07T19:06:56.139666Z","shell.execute_reply":"2021-06-07T19:06:56.794959Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(df_all, 'latDeg_basepred')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:56.796779Z","iopub.execute_input":"2021-06-07T19:06:56.797060Z","iopub.status.idle":"2021-06-07T19:06:57.485823Z","shell.execute_reply.started":"2021-06-07T19:06:56.797033Z","shell.execute_reply":"2021-06-07T19:06:57.485019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(df_all, 'lngDeg_truth')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:57.486869Z","iopub.execute_input":"2021-06-07T19:06:57.487292Z","iopub.status.idle":"2021-06-07T19:06:58.150859Z","shell.execute_reply.started":"2021-06-07T19:06:57.487259Z","shell.execute_reply":"2021-06-07T19:06:58.149863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(df_all, 'millisSinceGpsEpoch')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:58.152301Z","iopub.execute_input":"2021-06-07T19:06:58.152597Z","iopub.status.idle":"2021-06-07T19:06:58.809371Z","shell.execute_reply.started":"2021-06-07T19:06:58.152569Z","shell.execute_reply":"2021-06-07T19:06:58.808561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot(df_all, 'phoneName')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:58.810424Z","iopub.execute_input":"2021-06-07T19:06:58.810826Z","iopub.status.idle":"2021-06-07T19:06:59.796877Z","shell.execute_reply.started":"2021-06-07T19:06:58.810787Z","shell.execute_reply":"2021-06-07T19:06:59.795294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nplot(df_all, 'collectionName')","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:06:59.798469Z","iopub.execute_input":"2021-06-07T19:06:59.798842Z","iopub.status.idle":"2021-06-07T19:07:00.940939Z","shell.execute_reply.started":"2021-06-07T19:06:59.798806Z","shell.execute_reply":"2021-06-07T19:07:00.939380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"create_report(df_all)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:00.942930Z","iopub.execute_input":"2021-06-07T19:07:00.943337Z","iopub.status.idle":"2021-06-07T19:07:07.212967Z","shell.execute_reply.started":"2021-06-07T19:07:00.943295Z","shell.execute_reply":"2021-06-07T19:07:07.211979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,8))\nsns.heatmap(df_all.corr(), annot=True)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:07.214166Z","iopub.execute_input":"2021-06-07T19:07:07.214498Z","iopub.status.idle":"2021-06-07T19:07:07.644356Z","shell.execute_reply.started":"2021-06-07T19:07:07.214467Z","shell.execute_reply":"2021-06-07T19:07:07.643385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all[:10].T","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:07.645653Z","iopub.execute_input":"2021-06-07T19:07:07.645958Z","iopub.status.idle":"2021-06-07T19:07:07.664493Z","shell.execute_reply.started":"2021-06-07T19:07:07.645928Z","shell.execute_reply":"2021-06-07T19:07:07.663765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"phoneNamene = {'Pixel4':1, \n 'Pixel4XL':2, \n 'SamsungS20Ultra':3, \n 'Mi8':4, \n 'Pixel5':5, \n 'Pixel4XLModded':6,\n 'Pixel4Modded':7}\n\n# apply using map\ndf_all['phoneName'] = df_all['phoneName'].map(phoneNamene)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:07.665525Z","iopub.execute_input":"2021-06-07T19:07:07.665954Z","iopub.status.idle":"2021-06-07T19:07:07.692696Z","shell.execute_reply.started":"2021-06-07T19:07:07.665917Z","shell.execute_reply":"2021-06-07T19:07:07.691819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all[:10]","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:07.693872Z","iopub.execute_input":"2021-06-07T19:07:07.694355Z","iopub.status.idle":"2021-06-07T19:07:07.720830Z","shell.execute_reply.started":"2021-06-07T19:07:07.694298Z","shell.execute_reply":"2021-06-07T19:07:07.719751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_all.info()","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:07.722364Z","iopub.execute_input":"2021-06-07T19:07:07.722820Z","iopub.status.idle":"2021-06-07T19:07:07.758530Z","shell.execute_reply.started":"2021-06-07T19:07:07.722774Z","shell.execute_reply":"2021-06-07T19:07:07.757404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_correlation(df_all)","metadata":{"execution":{"iopub.status.busy":"2021-06-07T19:07:07.759891Z","iopub.execute_input":"2021-06-07T19:07:07.760248Z","iopub.status.idle":"2021-06-07T19:07:09.041880Z","shell.execute_reply.started":"2021-06-07T19:07:07.760215Z","shell.execute_reply":"2021-06-07T19:07:09.041067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Good Luck All","metadata":{}}]}