{"cells":[{"metadata":{},"cell_type":"markdown","source":"

\n

🧭Indoor Location and Navigation🧭

\n\n# 1. Introduction\n> 📌 **Goal**: Predicting the indoor position of smartphones 📱 based on a *real-time* sensor 🎯.\n\n> We'll also learn how to use the **GitHub Repository** available through this competition and call the custom functions **without** copy-pasting them into the notebook.\n\n### Libraries📚"},{"metadata":{"trusted":true},"cell_type":"code","source":"# CPU libraries\nimport os\nimport json\nimport glob\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib\nimport matplotlib.pyplot as plt\nimport plotly.graph_objs as go\n\nfrom PIL import Image, ImageOps\nfrom skimage import io\nfrom skimage.color import rgba2rgb, rgb2xyz\nfrom tqdm import tqdm\nfrom dataclasses import dataclass\nfrom math import floor, ceil\n\nmycolors = [\"#797D62\", \"#9B9B7A\", \"#D9AE94\", \"#FFCB69\", \"#D08C60\", \"#997B66\"]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"def show_values_on_bars(axs, h_v=\"v\", space=0.4):\n '''Plots the value at the end of the a seaborn barplot.\n axs: the ax of the plot\n h_v: weather or not the barplot is vertical/ horizontal'''\n \n def _show_on_single_plot(ax):\n if h_v == \"v\":\n for p in ax.patches:\n _x = p.get_x() + p.get_width() / 2\n _y = p.get_y() + p.get_height()\n value = int(p.get_height())\n ax.text(_x, _y, format(value, ','), ha=\"center\") \n elif h_v == \"h\":\n for p in ax.patches:\n _x = p.get_x() + p.get_width() + float(space)\n _y = p.get_y() + p.get_height()\n value = int(p.get_width())\n ax.text(_x, _y, format(value, ','), ha=\"left\")\n\n if isinstance(axs, np.ndarray):\n for idx, ax in np.ndenumerate(axs):\n _show_on_single_plot(ax)\n else:\n _show_on_single_plot(axs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### W&B Setup\n* Create an account on https://wandb.ai (it's free)\n* Install `wandb`\n* Input your `personal key` of the project ( mine will be secret, as it is confidential :) )"},{"metadata":{"trusted":true},"cell_type":"code","source":"import wandb\nos.environ[\"WANDB_SILENT\"] = \"true\"\n\nfrom kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\nsecret_value_0 = user_secrets.get_secret(\"wandb\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"! wandb login $secret_value_0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Initialize new project\n### project - the name of the overajj project (like the name of the repo in GitHub)\n### name/experiment - the name of the run (we'll have multiple runs in 1 project)\nrun = wandb.init(project=\"indoor-location-kaggle\", name=\"data-understanding\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> We're all set!\n\n# 2. Data Understanding\n\n* `train` - train path files, organized by *site* and *floor*. Each path file contains the data of a single path on a single floor.\n* `test` - test path files, organized by *site* and *floor*. Each path file contains the data of a single path on a single floor, **but without the waypoint (x, y) data**. \n* `metadata` - floor metadata folder, organized by site and floor, which includes the following for each floor:\n * `floor_image.png`\n * `floor_info.json`\n * `geojson_map.json`\n \n

\n\n> 📌 **Goal**: The task of this competition is, for a given site-path file, **predict the floor** and **waypoint locations** at the **timestamps** given in the `sample_submission.csv` file.\n\n> ❗ **Note on data quality** ❗: In the training files, you may find occasionally that a line is missing the ending newline character, causing it to run on to the next line. It is up to you how you want to handle this issue. This issue is not found in the test data."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get path to all TRAIN & TEST files\ntrain_paths = glob.glob('../input/indoor-location-navigation/train/*/*/*')\ntest_paths = glob.glob('../input/indoor-location-navigation/test/*')\nsites = glob.glob('../input/indoor-location-navigation/metadata/*')\n\nprint(\"No. Files in Train: {:,}\".format(len(train_paths)), \"\\n\" +\n \"No. Files in Test: {:,}\".format(len(test_paths)), \"\\n\" +\n \"Total Sites (metadata): {:,}\".format(len(sites)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Log to W&B in \"data-understanding\" experiment\nwandb.log({'No. Files in Train': len(train_paths), \n 'No. Files in Test:' : len(test_paths),\n 'Total Sites (metadata)' : len(sites)})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Reading in the data\n\n> ❗ **Note**: The presented data is in `.txt` format and contains A LOT of information. In only 1 `.txt` file (meaning 1 path), there were over 126k lines of information. From 1 file to another, this information varies, so trying to structure it all could be a hassle."},{"metadata":{"trusted":true},"cell_type":"code","source":"# How 1 path looks\nbase = '../input/indoor-location-navigation'\npath = f'{base}/train/5cd56b5ae2acfd2d33b58549/5F/5d06134c4a19c000086c4324.txt'\n\nwith open(path) as p:\n lines = p.readlines()\n\nprint(\"No. Lines in 1 example: {:,}\". format(len(lines)), \"\\n\" +\n \"Example (5 lines): \", lines[0:5])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 🔗 2. How to use a GitHub repo on Kaggle?\n\n> 📌 **Goal**: This competition has a [GitHub repo](https://github.com/location-competition/indoor-location-competition-20) available. We can use the `read_data_file` function in the `io_f` file to read in the information ( no additional struggle on our side + no copy-paste and cluttered code :) ).\n\n#### *🙏 🏻Special thanks to [Laura](https://www.kaggle.com/allunia) for teaching me this awesome trick. 🙏🏻*\n\n**Steps**:\n* 🦶🏻 - download the repo from [this link](https://github.com/location-competition/indoor-location-competition-20)\n* 🦶🏻 - copy the package to the Kaggle environment (`!cp -r path/* ./`)\n* 🦶🏻 - import it and use it as you please"},{"metadata":{"trusted":true},"cell_type":"code","source":"!cp -r ../input/indoor-locationnavigation-2021/indoor-location-competition-20-master/indoor-location-competition-20-master/* ./","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Import custom function from the repository\nfrom io_f import read_data_file\n\n# Read in 1 random example\nsample_file = read_data_file(path)\n\n# You can access the information for each variable:\nprint(\"~~~ Example ~~~\")\nprint(\"acce: {}\".format(sample_file.acce.shape), \"\\n\" +\n \"acacce_uncalice: {}\".format(sample_file.acce_uncali.shape), \"\\n\" +\n \"ahrs: {}\".format(sample_file.ahrs.shape), \"\\n\" +\n \"gyro: {}\".format(sample_file.gyro.shape), \"\\n\" +\n \"gyro_uncali: {}\".format(sample_file.gyro_uncali.shape), \"\\n\" +\n \"ibeacon: {}\".format(sample_file.ibeacon.shape), \"\\n\" +\n \"magn: {}\".format(sample_file.magn.shape), \"\\n\" +\n \"magn_uncali: {}\".format(sample_file.magn_uncali.shape), \"\\n\" +\n \"waypoint: {}\".format(sample_file.waypoint.shape), \"\\n\" +\n \"wifi: {}\".format(sample_file.wifi.shape))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we know how our data is structured, we can start to explore the floor plans, in other words the paths and different features that could be used to predict the `waypoint`, a particular point in the trace.\n\nOk, let's take the EDA step by step!\n\n## 🏢 Sites\n\nFirst, let's look at a random shopping mall and visualize the plan of each floor. For this I've created a custom function, which takes the `site_no` of a site as input, and returns a plt plot of the floor plans in that site.\n\nBelow you can see that for this site example there are 8 floors:\n* 3 underground: *B1, B2* and *B3*\n* 5 above ground: *F1, F2, F3, F4* and *F5*"},{"metadata":{"trusted":true},"cell_type":"code","source":"def show_site_png(site):\n '''This functions outputs the visualization of the .png images available\n in the metadata.\n sites: the code coresponding to 1 site (or building)'''\n \n base = '../input/indoor-location-navigation'\n site_path = f\"{base}/metadata/{site}/*/floor_image.png\"\n floor_paths = glob.glob(site_path)\n n = len(floor_paths)\n\n # Create the custom number of rows & columns\n ncols = [ceil(n / 3) if n > 4 else 4][0]\n nrows = [ceil(n / ncols) if n > 4 else 1][0]\n\n plt.figure(figsize=(16, 10))\n plt.suptitle(f\"Site no. '{site}'\", fontsize=18)\n\n # Plot image for each floor\n for k, floor in enumerate(floor_paths):\n plt.subplot(nrows, ncols, k+1)\n\n image = Image.open(floor)\n image = ImageOps.expand(image, border=15, fill=mycolors[5])\n\n plt.imshow(image)\n plt.axis(\"off\")\n title = floor.split(\"/\")[5]\n plt.title(title, fontsize=15)\n \n # Log to W&B in \"data-understanding\" experiment\n wandb.log({\"Site Floors Example\": plt})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Let's observe 1 example\n# site = '5cd56b64e2acfd2d33b59246'\nshow_site_png(site='5cd56b64e2acfd2d33b592b3')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 🛩 How many floors? \n\nOk, the site plan looks very interesting, but how many floors do we really have on average for each building?\n\nThis information is extremely important, first to observe the *variance* between the sites, and second because the variable `floor` variable is also one of our **target variables**."},{"metadata":{"trusted":true},"cell_type":"code","source":"all_floors = glob.glob(\"../input/indoor-location-navigation/metadata/*/*\")\nfloor_no = []\n\n# Extract only the floor number\nfor floor in all_floors:\n no = floor.split(\"/\")[5]\n floor_no.append(no)\n \nfloor_no = pd.DataFrame(floor_no, columns=[\"No\"])\nfloor_no = floor_no[\"No\"].value_counts().reset_index()\nfloor_no = floor_no.sort_values(\"No\", ascending=False)\n\n# ~~~~\n# PLOT\n# ~~~~\nplt.figure(figsize=(16, 12))\nax = sns.barplot(data=floor_no, x=\"No\", y=\"index\", palette=\"Greens_r\",\n saturation=0.4)\nshow_values_on_bars(ax, h_v=\"h\", space=0.4)\nax.set_title(\"Frequency of Floors\", size = 26, color = mycolors[0], weight='bold')\nax.set_xlabel(\"\")\nax.set_ylabel(\"Floor No.\", size = 18, color = mycolors[0], weight='bold')\nplt.xticks([])\nplt.yticks(fontsize=11)\nsns.despine(left=True, bottom=True);","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Log to W&B in \"data-understanding\" experiment\n### As seaborn direct log is not supported at the moment\n### we'll create a custom barplot\ndata = [[index, no] for (index, no) in zip(floor_no[\"index\"], floor_no[\"No\"])]\ntable = wandb.Table(data=data, columns=[\"index\", \"no\"])\nwandb.log({\"Frequency of Floors\" : wandb.plot.bar(table, \"index\", \"no\", title=\"Frequency of Floors\")})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> This is how it looks in the W&B dashboard:\n

\n\n## Waypoint\n\nThe waypoint is our **second and third target** variables. A waypoint is *\"a stopping place on a journey\"*. Hence, that point would have `x` and `y` coordinates and would belong to a bigger trace.\n\nLet's visualize a random trace in a random building in our data. In the figure below there are:\n* A starting point: the smartphone starts at Start Point 0, marked as a big green bullet point.\n* A trace: the smartphone moves along the edge of the floor's building, takes a right and the walks straight ahead, sometimes moving back and forth. There are around 30 waypoints along this trace.\n* An ending point: the smartphone settles at the End Point, marked with a big red bullet point.\n\nEach point mentioned above is a waypoint and can represent a location on the map that we would need to predict."},{"metadata":{"trusted":true},"cell_type":"code","source":"# GitHub functions\nfrom visualize_f import visualize_trajectory, visualize_heatmap\n\npath = f'{base}/train/5cd56b5ae2acfd2d33b58549/5F/5d06134c4a19c000086c4324.txt'\n\n# Read in a sample\nexample = read_data_file(path)\n\n# ~~~~~~~~~\n\n# Returns timestamp, x, y values\ntrajectory = example.waypoint\n# Removes timestamp (we only need the coordinates)\ntrajectory = trajectory[:, 1:3]\n\n# Prepare floor_plan coresponding with our example\nsite = path.split(\"/\")[4]\nfloorNo = path.split(\"/\")[5]\nfloor_plan_filename = f'{base}/metadata/{site}/{floorNo}/floor_image.png'\n\n# Prepare width_meter & height_meter\n### (taken from the .json file)\njson_plan_filename = f'{base}/metadata/{site}/{floorNo}/floor_info.json'\nwith open(json_plan_filename) as json_file:\n json_data = json.load(json_file)\n \nwidth_meter = json_data[\"map_info\"][\"width\"]\nheight_meter = json_data[\"map_info\"][\"height\"]\n\n# Title\ntitle = \"Example of Waypoint\"\n\n# ~~~~~~~~~\n\n# Finally, let's plot\nvisualize_trajectory(trajectory = trajectory,\n floor_plan_filename = floor_plan_filename,\n width_meter = width_meter,\n height_meter = height_meter,\n title = title,\n g_size=755,\n point_color='#76C1A0',\n start_color='#007B51',\n end_color='#9B0000')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 🧲 Magnetic Strength \n\n> 📌 **Magnetic Strength**: Any point inside a building is subject to unique magnetic forces. Floors, walls, and objects around the room create a *four-dimensional* map of three-dimensional space and magnetic magnitudes. The magnetic magnitude at any point in space can be measured by reading the x, y, and z magnetic vectors at that point.\n\nHence, the phone detects *fluctuations* in the magnetic field as the user moves (and as the phone rotates). In the example below, the magnetic field is much more powerful at the beginning of the trajectory, but less strong at the left side of the floor.\n\n#### 📐 mu tesla (1×104 G) - is a derived unit of the magnetic induction 📐"},{"metadata":{"trusted":true},"cell_type":"code","source":"# GitHub functions\nfrom main import calibrate_magnetic_wifi_ibeacon_to_position\nfrom main import extract_magnetic_strength\n\n# Extracting the magnetic strength\nmwi_datas = calibrate_magnetic_wifi_ibeacon_to_position([path])\nmagnetic_strength = extract_magnetic_strength(mwi_datas)\n\nheat_positions = np.array(list(magnetic_strength.keys()))\nheat_values = np.array(list(magnetic_strength.values()))\n\n# Visualize the heatmap\nvisualize_heatmap(heat_positions, \n heat_values, \n floor_plan_filename,\n width_meter, \n height_meter, \n colorbar_title='mu tesla', \n title='Magnetic Strength',\n g_size=755,\n colorscale='temps')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 📶 WiFi \n\n> 📌 **WiFi Access Points**: The floors of a site can have many WiFi Access Points. Hence, the signals and their strengths will vary widely in different areas.\n\nIn the example below you can see that the route is filled with such access points:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# GitHub Libraries\nfrom main import extract_wifi_rssi, extract_wifi_count\n\n# Get WiFi data\nwifi_rssi = extract_wifi_rssi(mwi_datas)\nprint(f'This floor has {len(wifi_rssi.keys())} wifi aps (access points).')\n\nwifi_counts = extract_wifi_count(mwi_datas)\nheat_positions = np.array(list(wifi_counts.keys()))\nheat_values = np.array(list(wifi_counts.values()))\n# filter out positions that no wifi detected\nmask = heat_values != 0\nheat_positions = heat_positions[mask]\nheat_values = heat_values[mask]\n\n# The heatmap\nvisualize_heatmap(heat_positions, \n heat_values, \n floor_plan_filename, \n width_meter, \n height_meter, \n colorbar_title='count', \n title=f'WiFi Count',\n g_size=755,\n colorscale='temps')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 🔵🦷 IBeacon (Bluetooth) \n\n> 📌 **IBeacon**: Beacons work in tandem with mobile apps to trigger particular messages or actions based on rules, such as triggering a push notification when a user is within a certain distance from a beacon.\n\nThere are many beacons along the example path, however there is a big portion that has no iBeacons at all (circled in orange), therefore the accuracy in this section for this particular example might be lower.\nNote: RSSE is a web feed that can allow a user to keep track of many different websites in a single news aggregator.\n\n#### 📐 dBm (decibel milliwatts) - used as a measure of absolute power; the closer the number is to 0, the better off your signal strength is 📐"},{"metadata":{"trusted":true},"cell_type":"code","source":"# The GitHub function\nfrom main import extract_ibeacon_rssi\n\n# Getting the iBeacon data\nibeacon_rssi = extract_ibeacon_rssi(mwi_datas)\nprint(f'This floor has {len(ibeacon_rssi.keys())} ibeacons.')\nibeacon_ummids = list(ibeacon_rssi.keys())\ntarget_ibeacon = ibeacon_ummids[0]\nheat_positions = np.array(list(ibeacon_rssi[target_ibeacon].keys()))\nheat_values = np.array(list(ibeacon_rssi[target_ibeacon].values()))[:, 0]\n\n# The heatmap\nvisualize_heatmap(heat_positions, \n heat_values, \n floor_plan_filename, \n width_meter, \n height_meter, \n colorbar_title='dBm', \n title='iBeacon RSSE',\n g_size=755,\n colorscale='temps')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ~ END of EXPERIMENT ~\nwandb.finish()\n# ~~~~~~~~~~~~~~~~~~~~~","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. Understanding the Competition Metric\n\n> 📌 **Note**: Although many notebooks I've seen use the same `comp_metric` function to compute the *Evaluation metric* (as below), it took me a a bit to understand the variable and what they mean. Hence, I've decided to share a bit of information around it for those who skipped a bit too early the informations in the description of the competition (like I did 😅).\n\n`def comp_metric(xhat, yhat, fhat, x, y, f):\n intermediate = np.sqrt(np.power(xhat-x, 2) + np.power(yhat-y, 2)) + 15 * np.abs(fhat-f)\n return intermediate.sum()/xhat.shape[0]`\n \n### Explanation\n\nThe above formula is simply the Python version of the mathematical function below (provided by the competition):\n

"},{"metadata":{"trusted":true},"cell_type":"code","source":"from numpy import sqrt, power\nfrom numpy import abs as absolute","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def mean_position_error(x_pred, y_pred, f_pred, x_true, y_true, f_true, p=15):\n '''Custom function to evaluate Mean Position Error.\n x: x coordinate of the waypoint position; dtype list()\n y: y coordinate of the waypoint position; dtype list()\n f: exact floor or the building; dtype list()\n p: floor penalty, set to 15 (always)'''\n \n N = len(x_true)\n #1\n formula = sqrt( power(x_pred - x_true, 2) + power(y_pred - y_true, 2) )\n #2\n formula = formula + p * absolute(f_pred - f_true)\n #3\n formula = formula.sum() / N\n \n return formula","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### RAPIDS function"},{"metadata":{"trusted":true},"cell_type":"code","source":"from cupy import sqrt as sqrt_g\nfrom cupy import power as power_g\nfrom cupy import abs as abs_g","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def mean_position_error_gpu(x_pred, y_pred, f_pred, x_true, y_true, f_true, p=15):\n '''Same, but Faster ;)\n Using RAPIDS here for our XGBoost model later.'''\n \n N = len(x_true)\n formula = sqrt_g( power_g(x_pred - x_true, 2) + power_g(y_pred - y_true, 2) )\n formula = formula + p * abs_g(f_pred - f_true)\n formula = formula.sum() / N\n return formula","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 4. Baseline Model\n\n> 📌 **Note**: Preprocessed data is from [this dataset](https://www.kaggle.com/devinanzelmo/indoor-navigation-and-location-wifi-features) by [Devin Anzelmo](https://www.kaggle.com/devinanzelmo)."},{"metadata":{"trusted":true},"cell_type":"code","source":"def make_submission(predictions, sample_subm, name=\"base.csv\"):\n '''Receives a list of predictions in dataframe format.'''\n final_submission = pd.concat(predictions).reset_index(drop=True)\n final_submission.index = sample_subm.index\n final_submission.to_csv(name)\n print(\"Submission ready.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## I. Light GBM"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Import Libraries\nimport lightgbm as lgb\n\n# ~~~~\n# Data\n# ~~~~\nbase_dir = \"../input/indoor-navigation-and-location-wifi-features/wifi_features\"\ntrain_dir = \"/train/*_train.csv\"\ntest_dir = \"/test/*_test.csv\"\n\n\n# Paths for train & test files\ntrain_paths = sorted(glob.glob(base_dir + train_dir))\ntest_paths = sorted(glob.glob(base_dir + test_dir))\nsample_subm = pd.read_csv('../input/indoor-location-navigation/sample_submission.csv',\n index_col=0)\n\nprint(\"Len Train Files: {}\".format(len(train_paths)), \"\\n\" +\n \"Len Test Files: {}\".format(len(test_paths)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Initialize new experiment (LGBM)\nrun = wandb.init(project=\"indoor-location-kaggle\", name=\"lgbm_train\")\n\nwandb.log({'Len Train Files' : len(train_paths),\n 'Len Test Files' : len(test_paths)})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> Schema of the \"Training loop\" (inspired by [Jiwei Liu's work](https://www.kaggle.com/jiweiliu)):\n

\n\n#### Code Below"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":false},"cell_type":"code","source":"def train_lgbm(train_perc=0.75, version=1, n_estimators=150, num_leaves=127):\n '''\n Training loop\n '''\n\n f = open(f\"lgbm_logs_{version}.txt\", \"w+\")\n lgbm_predictions = []\n \n # Log in W&B\n wandb.log({'n_estimators': n_estimators, 'num_leaves': num_leaves})\n \n k = 1\n for train_path, test_path in zip(train_paths, test_paths):\n\n\n # --- Read in data ---\n train_df = pd.read_csv(train_path, index_col=0)\n train_df = train_df.sample(frac=1, random_state=10)\n\n # Erase last column (which is \"site_path_timestamp\")\n test_df = pd.read_csv(test_path, index_col=0).iloc[:, :-1]\n\n # Sample out training and validation data\n ### we need to be careful to choose same information for ALL 3 models\n ### 1 for x, 1 for y and 1 for floor\n\n train_size = int(len(train_df) * train_perc)\n\n\n # --- Data Validation ---\n # Train features + targets\n X_train = train_df.iloc[:train_size, :-4]\n y_train_x = train_df.iloc[:train_size, -4]\n y_train_y = train_df.iloc[:train_size, -3]\n y_train_f = train_df.iloc[:train_size, -2]\n\n # Valid features + targets\n X_valid = train_df.iloc[train_size:, :-4]\n y_valid_x = train_df.iloc[train_size:, -4]\n y_valid_y = train_df.iloc[train_size:, -3]\n y_valid_f = train_df.iloc[train_size:, -2]\n\n\n # --- Model Training ---\n lgbm_x = lgb.LGBMRegressor(n_estimators=n_estimators, num_leaves=num_leaves)\n lgbm_x.fit(X_train, y_train_x)\n\n lgbm_y = lgb.LGBMRegressor(n_estimators=n_estimators, num_leaves=num_leaves)\n lgbm_y.fit(X_train, y_train_y)\n\n lgbm_f = lgb.LGBMClassifier(n_estimators=n_estimators, num_leaves=num_leaves)\n lgbm_f.fit(X_train, y_train_f)\n\n\n # --- Model Validation Predictions ---\n preds_x = lgbm_x.predict(X_valid)\n preds_y = lgbm_y.predict(X_valid)\n preds_f = lgbm_f.predict(X_valid).astype(int)\n \n mpe = mean_position_error(preds_x, preds_y, preds_f,\n y_valid_x, y_valid_y, y_valid_f)\n print(\"{} | MPE: {}\".format(k, mpe))\n # Save logs\n with open(f\"lgbm_logs_{version}.txt\", 'a+') as f:\n print(\"{} | MPE: {}\".format(k, mpe), file=f)\n \n # Log MPE of this experiment\n wandb.log({'MPE' : mpe, 'step' : k})\n \n k+=1\n\n\n # --- Model Test Predictions ---\n test_preds_x = lgbm_x.predict(test_df)\n test_preds_y = lgbm_y.predict(test_df)\n test_preds_f = lgbm_f.predict(test_df).astype(int)\n\n all_test_preds = pd.DataFrame({'floor' : test_preds_f,\n 'x' : test_preds_x, \n 'y' : test_preds_y})\n lgbm_predictions.append(all_test_preds)\n \n \n return lgbm_predictions","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Uncomment line below to train your own model\n# lgbm_predictions = train_lgbm(train_perc = 0.75, version=1)\n\n# # Logs from my training:\nprint(open('../input/indoor-locationnavigation-2021/lgbm_logs_1.txt', \"r\").read())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> ❗**Attention**: *5th* dataframe had a BIG error (jumped from ~4 on average to 18). This case HAS to be taken into consideration, as the models seems to be underfitting. The *10th, 13th, 21st* have big MPE as well.\n\n### Submission LGBM"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Uncomment line below to make your own submission\n# make_submission(lgbm_predictions, sample_subm, name=\"lgbm_base.csv\")\n\n# My submission:\nlgbm_predictions = pd.read_csv(\"../input/indoor-locationnavigation-2021/lgbm_base.csv\")\nlgbm_predictions.to_csv(\"lgbm_base.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ~ END of EXPERIMENT ~\nwandb.finish()\n# ~~~~~~~~~~~~~~~~~~~~~","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## II. XGBoost - Faster with RAPIDS\n\n> 📌 **Note**: I will use a combination of **RAPIDS** libraries on GPU and XGBoost as one of my base models. More information on this open source suite of libraries [here](https://rapids.ai/)."},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"# Libraries\nimport cudf\nimport cupy\nimport cuml\nimport xgboost\n\n# Adjust floor function\n### As the Multiclass XGBoost takes only labels between [0, n)\n### But we have negative floor values\ndef adjust_floor(df, col_name):\n '''Adjusts the floor to be >= 0.\n Also returns the number fo classes (also used to complete classification).'''\n num_classes = df[col_name].nunique()\n smallest = df[col_name].unique().min()\n df[col_name] = df[col_name] - smallest\n \n return df[col_name], num_classes, smallest","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Initialize new experiment (XGB)\nrun = wandb.init(project=\"indoor-location-kaggle\", name=\"xgb_train\")\n\nwandb.log({'Len Train Files' : len(train_paths),\n 'Len Test Files' : len(test_paths)})","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def train_xgb(train_perc=0.75, version=1):\n '''\n Training loop\n '''\n\n f = open(f\"xgb_logs_{version}.txt\", \"w+\")\n xgb_predictions = []\n \n \n k = 1\n for train_path, test_path in zip(train_paths, test_paths):\n\n\n # --- Read in data ---\n train_df = cudf.read_csv(train_path, index_col=0)\n train_df = train_df.sample(frac=1, random_state=10)\n train_df[\"f\"], num_classes, smallest = adjust_floor(train_df, 'f')\n\n # Erase last column (which is \"site_path_timestamp\")\n test_df = cudf.read_csv(test_path, index_col=0).iloc[:, :-1]\n\n # Sample out training and validation data\n ### we need to be careful to choose same information for ALL 3 models\n ### 1 for x, 1 for y and 1 for floor\n\n train_size = int(len(train_df) * train_perc)\n\n\n # --- Data Validation ---\n # Train features + targets\n X_train = train_df.iloc[:train_size, :-4]\n y_train_x = train_df.iloc[:train_size, -4]\n y_train_y = train_df.iloc[:train_size, -3]\n y_train_f = train_df.iloc[:train_size, -2]\n\n # Valid features + targets\n X_valid = train_df.iloc[train_size:, :-4]\n y_valid_x = cupy.asanyarray(train_df.iloc[train_size:, -4])\n y_valid_y = cupy.asanyarray(train_df.iloc[train_size:, -3])\n y_valid_f = cupy.asanyarray(train_df.iloc[train_size:, -2])\n \n \n # --- Parameters ---\n regr_params = {'max_depth' : 4, 'max_leaves' : 2**4, \n 'tree_method' : 'gpu_hist', 'objective' : 'reg:squarederror',\n 'grow_policy' : 'lossguide', 'colsample_bynode': 0.8}\n classif_params = {'max_depth' : 4, 'max_leaves' : 2**4,\n 'tree_method' : 'gpu_hist', 'objective' : 'multi:softmax',\n 'num_class' : num_classes, 'grow_policy' : 'lossguide',\n 'colsample_bynode': 0.8, 'verbosity' : 0}\n \n # Log once to W&B\n if k == 1:\n wandb.log(regr_params)\n wandb.log(classif_params)\n\n\n # --- Model Training ---\n trainMatrix_x = xgboost.DMatrix(data=X_train, label=y_train_x)\n xgboost_x = xgboost.train(params=regr_params, dtrain=trainMatrix_x)\n\n trainMatrix_y = xgboost.DMatrix(data=X_train, label=y_train_y)\n xgboost_y = xgboost.train(params=regr_params, dtrain=trainMatrix_y)\n\n trainMatrix_f = xgboost.DMatrix(data=X_train, label=y_train_f)\n xgboost_f = xgboost.train(params=classif_params, dtrain=trainMatrix_f)\n\n\n # --- Model Validation Predictions ---\n preds_x = cupy.asanyarray(xgboost_x.predict(xgboost.DMatrix(X_valid)))\n preds_y = cupy.asanyarray(xgboost_y.predict(xgboost.DMatrix(X_valid)))\n preds_f = cupy.asanyarray(xgboost_f.predict(xgboost.DMatrix(X_valid)).astype(int))\n\n mpe = mean_position_error_gpu(preds_x, preds_y, preds_f,\n y_valid_x, y_valid_y, y_valid_f)\n print(\"{} | MPE: {}\".format(k, mpe))\n # Save logs\n with open(f\"xgb_logs_{version}.txt\", 'a+') as f:\n print(\"{} | MPE: {}\".format(k, mpe), file=f)\n \n # Log MPE of this experiment\n wandb.log({'MPE' : mpe, 'step' : k})\n \n k+=1\n\n\n # --- Model Test Predictions ---\n test_preds_x = xgboost_x.predict(xgboost.DMatrix(test_df))\n test_preds_y = xgboost_y.predict(xgboost.DMatrix(test_df))\n test_preds_f = xgboost_f.predict(xgboost.DMatrix(test_df)).astype(int) + smallest\n\n all_test_preds = pd.DataFrame({'floor' : test_preds_f,\n 'x' : test_preds_x, \n 'y' : test_preds_y})\n xgb_predictions.append(all_test_preds)\n \n \n return xgb_predictions","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training"},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"# Uncomment line below to train your own model\n# xgb_predictions = train_xgb(train_perc=0.75, version=1)\n\n# Logs from my training:\nprint(open('../input/indoor-locationnavigation-2021/xgb_logs_1.txt', \"r\").read())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Uncomment line below to make your own submission\n# make_submission(xgb_predictions, sample_subm, name=\"xgb_base.csv\")\n\n# My submission:\nxgb_predictions = pd.read_csv(\"../input/indoor-locationnavigation-2021/xgb_base.csv\")\nxgb_predictions.to_csv(\"xgb_base.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# ~ END of EXPERIMENT ~\nwandb.finish()\n# ~~~~~~~~~~~~~~~~~~~~~","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 5. Save W&B Submissions and Logs\n\n> We can save the predictions and logs to W&B."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Submissions\n### we can save the predictions in W&B\nrun = wandb.init(project='indoor-location-kaggle', name='submissions')\nartifact = wandb.Artifact(name='submissions', \n type='dataset')\n\nartifact.add_file(\"../input/indoor-locationnavigation-2021/lgbm_base.csv\")\nartifact.add_file(\"../input/indoor-locationnavigation-2021/xgb_base.csv\")\n\nwandb.log_artifact(artifact)\nwandb.finish()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Logs\n### we can save the predictions in W&B\nrun = wandb.init(project='indoor-location-kaggle', name='training_logs')\nartifact = wandb.Artifact(name='training_logs', \n type='dataset')\n\nartifact.add_file(\"../input/indoor-locationnavigation-2021/lgbm_logs_1.txt\")\nartifact.add_file(\"../input/indoor-locationnavigation-2021/xgb_logs_1.txt\")\n\nwandb.log_artifact(artifact)\nwandb.finish()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> The Artifact section of the project:\n

\n\n# 🥙 Blending Stirring Cooking \n\n> First let's compare the 2 model's predictions. (needs more work)"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# # Read in data\n# lgb_preds = pd.read_csv(\"../input/indoor-locationnavigation-2021/lgbm_base.csv\")\n# xgb_preds = pd.read_csv(\"../input/indoor-locationnavigation-2021/xgb_base.csv\")\n\n# sample_submission = pd.read_csv(\"../input/indoor-location-navigation/sample_submission.csv\")\n\n# # Sample Submission\n# sample_submission[\"x\"] = lgb_preds[\"x\"] * 0.9 + xgb_preds[\"x\"] * 0.1\n# sample_submission[\"y\"] = lgb_preds[\"y\"] * 0.9 + xgb_preds[\"y\"] * 0.1\n\n# sample_submission.to_csv(\"blend1.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

\n\n# ⌨️🎨 Specs on how I prepped & trained \n### (on my local machine)\n* Z8 G4 Workstation 🖥\n* 2 CPUs & 96GB Memory 💾\n* NVIDIA Quadro RTX 8000 🎮\n* RAPIDS version 0.17 🏃🏾‍♀️"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}