{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Where are your predictions located?\n\nI have read the discussion: [How to create grid points automatically](https://www.kaggle.com/c/indoor-location-navigation/discussion/237776) where I found interesting answers. My work is inspired by such answers.\n\nIn this notebook, you can check whether your predicted waypoints really stand or not. \n*Are your waypoints located between your predicted floor bounds?*\n*Are your waypoints inside stores?*\n\nSmarter usage of post-process techniques such as snap-to-grid can be applied after a careful exploration of your predictions. This notebook aims to help you decide which waypoints should be post-processed and which ones should remain as they are. \n\nDo not hesitate to comment if you spot anything wrong. For instance, I did not understand why some of the Polygons of the GeoJSON data were defined as MultiPolygons. I treated them equally. ","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport shapely\nimport json","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Example - How to use Shapely to work with GeoJSON data\n\nLets take an example to show how to use Shapely to get: floor's polygons, store's polygons and their difference.\n\nWe are going to use GeoJSON data located in metadata folder. Then, we will use shapely to compute the shapes of our points of interest. ","metadata":{}},{"cell_type":"code","source":"site = '5a0546857ecc773753327266'\nfloorNo = 'B1' \nwith open(f\"../input/indoor-location-navigation/metadata/{site}/{floorNo}/geojson_map.json\") as json_file:\n    geofloor_data = json.load(json_file)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The first value of the geometry coordinates is always the **floor coordinates**. The rest is data from stores.","metadata":{}},{"cell_type":"code","source":"from shapely.geometry import Point\nfrom shapely.geometry.polygon import Polygon\nimport shapely.ops as so\n\nfloor_polygon = Polygon(np.array(geofloor_data['features'][0]['geometry']['coordinates'][0]))  # Floor polygon is always the first value\nstore_polygons_l = [Polygon(features['geometry']['coordinates'][0]) for features in geofloor_data['features'][1:]]  # The rest are stores data","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"While the floor is computed just with one polygon the stores require several. Then, we use *unary_union* function from shapely to join them. ","metadata":{}},{"cell_type":"code","source":"store_polygons = so.unary_union(store_polygons_l)\nsafe_area_polygons = floor_polygon.difference(store_polygons)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Floor Polygon","metadata":{}},{"cell_type":"code","source":"floor_polygon","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Store Polygons","metadata":{}},{"cell_type":"code","source":"store_polygons","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Safe Zone (floor - store)","metadata":{}},{"cell_type":"code","source":"safe_area_polygons","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Check where are located your predictions\n\nFirst, we will compute the polygons for each floor. Then we can use Shapely to detect where our predicted points are located.","metadata":{}},{"cell_type":"code","source":"def split_col(df):\n    \"\"\"\n    Split submission site/path/timestamp into individual columns.\n    \"\"\"\n    df = pd.concat(\n        [\n            df[\"site_path_timestamp\"]\n            .str.split(\"_\", expand=True)\n            .rename(columns={0: \"site\", 1: \"path\", 2: \"timestamp\"}),\n            df,\n        ],\n        axis=1,\n    ).copy()\n    return df\n\nsub = split_col(\n    pd.read_csv(\"../input/multioutput-mlp-weighted-loss/submission.csv\")\n)\ntrue_locs = pd.read_csv(\"../input/indoor-location-train-waypoints/train_waypoints.csv\")\n# Add floor No to sub file\nsub = sub.merge(true_locs[[\"site\", \"floor\", \"floorNo\"]].drop_duplicates())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_predictions_location(args):\n    (site, floorNo) , df_submission = args\n    df_result = df_submission.copy()\n    with open(f\"../input/indoor-location-navigation/metadata/{site}/{floorNo}/geojson_map.json\") as json_file:\n        geofloor_data = json.load(json_file)\n    with open(f\"../input/indoor-location-navigation/metadata/{site}/{floorNo}/floor_info.json\") as json_file:\n        floor_info = json.load(json_file)\n    type_poly = geofloor_data['features'][0]['geometry']['type']\n    if type_poly == 'Polygon':\n        polygon = np.array(geofloor_data['features'][0]['geometry']['coordinates'][0])\n    else:\n        polygon = np.array(geofloor_data['features'][0]['geometry']['coordinates'][0][0])\n    floor_polygons = Polygon(polygon)\n    store_polygons_l = [Polygon(features['geometry']['coordinates'][0]) for features in geofloor_data['features'][1:]]\n    store_polygons = so.unary_union(store_polygons_l)\n    safe_area_polygons = floor_polygons.difference(store_polygons)\n    x_max, x_min = polygon[:, 0].max(), polygon[:, 0].min()\n    y_max, y_min = polygon[:, 1].max(), polygon[:, 1].min()\n    df_result['x_scaled'] = x_min + df_result['x'] * (x_max - x_min) / floor_info['map_info']['width']\n    df_result['y_scaled'] = y_min + df_result['y'] * (y_max - y_min) / floor_info['map_info']['height']\n    df_result['InFloor'] = df_result.apply(lambda row: floor_polygons.contains(Point(row['x_scaled'], row['y_scaled'])), axis=1)\n    df_result['InStore'] = df_result.apply(lambda row: store_polygons.contains(Point(row['x_scaled'], row['y_scaled'])), axis=1)\n    df_result['InSafe'] = df_result.apply(lambda row: safe_area_polygons.contains(Point(row['x_scaled'], row['y_scaled'])), axis=1)\n    return df_result","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import multiprocessing\nfrom tqdm import tqdm\n\nprocesses = multiprocessing.cpu_count()\nwith multiprocessing.Pool(processes=processes) as pool:\n    dfs = pool.imap_unordered(add_predictions_location, sub.groupby(['site', 'floorNo']))\n    dfs = tqdm(dfs)\n    dfs = list(dfs)\nsub = pd.concat(dfs).sort_values('site_path_timestamp')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['InFloor'].value_counts(normalize=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['InStore'].value_counts(normalize=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['InSafe'].value_counts(normalize=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Thanks for reading! Hopefully this notebook can help you make a smarter post-process of your raw predictions.","metadata":{}}]}