{"cells":[{"metadata":{},"cell_type":"markdown","source":"This notebook is inspired by [Yaroslav Isaienkov's](https://www.kaggle.com/ihelon) great [EDA].(https://www.kaggle.com/ihelon/indoor-location-exploratory-data-analysis)\nPlease upvote his notebook as well."},{"metadata":{},"cell_type":"markdown","source":"# # Indoor Location - Show all trajectories\n\n\nIn this competition, your task is to predict the indoor position of smartphones. **Since the trajectories are collected by a site-surveyor, I have a feeling that the surveyor has a fixed `region` within the building to explore and it is likely that we should always predict the trajectories in that region.**\n\n**In this notebook, we show all the trajectories of a floor plan in training data and check how much area the surveyor covers.**"},{"metadata":{},"cell_type":"markdown","source":"**<font color='blue'>From the results, it is clear that the surveyor has a well pre-defined regions to cover, which could help us refine where to predict.</font>**"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import os\nimport glob\nimport math\nimport json\n\nfrom dataclasses import dataclass\n\nimport numpy as np\nimport pandas as pd\nimport cv2\nimport matplotlib.pyplot as plt\nimport plotly.graph_objs as go\nfrom PIL import Image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PATH = \"../input/indoor-location-navigation\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def plot_all_floors(path_to_building):\n    plt.figure(figsize=(16, 10))\n    floor_paths = glob.glob(os.path.join(path_to_building, \"*/floor_image.png\"))\n    for ind, floor_path in enumerate(floor_paths):\n        w = math.ceil(len(floor_paths) / 2)\n        h = math.ceil(len(floor_paths) / w)\n        if len(floor_paths) <= 4:\n            w = 4\n            h = 1\n        plt.subplot(h, w, ind + 1)\n        image = cv2.imread(floor_path)  \n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        plt.imshow(image)\n        plt.axis(\"off\")\n        plt.title(floor_path.split(\"/\")[-2], fontsize=16)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# copy from https://github.com/location-competition/indoor-location-competition-20/blob/master/io_f.py\n\n\n@dataclass\nclass ReadData:\n    acce: np.ndarray\n    acce_uncali: np.ndarray\n    gyro: np.ndarray\n    gyro_uncali: np.ndarray\n    magn: np.ndarray\n    magn_uncali: np.ndarray\n    ahrs: np.ndarray\n    wifi: np.ndarray\n    ibeacon: np.ndarray\n    waypoint: np.ndarray\n\n\ndef read_data_file(data_filename):\n    acce = []\n    acce_uncali = []\n    gyro = []\n    gyro_uncali = []\n    magn = []\n    magn_uncali = []\n    ahrs = []\n    wifi = []\n    ibeacon = []\n    waypoint = []\n\n    with open(data_filename, 'r', encoding='utf-8') as file:\n        lines = file.readlines()\n\n    for line_data in lines:\n        line_data = line_data.strip()\n        if not line_data or line_data[0] == '#':\n            continue\n\n        line_data = line_data.split('\\t')\n\n        if line_data[1] == 'TYPE_ACCELEROMETER':\n            acce.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_ACCELEROMETER_UNCALIBRATED':\n            acce_uncali.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_GYROSCOPE':\n            gyro.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_GYROSCOPE_UNCALIBRATED':\n            gyro_uncali.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_MAGNETIC_FIELD':\n            magn.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_MAGNETIC_FIELD_UNCALIBRATED':\n            magn_uncali.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_ROTATION_VECTOR':\n            ahrs.append([int(line_data[0]), float(line_data[2]), float(line_data[3]), float(line_data[4])])\n            continue\n\n        if line_data[1] == 'TYPE_WIFI':\n            sys_ts = line_data[0]\n            ssid = line_data[2]\n            bssid = line_data[3]\n            rssi = line_data[4]\n            lastseen_ts = line_data[6]\n            wifi_data = [sys_ts, ssid, bssid, rssi, lastseen_ts]\n            wifi.append(wifi_data)\n            continue\n\n        if line_data[1] == 'TYPE_BEACON':\n            ts = line_data[0]\n            uuid = line_data[2]\n            major = line_data[3]\n            minor = line_data[4]\n            rssi = line_data[6]\n            ibeacon_data = [ts, '_'.join([uuid, major, minor]), rssi]\n            ibeacon.append(ibeacon_data)\n            continue\n\n        if line_data[1] == 'TYPE_WAYPOINT':\n            waypoint.append([int(line_data[0]), float(line_data[2]), float(line_data[3])])\n\n    acce = np.array(acce)\n    acce_uncali = np.array(acce_uncali)\n    gyro = np.array(gyro)\n    gyro_uncali = np.array(gyro_uncali)\n    magn = np.array(magn)\n    magn_uncali = np.array(magn_uncali)\n    ahrs = np.array(ahrs)\n    wifi = np.array(wifi)\n    ibeacon = np.array(ibeacon)\n    waypoint = np.array(waypoint)\n    \n#     print(acce.shape)\n#     print(acce_uncali.shape)\n#     print(gyro.shape)\n#     print(gyro_uncali.shape)\n#     print(magn.shape)\n#     print(magn_uncali.shape)\n#     print(ahrs.shape)\n#     print(wifi.shape)\n#     print(ibeacon.shape)\n#     print(waypoint.shape)\n    \n    return ReadData(acce, acce_uncali, gyro, gyro_uncali, magn, magn_uncali, ahrs, wifi, ibeacon, waypoint)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# copy from https://github.com/location-competition/indoor-location-competition-20/blob/master/visualize_f.py\n\n\ndef save_figure_to_html(fig, filename):\n    fig.write_html(filename)\n\ndef get_lists_of_one_trajectory(trajectory, full_name=True):\n    # add trajectory\n    size_list = [6] * trajectory.shape[0]\n    size_list[0] = 10\n    size_list[-1] = 10\n\n    color_list = ['rgba(4, 174, 4, 0.5)'] * trajectory.shape[0]\n    color_list[0] = 'rgba(12, 5, 235, 1)'\n    color_list[-1] = 'rgba(235, 5, 5, 1)'\n\n    position_count = {}\n    text_list = []\n    for i in range(trajectory.shape[0]):\n        if str(trajectory[i]) in position_count:\n            position_count[str(trajectory[i])] += 1\n        else:\n            position_count[str(trajectory[i])] = 0\n        text_list.append('        ' * position_count[str(trajectory[i])] + f'{i}')\n    if full_name:\n        text_list[0] = f'{name} Start Point: 0'\n        text_list[-1] = f'{name} End Point: {trajectory.shape[0] - 1}'\n    return size_list, color_list, text_list\n    \n\ndef visualize_trajectory(trajectory, floor_plan_filename, width_meter, height_meter, title=None, mode='lines + markers + text', show=False):\n    fig = go.Figure()\n\n    size_list, color_list, text_list = get_lists_of_one_trajectory(trajectory)\n\n    fig.add_trace(\n        go.Scattergl(\n            x=trajectory[:, 0],\n            y=trajectory[:, 1],\n            mode=mode,\n            marker=dict(size=size_list, color=color_list),\n            line=dict(shape='linear', color='rgb(100, 10, 100)', width=2, dash='dot'),\n            text=text_list,\n            textposition=\"top center\",\n            name='trajectory',\n        ))\n\n    # add floor plan\n    floor_plan = Image.open(floor_plan_filename)\n    fig.update_layout(images=[\n        go.layout.Image(\n            source=floor_plan,\n            xref=\"x\",\n            yref=\"y\",\n            x=0,\n            y=height_meter,\n            sizex=width_meter,\n            sizey=height_meter,\n            sizing=\"contain\",\n            opacity=1,\n            layer=\"below\",\n        )\n    ])\n\n    # configure\n    fig.update_xaxes(autorange=False, range=[0, width_meter])\n    fig.update_yaxes(autorange=False, range=[0, height_meter], scaleanchor=\"x\", scaleratio=1)\n    fig.update_layout(\n        title=go.layout.Title(\n            text=title or \"No title.\",\n            xref=\"paper\",\n            x=0,\n        ),\n        autosize=True,\n        width=900,\n        height=200 + 900 * height_meter / width_meter,\n        template=\"plotly_white\",\n    )\n\n    if show:\n        fig.show()\n\n    return fig\n\n\ndef visualize_trajectories(trajectories, floor_plan_filename, width_meter, height_meter, title=None, mode='lines + markers + text', show=False):\n    fig = go.Figure()\n\n    for c,trajectory in enumerate(trajectories):\n        size_list, color_list, text_list = get_lists_of_one_trajectory(trajectory,False)\n        fig.add_trace(\n            go.Scattergl(\n                x=trajectory[:, 0],\n                y=trajectory[:, 1],\n                mode=mode,\n                marker=dict(size=size_list, color=color_list),\n                line=dict(shape='linear', color='rgb(100, 10, 100)', width=2, dash='dot'),\n                text=text_list,\n                textposition=\"top center\",\n                name=None#'trajectory',\n            ))\n\n    # add floor plan\n    floor_plan = Image.open(floor_plan_filename)\n    fig.update_layout(images=[\n        go.layout.Image(\n            source=floor_plan,\n            xref=\"x\",\n            yref=\"y\",\n            x=0,\n            y=height_meter,\n            sizex=width_meter,\n            sizey=height_meter,\n            sizing=\"contain\",\n            opacity=1,\n            layer=\"below\",\n        )\n    ])\n\n    # configure\n    fig.update_xaxes(autorange=False, range=[0, width_meter])\n    fig.update_yaxes(autorange=False, range=[0, height_meter], scaleanchor=\"x\", scaleratio=1)\n    fig.update_layout(\n        title=go.layout.Title(\n            text=title or \"No title.\",\n            xref=\"paper\",\n            x=0,\n        ),\n        autosize=True,\n        width=900,\n        height=200 + 900 * height_meter / width_meter,\n        template=\"plotly_white\",\n    )\n\n    if show:\n        fig.show()\n\n    return fig\n\n\ndef visualize_heatmap(position, value, floor_plan_filename, width_meter, height_meter, colorbar_title=\"colorbar\", title=None, show=False):\n    fig = go.Figure()\n\n    # add heat map\n    fig.add_trace(\n        go.Scatter(x=position[:, 0],\n                   y=position[:, 1],\n                   mode='markers',\n                   marker=dict(size=7,\n                               color=value,\n                               colorbar=dict(title=colorbar_title),\n                               colorscale=\"Rainbow\"),\n                   text=value,\n                   name=title))\n\n    # add floor plan\n    floor_plan = Image.open(floor_plan_filename)\n    fig.update_layout(images=[\n        go.layout.Image(\n            source=floor_plan,\n            xref=\"x\",\n            yref=\"y\",\n            x=0,\n            y=height_meter,\n            sizex=width_meter,\n            sizey=height_meter,\n            sizing=\"contain\",\n            opacity=1,\n            layer=\"below\",\n        )\n    ])\n\n    # configure\n    fig.update_xaxes(autorange=False, range=[0, width_meter])\n    fig.update_yaxes(autorange=False, range=[0, height_meter], scaleanchor=\"x\", scaleratio=1)\n    fig.update_layout(\n        title=go.layout.Title(\n            text=title or \"No title.\",\n            xref=\"paper\",\n            x=0,\n        ),\n        autosize=True,\n        width=900,\n        height=200 + 900 * height_meter / width_meter,\n        template=\"plotly_white\",\n    )\n\n    if show:\n        fig.show()\n\n    return fig","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def visualize_train_trajectory(path):\n    _id, floor = path.split(\"/\")[:2]\n    \n    train_floor_data = read_data_file(os.path.join(\"../input/indoor-location-navigation/train/\", path))\n    with open(f\"../input/indoor-location-navigation/metadata/{_id}/{floor}/floor_info.json\") as f:\n        train_floor_info = json.load(f)\n    #print(train_floor_data.waypoint)\n    return visualize_trajectory(\n        train_floor_data.waypoint[:, 1:3], \n        f\"../input/indoor-location-navigation/metadata/{_id}/{floor}/floor_image.png\",\n        train_floor_info[\"map_info\"][\"width\"], \n        train_floor_info[\"map_info\"][\"height\"],\n    )\n\ndef visualize_train_trajectories(paths):\n    # must be from the same building and the same floor\n    fps = set([os.path.dirname(path) for path in paths])\n    assert len(fps) == 1\n    \n    _id, floor = paths[0].split(\"/\")[-3:-1]\n    print(_id, floor)\n    \n    with open(f\"{PATH}/metadata/{_id}/{floor}/floor_info.json\") as f:\n        train_floor_info = json.load(f)\n    \n    trajectories = []\n    for path in paths:\n        train_floor_data = read_data_file(path)\n        trajectories.append(train_floor_data.waypoint[:, 1:3])\n\n        \n    return visualize_trajectories(\n        trajectories, \n        f\"{PATH}/metadata/{_id}/{floor}/floor_image.png\",\n        train_floor_info[\"map_info\"][\"width\"], \n        train_floor_info[\"map_info\"][\"height\"],\n        title = f\"{_id} {floor}\", show=True\n    )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"floorplans = sorted(glob.glob(f\"{PATH}/train/*/*\"))\nprint(\"Number of floor plans:\", len(floorplans))\nfloorplans[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"paths = {fp:glob.glob(f\"{fp}/*.txt\") for fp in floorplans}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Show one building\n\nSince there are many floorplans, let's just show one building in this notebook, you can also show them all if you want."},{"metadata":{"trusted":true},"cell_type":"code","source":"building_0 = floorplans[0].split('/')[-2]\nprint('Building', building_0)\nfor fp in floorplans:\n    if fp.split('/')[-2]!=building_0:\n        break\n    paths = glob.glob(f\"{fp}/*.txt\")\n    visualize_train_trajectories(paths)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Show them all!!\nUncomment and run the following code"},{"metadata":{"trusted":true},"cell_type":"code","source":"\"\"\"\nfor fp in floorplans:\n    paths = glob.glob(f\"{fp}/*.txt\")\n    visualize_train_trajectories(paths)\n\"\"\"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}