This was a very fun competition! I actually have a background in this industry (8 years at an indoor location startup) which is what attracted me to participate. I didn't work on the actual location algorithms there however, so this was mostly all new and interesting.
\n
\nI planned to write a much longer post with more details and diagrams, but I spent all day today trying to improve my score, so I didn't have time to write it today 😆
\n
\n(If you'd like to see more info about any of these steps in a longer post, let me know - I plan to write up more actual details about my models and process)
\n
\nHere are the basics of what I did:
\n
I made a mini web app to explore the data, which was incredibly helpful! Here's a screenshot of what that looked like:
\n![\"Indoor](\"https://i.imgur.com/1sN94Q1.png\")
\nIs it ugly? yes. 😂 but it's also very functional.
\n
\nIt shows the selected path and all the waypoints for the floor, as well as some sensor data. It also shows the previous end point and next start point of the paths on that floor, as well as the time since the previous end and time until the next path start.
\n
\nAll of that data was incredibly helpful in understanding how the data was collected, and how different sites and floors varied from each other.
\n
I used a simple CNN based wifi model, combined with some public RNN notebooks (thank you!) and a bit of the device + timestamp leakage I outlined here: https://www.kaggle.com/c/indoor-location-navigation/discussion/234543 to predict the floor for each path.
\n
\nThe data leakage probably only affected < 1% of the floor predictions, so it was fairly minor I think.\n
\nI felt pretty good about those floor predictions, so for the remainder of the steps I then kept those predictions constant.
\n
I tried several RNNs but they either didn't train, or didn't give good results, so I settled on a CNN based model for wifi prediction.
\n
\nI predicted both individual waypoint x,y points as well as the entire path centroid (multiple models) and then ensembled that all with a few of the best public notebooks to get approximate x,y positions for the entire path
\n
Here's where I think my solution might be a bit unique:
\n
\nI split each path into multiple \"legs\", and then used the sensor data to predict the delta_x and delta_y for each leg individually. I did this using a CNN for the sensor data + a lot of extra features in a giant model (that I'll detail in a future post), and was able to get a really solid representation of the entire path \"shape\". (down to about +- 1m MAE per leg).
\n
\nI also fine-tuned these models on a per-site basis, which gave it about a 20% boost (according to my training validation split).
\n
\nI was worried that the error would compound when I put it all together (especially for longer paths), but that didn't seem to be the case.
\n
Probably 75% of my time was spent on post-processing, trying to fit the paths (of a predicted shape) onto the floorplan at the predicted wifi x,y point.
\n
\nAgain, I'll write up more details, but some of the post-processing operations I created were:
\n
My final submission were ensembles of those ensembles - either just averaging, or a few extra rules, including only averaging the closest 2 out of 3 predicted points, or checking the path relative waypoint distance or distance from the predicted wifi.
\n
From my background in indoor location, I knew that uncalibrated wifi would only ever get to about 4m - 5m accuracy, which I think played out in the results.
\n
\nI knew that I would have to switch to sensor data sooner rather than later, and so the decision to do that quickly gave me more time to explore the sensor data.
\n
I never really got a good cross validation approach working for the post-processing.
\n
\nI was able to do validation for the CNN models I made, but trying to post process the training data never quite worked out.
\n
\nI should have spent more time setting up proper CV for the post-processing of the training data, so I didn't have to rely so much on the public LB for feedback.
\n
I feel like I extracted just about all I could with a dual model + post processing setup… I really tried hard to make a unified single model work, but I kept getting stuck. Good job for any team that made it work!!
\n
Thanks to everyone who participated - this was my first Kaggle competition, and a bunch of fun! I'll post more details in the next few days, and I look forward to competing in the future :)
\n
This was a very fun competition! I actually have a background in this industry (8 years at an indoor location startup) which is what attracted me to participate. I didn't work on the actual location algorithms there however, so this was mostly all new and interesting.
\n
\nI planned to write a much longer post with more details and diagrams, but I spent all day today trying to improve my score, so I didn't have time to write it today 😆
\n
\n(If you'd like to see more info about any of these steps in a longer post, let me know - I plan to write up more actual details about my models and process)
\n
\nHere are the basics of what I did:
\n
I made a mini web app to explore the data, which was incredibly helpful! Here's a screenshot of what that looked like:
\n
\nIs it ugly? yes. 😂 but it's also very functional.
\n
\nIt shows the selected path and all the waypoints for the floor, as well as some sensor data. It also shows the previous end point and next start point of the paths on that floor, as well as the time since the previous end and time until the next path start.
\n
\nAll of that data was incredibly helpful in understanding how the data was collected, and how different sites and floors varied from each other.
\n
I used a simple CNN based wifi model, combined with some public RNN notebooks (thank you!) and a bit of the device + timestamp leakage I outlined here: https://www.kaggle.com/c/indoor-location-navigation/discussion/234543 to predict the floor for each path.
\n
\nThe data leakage probably only affected < 1% of the floor predictions, so it was fairly minor I think.\n
\nI felt pretty good about those floor predictions, so for the remainder of the steps I then kept those predictions constant.
\n
I tried several RNNs but they either didn't train, or didn't give good results, so I settled on a CNN based model for wifi prediction.
\n
\nI predicted both individual waypoint x,y points as well as the entire path centroid (multiple models) and then ensembled that all with a few of the best public notebooks to get approximate x,y positions for the entire path
\n
Here's where I think my solution might be a bit unique:
\n
\nI split each path into multiple \"legs\", and then used the sensor data to predict the delta_x and delta_y for each leg individually. I did this using a CNN for the sensor data + a lot of extra features in a giant model (that I'll detail in a future post), and was able to get a really solid representation of the entire path \"shape\". (down to about +- 1m MAE per leg).
\n
\nI also fine-tuned these models on a per-site basis, which gave it about a 20% boost (according to my training validation split).
\n
\nI was worried that the error would compound when I put it all together (especially for longer paths), but that didn't seem to be the case.
\n
Probably 75% of my time was spent on post-processing, trying to fit the paths (of a predicted shape) onto the floorplan at the predicted wifi x,y point.
\n
\nAgain, I'll write up more details, but some of the post-processing operations I created were:
\n
My final submission were ensembles of those ensembles - either just averaging, or a few extra rules, including only averaging the closest 2 out of 3 predicted points, or checking the path relative waypoint distance or distance from the predicted wifi.
\n
From my background in indoor location, I knew that uncalibrated wifi would only ever get to about 4m - 5m accuracy, which I think played out in the results.
\n
\nI knew that I would have to switch to sensor data sooner rather than later, and so the decision to do that quickly gave me more time to explore the sensor data.
\n
I never really got a good cross validation approach working for the post-processing.
\n
\nI was able to do validation for the CNN models I made, but trying to post process the training data never quite worked out.
\n
\nI should have spent more time setting up proper CV for the post-processing of the training data, so I didn't have to rely so much on the public LB for feedback.
\n
I feel like I extracted just about all I could with a dual model + post processing setup… I really tried hard to make a unified single model work, but I kept getting stuck. Good job for any team that made it work!!
\n
Thanks to everyone who participated - this was my first Kaggle competition, and a bunch of fun! I'll post more details in the next few days, and I look forward to competing in the future :)
\n
thanks for the writeup @chris62 , very interesting. Can you share the score of your model without any postprocessing (or basic) ?
", "rawMarkdown": "thanks for the writeup @chris62 , very interesting. Can you share the score of your model without any postprocessing (or basic) ?", "votes": null }, { "id": "1312271", "postDate": "05/18/2021 00:30:07", "content": "Wifi alone was 4m - 5m (public LB)
\nWifi + sensor data was 3m - 4m (public LB)
\npost processing brought that down < 3m (public LB)
\n\nIs it ugly? yes. 😂 but it's also very functional.
\n
Not ugly at all. It is kicking ass! Congrats!
", "rawMarkdown": "> Is it ugly? yes. 😂 but it's also very functional.\n\nNot ugly at all. It is kicking ass! Congrats!", "votes": null }, { "id": "1312383", "postDate": "05/18/2021 02:54:22", "content": "I totally agree about cv being challenging w/ post processing. A simple example is how every snap-to-grid step had to have separate lists of out of fold waypoints. Similarly, I recalculated leaky waypoints based only on the out-of-fold paths that were adjacent to in fold ones. And then repeat the process for submission paths. Managing this pipeline and tracking all the versions of things took up a major fraction of my time and attention.
", "rawMarkdown": "I totally agree about cv being challenging w/ post processing. A simple example is how every snap-to-grid step had to have separate lists of out of fold waypoints. Similarly, I recalculated leaky waypoints based only on the out-of-fold paths that were adjacent to in fold ones. And then repeat the process for submission paths. Managing this pipeline and tracking all the versions of things took up a major fraction of my time and attention.", "votes": null }, { "id": "1312388", "postDate": "05/18/2021 02:57:47", "content": "I also like that you modeled the shape independently. I suspected shape was very important, but I relied heavily on the “cost min” method and the provided GitHub code as gospel for the shape, I didn’t think to model it myself. Nice work, and congrats!
", "rawMarkdown": "I also like that you modeled the shape independently. I suspected shape was very important, but I relied heavily on the “cost min” method and the provided GitHub code as gospel for the shape, I didn’t think to model it myself. Nice work, and congrats!", "votes": null }, { "id": "1312748", "postDate": "05/18/2021 08:15:36", "content": "This is an interesting write-up @chris62. Congrats!!
", "rawMarkdown": "This is an interesting write-up @chris62. Congrats!!", "votes": null }, { "id": "1312793", "postDate": "05/18/2021 08:51:48", "content": "Congrats Chris! Looking forward to more details
", "rawMarkdown": "Congrats Chris! Looking forward to more details", "votes": null }, { "id": "1314117", "postDate": "05/19/2021 00:55:13", "content": "Building a webapp to to explore the data 💯. Awesome solution and congrats on the finish.
", "rawMarkdown": "Building a webapp to to explore the data 💯. Awesome solution and congrats on the finish.", "votes": null } ], "comments": [ { "id": 1312265, "author_name": "jesucristo", "author_url": "", "post_date": "05/18/2021 00:26:46", "content": "thanks for the writeup @chris62 , very interesting. Can you share the score of your model without any postprocessing (or basic) ?
", "votes": null, "replies": [ { "id": 1312271, "author_name": "chris62", "author_url": "", "post_date": "05/18/2021 00:30:07", "content": "Wifi alone was 4m - 5m (public LB)
\nWifi + sensor data was 3m - 4m (public LB)
\npost processing brought that down < 3m (public LB)
\n\nIs it ugly? yes. 😂 but it's also very functional.
\n
Not ugly at all. It is kicking ass! Congrats!
", "votes": null, "replies": [] }, { "id": 1312383, "author_name": "paulfornia", "author_url": "", "post_date": "05/18/2021 02:54:22", "content": "I totally agree about cv being challenging w/ post processing. A simple example is how every snap-to-grid step had to have separate lists of out of fold waypoints. Similarly, I recalculated leaky waypoints based only on the out-of-fold paths that were adjacent to in fold ones. And then repeat the process for submission paths. Managing this pipeline and tracking all the versions of things took up a major fraction of my time and attention.
", "votes": null, "replies": [ { "id": 1312388, "author_name": "paulfornia", "author_url": "", "post_date": "05/18/2021 02:57:47", "content": "I also like that you modeled the shape independently. I suspected shape was very important, but I relied heavily on the “cost min” method and the provided GitHub code as gospel for the shape, I didn’t think to model it myself. Nice work, and congrats!
", "votes": null, "replies": [] } ] }, { "id": 1312748, "author_name": "ankitp013", "author_url": "", "post_date": "05/18/2021 08:15:36", "content": "This is an interesting write-up @chris62. Congrats!!
", "votes": null, "replies": [] }, { "id": 1312793, "author_name": "olaf2000", "author_url": "", "post_date": "05/18/2021 08:51:48", "content": "Congrats Chris! Looking forward to more details
", "votes": null, "replies": [] }, { "id": 1314117, "author_name": "robikscube", "author_url": "", "post_date": "05/19/2021 00:55:13", "content": "Building a webapp to to explore the data 💯. Awesome solution and congrats on the finish.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1312262": "This was a very fun competition! I actually have a background in this industry (8 years at an indoor location startup) which is what attracted me to participate. I didn't work on the actual location algorithms there however, so this was mostly all new and interesting.\n\n