![\"slide\"](\"https://i.imgur.com/tAhTdUG.png\")
First of all, I would like to thank the host for organizing such a wonderful conference. I would also like to take this opportunity to thank all the wonderful members( @mmotoki, @ryches, @takoihiraokazu , @masatomatsui ) have worked so hard throughout the competition. Finally, I would like to congratulate for @mmotoki becoming a GM with this medal. Congratulations! I'm as happy as myself!
\nThis is our team's solution. Have fun, everyone!
\nOur solution is relatively simliar to some of other teams in the same range as us. Our final submission is a blend of LSTM and LGBM models trained with repeated pseudo labeling and postprocesing. Here we provide a brief description of some of the ideas that worked for us and rough estimates of the gains that we got from them.
\n
We used an MLP inspired by Nigel’s 99% Accuracy public notebook. The idea seemed similar to target encoding where each bssid is mapped to the floor where its max rssi was observed. The MLP generalized this approach using the hard floor estimates (descried earlier), a soft floor estimates derived using a weighted average of bssid-floors where the weight comes from the softmax of rssi, a learned bssid embedding and a beacon weight derived from beacon measurements (e.g., rssi, last_seen). Our model changed the floor predictions on 11 of the paths predicted by the public notebook. Our model had 100% correct predictions on the public and private test sets.
\nWe used a wavenet model for predicting the relative position between two waypoints.
\nInputs are 12 Sensor data (accem gyro, ahrs, and magn). This model resulted in a mean delta positioning error of 1.68 m, and the host’s “compute_rel_positions” code results in a mean error of 2.84 m.
Like most other teams, this step was really important for us. We used a lot of the ideas shared publicly with a few modifications and additional steps
\nIt is difficult to quantify how much each step in our postprocessing pipeline helped, but most of our gains came from the non-leak postprocessing steps. Postprocessing improved the score of our raw predictions by about as 1.0 (public:3.5 and private:3.8).
\nThis was also a really important step for us. After training our base LSTM models, we averaged their predictions, and used the postprocessed predictions as pseudo labels for LGBM. We retrained our LSTM models with the LGBM pseudo labels and repeated the process until it converged. Repeated pseudo labeling improved our score by another 1.0 (public:2.5 and private:2.8)
", "messages": [ { "id": "1314278", "postDate": "05/19/2021 04:48:06", "content": "First of all, I would like to thank the host for organizing such a wonderful conference. I would also like to take this opportunity to thank all the wonderful members( @mmotoki, @ryches, @takoihiraokazu , @masatomatsui ) have worked so hard throughout the competition. Finally, I would like to congratulate for @mmotoki becoming a GM with this medal. Congratulations! I'm as happy as myself!
\nThis is our team's solution. Have fun, everyone!
\nOur solution is relatively simliar to some of other teams in the same range as us. Our final submission is a blend of LSTM and LGBM models trained with repeated pseudo labeling and postprocesing. Here we provide a brief description of some of the ideas that worked for us and rough estimates of the gains that we got from them.
\n
We used an MLP inspired by Nigel’s 99% Accuracy public notebook. The idea seemed similar to target encoding where each bssid is mapped to the floor where its max rssi was observed. The MLP generalized this approach using the hard floor estimates (descried earlier), a soft floor estimates derived using a weighted average of bssid-floors where the weight comes from the softmax of rssi, a learned bssid embedding and a beacon weight derived from beacon measurements (e.g., rssi, last_seen). Our model changed the floor predictions on 11 of the paths predicted by the public notebook. Our model had 100% correct predictions on the public and private test sets.
\nWe used a wavenet model for predicting the relative position between two waypoints.
\nInputs are 12 Sensor data (accem gyro, ahrs, and magn). This model resulted in a mean delta positioning error of 1.68 m, and the host’s “compute_rel_positions” code results in a mean error of 2.84 m.
Like most other teams, this step was really important for us. We used a lot of the ideas shared publicly with a few modifications and additional steps
\nIt is difficult to quantify how much each step in our postprocessing pipeline helped, but most of our gains came from the non-leak postprocessing steps. Postprocessing improved the score of our raw predictions by about as 1.0 (public:3.5 and private:3.8).
\nThis was also a really important step for us. After training our base LSTM models, we averaged their predictions, and used the postprocessed predictions as pseudo labels for LGBM. We retrained our LSTM models with the LGBM pseudo labels and repeated the process until it converged. Repeated pseudo labeling improved our score by another 1.0 (public:2.5 and private:2.8)
", "rawMarkdown": "First of all, I would like to thank the host for organizing such a wonderful conference. I would also like to take this opportunity to thank all the wonderful members( @mmotoki, @ryches, @takoihiraokazu , @masatomatsui ) have worked so hard throughout the competition. Finally, I would like to congratulate for @mmotoki becoming a GM with this medal. Congratulations! I'm as happy as myself!\n\nThis is our team's solution. Have fun, everyone!\n\n# Quick summary\nOur solution is relatively simliar to some of other teams in the same range as us. Our final submission is a blend of LSTM and LGBM models trained with repeated pseudo labeling and postprocesing. Here we provide a brief description of some of the ideas that worked for us and rough estimates of the gains that we got from them.\n\n\n\n\n# Preprocessing\n- Linear interpolation of coordinate at wifi steps\n- Timestamp using the fix from [Jiwei’s Fix the timestamps of test data using DASK](https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask)\n\n# Model\n## Floor Model\nWe used an MLP inspired by [Nigel’s 99% Accuracy public notebook](https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model). The idea seemed similar to target encoding where each bssid is mapped to the floor where its max rssi was observed. The MLP generalized this approach using the hard floor estimates (descried earlier), a soft floor estimates derived using a weighted average of bssid-floors where the weight comes from the softmax of rssi, a learned bssid embedding and a beacon weight derived from beacon measurements (e.g., rssi, last_seen). Our model changed the floor predictions on 11 of the paths predicted by the public notebook. Our model had 100% correct predictions on the public and private test sets. \n\n## Relative Position Model\nWe used a wavenet model for predicting the relative position between two waypoints.\nInputs are 12 Sensor data (accem gyro, ahrs, and magn). This model resulted in a mean delta positioning error of 1.68 m, and the host’s “compute_rel_positions” code results in a mean error of 2.84 m.\n\n# Position Model\n- LGBM - This model was based on the [BIZEN’s public notebook](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold). The base LGBM model was not very effective, but it was particularly effective when pseudo labeling was used.\n- LSTM - Our LSTM model used features from [Kouki’s LSTM by Keras with Unified Wi-Fi Feats notebook](https://www.kaggle.com/kokitanisaka/lstm-by-keras-with-unified-wi-fi-feats) and extra features including the delta of x,y calculated by the host’s “compute_rel_positions”. The length of the input sequence was 2*n+1 (n each before and n after the target to be predicted). We used device id to connect the paths of train and test, and created the input sequence. Our final submission used an average of two slight variants, the score before the post process was public:4.6 and private:4.9. Averaging the raw predictions of our 2 LSTM models (no pseudo labeling) reduced the error by about 0.1 (public:4.5 and private:4.8)\n\n# Postprocessing:\nLike most other teams, this step was really important for us. We used a lot of the ideas shared publicly with a few modifications and additional steps\n- tomooinubushi’s[ time leakage](https://www.kaggle.com/tomooinubushi/postprocessing-based-on-leakage) (1st)\n- nn cost minimization (e-toppos) - similar to the public magn cost minimization but using a blend of relative position predictions and the official code estimates\n- Iterative closest point algorithm. [ICP](https://en.wikipedia.org/wiki/Iterative_closest_point) is often used for robot path planning and alignment of scanned point cloud data, but we thought it could also be used for alignment between estimated paths and grids. We used the training grids and the auto generated grids proposed by e-toppo \n- snap to corridor - Move estimated step positions outside corridor or building to be inside nearest corridor. We used the polygon data of the corridor, and snapped to the corridor using shapely's method “nearest_points”.\n- snap to grid - Almost the same as [the public notebook](https://www.kaggle.com/robikscube/indoor-navigation-snap-to-grid-post-processing), but we divided the snapping into two stages, first snapping to the training grids and then snapping the rest to the auto-generated grids.\n- tomooinubushi’s time leakage (2nd)\n- ganchan’s [Device leakage](https://www.kaggle.com/iwatatakuya/use-leakage-considering-device-id-postprocess)\n\nIt is difficult to quantify how much each step in our postprocessing pipeline helped, but most of our gains came from the non-leak postprocessing steps. Postprocessing improved the score of our raw predictions by about as 1.0 (public:3.5 and private:3.8).\n\n# Repeated Pseudo Labeling\nThis was also a really important step for us. After training our base LSTM models, we averaged their predictions, and used the postprocessed predictions as pseudo labels for LGBM. We retrained our LSTM models with the LGBM pseudo labels and repeated the process until it converged. Repeated pseudo labeling improved our score by another 1.0 (public:2.5 and private:2.8)", "votes": null }, { "id": "1314357", "postDate": "05/19/2021 06:06:08", "content": "Great！Repeated pseudo labeling improved your socore incredibly. Other teams seem to have done this, but not as much as yours.
\nDid you make lgb learned lstm's output and then make lstm learned lstm's output?
\nIt sounds a little strange, but maybe I got it wrong.
\nCould you please give us more detail?
Wow, that's a great boost with pseudo labelling. We also got around ~0.1 boost but not that much, how many times did you repeated it? Also, congratulations for gold finish and @mmotoki on becoming grandmaster. 🙌
", "rawMarkdown": "Wow, that's a great boost with pseudo labelling. We also got around ~0.1 boost but not that much, how many times did you repeated it? Also, congratulations for gold finish and @mmotoki on becoming grandmaster. 🙌", "votes": null }, { "id": "1314813", "postDate": "05/19/2021 11:56:12", "content": "Thanks for a great write-up @yamsam. And it seems that we had exactly the same floor model (since I can now see that the +/-1 floor trick gave me +15 both ways around on the private LB as well as on the public one).
", "rawMarkdown": "Thanks for a great write-up @yamsam. And it seems that we had exactly the same floor model (since I can now see that the +/-1 floor trick gave me +15 both ways around on the private LB as well as on the public one).", "votes": null }, { "id": "1315537", "postDate": "05/19/2021 22:07:38", "content": "Thanks for the comment. I think the repeat training converged after about two or three times.
", "rawMarkdown": "Thanks for the comment. I think the repeat training converged after about two or three times.", "votes": null }, { "id": "1315539", "postDate": "05/19/2021 22:14:34", "content": "Thanks for the comment. I've heard similar comments from other teams, it seems that higepon's team had also the similar effect with pseudo labeling like ours. We blended the predictions of two RNN models first and used them as pseudo label labels for LGBM and RNNs after performing postprocess.
", "rawMarkdown": "Thanks for the comment. I've heard similar comments from other teams, it seems that [higepon's](https://www.kaggle.com/c/indoor-location-navigation/discussion/239919) team had also the similar effect with pseudo labeling like ours. We blended the predictions of two RNN models first and used them as pseudo label labels for LGBM and RNNs after performing postprocess.", "votes": null }, { "id": "1315542", "postDate": "05/19/2021 22:16:35", "content": "Thanks for the validation! Nice!👍
", "rawMarkdown": "Thanks for the validation! Nice!👍", "votes": null } ], "comments": [ { "id": 1314357, "author_name": "max2020", "author_url": "", "post_date": "05/19/2021 06:06:08", "content": "Great！Repeated pseudo labeling improved your socore incredibly. Other teams seem to have done this, but not as much as yours.
\nDid you make lgb learned lstm's output and then make lstm learned lstm's output?
\nIt sounds a little strange, but maybe I got it wrong.
\nCould you please give us more detail?
Thanks for the comment. I've heard similar comments from other teams, it seems that higepon's team had also the similar effect with pseudo labeling like ours. We blended the predictions of two RNN models first and used them as pseudo label labels for LGBM and RNNs after performing postprocess.
", "votes": null, "replies": [] } ] }, { "id": 1314781, "author_name": "nischaydnk", "author_url": "", "post_date": "05/19/2021 11:31:01", "content": "Wow, that's a great boost with pseudo labelling. We also got around ~0.1 boost but not that much, how many times did you repeated it? Also, congratulations for gold finish and @mmotoki on becoming grandmaster. 🙌
", "votes": null, "replies": [ { "id": 1315537, "author_name": "yamsam", "author_url": "", "post_date": "05/19/2021 22:07:38", "content": "Thanks for the comment. I think the repeat training converged after about two or three times.
", "votes": null, "replies": [] } ] }, { "id": 1314813, "author_name": "jbomitchell", "author_url": "", "post_date": "05/19/2021 11:56:12", "content": "Thanks for a great write-up @yamsam. And it seems that we had exactly the same floor model (since I can now see that the +/-1 floor trick gave me +15 both ways around on the private LB as well as on the public one).
", "votes": null, "replies": [ { "id": 1315542, "author_name": "yamsam", "author_url": "", "post_date": "05/19/2021 22:16:35", "content": "Thanks for the validation! Nice!👍
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1314278": "First of all, I would like to thank the host for organizing such a wonderful conference. I would also like to take this opportunity to thank all the wonderful members( @mmotoki, @ryches, @takoihiraokazu , @masatomatsui ) have worked so hard throughout the competition. Finally, I would like to congratulate for @mmotoki becoming a GM with this medal. Congratulations! I'm as happy as myself!\n\nThis is our team's solution. Have fun, everyone!\n\n# Quick summary\nOur solution is relatively simliar to some of other teams in the same range as us. Our final submission is a blend of LSTM and LGBM models trained with repeated pseudo labeling and postprocesing. Here we provide a brief description of some of the ideas that worked for us and rough estimates of the gains that we got from them.\n\n\n\n\n# Preprocessing\n- Linear interpolation of coordinate at wifi steps\n- Timestamp using the fix from [Jiwei’s Fix the timestamps of test data using DASK](https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask)\n\n# Model\n## Floor Model\nWe used an MLP inspired by [Nigel’s 99% Accuracy public notebook](https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model). The idea seemed similar to target encoding where each bssid is mapped to the floor where its max rssi was observed. The MLP generalized this approach using the hard floor estimates (descried earlier), a soft floor estimates derived using a weighted average of bssid-floors where the weight comes from the softmax of rssi, a learned bssid embedding and a beacon weight derived from beacon measurements (e.g., rssi, last_seen). Our model changed the floor predictions on 11 of the paths predicted by the public notebook. Our model had 100% correct predictions on the public and private test sets. \n\n## Relative Position Model\nWe used a wavenet model for predicting the relative position between two waypoints.\nInputs are 12 Sensor data (accem gyro, ahrs, and magn). This model resulted in a mean delta positioning error of 1.68 m, and the host’s “compute_rel_positions” code results in a mean error of 2.84 m.\n\n# Position Model\n- LGBM - This model was based on the [BIZEN’s public notebook](https://www.kaggle.com/hiro5299834/wifi-features-with-lightgbm-kfold). The base LGBM model was not very effective, but it was particularly effective when pseudo labeling was used.\n- LSTM - Our LSTM model used features from [Kouki’s LSTM by Keras with Unified Wi-Fi Feats notebook](https://www.kaggle.com/kokitanisaka/lstm-by-keras-with-unified-wi-fi-feats) and extra features including the delta of x,y calculated by the host’s “compute_rel_positions”. The length of the input sequence was 2*n+1 (n each before and n after the target to be predicted). We used device id to connect the paths of train and test, and created the input sequence. Our final submission used an average of two slight variants, the score before the post process was public:4.6 and private:4.9. Averaging the raw predictions of our 2 LSTM models (no pseudo labeling) reduced the error by about 0.1 (public:4.5 and private:4.8)\n\n# Postprocessing:\nLike most other teams, this step was really important for us. We used a lot of the ideas shared publicly with a few modifications and additional steps\n- tomooinubushi’s[ time leakage](https://www.kaggle.com/tomooinubushi/postprocessing-based-on-leakage) (1st)\n- nn cost minimization (e-toppos) - similar to the public magn cost minimization but using a blend of relative position predictions and the official code estimates\n- Iterative closest point algorithm. [ICP](https://en.wikipedia.org/wiki/Iterative_closest_point) is often used for robot path planning and alignment of scanned point cloud data, but we thought it could also be used for alignment between estimated paths and grids. We used the training grids and the auto generated grids proposed by e-toppo \n- snap to corridor - Move estimated step positions outside corridor or building to be inside nearest corridor. We used the polygon data of the corridor, and snapped to the corridor using shapely's method “nearest_points”.\n- snap to grid - Almost the same as [the public notebook](https://www.kaggle.com/robikscube/indoor-navigation-snap-to-grid-post-processing), but we divided the snapping into two stages, first snapping to the training grids and then snapping the rest to the auto-generated grids.\n- tomooinubushi’s time leakage (2nd)\n- ganchan’s [Device leakage](https://www.kaggle.com/iwatatakuya/use-leakage-considering-device-id-postprocess)\n\nIt is difficult to quantify how much each step in our postprocessing pipeline helped, but most of our gains came from the non-leak postprocessing steps. Postprocessing improved the score of our raw predictions by about as 1.0 (public:3.5 and private:3.8).\n\n# Repeated Pseudo Labeling\nThis was also a really important step for us. After training our base LSTM models, we averaged their predictions, and used the postprocessed predictions as pseudo labels for LGBM. We retrained our LSTM models with the LGBM pseudo labels and repeated the process until it converged. Repeated pseudo labeling improved our score by another 1.0 (public:2.5 and private:2.8)", "1314357": "Great！Repeated pseudo labeling improved your socore incredibly. Other teams seem to have done this, but not as much as yours.\nDid you make lgb learned lstm's output and then make lstm learned lstm's output?\nIt sounds a little strange, but maybe I got it wrong.\nCould you please give us more detail?", "1314781": "Wow, that's a great boost with pseudo labelling. We also got around ~0.1 boost but not that much, how many times did you repeated it? Also, congratulations for gold finish and @mmotoki on becoming grandmaster. 🙌", "1314813": "Thanks for a great write-up @yamsam. And it seems that we had exactly the same floor model (since I can now see that the +/-1 floor trick gave me +15 both ways around on the private LB as well as on the public one).", "1315537": "Thanks for the comment. I think the repeat training converged after about two or three times.", "1315539": "Thanks for the comment. I've heard similar comments from other teams, it seems that [higepon's](https://www.kaggle.com/c/indoor-location-navigation/discussion/239919) team had also the similar effect with pseudo labeling like ours. We blended the predictions of two RNN models first and used them as pseudo label labels for LGBM and RNNs after performing postprocess.", "1315542": "Thanks for the validation! Nice!👍" }, "source": "meta" }