{
  "id": 240116,
  "title": "18th place solution (Moro & taksai)",
  "url": "/competitions/indoor-location-navigation/writeups/nakano-18th-place-solution-moro-taksai",
  "author_name": "",
  "post_date": "2021-05-22T15:07:20.570Z",
  "votes": 12,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Thank you to the organizers for the fun competition and everyone who participated. And thank you to my teammate ( taksai <a href=\"https://www.kaggle.com/tsaito21219\" target=\"_blank\">@tsaito21219</a> ).<br>\nI share our team's solution.</p>\n<h1>summary</h1>\n<ul>\n<li>train xy-model with only wifi-data</li>\n<li>train delta-model with sensor-data for cost-minimization</li>\n<li>postprocess repeatly 20times</li>\n</ul>\n<h1>1. preprocess</h1>\n<p>we make 4 dataset for the two model described later.</p>\n<table>\n<thead>\n<tr>\n<th>dataset</th>\n<th>describe</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>(a)bssid-ranking</td>\n<td>list of bssids arranged in descending order of rssi. (common to buildings)</td>\n</tr>\n<tr>\n<td>(b)bssid-rssi-matrix</td>\n<td>bssid as a columns and rssi as data (for each building).  Fill missing-value in -120.  Drop rssi-value over 10000ms from last-seen-timestamp.</td>\n</tr>\n<tr>\n<td>(c)sensor-data(aggregate)</td>\n<td>aggregate data between wifi. (1 record for 1 target)</td>\n</tr>\n<tr>\n<td>(d)sensor-data(sampling)</td>\n<td>sampling raw data between wifi every 100ms (N record for 1 target).  Padding to a fixed length.</td>\n</tr>\n</tbody>\n</table>\n<h1>2. models</h1>\n<p>we make 2 type of model. First is xy-model which predict position of waypoint, Second is delta-model which predict distance between two waypoints.</p>\n<h2>(1) xy-model</h2>\n<ul>\n<li>GBDT (lightgbm)  with dataset-(b): each site</li>\n<li>MLP (keras) with dataset-(a): 1model</li>\n<li>MLP (keras) with dataset-(b): each site  -&gt; LB=6.5 (MLP no postprocess)</li>\n</ul>\n<h2>(2) delta-model</h2>\n<ul>\n<li>This model is used for cost-minimization(postprocess). Since the delta calculated using the github function  has a little error, the error is reduced by creating a prediction model. <ul>\n<li>error(mean of sum of squared error): 13.1(use github) -&gt; 4.48(our model)</li>\n<li>MAE(mean over x and y of absolute error) : 1.69(use github) -&gt; 1.08(our model) </li></ul></li>\n<li>MLP (keras) with dataset-(c): Dense(128) &gt; Dense(256) &gt; Dense(128) &gt; Dense(64) &gt; Dense(2)</li>\n<li>1d-cnn (keras) with dataset-(d):  Conv1D(filters=32,kernel_size=5,padding=2) &gt; Conv1D(64,5,2) &gt; Conv1D(128,3,2) &gt; Conv1D(256,3,2) &gt; Conv1D(512,2,2) &gt; GlobalMaxPool1D &gt; Dense(64) &gt; Dense(2)</li>\n</ul>\n<h1>3. postprocess</h1>\n<p>We customized postprocessing based on some useful public kernel.</p>\n<ul>\n<li>1st step: (LB=6.5 -&gt; 3.0)<br>\n1) ensemble: weighted averaged the predicted value of some xy-models. The same applies to delta-model.<br>\n2) tune xy for leakage considering device-id<br>\n3) cost-minimization: use delta calculated by delta-model instead of github function.<br>\n4) snap-to-grid<br>\n5) repeat 2)-4) 20times while adjusting the threshold of snap-to-grid. Move to the grid little by little.   threshold: 2 (1-5 times) &gt; 3 (6-10 times)&gt; 4 (11-20 times)<br>\n6) execute 2) again</li>\n<li>2nd step: (LB=3.0 -&gt; 2.75)<br>\n1) get output(xy) of 1st step about some patterns(ensemble weight etc.)<br>\n2) execute 1st step again with 1)</li>\n</ul>\n<h1>Not work</h1>\n<ul>\n<li>use data of other site(over 200). I think it is efficient for delta-model, but don't work.</li>\n<li>train one model which predict both xy and delta (use RNN and transformer)</li>\n<li>learn multiple waypoints of a path together</li>\n</ul>\n<p>we didn't make floor-model. Our team' floor-loss is big about private dataset. Big mistake… So private score is worse than public.<br>\nAlthough there were many ideas for this competition, most of them were ineffective and it was a very difficult competition.I almost gave up on the way, but since I came up with the delta-model a week ago, the score has risen by more than 1m. I'm glad I didn't give up.</p>\n<p>Thank you to my teammate. <br>\nI'd like to get the gold medal next time.</p>",
  "messages": [
    {
      "id": "1313530",
      "postDate": "05/18/2021 15:44:11",
      "content": "<p>Thank you to the organizers for the fun competition and everyone who participated. And thank you to my teammate ( taksai <a href=\"https://www.kaggle.com/tsaito21219\" target=\"_blank\">@tsaito21219</a> ).<br>\nI share our team's solution.</p>\n<h1>summary</h1>\n<ul>\n<li>train xy-model with only wifi-data</li>\n<li>train delta-model with sensor-data for cost-minimization</li>\n<li>postprocess repeatly 20times</li>\n</ul>\n<h1>1. preprocess</h1>\n<p>we make 4 dataset for the two model described later.</p>\n<table>\n<thead>\n<tr>\n<th>dataset</th>\n<th>describe</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>(a)bssid-ranking</td>\n<td>list of bssids arranged in descending order of rssi. (common to buildings)</td>\n</tr>\n<tr>\n<td>(b)bssid-rssi-matrix</td>\n<td>bssid as a columns and rssi as data (for each building).  Fill missing-value in -120.  Drop rssi-value over 10000ms from last-seen-timestamp.</td>\n</tr>\n<tr>\n<td>(c)sensor-data(aggregate)</td>\n<td>aggregate data between wifi. (1 record for 1 target)</td>\n</tr>\n<tr>\n<td>(d)sensor-data(sampling)</td>\n<td>sampling raw data between wifi every 100ms (N record for 1 target).  Padding to a fixed length.</td>\n</tr>\n</tbody>\n</table>\n<h1>2. models</h1>\n<p>we make 2 type of model. First is xy-model which predict position of waypoint, Second is delta-model which predict distance between two waypoints.</p>\n<h2>(1) xy-model</h2>\n<ul>\n<li>GBDT (lightgbm)  with dataset-(b): each site</li>\n<li>MLP (keras) with dataset-(a): 1model</li>\n<li>MLP (keras) with dataset-(b): each site  -&gt; LB=6.5 (MLP no postprocess)</li>\n</ul>\n<h2>(2) delta-model</h2>\n<ul>\n<li>This model is used for cost-minimization(postprocess). Since the delta calculated using the github function  has a little error, the error is reduced by creating a prediction model. <ul>\n<li>error(mean of sum of squared error): 13.1(use github) -&gt; 4.48(our model)</li>\n<li>MAE(mean over x and y of absolute error) : 1.69(use github) -&gt; 1.08(our model) </li></ul></li>\n<li>MLP (keras) with dataset-(c): Dense(128) &gt; Dense(256) &gt; Dense(128) &gt; Dense(64) &gt; Dense(2)</li>\n<li>1d-cnn (keras) with dataset-(d):  Conv1D(filters=32,kernel_size=5,padding=2) &gt; Conv1D(64,5,2) &gt; Conv1D(128,3,2) &gt; Conv1D(256,3,2) &gt; Conv1D(512,2,2) &gt; GlobalMaxPool1D &gt; Dense(64) &gt; Dense(2)</li>\n</ul>\n<h1>3. postprocess</h1>\n<p>We customized postprocessing based on some useful public kernel.</p>\n<ul>\n<li>1st step: (LB=6.5 -&gt; 3.0)<br>\n1) ensemble: weighted averaged the predicted value of some xy-models. The same applies to delta-model.<br>\n2) tune xy for leakage considering device-id<br>\n3) cost-minimization: use delta calculated by delta-model instead of github function.<br>\n4) snap-to-grid<br>\n5) repeat 2)-4) 20times while adjusting the threshold of snap-to-grid. Move to the grid little by little.   threshold: 2 (1-5 times) &gt; 3 (6-10 times)&gt; 4 (11-20 times)<br>\n6) execute 2) again</li>\n<li>2nd step: (LB=3.0 -&gt; 2.75)<br>\n1) get output(xy) of 1st step about some patterns(ensemble weight etc.)<br>\n2) execute 1st step again with 1)</li>\n</ul>\n<h1>Not work</h1>\n<ul>\n<li>use data of other site(over 200). I think it is efficient for delta-model, but don't work.</li>\n<li>train one model which predict both xy and delta (use RNN and transformer)</li>\n<li>learn multiple waypoints of a path together</li>\n</ul>\n<p>we didn't make floor-model. Our team' floor-loss is big about private dataset. Big mistake… So private score is worse than public.<br>\nAlthough there were many ideas for this competition, most of them were ineffective and it was a very difficult competition.I almost gave up on the way, but since I came up with the delta-model a week ago, the score has risen by more than 1m. I'm glad I didn't give up.</p>\n<p>Thank you to my teammate. <br>\nI'd like to get the gold medal next time.</p>",
      "rawMarkdown": "Thank you to the organizers for the fun competition and everyone who participated. And thank you to my teammate ( taksai @tsaito21219 ).\nI share our team's solution.\n\n# summary\n- train xy-model with only wifi-data\n- train delta-model with sensor-data for cost-minimization\n- postprocess repeatly 20times\n\n# 1. preprocess\nwe make 4 dataset for the two model described later.\n|  dataset   |describe  |\n| --- | --- |\n| (a)bssid-ranking     | list of bssids arranged in descending order of rssi. (common to buildings) |\n| (b)bssid-rssi-matrix| bssid as a columns and rssi as data (for each building).  Fill missing-value in -120.  Drop rssi-value over 10000ms from last-seen-timestamp. |\n| (c)sensor-data(aggregate) |  aggregate data between wifi. (1 record for 1 target) |\n| (d)sensor-data(sampling) | sampling raw data between wifi every 100ms (N record for 1 target).  Padding to a fixed length. |\n\n# 2. models\nwe make 2 type of model. First is xy-model which predict position of waypoint, Second is delta-model which predict distance between two waypoints.\n\n## (1) xy-model\n- GBDT (lightgbm)  with dataset-(b): each site\n- MLP (keras) with dataset-(a): 1model\n- MLP (keras) with dataset-(b): each site  -> LB=6.5 (MLP no postprocess)\n\n## (2) delta-model\n- This model is used for cost-minimization(postprocess). Since the delta calculated using the github function  has a little error, the error is reduced by creating a prediction model. \n  - error(mean of sum of squared error): 13.1(use github) -> 4.48(our model)\n  - MAE(mean over x and y of absolute error) : 1.69(use github) -> 1.08(our model) \n- MLP (keras) with dataset-(c): Dense(128) > Dense(256) > Dense(128) > Dense(64) > Dense(2)\n- 1d-cnn (keras) with dataset-(d):  Conv1D(filters=32,kernel_size=5,padding=2) > Conv1D(64,5,2) > Conv1D(128,3,2) > Conv1D(256,3,2) > Conv1D(512,2,2) > GlobalMaxPool1D > Dense(64) > Dense(2)\n\n# 3. postprocess\nWe customized postprocessing based on some useful public kernel.\n- 1st step: (LB=6.5 -> 3.0)\n1) ensemble: weighted averaged the predicted value of some xy-models. The same applies to delta-model.\n2) tune xy for leakage considering device-id\n3) cost-minimization: use delta calculated by delta-model instead of github function.\n4) snap-to-grid\n5) repeat 2)-4) 20times while adjusting the threshold of snap-to-grid. Move to the grid little by little.   threshold: 2 (1-5 times) > 3 (6-10 times)> 4 (11-20 times)\n6) execute 2) again\n- 2nd step: (LB=3.0 -> 2.75)\n1) get output(xy) of 1st step about some patterns(ensemble weight etc.)\n2) execute 1st step again with 1)\n\n# Not work\n- use data of other site(over 200). I think it is efficient for delta-model, but don't work.\n- train one model which predict both xy and delta (use RNN and transformer)\n- learn multiple waypoints of a path together\n\nwe didn't make floor-model. Our team' floor-loss is big about private dataset. Big mistake... So private score is worse than public.\nAlthough there were many ideas for this competition, most of them were ineffective and it was a very difficult competition.I almost gave up on the way, but since I came up with the delta-model a week ago, the score has risen by more than 1m. I'm glad I didn't give up.\n\nThank you to my teammate. \nI'd like to get the gold medal next time.",
      "votes": null
    },
    {
      "id": "1323513",
      "postDate": "05/26/2021 09:31:10",
      "content": "<p>Thank you. This is very helpful!</p>",
      "rawMarkdown": "Thank you. This is very helpful!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1323513,
      "author_name": "suzuvaki",
      "author_url": "",
      "post_date": "05/26/2021 09:31:10",
      "content": "<p>Thank you. This is very helpful!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1313530": "Thank you to the organizers for the fun competition and everyone who participated. And thank you to my teammate ( taksai @tsaito21219 ).\nI share our team's solution.\n\n# summary\n- train xy-model with only wifi-data\n- train delta-model with sensor-data for cost-minimization\n- postprocess repeatly 20times\n\n# 1. preprocess\nwe make 4 dataset for the two model described later.\n|  dataset   |describe  |\n| --- | --- |\n| (a)bssid-ranking     | list of bssids arranged in descending order of rssi. (common to buildings) |\n| (b)bssid-rssi-matrix| bssid as a columns and rssi as data (for each building).  Fill missing-value in -120.  Drop rssi-value over 10000ms from last-seen-timestamp. |\n| (c)sensor-data(aggregate) |  aggregate data between wifi. (1 record for 1 target) |\n| (d)sensor-data(sampling) | sampling raw data between wifi every 100ms (N record for 1 target).  Padding to a fixed length. |\n\n# 2. models\nwe make 2 type of model. First is xy-model which predict position of waypoint, Second is delta-model which predict distance between two waypoints.\n\n## (1) xy-model\n- GBDT (lightgbm)  with dataset-(b): each site\n- MLP (keras) with dataset-(a): 1model\n- MLP (keras) with dataset-(b): each site  -> LB=6.5 (MLP no postprocess)\n\n## (2) delta-model\n- This model is used for cost-minimization(postprocess). Since the delta calculated using the github function  has a little error, the error is reduced by creating a prediction model. \n  - error(mean of sum of squared error): 13.1(use github) -> 4.48(our model)\n  - MAE(mean over x and y of absolute error) : 1.69(use github) -> 1.08(our model) \n- MLP (keras) with dataset-(c): Dense(128) > Dense(256) > Dense(128) > Dense(64) > Dense(2)\n- 1d-cnn (keras) with dataset-(d):  Conv1D(filters=32,kernel_size=5,padding=2) > Conv1D(64,5,2) > Conv1D(128,3,2) > Conv1D(256,3,2) > Conv1D(512,2,2) > GlobalMaxPool1D > Dense(64) > Dense(2)\n\n# 3. postprocess\nWe customized postprocessing based on some useful public kernel.\n- 1st step: (LB=6.5 -> 3.0)\n1) ensemble: weighted averaged the predicted value of some xy-models. The same applies to delta-model.\n2) tune xy for leakage considering device-id\n3) cost-minimization: use delta calculated by delta-model instead of github function.\n4) snap-to-grid\n5) repeat 2)-4) 20times while adjusting the threshold of snap-to-grid. Move to the grid little by little.   threshold: 2 (1-5 times) > 3 (6-10 times)> 4 (11-20 times)\n6) execute 2) again\n- 2nd step: (LB=3.0 -> 2.75)\n1) get output(xy) of 1st step about some patterns(ensemble weight etc.)\n2) execute 1st step again with 1)\n\n# Not work\n- use data of other site(over 200). I think it is efficient for delta-model, but don't work.\n- train one model which predict both xy and delta (use RNN and transformer)\n- learn multiple waypoints of a path together\n\nwe didn't make floor-model. Our team' floor-loss is big about private dataset. Big mistake... So private score is worse than public.\nAlthough there were many ideas for this competition, most of them were ineffective and it was a very difficult competition.I almost gave up on the way, but since I came up with the delta-model a week ago, the score has risen by more than 1m. I'm glad I didn't give up.\n\nThank you to my teammate. \nI'd like to get the gold medal next time.",
    "1323513": "Thank you. This is very helpful!"
  },
  "source": "meta"
}