{
  "id": 218074,
  "title": "TYPE_WIFI lasttime error (?) in test",
  "url": "/competitions/indoor-location-navigation/discussion/218074",
  "author_name": "",
  "post_date": "2021-02-09T07:07:36.068725600Z",
  "votes": 15,
  "comment_count": 10,
  "views": 0,
  "content": "<p>It appears that the last time seen in the <code>test</code> files are incompatible with the way they are presented in the <code>train</code> files.</p>\n<p>In the <code>train</code> files, <strong>both</strong> the <code>time</code>s and last time seen appear to be UNIX timestamps</p>\n<table>\n<thead>\n<tr>\n<th>time</th>\n<th>TYPE_WIFI</th>\n<th>ssid</th>\n<th>bssid</th>\n<th>RSSI</th>\n<th>frequency</th>\n<th>last seen timestamp</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>da39a3ee5e6b4b0d3255bfef95601890afd80709</td>\n<td>c08ad78a45798cfe176a42b35c7381ae602711c5</td>\n<td>-46</td>\n<td>5825</td>\n<td>1578462603277</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>7182afc4e5c212133d5d7d76eb3df6c24618302b</td>\n<td>4d89139ca69acc0a8a762672a822411a769ac266</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618272</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>d839a45ebe64ab48b60a407d837fb01d3c0dfef9</td>\n<td>30f85a5e14351468a6dd13718a9da3b0d7b73685</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618268</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>b6ffe5619e02871fcd04f61c9bb4b5c53a3f46b7</td>\n<td>fd0bdf5a4dca2566935b14a78c441846b4fbda57</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618270</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>b7e6027447eb1f81327d66cfd3adbe557aabf26c</td>\n<td>bce435ee12b29ad4d543e1418e48fbdea5dfcce2</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618271</td>\n</tr>\n</tbody>\n</table>\n<p>but in the <code>test</code> files, the <code>time</code> seems to start from <code>0</code> (i.e. NOT a UNIX timestamp) while the last seen timestamp is still a UNIX timestamp!</p>\n<table>\n<thead>\n<tr>\n<th>time</th>\n<th>TYPE_WIFI</th>\n<th>ssid</th>\n<th>bssid</th>\n<th>RSSI</th>\n<th>frequency</th>\n<th>last seen timestamp</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>3c1e7602176e050694e3a5cf8ba5f6f725e3ec51</td>\n<td>889bfa434d66eed8c386ccbc90f445932c43f8dd</td>\n<td>-58</td>\n<td>2437</td>\n<td>1573190310545</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>5c072340f8e500f7e62819ab82bb8998ecd0ef4e</td>\n<td>29c7d9e757292e7b2b3d00dc4dae7514531b20b4</td>\n<td>-63</td>\n<td>2462</td>\n<td>1573190310849</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>a4e38996343460efde1140975529e97c9f9aa60b</td>\n<td>98d67fadac518296992afddd24e97a2855af9472</td>\n<td>-64</td>\n<td>2412</td>\n<td>1573190310225</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>d0af9d9c2709796ee07a0432de0e26298a64e3e8</td>\n<td>11567178cc5ca582a37c4733207c77739e1bf5fd</td>\n<td>-64</td>\n<td>2412</td>\n<td>573190310235</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>a4e38996343460efde1140975529e97c9f9aa60b</td>\n<td>bd400fbef9b9b15143e93f8ad2efb07c076e2f5b</td>\n<td>-66</td>\n<td>5745</td>\n<td>1573190311283</td>\n</tr>\n</tbody>\n</table>\n<p>**We therefore cannot use the field in the same way in the test files to see how recent the wifi reading is.<br>\n**<br>\nI suspect the metadata <code>starttime</code> was intended to deal with this (to include the UNIX timestamp corresponding to 0 time), but instead, they all have 0 as the <code>starttime</code>.</p>\n<p>Any ideas?</p>",
  "messages": [
    {
      "id": "1192542",
      "postDate": "02/09/2021 07:07:36",
      "content": "<p>It appears that the last time seen in the <code>test</code> files are incompatible with the way they are presented in the <code>train</code> files.</p>\n<p>In the <code>train</code> files, <strong>both</strong> the <code>time</code>s and last time seen appear to be UNIX timestamps</p>\n<table>\n<thead>\n<tr>\n<th>time</th>\n<th>TYPE_WIFI</th>\n<th>ssid</th>\n<th>bssid</th>\n<th>RSSI</th>\n<th>frequency</th>\n<th>last seen timestamp</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>da39a3ee5e6b4b0d3255bfef95601890afd80709</td>\n<td>c08ad78a45798cfe176a42b35c7381ae602711c5</td>\n<td>-46</td>\n<td>5825</td>\n<td>1578462603277</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>7182afc4e5c212133d5d7d76eb3df6c24618302b</td>\n<td>4d89139ca69acc0a8a762672a822411a769ac266</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618272</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>d839a45ebe64ab48b60a407d837fb01d3c0dfef9</td>\n<td>30f85a5e14351468a6dd13718a9da3b0d7b73685</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618268</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>b6ffe5619e02871fcd04f61c9bb4b5c53a3f46b7</td>\n<td>fd0bdf5a4dca2566935b14a78c441846b4fbda57</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618270</td>\n</tr>\n<tr>\n<td>1578462618826</td>\n<td>TYPE_WIFI</td>\n<td>b7e6027447eb1f81327d66cfd3adbe557aabf26c</td>\n<td>bce435ee12b29ad4d543e1418e48fbdea5dfcce2</td>\n<td>-49</td>\n<td>5825</td>\n<td>1578462618271</td>\n</tr>\n</tbody>\n</table>\n<p>but in the <code>test</code> files, the <code>time</code> seems to start from <code>0</code> (i.e. NOT a UNIX timestamp) while the last seen timestamp is still a UNIX timestamp!</p>\n<table>\n<thead>\n<tr>\n<th>time</th>\n<th>TYPE_WIFI</th>\n<th>ssid</th>\n<th>bssid</th>\n<th>RSSI</th>\n<th>frequency</th>\n<th>last seen timestamp</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>3c1e7602176e050694e3a5cf8ba5f6f725e3ec51</td>\n<td>889bfa434d66eed8c386ccbc90f445932c43f8dd</td>\n<td>-58</td>\n<td>2437</td>\n<td>1573190310545</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>5c072340f8e500f7e62819ab82bb8998ecd0ef4e</td>\n<td>29c7d9e757292e7b2b3d00dc4dae7514531b20b4</td>\n<td>-63</td>\n<td>2462</td>\n<td>1573190310849</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>a4e38996343460efde1140975529e97c9f9aa60b</td>\n<td>98d67fadac518296992afddd24e97a2855af9472</td>\n<td>-64</td>\n<td>2412</td>\n<td>1573190310225</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>d0af9d9c2709796ee07a0432de0e26298a64e3e8</td>\n<td>11567178cc5ca582a37c4733207c77739e1bf5fd</td>\n<td>-64</td>\n<td>2412</td>\n<td>573190310235</td>\n</tr>\n<tr>\n<td>0000000001180</td>\n<td>TYPE_WIFI</td>\n<td>a4e38996343460efde1140975529e97c9f9aa60b</td>\n<td>bd400fbef9b9b15143e93f8ad2efb07c076e2f5b</td>\n<td>-66</td>\n<td>5745</td>\n<td>1573190311283</td>\n</tr>\n</tbody>\n</table>\n<p>**We therefore cannot use the field in the same way in the test files to see how recent the wifi reading is.<br>\n**<br>\nI suspect the metadata <code>starttime</code> was intended to deal with this (to include the UNIX timestamp corresponding to 0 time), but instead, they all have 0 as the <code>starttime</code>.</p>\n<p>Any ideas?</p>",
      "rawMarkdown": "It appears that the last time seen in the `test` files are incompatible with the way they are presented in the `train` files.\n\nIn the `train` files, **both** the `time`s and last time seen appear to be UNIX timestamps\n| time | TYPE_WIFI  | ssid | bssid | RSSI | frequency | last seen timestamp|\n| --- | --- | --- | --- |--- | --- |\n1578462618826|TYPE_WIFI|da39a3ee5e6b4b0d3255bfef95601890afd80709|c08ad78a45798cfe176a42b35c7381ae602711c5|-46|5825|1578462603277\n1578462618826|TYPE_WIFI|7182afc4e5c212133d5d7d76eb3df6c24618302b|4d89139ca69acc0a8a762672a822411a769ac266|-49|5825|1578462618272\n1578462618826|TYPE_WIFI|d839a45ebe64ab48b60a407d837fb01d3c0dfef9|30f85a5e14351468a6dd13718a9da3b0d7b73685|-49|5825|1578462618268\n1578462618826|TYPE_WIFI|b6ffe5619e02871fcd04f61c9bb4b5c53a3f46b7|fd0bdf5a4dca2566935b14a78c441846b4fbda57|-49|5825|1578462618270\n1578462618826|TYPE_WIFI|b7e6027447eb1f81327d66cfd3adbe557aabf26c|bce435ee12b29ad4d543e1418e48fbdea5dfcce2|-49|5825|1578462618271\n\nbut in the `test` files, the `time` seems to start from `0` (i.e. NOT a UNIX timestamp) while the last seen timestamp is still a UNIX timestamp!\n\n| time | TYPE_WIFI  | ssid | bssid | RSSI | frequency | last seen timestamp|\n| --- | --- | --- | --- |--- | --- |\n0000000001180|TYPE_WIFI|3c1e7602176e050694e3a5cf8ba5f6f725e3ec51|889bfa434d66eed8c386ccbc90f445932c43f8dd|-58|2437|1573190310545\n0000000001180|TYPE_WIFI|5c072340f8e500f7e62819ab82bb8998ecd0ef4e|29c7d9e757292e7b2b3d00dc4dae7514531b20b4|-63|2462|1573190310849\n0000000001180|TYPE_WIFI|a4e38996343460efde1140975529e97c9f9aa60b|98d67fadac518296992afddd24e97a2855af9472|-64|2412|1573190310225\n0000000001180|TYPE_WIFI|d0af9d9c2709796ee07a0432de0e26298a64e3e8|11567178cc5ca582a37c4733207c77739e1bf5fd|-64|2412|573190310235\n0000000001180|TYPE_WIFI|a4e38996343460efde1140975529e97c9f9aa60b|bd400fbef9b9b15143e93f8ad2efb07c076e2f5b|-66|5745|1573190311283\n\n**We therefore cannot use the field in the same way in the test files to see how recent the wifi reading is.\n**\nI suspect the metadata `starttime` was intended to deal with this (to include the UNIX timestamp corresponding to 0 time), but instead, they all have 0 as the `starttime`.\n\nAny ideas?",
      "votes": null
    },
    {
      "id": "1193320",
      "postDate": "02/09/2021 15:15:36",
      "content": "<p>if you had the same timestamp, would you be able to re-build the paths of the cell phones without ML ? </p>",
      "rawMarkdown": "if you had the same timestamp, would you be able to re-build the paths of the cell phones without ML ?",
      "votes": null
    },
    {
      "id": "1193771",
      "postDate": "02/09/2021 21:08:08",
      "content": "<p>No, I don't think providing the correct timestamps would create target leakage, if that's what you're asking. If providing it would open up a wider variety of computational methods such as algebraic or geometric modeling solutions to rival ML, I think the answer is yes…. but we would still need some kind of \"big data\" algorithmic approach.</p>",
      "rawMarkdown": "No, I don't think providing the correct timestamps would create target leakage, if that's what you're asking. If providing it would open up a wider variety of computational methods such as algebraic or geometric modeling solutions to rival ML, I think the answer is yes.... but we would still need some kind of \"big data\" algorithmic approach.",
      "votes": null
    },
    {
      "id": "1196615",
      "postDate": "02/11/2021 14:05:59",
      "content": "<p>Take a look at the beacon measurements. In the video where they explain the dataset, they mention that the last field of the beacon measurements are a repeat of the measurement timestamp. You should be able to compute the time delta by looking at the two.</p>",
      "rawMarkdown": "Take a look at the beacon measurements. In the video where they explain the dataset, they mention that the last field of the beacon measurements are a repeat of the measurement timestamp. You should be able to compute the time delta by looking at the two.",
      "votes": null
    },
    {
      "id": "1200852",
      "postDate": "02/15/2021 03:48:29",
      "content": "<p>Just stumbled on this too… In the test files, the sample timestamps have been normalized (starting with 0ms) whereas both the WIFI and Beacon timestamps are still in UNIX.</p>\n<p>Any chance the competition hosts are going to correct that?</p>",
      "rawMarkdown": "Just stumbled on this too... In the test files, the sample timestamps have been normalized (starting with 0ms) whereas both the WIFI and Beacon timestamps are still in UNIX.\n\nAny chance the competition hosts are going to correct that?",
      "votes": null
    },
    {
      "id": "1214358",
      "postDate": "02/22/2021 19:24:16",
      "content": "<p>I just want to confirm for anyone here reading, that this seems to work<br>\n✅</p>",
      "rawMarkdown": "I just want to confirm for anyone here reading, that this seems to work\n✅",
      "votes": null
    },
    {
      "id": "1235455",
      "postDate": "03/12/2021 06:45:39",
      "content": "<p>Thanks for the discussion. I'm now catching this up.<br>\nIIUC, TYPE_BEACON row has normalized timestamp as the first column and unix timestamp as the last column. So by comparing the two value we can know the delta.</p>\n<p><a href=\"https://www.kaggle.com/nigelhenry\" target=\"_blank\">@nigelhenry</a> how do you handle a path file w/o any TYPE_BEACON data?</p>\n<p>thank you!</p>",
      "rawMarkdown": "Thanks for the discussion. I'm now catching this up.\nIIUC, TYPE_BEACON row has normalized timestamp as the first column and unix timestamp as the last column. So by comparing the two value we can know the delta.\n\n@nigelhenry how do you handle a path file w/o any TYPE_BEACON data?\n\nthank you!",
      "votes": null
    },
    {
      "id": "1235581",
      "postDate": "03/12/2021 09:17:01",
      "content": "<p><a href=\"https://www.kaggle.com/higepon\" target=\"_blank\">@higepon</a> Thank you for pointing this out. 34 out of 626 paths in the test data are missing BEACON, which represents 5% of test data. Quite non-trivial. </p>",
      "rawMarkdown": "higepon Thank you for pointing this out. 34 out of 626 paths in the test data are missing BEACON, which represents 5% of test data. Quite non-trivial.",
      "votes": null
    },
    {
      "id": "1235701",
      "postDate": "03/12/2021 11:47:53",
      "content": "<p>Good point.<br>\nWell if you look carefully in <a href=\"https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model\" target=\"_blank\">my notebook</a>, you'll see a very rough short-hand . but I'm open to ideas.🙄</p>",
      "rawMarkdown": "Good point.\nWell if you look carefully in [my notebook](https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model), you'll see a very rough short-hand . but I'm open to ideas.🙄",
      "votes": null
    },
    {
      "id": "1236086",
      "postDate": "03/12/2021 18:40:24",
      "content": "<p>I also published a notebook: <a href=\"https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask\" target=\"_blank\">https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask</a><br>\nThank you all!</p>",
      "rawMarkdown": "I also published a notebook: https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask\nThank you all!",
      "votes": null
    },
    {
      "id": "1236115",
      "postDate": "03/12/2021 19:23:45",
      "content": "<p>Thank you all. I got some boost by using this fix. I leave timestamps as is for data w/o iBeacon at this moment.</p>",
      "rawMarkdown": "Thank you all. I got some boost by using this fix. I leave timestamps as is for data w/o iBeacon at this moment.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1193320,
      "author_name": "chabir",
      "author_url": "",
      "post_date": "02/09/2021 15:15:36",
      "content": "<p>if you had the same timestamp, would you be able to re-build the paths of the cell phones without ML ? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1193771,
          "author_name": "nigelhenry",
          "author_url": "",
          "post_date": "02/09/2021 21:08:08",
          "content": "<p>No, I don't think providing the correct timestamps would create target leakage, if that's what you're asking. If providing it would open up a wider variety of computational methods such as algebraic or geometric modeling solutions to rival ML, I think the answer is yes…. but we would still need some kind of \"big data\" algorithmic approach.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1196615,
      "author_name": "efuentes",
      "author_url": "",
      "post_date": "02/11/2021 14:05:59",
      "content": "<p>Take a look at the beacon measurements. In the video where they explain the dataset, they mention that the last field of the beacon measurements are a repeat of the measurement timestamp. You should be able to compute the time delta by looking at the two.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1214358,
          "author_name": "nigelhenry",
          "author_url": "",
          "post_date": "02/22/2021 19:24:16",
          "content": "<p>I just want to confirm for anyone here reading, that this seems to work<br>\n✅</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235455,
          "author_name": "higepon",
          "author_url": "",
          "post_date": "03/12/2021 06:45:39",
          "content": "<p>Thanks for the discussion. I'm now catching this up.<br>\nIIUC, TYPE_BEACON row has normalized timestamp as the first column and unix timestamp as the last column. So by comparing the two value we can know the delta.</p>\n<p><a href=\"https://www.kaggle.com/nigelhenry\" target=\"_blank\">@nigelhenry</a> how do you handle a path file w/o any TYPE_BEACON data?</p>\n<p>thank you!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235581,
          "author_name": "jiweiliu",
          "author_url": "",
          "post_date": "03/12/2021 09:17:01",
          "content": "<p><a href=\"https://www.kaggle.com/higepon\" target=\"_blank\">@higepon</a> Thank you for pointing this out. 34 out of 626 paths in the test data are missing BEACON, which represents 5% of test data. Quite non-trivial. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235701,
          "author_name": "nigelhenry",
          "author_url": "",
          "post_date": "03/12/2021 11:47:53",
          "content": "<p>Good point.<br>\nWell if you look carefully in <a href=\"https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model\" target=\"_blank\">my notebook</a>, you'll see a very rough short-hand . but I'm open to ideas.🙄</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1236086,
          "author_name": "jiweiliu",
          "author_url": "",
          "post_date": "03/12/2021 18:40:24",
          "content": "<p>I also published a notebook: <a href=\"https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask\" target=\"_blank\">https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask</a><br>\nThank you all!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1236115,
          "author_name": "higepon",
          "author_url": "",
          "post_date": "03/12/2021 19:23:45",
          "content": "<p>Thank you all. I got some boost by using this fix. I leave timestamps as is for data w/o iBeacon at this moment.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1200852,
      "author_name": "depthfirstsearch",
      "author_url": "",
      "post_date": "02/15/2021 03:48:29",
      "content": "<p>Just stumbled on this too… In the test files, the sample timestamps have been normalized (starting with 0ms) whereas both the WIFI and Beacon timestamps are still in UNIX.</p>\n<p>Any chance the competition hosts are going to correct that?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1192542": "It appears that the last time seen in the `test` files are incompatible with the way they are presented in the `train` files.\n\nIn the `train` files, **both** the `time`s and last time seen appear to be UNIX timestamps\n| time | TYPE_WIFI  | ssid | bssid | RSSI | frequency | last seen timestamp|\n| --- | --- | --- | --- |--- | --- |\n1578462618826|TYPE_WIFI|da39a3ee5e6b4b0d3255bfef95601890afd80709|c08ad78a45798cfe176a42b35c7381ae602711c5|-46|5825|1578462603277\n1578462618826|TYPE_WIFI|7182afc4e5c212133d5d7d76eb3df6c24618302b|4d89139ca69acc0a8a762672a822411a769ac266|-49|5825|1578462618272\n1578462618826|TYPE_WIFI|d839a45ebe64ab48b60a407d837fb01d3c0dfef9|30f85a5e14351468a6dd13718a9da3b0d7b73685|-49|5825|1578462618268\n1578462618826|TYPE_WIFI|b6ffe5619e02871fcd04f61c9bb4b5c53a3f46b7|fd0bdf5a4dca2566935b14a78c441846b4fbda57|-49|5825|1578462618270\n1578462618826|TYPE_WIFI|b7e6027447eb1f81327d66cfd3adbe557aabf26c|bce435ee12b29ad4d543e1418e48fbdea5dfcce2|-49|5825|1578462618271\n\nbut in the `test` files, the `time` seems to start from `0` (i.e. NOT a UNIX timestamp) while the last seen timestamp is still a UNIX timestamp!\n\n| time | TYPE_WIFI  | ssid | bssid | RSSI | frequency | last seen timestamp|\n| --- | --- | --- | --- |--- | --- |\n0000000001180|TYPE_WIFI|3c1e7602176e050694e3a5cf8ba5f6f725e3ec51|889bfa434d66eed8c386ccbc90f445932c43f8dd|-58|2437|1573190310545\n0000000001180|TYPE_WIFI|5c072340f8e500f7e62819ab82bb8998ecd0ef4e|29c7d9e757292e7b2b3d00dc4dae7514531b20b4|-63|2462|1573190310849\n0000000001180|TYPE_WIFI|a4e38996343460efde1140975529e97c9f9aa60b|98d67fadac518296992afddd24e97a2855af9472|-64|2412|1573190310225\n0000000001180|TYPE_WIFI|d0af9d9c2709796ee07a0432de0e26298a64e3e8|11567178cc5ca582a37c4733207c77739e1bf5fd|-64|2412|573190310235\n0000000001180|TYPE_WIFI|a4e38996343460efde1140975529e97c9f9aa60b|bd400fbef9b9b15143e93f8ad2efb07c076e2f5b|-66|5745|1573190311283\n\n**We therefore cannot use the field in the same way in the test files to see how recent the wifi reading is.\n**\nI suspect the metadata `starttime` was intended to deal with this (to include the UNIX timestamp corresponding to 0 time), but instead, they all have 0 as the `starttime`.\n\nAny ideas?",
    "1193320": "if you had the same timestamp, would you be able to re-build the paths of the cell phones without ML ?",
    "1193771": "No, I don't think providing the correct timestamps would create target leakage, if that's what you're asking. If providing it would open up a wider variety of computational methods such as algebraic or geometric modeling solutions to rival ML, I think the answer is yes.... but we would still need some kind of \"big data\" algorithmic approach.",
    "1196615": "Take a look at the beacon measurements. In the video where they explain the dataset, they mention that the last field of the beacon measurements are a repeat of the measurement timestamp. You should be able to compute the time delta by looking at the two.",
    "1200852": "Just stumbled on this too... In the test files, the sample timestamps have been normalized (starting with 0ms) whereas both the WIFI and Beacon timestamps are still in UNIX.\n\nAny chance the competition hosts are going to correct that?",
    "1214358": "I just want to confirm for anyone here reading, that this seems to work\n✅",
    "1235455": "Thanks for the discussion. I'm now catching this up.\nIIUC, TYPE_BEACON row has normalized timestamp as the first column and unix timestamp as the last column. So by comparing the two value we can know the delta.\n\n@nigelhenry how do you handle a path file w/o any TYPE_BEACON data?\n\nthank you!",
    "1235581": "higepon Thank you for pointing this out. 34 out of 626 paths in the test data are missing BEACON, which represents 5% of test data. Quite non-trivial.",
    "1235701": "Good point.\nWell if you look carefully in [my notebook](https://www.kaggle.com/nigelhenry/simple-99-accurate-floor-model), you'll see a very rough short-hand . but I'm open to ideas.🙄",
    "1236086": "I also published a notebook: https://www.kaggle.com/jiweiliu/fix-the-timestamps-of-test-data-using-dask\nThank you all!",
    "1236115": "Thank you all. I got some boost by using this fix. I leave timestamps as is for data w/o iBeacon at this moment."
  },
  "source": "meta"
}