{
  "id": 412667,
  "title": "Several data inconsistencies you might be wondering about",
  "url": "/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/412667",
  "author_name": "",
  "post_date": "2023-05-24T17:51:03.153080Z",
  "votes": 5,
  "comment_count": 4,
  "views": 0,
  "content": "<p>1) Events.csv in the Data tab include a few patients that do not exist in the dataset you can reach from the python Notebook, namely the 60dfb26b2c and 2054f1d5df records.</p>\n<p>2) Begginings and endings of recordings nearly universally include a period of \"wind up\" and \"wind down\", seeminlgy created by the act of putting on and taking off the sensor or turning it on. You should probably not include those pieces of data.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2Fa80c4867300321bf4ab3ff3235c8c8a1%2FFOG_begginingEnd.png?generation=1684950361214384&amp;alt=media\" alt=\"\"></p>\n<p>It was created using this code:</p>\n<p>df.AccV.loc[start:end].plot()<br>\n&nbsp; &nbsp; &nbsp; &nbsp; df.AccML.loc[start:end].plot()<br>\n&nbsp; &nbsp; &nbsp; &nbsp; df.AccAP.loc[start:end].plot()</p>\n<p>3) There is at least one case in which the FOG events overlap, in this case even having the same type, here turn</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2F626216bfefc36a8d40194d1e1150f862%2FOverlap.png?generation=1684950569272481&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "2272751",
      "postDate": "05/24/2023 17:51:03",
      "content": "<p>1) Events.csv in the Data tab include a few patients that do not exist in the dataset you can reach from the python Notebook, namely the 60dfb26b2c and 2054f1d5df records.</p>\n<p>2) Begginings and endings of recordings nearly universally include a period of \"wind up\" and \"wind down\", seeminlgy created by the act of putting on and taking off the sensor or turning it on. You should probably not include those pieces of data.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2Fa80c4867300321bf4ab3ff3235c8c8a1%2FFOG_begginingEnd.png?generation=1684950361214384&amp;alt=media\" alt=\"\"></p>\n<p>It was created using this code:</p>\n<p>df.AccV.loc[start:end].plot()<br>\n&nbsp; &nbsp; &nbsp; &nbsp; df.AccML.loc[start:end].plot()<br>\n&nbsp; &nbsp; &nbsp; &nbsp; df.AccAP.loc[start:end].plot()</p>\n<p>3) There is at least one case in which the FOG events overlap, in this case even having the same type, here turn</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2F626216bfefc36a8d40194d1e1150f862%2FOverlap.png?generation=1684950569272481&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "1) Events.csv in the Data tab include a few patients that do not exist in the dataset you can reach from the python Notebook, namely the 60dfb26b2c and 2054f1d5df records.\n\n2) Begginings and endings of recordings nearly universally include a period of \"wind up\" and \"wind down\", seeminlgy created by the act of putting on and taking off the sensor or turning it on. You should probably not include those pieces of data.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2Fa80c4867300321bf4ab3ff3235c8c8a1%2FFOG_begginingEnd.png?generation=1684950361214384&alt=media)\n\nIt was created using this code:\n\ndf.AccV.loc[start:end].plot()\n        df.AccML.loc[start:end].plot()\n        df.AccAP.loc[start:end].plot()\n\n3) There is at least one case in which the FOG events overlap, in this case even having the same type, here turn\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2F626216bfefc36a8d40194d1e1150f862%2FOverlap.png?generation=1684950569272481&alt=media)",
      "votes": null
    },
    {
      "id": "2273065",
      "postDate": "05/25/2023 01:51:27",
      "content": "<p>there is no event type for 60dfb26b2c ?</p>",
      "rawMarkdown": "there is no event type for 60dfb26b2c ?",
      "votes": null
    },
    {
      "id": "2273984",
      "postDate": "05/25/2023 14:30:22",
      "content": "<p>Yes, the two mentioned above can't be acessed from events.csv in the Notebook and in the Data tab they have no events recorded, for these two both type and kinetic columns are empty. It seems they were deleted from the final data.</p>",
      "rawMarkdown": "Yes, the two mentioned above can't be acessed from events.csv in the Notebook and in the Data tab they have no events recorded, for these two both type and kinetic columns are empty. It seems they were deleted from the final data.",
      "votes": null
    },
    {
      "id": "2274239",
      "postDate": "05/25/2023 18:03:57",
      "content": "<p>Maybe I didn't quite understand your question, </p>\n<ul>\n<li>these  ['60dfb26b2c', '2054f1d5df']  time series are in <code>notype</code> folder, paths: /kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/notype/…,  and you can load these files to kaggle-notebook;</li>\n<li>series in the <code>notype</code> folder are from the <code>defog</code> dataset but <strong>lack event-type annotations</strong>, hence there are no event Type and Kinetic for Ids '60dfb26b2c', '2054f1d5df' in events.</li>\n</ul>\n<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a><br>\n<strong>Overlapping events</strong></p>\n<ul>\n<li>there are not only overlapping same type events here, but also <strong>Turn and Walking overlapping</strong> events (Id 'c784d2f5d6' <code>tdcsfog</code> dataset)</li>\n<li>there are not only overlapping events here, but also those that <strong>start and end at a negative time</strong>,<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F09459e2f29f4b2b1a0bdffebddd113cf%2FEvents_overlapTurnWalk.png?generation=1685039416468576&amp;alt=media\" alt=\"Events_overlap T&amp;W\"><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F01d0f90645ef36e072cf8c3bc4cf383c%2FEvents_overlap.png?generation=1685037901566726&amp;alt=media\" alt=\"Events_overlap &amp; negative time\"><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2Ff17ece8be8534da12843f0e89985b768%2FEvents_overlap1.png?generation=1685037929062602&amp;alt=media\" alt=\"Events_overlap\"> </li>\n</ul>\n<p><strong>Turn and Walking overlapping</strong><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F42b01ea1bb8f341b87f3b72adcef45b1%2FEvents_overlapTurnWalk1.png?generation=1685040590997834&amp;alt=media\" alt=\"TurnWalk1\"></p>",
      "rawMarkdown": "Maybe I didn't quite understand your question, \n- these  ['60dfb26b2c', '2054f1d5df']  time series are in `notype` folder, paths: /kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/notype/…,  and you can load these files to kaggle-notebook;\n- series in the `notype` folder are from the `defog` dataset but **lack event-type annotations**, hence there are no event Type and Kinetic for Ids '60dfb26b2c', '2054f1d5df' in events.\n\n@ryanholbrook\n**Overlapping events**\n\n\n- there are not only overlapping same type events here, but also **Turn and Walking overlapping** events (Id 'c784d2f5d6' `tdcsfog` dataset)\n- there are not only overlapping events here, but also those that **start and end at a negative time**,\n![Events_overlap T&W](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F09459e2f29f4b2b1a0bdffebddd113cf%2FEvents_overlapTurnWalk.png?generation=1685039416468576&alt=media)\n![Events_overlap & negative time](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F01d0f90645ef36e072cf8c3bc4cf383c%2FEvents_overlap.png?generation=1685037901566726&alt=media)\n![Events_overlap](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2Ff17ece8be8534da12843f0e89985b768%2FEvents_overlap1.png?generation=1685037929062602&alt=media) \n\n**Turn and Walking overlapping**\n![TurnWalk1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F42b01ea1bb8f341b87f3b72adcef45b1%2FEvents_overlapTurnWalk1.png?generation=1685040590997834&alt=media)",
      "votes": null
    },
    {
      "id": "2274837",
      "postDate": "05/26/2023 09:32:52",
      "content": "<p>thx for sharing👍</p>",
      "rawMarkdown": "thx for sharing👍",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2273065,
      "author_name": "yuanzhezhou",
      "author_url": "",
      "post_date": "05/25/2023 01:51:27",
      "content": "<p>there is no event type for 60dfb26b2c ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2273984,
          "author_name": "solkaczmarek",
          "author_url": "",
          "post_date": "05/25/2023 14:30:22",
          "content": "<p>Yes, the two mentioned above can't be acessed from events.csv in the Notebook and in the Data tab they have no events recorded, for these two both type and kinetic columns are empty. It seems they were deleted from the final data.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2274239,
      "author_name": "maatkara",
      "author_url": "",
      "post_date": "05/25/2023 18:03:57",
      "content": "<p>Maybe I didn't quite understand your question, </p>\n<ul>\n<li>these  ['60dfb26b2c', '2054f1d5df']  time series are in <code>notype</code> folder, paths: /kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/notype/…,  and you can load these files to kaggle-notebook;</li>\n<li>series in the <code>notype</code> folder are from the <code>defog</code> dataset but <strong>lack event-type annotations</strong>, hence there are no event Type and Kinetic for Ids '60dfb26b2c', '2054f1d5df' in events.</li>\n</ul>\n<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a><br>\n<strong>Overlapping events</strong></p>\n<ul>\n<li>there are not only overlapping same type events here, but also <strong>Turn and Walking overlapping</strong> events (Id 'c784d2f5d6' <code>tdcsfog</code> dataset)</li>\n<li>there are not only overlapping events here, but also those that <strong>start and end at a negative time</strong>,<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F09459e2f29f4b2b1a0bdffebddd113cf%2FEvents_overlapTurnWalk.png?generation=1685039416468576&amp;alt=media\" alt=\"Events_overlap T&amp;W\"><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F01d0f90645ef36e072cf8c3bc4cf383c%2FEvents_overlap.png?generation=1685037901566726&amp;alt=media\" alt=\"Events_overlap &amp; negative time\"><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2Ff17ece8be8534da12843f0e89985b768%2FEvents_overlap1.png?generation=1685037929062602&amp;alt=media\" alt=\"Events_overlap\"> </li>\n</ul>\n<p><strong>Turn and Walking overlapping</strong><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F42b01ea1bb8f341b87f3b72adcef45b1%2FEvents_overlapTurnWalk1.png?generation=1685040590997834&amp;alt=media\" alt=\"TurnWalk1\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2274837,
      "author_name": "liuxin22",
      "author_url": "",
      "post_date": "05/26/2023 09:32:52",
      "content": "<p>thx for sharing👍</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2272751": "1) Events.csv in the Data tab include a few patients that do not exist in the dataset you can reach from the python Notebook, namely the 60dfb26b2c and 2054f1d5df records.\n\n2) Begginings and endings of recordings nearly universally include a period of \"wind up\" and \"wind down\", seeminlgy created by the act of putting on and taking off the sensor or turning it on. You should probably not include those pieces of data.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2Fa80c4867300321bf4ab3ff3235c8c8a1%2FFOG_begginingEnd.png?generation=1684950361214384&alt=media)\n\nIt was created using this code:\n\ndf.AccV.loc[start:end].plot()\n        df.AccML.loc[start:end].plot()\n        df.AccAP.loc[start:end].plot()\n\n3) There is at least one case in which the FOG events overlap, in this case even having the same type, here turn\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14178795%2F626216bfefc36a8d40194d1e1150f862%2FOverlap.png?generation=1684950569272481&alt=media)",
    "2273065": "there is no event type for 60dfb26b2c ?",
    "2273984": "Yes, the two mentioned above can't be acessed from events.csv in the Notebook and in the Data tab they have no events recorded, for these two both type and kinetic columns are empty. It seems they were deleted from the final data.",
    "2274239": "Maybe I didn't quite understand your question, \n- these  ['60dfb26b2c', '2054f1d5df']  time series are in `notype` folder, paths: /kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/train/notype/…,  and you can load these files to kaggle-notebook;\n- series in the `notype` folder are from the `defog` dataset but **lack event-type annotations**, hence there are no event Type and Kinetic for Ids '60dfb26b2c', '2054f1d5df' in events.\n\n@ryanholbrook\n**Overlapping events**\n\n\n- there are not only overlapping same type events here, but also **Turn and Walking overlapping** events (Id 'c784d2f5d6' `tdcsfog` dataset)\n- there are not only overlapping events here, but also those that **start and end at a negative time**,\n![Events_overlap T&W](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F09459e2f29f4b2b1a0bdffebddd113cf%2FEvents_overlapTurnWalk.png?generation=1685039416468576&alt=media)\n![Events_overlap & negative time](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F01d0f90645ef36e072cf8c3bc4cf383c%2FEvents_overlap.png?generation=1685037901566726&alt=media)\n![Events_overlap](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2Ff17ece8be8534da12843f0e89985b768%2FEvents_overlap1.png?generation=1685037929062602&alt=media) \n\n**Turn and Walking overlapping**\n![TurnWalk1](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F292812%2F42b01ea1bb8f341b87f3b72adcef45b1%2FEvents_overlapTurnWalk1.png?generation=1685040590997834&alt=media)",
    "2274837": "thx for sharing👍"
  },
  "source": "meta"
}