{
  "id": 407875,
  "title": "defog cv score way lower than tdcsfog cv score?",
  "url": "/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/407875",
  "author_name": "",
  "post_date": "2023-05-08T15:28:37.432662500Z",
  "votes": null,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I use StratifiedGroupKFold  (groups = Subjects)<br>\nFor defog data, I get CV score over 0.2<br>\nFor tdcsfog data, I get CV score under 0.1</p>\n<p>I wonder if that's because I missed something, or maybe tdcsfog is harder to predict (more sparse, perhaps)? </p>\n<p>Did anyone experience the same?</p>",
  "messages": [
    {
      "id": "2250491",
      "postDate": "05/08/2023 15:28:37",
      "content": "<p>I use StratifiedGroupKFold  (groups = Subjects)<br>\nFor defog data, I get CV score over 0.2<br>\nFor tdcsfog data, I get CV score under 0.1</p>\n<p>I wonder if that's because I missed something, or maybe tdcsfog is harder to predict (more sparse, perhaps)? </p>\n<p>Did anyone experience the same?</p>",
      "rawMarkdown": "I use StratifiedGroupKFold  (groups = Subjects)\nFor defog data, I get CV score over 0.2\nFor tdcsfog data, I get CV score under 0.1\n\nI wonder if that's because I missed something, or maybe tdcsfog is harder to predict (more sparse, perhaps)? \n\nDid anyone experience the same?",
      "votes": null
    },
    {
      "id": "2256883",
      "postDate": "05/12/2023 19:55:52",
      "content": "<p>Defog dataset has much less data series (91 files) than tdcsfog (833 files).</p>",
      "rawMarkdown": "Defog dataset has much less data series (91 files) than tdcsfog (833 files).",
      "votes": null
    },
    {
      "id": "2256907",
      "postDate": "05/12/2023 20:19:42",
      "content": "<p>Nice point, thank you!</p>",
      "rawMarkdown": "Nice point, thank you!",
      "votes": null
    },
    {
      "id": "2257779",
      "postDate": "05/13/2023 16:56:15",
      "content": "<p>But defog dataset has ~37,5 hours, tdcsfog has ~15 hours. </p>\n<p>The point is that defog and tdcsfog have <strong>different target distributions</strong>. <br>\n <em>Mean target,  roughly:</em></p>\n<table>\n<thead>\n<tr>\n<th>data</th>\n<th>ST</th>\n<th>Turn</th>\n<th>Walking</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>defog</td>\n<td>0</td>\n<td>0.04</td>\n<td>0.01</td>\n</tr>\n<tr>\n<td>tdcsfog</td>\n<td>0.04</td>\n<td><strong>0.24</strong></td>\n<td>0.03</td>\n</tr>\n</tbody>\n</table>\n<p>But you have a worse result on more balanced data (tdcsfog), which is strange.</p>",
      "rawMarkdown": "But defog dataset has ~37,5 hours, tdcsfog has ~15 hours. \n\nThe point is that defog and tdcsfog have **different target distributions**. \n *Mean target,  roughly:*\n\n| data |ST  |Turn  |  Walking |\n| --- | --- | --- | --- |\n| defog | 0 | 0.04  | 0.01 |\n| tdcsfog | 0.04 |  **0.24**  | 0.03 |\n\n\nBut you have a worse result on more balanced data (tdcsfog), which is strange.",
      "votes": null
    },
    {
      "id": "2257796",
      "postDate": "05/13/2023 17:03:22",
      "content": "<p>What do you think makes this target distribution so hard?</p>",
      "rawMarkdown": "What do you think makes this target distribution so hard?",
      "votes": null
    },
    {
      "id": "2257835",
      "postDate": "05/13/2023 17:26:39",
      "content": "<ul>\n<li>Time-series in defog are significantly longer (~12 times).</li>\n<li>Patients feel much more comfortable in their home environment, resulting in fewer and shorter FOG-events</li>\n</ul>",
      "rawMarkdown": "Time-series in defog are significantly longer (~12 times).\n- Patients feel much more comfortable in their home environment, resulting in fewer and shorter FOG-events",
      "votes": null
    },
    {
      "id": "2257868",
      "postDate": "05/13/2023 17:46:29",
      "content": "<p>Good points. Thank you!</p>",
      "rawMarkdown": "Good points. Thank you!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2256883,
      "author_name": "atamazian",
      "author_url": "",
      "post_date": "05/12/2023 19:55:52",
      "content": "<p>Defog dataset has much less data series (91 files) than tdcsfog (833 files).</p>",
      "votes": null,
      "replies": [
        {
          "id": 2256907,
          "author_name": "avivlevi815",
          "author_url": "",
          "post_date": "05/12/2023 20:19:42",
          "content": "<p>Nice point, thank you!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2257779,
          "author_name": "maatkara",
          "author_url": "",
          "post_date": "05/13/2023 16:56:15",
          "content": "<p>But defog dataset has ~37,5 hours, tdcsfog has ~15 hours. </p>\n<p>The point is that defog and tdcsfog have <strong>different target distributions</strong>. <br>\n <em>Mean target,  roughly:</em></p>\n<table>\n<thead>\n<tr>\n<th>data</th>\n<th>ST</th>\n<th>Turn</th>\n<th>Walking</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>defog</td>\n<td>0</td>\n<td>0.04</td>\n<td>0.01</td>\n</tr>\n<tr>\n<td>tdcsfog</td>\n<td>0.04</td>\n<td><strong>0.24</strong></td>\n<td>0.03</td>\n</tr>\n</tbody>\n</table>\n<p>But you have a worse result on more balanced data (tdcsfog), which is strange.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2257796,
              "author_name": "avivlevi815",
              "author_url": "",
              "post_date": "05/13/2023 17:03:22",
              "content": "<p>What do you think makes this target distribution so hard?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2257835,
                  "author_name": "maatkara",
                  "author_url": "",
                  "post_date": "05/13/2023 17:26:39",
                  "content": "<ul>\n<li>Time-series in defog are significantly longer (~12 times).</li>\n<li>Patients feel much more comfortable in their home environment, resulting in fewer and shorter FOG-events</li>\n</ul>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2257868,
                      "author_name": "avivlevi815",
                      "author_url": "",
                      "post_date": "05/13/2023 17:46:29",
                      "content": "<p>Good points. Thank you!</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2250491": "I use StratifiedGroupKFold  (groups = Subjects)\nFor defog data, I get CV score over 0.2\nFor tdcsfog data, I get CV score under 0.1\n\nI wonder if that's because I missed something, or maybe tdcsfog is harder to predict (more sparse, perhaps)? \n\nDid anyone experience the same?",
    "2256883": "Defog dataset has much less data series (91 files) than tdcsfog (833 files).",
    "2256907": "Nice point, thank you!",
    "2257779": "But defog dataset has ~37,5 hours, tdcsfog has ~15 hours. \n\nThe point is that defog and tdcsfog have **different target distributions**. \n *Mean target,  roughly:*\n\n| data |ST  |Turn  |  Walking |\n| --- | --- | --- | --- |\n| defog | 0 | 0.04  | 0.01 |\n| tdcsfog | 0.04 |  **0.24**  | 0.03 |\n\n\nBut you have a worse result on more balanced data (tdcsfog), which is strange.",
    "2257796": "What do you think makes this target distribution so hard?",
    "2257835": "Time-series in defog are significantly longer (~12 times).\n- Patients feel much more comfortable in their home environment, resulting in fewer and shorter FOG-events",
    "2257868": "Good points. Thank you!"
  },
  "source": "meta"
}