{
  "id": 478459,
  "title": "5 fold vs 10 fold CV",
  "url": "/competitions/hms-harmful-brain-activity-classification/discussion/478459",
  "author_name": "",
  "post_date": "2024-02-20T20:31:44.744768500Z",
  "votes": 7,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I'm just getting started with this competition. The data seems small &amp; noisy so I did a 5 vs 10-fold comparison for GroupKFold and saw CV scores of 0.685 vs 0.662.</p>\n<p>I think the extra 10% training data might be worth it given how fast models seem to be training.</p>\n<p>Does anyone else's results look similar?</p>",
  "messages": [
    {
      "id": "2660825",
      "postDate": "02/20/2024 20:31:44",
      "content": "<p>I'm just getting started with this competition. The data seems small &amp; noisy so I did a 5 vs 10-fold comparison for GroupKFold and saw CV scores of 0.685 vs 0.662.</p>\n<p>I think the extra 10% training data might be worth it given how fast models seem to be training.</p>\n<p>Does anyone else's results look similar?</p>",
      "rawMarkdown": "I'm just getting started with this competition. The data seems small & noisy so I did a 5 vs 10-fold comparison for GroupKFold and saw CV scores of 0.685 vs 0.662.\n\nI think the extra 10% training data might be worth it given how fast models seem to be training.\n\nDoes anyone else's results look similar?",
      "votes": null
    },
    {
      "id": "2660865",
      "postDate": "02/20/2024 21:02:09",
      "content": "<p>I haven't tried anything above 5 folds. As you noted, the dataset is small, but it also has class imbalance. Did you stratify when creating the folds?</p>",
      "rawMarkdown": "I haven't tried anything above 5 folds. As you noted, the dataset is small, but it also has class imbalance. Did you stratify when creating the folds?",
      "votes": null
    },
    {
      "id": "2660871",
      "postDate": "02/20/2024 21:17:48",
      "content": "<p>Not yet - just GKF on patient_id. Stratification is on my list of experiments :)</p>",
      "rawMarkdown": "Not yet - just GKF on patient_id. Stratification is on my list of experiments :)",
      "votes": null
    },
    {
      "id": "2660875",
      "postDate": "02/20/2024 21:23:16",
      "content": "<p>I've tried SGKF a few times and did not get better results with EfficientNet</p>",
      "rawMarkdown": "I've tried SGKF a few times and did not get better results with EfficientNet",
      "votes": null
    },
    {
      "id": "2662264",
      "postDate": "02/21/2024 19:41:41",
      "content": "<ul>\n<li>5 folds CV: 0.613 / LB: 0.39</li>\n<li>10 folds CV: 0.616 / LB: 0.4</li>\n</ul>",
      "rawMarkdown": "5 folds CV: 0.613 / LB: 0.39\n- 10 folds CV: 0.616 / LB: 0.4",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2660865,
      "author_name": "seanbearden",
      "author_url": "",
      "post_date": "02/20/2024 21:02:09",
      "content": "<p>I haven't tried anything above 5 folds. As you noted, the dataset is small, but it also has class imbalance. Did you stratify when creating the folds?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2660871,
          "author_name": "anjum48",
          "author_url": "",
          "post_date": "02/20/2024 21:17:48",
          "content": "<p>Not yet - just GKF on patient_id. Stratification is on my list of experiments :)</p>",
          "votes": null,
          "replies": [
            {
              "id": 2660875,
              "author_name": "seanbearden",
              "author_url": "",
              "post_date": "02/20/2024 21:23:16",
              "content": "<p>I've tried SGKF a few times and did not get better results with EfficientNet</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2662264,
      "author_name": "ihebch",
      "author_url": "",
      "post_date": "02/21/2024 19:41:41",
      "content": "<ul>\n<li>5 folds CV: 0.613 / LB: 0.39</li>\n<li>10 folds CV: 0.616 / LB: 0.4</li>\n</ul>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2660825": "I'm just getting started with this competition. The data seems small & noisy so I did a 5 vs 10-fold comparison for GroupKFold and saw CV scores of 0.685 vs 0.662.\n\nI think the extra 10% training data might be worth it given how fast models seem to be training.\n\nDoes anyone else's results look similar?",
    "2660865": "I haven't tried anything above 5 folds. As you noted, the dataset is small, but it also has class imbalance. Did you stratify when creating the folds?",
    "2660871": "Not yet - just GKF on patient_id. Stratification is on my list of experiments :)",
    "2660875": "I've tried SGKF a few times and did not get better results with EfficientNet",
    "2662264": "5 folds CV: 0.613 / LB: 0.39\n- 10 folds CV: 0.616 / LB: 0.4"
  },
  "source": "meta"
}