{
  "id": 342175,
  "title": "why do I have high variance in my cv?",
  "url": "/competitions/amex-default-prediction/discussion/342175",
  "author_name": "",
  "post_date": "2022-08-05T19:53:43.286043Z",
  "votes": 14,
  "comment_count": 6,
  "views": 0,
  "content": "<p>my CV std: 0.0031664491153340813<br>\nmy CV mean: 79586<br>\nfolds scores : [0.7956,0.7937,0.7927,0.7955,0.8018]<br>\nseed 42<br>\nwhy do I have high variance in my cv?</p>",
  "messages": [
    {
      "id": "1886423",
      "postDate": "08/05/2022 19:53:43",
      "content": "<p>my CV std: 0.0031664491153340813<br>\nmy CV mean: 79586<br>\nfolds scores : [0.7956,0.7937,0.7927,0.7955,0.8018]<br>\nseed 42<br>\nwhy do I have high variance in my cv?</p>",
      "rawMarkdown": "my CV std: 0.0031664491153340813\nmy CV mean: 79586\nfolds scores : [0.7956,0.7937,0.7927,0.7955,0.8018]\nseed 42\nwhy do I have high variance in my cv?",
      "votes": null
    },
    {
      "id": "1886426",
      "postDate": "08/05/2022 19:54:51",
      "content": "<p>Can you recheck how you performed the cross-validation? This seems to be fine…</p>",
      "rawMarkdown": "Can you recheck how you performed the cross-validation? This seems to be fine...",
      "votes": null
    },
    {
      "id": "1886443",
      "postDate": "08/05/2022 20:20:38",
      "content": "<p>This seems to me to be the resonable result. Different customers plus weak time series correlation, even if skf is used for data segmentation, there is no guarantee that the test score of each fold is very close to others.</p>",
      "rawMarkdown": "This seems to me to be the resonable result. Different customers plus weak time series correlation, even if skf is used for data segmentation, there is no guarantee that the test score of each fold is very close to others.",
      "votes": null
    },
    {
      "id": "1886452",
      "postDate": "08/05/2022 20:33:21",
      "content": "<p>Your variance isn't extremely high:<br>\nMy cv std: 0.0026<br>\nMy folds: 0.79823 0.79268 0.79492 0.79894 0.79958</p>",
      "rawMarkdown": "Your variance isn't extremely high:\nMy cv std: 0.0026\nMy folds: 0.79823 0.79268 0.79492 0.79894 0.79958",
      "votes": null
    },
    {
      "id": "1886846",
      "postDate": "08/06/2022 07:41:43",
      "content": "<p>But notice that when I submit my score on LB don't get above 0.795, even if I changed my models or features, I think I'm  facing a problem with the way I submit?</p>",
      "rawMarkdown": "But notice that when I submit my score on LB don't get above 0.795, even if I changed my models or features, I think I'm  facing a problem with the way I submit?",
      "votes": null
    },
    {
      "id": "1886848",
      "postDate": "08/06/2022 07:44:06",
      "content": "<p>What is your score on LB depending on this cv?<br>\nIs it 0.796?</p>\n<p>I mean did you  blend some models so you got 0.80 or you reached to this score from the same model.</p>",
      "rawMarkdown": "What is your score on LB depending on this cv?\nIs it 0.796?\n\nI mean did you  blend some models so you got 0.80 or you reached to this score from the same model.",
      "votes": null
    },
    {
      "id": "1888576",
      "postDate": "08/07/2022 17:12:23",
      "content": "<p>I didn't submit that model separately. I only used it in my ensemble.</p>",
      "rawMarkdown": "I didn't submit that model separately. I only used it in my ensemble.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1886426,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "08/05/2022 19:54:51",
      "content": "<p>Can you recheck how you performed the cross-validation? This seems to be fine…</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1886443,
      "author_name": "meli19",
      "author_url": "",
      "post_date": "08/05/2022 20:20:38",
      "content": "<p>This seems to me to be the resonable result. Different customers plus weak time series correlation, even if skf is used for data segmentation, there is no guarantee that the test score of each fold is very close to others.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1886452,
      "author_name": "ambrosm",
      "author_url": "",
      "post_date": "08/05/2022 20:33:21",
      "content": "<p>Your variance isn't extremely high:<br>\nMy cv std: 0.0026<br>\nMy folds: 0.79823 0.79268 0.79492 0.79894 0.79958</p>",
      "votes": null,
      "replies": [
        {
          "id": 1886846,
          "author_name": "xv7d111",
          "author_url": "",
          "post_date": "08/06/2022 07:41:43",
          "content": "<p>But notice that when I submit my score on LB don't get above 0.795, even if I changed my models or features, I think I'm  facing a problem with the way I submit?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1886848,
          "author_name": "xv7d111",
          "author_url": "",
          "post_date": "08/06/2022 07:44:06",
          "content": "<p>What is your score on LB depending on this cv?<br>\nIs it 0.796?</p>\n<p>I mean did you  blend some models so you got 0.80 or you reached to this score from the same model.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1888576,
          "author_name": "ambrosm",
          "author_url": "",
          "post_date": "08/07/2022 17:12:23",
          "content": "<p>I didn't submit that model separately. I only used it in my ensemble.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1886423": "my CV std: 0.0031664491153340813\nmy CV mean: 79586\nfolds scores : [0.7956,0.7937,0.7927,0.7955,0.8018]\nseed 42\nwhy do I have high variance in my cv?",
    "1886426": "Can you recheck how you performed the cross-validation? This seems to be fine...",
    "1886443": "This seems to me to be the resonable result. Different customers plus weak time series correlation, even if skf is used for data segmentation, there is no guarantee that the test score of each fold is very close to others.",
    "1886452": "Your variance isn't extremely high:\nMy cv std: 0.0026\nMy folds: 0.79823 0.79268 0.79492 0.79894 0.79958",
    "1886846": "But notice that when I submit my score on LB don't get above 0.795, even if I changed my models or features, I think I'm  facing a problem with the way I submit?",
    "1886848": "What is your score on LB depending on this cv?\nIs it 0.796?\n\nI mean did you  blend some models so you got 0.80 or you reached to this score from the same model.",
    "1888576": "I didn't submit that model separately. I only used it in my ensemble."
  },
  "source": "meta"
}