{
  "id": 329695,
  "title": "CV and LB do not seem to correlate well",
  "url": "/competitions/amex-default-prediction/discussion/329695",
  "author_name": "",
  "post_date": "2022-06-08T06:24:40.296768600Z",
  "votes": 8,
  "comment_count": 3,
  "views": 0,
  "content": "<p>CV 0.79432 0.79489<br>\nLB  0.797 0.796</p>\n<p>This is just an example<br>\nIs it only me?</p>",
  "messages": [
    {
      "id": "1814653",
      "postDate": "06/08/2022 06:24:40",
      "content": "<p>CV 0.79432 0.79489<br>\nLB  0.797 0.796</p>\n<p>This is just an example<br>\nIs it only me?</p>",
      "rawMarkdown": "CV 0.79432 0.79489\nLB  0.797 0.796\n\nThis is just an example\nIs it only me?",
      "votes": null
    },
    {
      "id": "1814692",
      "postDate": "06/08/2022 07:10:43",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/xiaowangiiiii\" target=\"_blank\">@xiaowangiiiii</a> A fluctuation of ±0.001 is normal in this competition. If you compare the scores of your five folds, you'll see even bigger differences. See the discussion about the noisy metric in <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/329088\" target=\"_blank\">Why is there so much test data?</a>.</p>",
      "rawMarkdown": "Hi @xiaowangiiiii A fluctuation of ±0.001 is normal in this competition. If you compare the scores of your five folds, you'll see even bigger differences. See the discussion about the noisy metric in [Why is there so much test data?](https://www.kaggle.com/competitions/amex-default-prediction/discussion/329088).",
      "votes": null
    },
    {
      "id": "1815576",
      "postDate": "06/09/2022 07:16:29",
      "content": "<p>It could be the generalization gap.</p>",
      "rawMarkdown": "It could be the generalization gap.",
      "votes": null
    },
    {
      "id": "1815852",
      "postDate": "06/09/2022 14:46:45",
      "content": "<p>It's a reasonable corrolation, except that the LB is so tightly packed.  </p>\n<p>Since there'll doubltless be a .00x shakeup - which is enough to go from no metal to at least silver - you might as well trust your CV since it's reasonably close to the public LB.</p>",
      "rawMarkdown": "It's a reasonable corrolation, except that the LB is so tightly packed.  \n\nSince there'll doubltless be a .00x shakeup - which is enough to go from no metal to at least silver - you might as well trust your CV since it's reasonably close to the public LB.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1814692,
      "author_name": "ambrosm",
      "author_url": "",
      "post_date": "06/08/2022 07:10:43",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/xiaowangiiiii\" target=\"_blank\">@xiaowangiiiii</a> A fluctuation of ±0.001 is normal in this competition. If you compare the scores of your five folds, you'll see even bigger differences. See the discussion about the noisy metric in <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/329088\" target=\"_blank\">Why is there so much test data?</a>.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1815576,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "06/09/2022 07:16:29",
      "content": "<p>It could be the generalization gap.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1815852,
      "author_name": "happycube",
      "author_url": "",
      "post_date": "06/09/2022 14:46:45",
      "content": "<p>It's a reasonable corrolation, except that the LB is so tightly packed.  </p>\n<p>Since there'll doubltless be a .00x shakeup - which is enough to go from no metal to at least silver - you might as well trust your CV since it's reasonably close to the public LB.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1814653": "CV 0.79432 0.79489\nLB  0.797 0.796\n\nThis is just an example\nIs it only me?",
    "1814692": "Hi @xiaowangiiiii A fluctuation of ±0.001 is normal in this competition. If you compare the scores of your five folds, you'll see even bigger differences. See the discussion about the noisy metric in [Why is there so much test data?](https://www.kaggle.com/competitions/amex-default-prediction/discussion/329088).",
    "1815576": "It could be the generalization gap.",
    "1815852": "It's a reasonable corrolation, except that the LB is so tightly packed.  \n\nSince there'll doubltless be a .00x shakeup - which is enough to go from no metal to at least silver - you might as well trust your CV since it's reasonably close to the public LB."
  },
  "source": "meta"
}