{
  "id": 160555,
  "title": "(CV  VS  LB) Which one should we trust?",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/160555",
  "author_name": "",
  "post_date": "2020-06-21T16:58:50.167724Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I'm really confused about this topic. I have got following variants,\n1. <code>CV: .925 LB: .936</code>\n2. <code>CV: .931 LB: .927</code>\n3. <code>CV: .919 LB: .918</code>\nWhat do you think which one is more trustworthy? How to figure it out?</p>",
  "messages": [
    {
      "id": "895871",
      "postDate": "06/21/2020 16:58:50",
      "content": "<p>I'm really confused about this topic. I have got following variants,\n1. <code>CV: .925 LB: .936</code>\n2. <code>CV: .931 LB: .927</code>\n3. <code>CV: .919 LB: .918</code>\nWhat do you think which one is more trustworthy? How to figure it out?</p>",
      "rawMarkdown": "I'm really confused about this topic. I have got following variants,\n1. `CV: .925 LB: .936`\n2. `CV: .931 LB: .927`\n3. `CV: .919 LB: .918`\nWhat do you think which one is more trustworthy? How to figure it out?",
      "votes": null
    },
    {
      "id": "909700",
      "postDate": "06/30/2020 18:54:55",
      "content": "<p>One could only possible say that knowing your validation scheme. In general, we can treat LB as n+1th fold. If your validation set is well calibrated then any improvements on the CV should result in a better LB score.</p>",
      "rawMarkdown": "One could only possible say that knowing your validation scheme. In general, we can treat LB as n+1th fold. If your validation set is well calibrated then any improvements on the CV should result in a better LB score.",
      "votes": null
    },
    {
      "id": "909712",
      "postDate": "06/30/2020 19:06:23",
      "content": "<p>If you trust you <strong>validation strategy</strong> (by anticipating the behavior of the private test), then you can trust you validation scores.\nIf you're using a random split in this competition, in my opinion you can't trust your cv and your public lb</p>",
      "rawMarkdown": "If you trust you **validation strategy** (by anticipating the behavior of the private test), then you can trust you validation scores.\nIf you're using a random split in this competition, in my opinion you can't trust your cv and your public lb",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 909700,
      "author_name": "janmalin",
      "author_url": "",
      "post_date": "06/30/2020 18:54:55",
      "content": "<p>One could only possible say that knowing your validation scheme. In general, we can treat LB as n+1th fold. If your validation set is well calibrated then any improvements on the CV should result in a better LB score.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 909712,
      "author_name": "seif95",
      "author_url": "",
      "post_date": "06/30/2020 19:06:23",
      "content": "<p>If you trust you <strong>validation strategy</strong> (by anticipating the behavior of the private test), then you can trust you validation scores.\nIf you're using a random split in this competition, in my opinion you can't trust your cv and your public lb</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "895871": "I'm really confused about this topic. I have got following variants,\n1. `CV: .925 LB: .936`\n2. `CV: .931 LB: .927`\n3. `CV: .919 LB: .918`\nWhat do you think which one is more trustworthy? How to figure it out?",
    "909700": "One could only possible say that knowing your validation scheme. In general, we can treat LB as n+1th fold. If your validation set is well calibrated then any improvements on the CV should result in a better LB score.",
    "909712": "If you trust you **validation strategy** (by anticipating the behavior of the private test), then you can trust you validation scores.\nIf you're using a random split in this competition, in my opinion you can't trust your cv and your public lb"
  },
  "source": "meta"
}