{
  "id": 126724,
  "title": "Validation up but lb down?",
  "url": "/competitions/tensorflow2-question-answering/discussion/126724",
  "author_name": "",
  "post_date": "2020-01-19T17:58:15.311423200Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I've been making some pretty large improvements on my private validation, but the improvements have led to lower and lower lb. Normally I would guess that I could be overfitting, but I am using validation on a holdout set, so overfitting shouldn't be a problem. Right now my validation is around 0.65, but my lb is 0.54. Does anyone have any idea what could be going wrong, and has anyone else been having this problem?</p>",
  "messages": [
    {
      "id": "723240",
      "postDate": "01/19/2020 17:58:15",
      "content": "<p>I've been making some pretty large improvements on my private validation, but the improvements have led to lower and lower lb. Normally I would guess that I could be overfitting, but I am using validation on a holdout set, so overfitting shouldn't be a problem. Right now my validation is around 0.65, but my lb is 0.54. Does anyone have any idea what could be going wrong, and has anyone else been having this problem?</p>",
      "rawMarkdown": "I've been making some pretty large improvements on my private validation, but the improvements have led to lower and lower lb. Normally I would guess that I could be overfitting, but I am using validation on a holdout set, so overfitting shouldn't be a problem. Right now my validation is around 0.65, but my lb is 0.54. Does anyone have any idea what could be going wrong, and has anyone else been having this problem?",
      "votes": null
    },
    {
      "id": "723267",
      "postDate": "01/19/2020 19:13:24",
      "content": "<p>Probably your local metric implementation just doesn't match the one used by Kaggle.</p>",
      "rawMarkdown": "Probably your local metric implementation just doesn't match the one used by Kaggle.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 723267,
      "author_name": "ddanevskyi",
      "author_url": "",
      "post_date": "01/19/2020 19:13:24",
      "content": "<p>Probably your local metric implementation just doesn't match the one used by Kaggle.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "723240": "I've been making some pretty large improvements on my private validation, but the improvements have led to lower and lower lb. Normally I would guess that I could be overfitting, but I am using validation on a holdout set, so overfitting shouldn't be a problem. Right now my validation is around 0.65, but my lb is 0.54. Does anyone have any idea what could be going wrong, and has anyone else been having this problem?",
    "723267": "Probably your local metric implementation just doesn't match the one used by Kaggle."
  },
  "source": "meta"
}