{
  "id": 313073,
  "title": "Overfitting Public LB and Shakeup",
  "url": "/competitions/birdclef-2022/discussion/313073",
  "author_name": "",
  "post_date": "2022-03-15T14:03:19.701502800Z",
  "votes": 13,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I know that we are still early in the competition but I wanted to raise awareness for something:</p>\n<hr>\n<blockquote>\n  <p>\"This leaderboard is calculated with approximately <strong>16%</strong> of the test data. The final results will be based on the other <strong>84%</strong>, so the final standings may be different.\"</p>\n</blockquote>\n<hr>\n<p>This is a really small amount of data (Last year used <strong>35%</strong> for reference) to use as validation and it would be very easy to overfit to the public LB and get tossed about in a violent Private LB shakeup. </p>\n<p>Long story short --&gt; figure out how to Cross Validate properly and rely on that instead of the public LB. Trust your CV.</p>",
  "messages": [
    {
      "id": "1723543",
      "postDate": "03/15/2022 14:03:19",
      "content": "<p>I know that we are still early in the competition but I wanted to raise awareness for something:</p>\n<hr>\n<blockquote>\n  <p>\"This leaderboard is calculated with approximately <strong>16%</strong> of the test data. The final results will be based on the other <strong>84%</strong>, so the final standings may be different.\"</p>\n</blockquote>\n<hr>\n<p>This is a really small amount of data (Last year used <strong>35%</strong> for reference) to use as validation and it would be very easy to overfit to the public LB and get tossed about in a violent Private LB shakeup. </p>\n<p>Long story short --&gt; figure out how to Cross Validate properly and rely on that instead of the public LB. Trust your CV.</p>",
      "rawMarkdown": "I know that we are still early in the competition but I wanted to raise awareness for something:\n\n---\n\n> \"This leaderboard is calculated with approximately **16%** of the test data. The final results will be based on the other **84%**, so the final standings may be different.\"\n\n---\n\nThis is a really small amount of data (Last year used **35%** for reference) to use as validation and it would be very easy to overfit to the public LB and get tossed about in a violent Private LB shakeup. \n\nLong story short --> figure out how to Cross Validate properly and rely on that instead of the public LB. Trust your CV.",
      "votes": null
    },
    {
      "id": "1725520",
      "postDate": "03/17/2022 07:09:17",
      "content": "<p>thanks. where is your baseline notebook?</p>\n<p>just  start here.</p>",
      "rawMarkdown": "thanks. where is your baseline notebook?\n\njust  start here.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1725520,
      "author_name": "dragonzhang",
      "author_url": "",
      "post_date": "03/17/2022 07:09:17",
      "content": "<p>thanks. where is your baseline notebook?</p>\n<p>just  start here.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1723543": "I know that we are still early in the competition but I wanted to raise awareness for something:\n\n---\n\n> \"This leaderboard is calculated with approximately **16%** of the test data. The final results will be based on the other **84%**, so the final standings may be different.\"\n\n---\n\nThis is a really small amount of data (Last year used **35%** for reference) to use as validation and it would be very easy to overfit to the public LB and get tossed about in a violent Private LB shakeup. \n\nLong story short --> figure out how to Cross Validate properly and rely on that instead of the public LB. Trust your CV.",
    "1725520": "thanks. where is your baseline notebook?\n\njust  start here."
  },
  "source": "meta"
}