{
  "id": 579823,
  "title": "What are your train/validation vs. public LB scores?",
  "url": "/competitions/birdclef-2025/discussion/579823",
  "author_name": "",
  "post_date": "2025-05-20T15:48:29.005868600Z",
  "votes": 3,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Hey all! I'm curious to know how your local validation scores compare to the public leaderboard scores. It would be interesting to see how much of a gap (if any) you're seeing between your offline CV and the actual LB. <br>\nPlease feel free to share your results if you're comfortable — it would be helpful towards understanding about generalization in this competition! 😊</p>",
  "messages": [
    {
      "id": "3205948",
      "postDate": "05/20/2025 15:48:29",
      "content": "<p>Hey all! I'm curious to know how your local validation scores compare to the public leaderboard scores. It would be interesting to see how much of a gap (if any) you're seeing between your offline CV and the actual LB. <br>\nPlease feel free to share your results if you're comfortable — it would be helpful towards understanding about generalization in this competition! 😊</p>",
      "rawMarkdown": "Hey all! I'm curious to know how your local validation scores compare to the public leaderboard scores. It would be interesting to see how much of a gap (if any) you're seeing between your offline CV and the actual LB. \nPlease feel free to share your results if you're comfortable — it would be helpful towards understanding about generalization in this competition! 😊",
      "votes": null
    },
    {
      "id": "3206044",
      "postDate": "05/20/2025 18:59:35",
      "content": "<p>I can say that I am not experiencing good CV-LB relations and I think some shakeup will happen. I am not so sure if CV score sharing now is useful as some strategies will be revealed <a href=\"https://www.kaggle.com/thanatoz\" target=\"_blank\">@thanatoz</a>!</p>",
      "rawMarkdown": "I can say that I am not experiencing good CV-LB relations and I think some shakeup will happen. I am not so sure if CV score sharing now is useful as some strategies will be revealed @thanatoz!",
      "votes": null
    },
    {
      "id": "3206637",
      "postDate": "05/21/2025 15:14:24",
      "content": "<p>I don't think the CV reference is of much significance, and it is very different from PB</p>",
      "rawMarkdown": "I don't think the CV reference is of much significance, and it is very different from PB",
      "votes": null
    },
    {
      "id": "3206789",
      "postDate": "05/21/2025 19:01:17",
      "content": "<p>for our case, we are not seeing a good correlation in cv and lb, Not sure about others</p>",
      "rawMarkdown": "for our case, we are not seeing a good correlation in cv and lb, Not sure about others",
      "votes": null
    },
    {
      "id": "3208633",
      "postDate": "05/24/2025 12:10:38",
      "content": "<p>My local test score is 0.9+ but I've only hit 0.7 on LB. I'm guessing the model is learning too much bias that overlap with labels. There seems to be an input bias that renders any changes to the model moot. I've tried a variety of things but failing to find the right trick. </p>",
      "rawMarkdown": "My local test score is 0.9+ but I've only hit 0.7 on LB. I'm guessing the model is learning too much bias that overlap with labels. There seems to be an input bias that renders any changes to the model moot. I've tried a variety of things but failing to find the right trick.",
      "votes": null
    },
    {
      "id": "3208661",
      "postDate": "05/24/2025 12:41:12",
      "content": "<p>I think a lot of people overfit the rare classes, shake-up happen soon.</p>",
      "rawMarkdown": "I think a lot of people overfit the rare classes, shake-up happen soon.",
      "votes": null
    },
    {
      "id": "3208772",
      "postDate": "05/24/2025 16:50:30",
      "content": "<p>The same as everyone commented, my local CV vs public LB do not correlate well. Most of my local CV shows almost 1.0 but the actual LB hits a bit above 0.8. (Though there still exists the case that my CV sets are not well-divided.)</p>",
      "rawMarkdown": "The same as everyone commented, my local CV vs public LB do not correlate well. Most of my local CV shows almost 1.0 but the actual LB hits a bit above 0.8. (Though there still exists the case that my CV sets are not well-divided.)",
      "votes": null
    },
    {
      "id": "3210166",
      "postDate": "05/26/2025 20:32:55",
      "content": "<p>Great point about the shakeup! For those still open to sharing, this thread could be a helpful resource. Appreciate your input either way!</p>",
      "rawMarkdown": "Great point about the shakeup! For those still open to sharing, this thread could be a helpful resource. Appreciate your input either way!",
      "votes": null
    },
    {
      "id": "3210170",
      "postDate": "05/26/2025 20:37:47",
      "content": "<p>Thanks for sharing! I've been having the same local scores, which is why I asked this question before submitting. Helps to know others see this too—appreciate the insight and good luck tuning further!</p>",
      "rawMarkdown": "Thanks for sharing! I've been having the same local scores, which is why I asked this question before submitting. Helps to know others see this too—appreciate the insight and good luck tuning further!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3206044,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "05/20/2025 18:59:35",
      "content": "<p>I can say that I am not experiencing good CV-LB relations and I think some shakeup will happen. I am not so sure if CV score sharing now is useful as some strategies will be revealed <a href=\"https://www.kaggle.com/thanatoz\" target=\"_blank\">@thanatoz</a>!</p>",
      "votes": null,
      "replies": [
        {
          "id": 3210166,
          "author_name": "thanatoz",
          "author_url": "",
          "post_date": "05/26/2025 20:32:55",
          "content": "<p>Great point about the shakeup! For those still open to sharing, this thread could be a helpful resource. Appreciate your input either way!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3206637,
      "author_name": "yandsbnb666zhang",
      "author_url": "",
      "post_date": "05/21/2025 15:14:24",
      "content": "<p>I don't think the CV reference is of much significance, and it is very different from PB</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3206789,
      "author_name": "mahdiseddigh",
      "author_url": "",
      "post_date": "05/21/2025 19:01:17",
      "content": "<p>for our case, we are not seeing a good correlation in cv and lb, Not sure about others</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3208633,
      "author_name": "bgeier",
      "author_url": "",
      "post_date": "05/24/2025 12:10:38",
      "content": "<p>My local test score is 0.9+ but I've only hit 0.7 on LB. I'm guessing the model is learning too much bias that overlap with labels. There seems to be an input bias that renders any changes to the model moot. I've tried a variety of things but failing to find the right trick. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3208661,
      "author_name": "tom99763",
      "author_url": "",
      "post_date": "05/24/2025 12:41:12",
      "content": "<p>I think a lot of people overfit the rare classes, shake-up happen soon.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3208772,
      "author_name": "nekot0",
      "author_url": "",
      "post_date": "05/24/2025 16:50:30",
      "content": "<p>The same as everyone commented, my local CV vs public LB do not correlate well. Most of my local CV shows almost 1.0 but the actual LB hits a bit above 0.8. (Though there still exists the case that my CV sets are not well-divided.)</p>",
      "votes": null,
      "replies": [
        {
          "id": 3210170,
          "author_name": "thanatoz",
          "author_url": "",
          "post_date": "05/26/2025 20:37:47",
          "content": "<p>Thanks for sharing! I've been having the same local scores, which is why I asked this question before submitting. Helps to know others see this too—appreciate the insight and good luck tuning further!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3205948": "Hey all! I'm curious to know how your local validation scores compare to the public leaderboard scores. It would be interesting to see how much of a gap (if any) you're seeing between your offline CV and the actual LB. \nPlease feel free to share your results if you're comfortable — it would be helpful towards understanding about generalization in this competition! 😊",
    "3206044": "I can say that I am not experiencing good CV-LB relations and I think some shakeup will happen. I am not so sure if CV score sharing now is useful as some strategies will be revealed @thanatoz!",
    "3206637": "I don't think the CV reference is of much significance, and it is very different from PB",
    "3206789": "for our case, we are not seeing a good correlation in cv and lb, Not sure about others",
    "3208633": "My local test score is 0.9+ but I've only hit 0.7 on LB. I'm guessing the model is learning too much bias that overlap with labels. There seems to be an input bias that renders any changes to the model moot. I've tried a variety of things but failing to find the right trick.",
    "3208661": "I think a lot of people overfit the rare classes, shake-up happen soon.",
    "3208772": "The same as everyone commented, my local CV vs public LB do not correlate well. Most of my local CV shows almost 1.0 but the actual LB hits a bit above 0.8. (Though there still exists the case that my CV sets are not well-divided.)",
    "3210166": "Great point about the shakeup! For those still open to sharing, this thread could be a helpful resource. Appreciate your input either way!",
    "3210170": "Thanks for sharing! I've been having the same local scores, which is why I asked this question before submitting. Helps to know others see this too—appreciate the insight and good luck tuning further!"
  },
  "source": "meta"
}