{
  "id": 572185,
  "title": "Robust local validation?",
  "url": "/competitions/birdclef-2025/discussion/572185",
  "author_name": "Yassine Alouini",
  "post_date": "2025-04-08T06:53:49.478000",
  "votes": 5,
  "comment_count": 5,
  "views": 0,
  "content": "<p>As stated by <a href=\"https://www.kaggle.com/christofhenkel\" target=\"_blank\">@christofhenkel</a> in this discussion thread <a href=\"https://www.kaggle.com/competitions/birdclef-2025/discussion/567495\" target=\"_blank\">https://www.kaggle.com/competitions/birdclef-2025/discussion/567495</a>, it is hard to have a robust validation set locally.</p>\n<p>Indeed, I have trained various versions of <a href=\"https://arxiv.org/abs/1905.11946\" target=\"_blank\"><code>efficientnet</code></a> (<code>B0</code>, <code>B1</code>, and <code>B2</code>) and for now <code>B1</code> is the best without alignment with local validation score (probably my local validation setup can be improved).</p>\n<p>What are your strategies in order to have a better local validation? Or is LB probing the only way?</p>\n<p>Thanks for your help. 🙏</p>",
  "messages": [
    {
      "id": 3173631,
      "postDate": "2025-04-08T06:53:49.480Z",
      "content": "<p>As stated by <a href=\"https://www.kaggle.com/christofhenkel\" target=\"_blank\">@christofhenkel</a> in this discussion thread <a href=\"https://www.kaggle.com/competitions/birdclef-2025/discussion/567495\" target=\"_blank\">https://www.kaggle.com/competitions/birdclef-2025/discussion/567495</a>, it is hard to have a robust validation set locally.</p>\n<p>Indeed, I have trained various versions of <a href=\"https://arxiv.org/abs/1905.11946\" target=\"_blank\"><code>efficientnet</code></a> (<code>B0</code>, <code>B1</code>, and <code>B2</code>) and for now <code>B1</code> is the best without alignment with local validation score (probably my local validation setup can be improved).</p>\n<p>What are your strategies in order to have a better local validation? Or is LB probing the only way?</p>\n<p>Thanks for your help. 🙏</p>",
      "rawMarkdown": "As stated by @christofhenkel in this discussion thread https://www.kaggle.com/competitions/birdclef-2025/discussion/567495, it is hard to have a robust validation set locally.\n\nIndeed, I have trained various versions of [`efficientnet`](https://arxiv.org/abs/1905.11946) (`B0`, `B1`, and `B2`) and for now `B1` is the best without alignment with local validation score (probably my local validation setup can be improved).\n\nWhat are your strategies in order to have a better local validation? Or is LB probing the only way?\n\nThanks for your help. 🙏",
      "votes": 5
    },
    {
      "id": 3173663,
      "postDate": "2025-04-08T07:45:49.373Z",
      "content": "<p>I think it's quite difficult to get good local validation here. In additional to the traditional problem of domain shift, this year competition features not only birds, but other species as well, making it even more difficult.</p>",
      "rawMarkdown": "I think it's quite difficult to get good local validation here. In additional to the traditional problem of domain shift, this year competition features not only birds, but other species as well, making it even more difficult.",
      "votes": 1,
      "replies": [
        {
          "id": 3179874,
          "postDate": "2025-04-15T19:59:59.250Z",
          "content": "<p>What do you mean by other species? </p>",
          "rawMarkdown": "What do you mean by other species? ",
          "votes": 1,
          "replies": [
            {
              "id": 3180377,
              "postDate": "2025-04-16T14:27:25.490Z",
              "content": "<p>Amphibia,Mammalia,Insecta in the train data.</p>",
              "rawMarkdown": "Amphibia,Mammalia,Insecta in the train data.",
              "votes": 1
            },
            {
              "id": 3202336,
              "postDate": "2025-05-15T08:19:09.813Z",
              "content": "<p>Thanks, makes sense.</p>",
              "rawMarkdown": "Thanks, makes sense."
            }
          ]
        },
        {
          "id": 3201006,
          "postDate": "2025-05-13T10:40:03.337Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 3173663,
      "author_name": "Araik Tamazian",
      "author_url": "",
      "post_date": "2025-04-08T07:45:49.373000",
      "content": "<p>I think it's quite difficult to get good local validation here. In additional to the traditional problem of domain shift, this year competition features not only birds, but other species as well, making it even more difficult.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3179874,
          "author_name": "Yassine Alouini",
          "author_url": "",
          "post_date": "2025-04-15T19:59:59.250000",
          "content": "<p>What do you mean by other species? </p>",
          "votes": 1,
          "replies": [
            {
              "id": 3180377,
              "author_name": "shanzhong8",
              "author_url": "",
              "post_date": "2025-04-16T14:27:25.490000",
              "content": "<p>Amphibia,Mammalia,Insecta in the train data.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3202336,
              "author_name": "Yassine Alouini",
              "author_url": "",
              "post_date": "2025-05-15T08:19:09.813000",
              "content": "<p>Thanks, makes sense.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 3201006,
          "author_name": "",
          "author_url": "",
          "post_date": "2025-05-13T10:40:03.337000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3173631": "As stated by @christofhenkel in this discussion thread https://www.kaggle.com/competitions/birdclef-2025/discussion/567495, it is hard to have a robust validation set locally.\n\nIndeed, I have trained various versions of [`efficientnet`](https://arxiv.org/abs/1905.11946) (`B0`, `B1`, and `B2`) and for now `B1` is the best without alignment with local validation score (probably my local validation setup can be improved).\n\nWhat are your strategies in order to have a better local validation? Or is LB probing the only way?\n\nThanks for your help. 🙏",
    "3173663": "I think it's quite difficult to get good local validation here. In additional to the traditional problem of domain shift, this year competition features not only birds, but other species as well, making it even more difficult."
  }
}