We are very interested in how many species are included in the public LB. It can be obtained by the following method.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4704212%2F5ff973d03a69ae72900dec80f6b7478c%2Fmath1.png?generation=1743387175769139&alt=media\")
Why is this equation derived? Please refer to the comments. I may be wrong. Please comment.
\nThe following is the above method I tried.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4704212%2F30518fe9efa5abcd486ac78a213114b9%2Fscore.png?generation=1743392802180609&alt=media\")
As a result of the calculation, I estimate that Public LB contains about 70 (0.4 / 0.006 ) species.
\nBy the way, the calculation with BirdCLEF2024 included about 100 bird species.
", "messages": [ { "id": "3163635", "postDate": "03/31/2025 03:48:58", "content": "We are very interested in how many species are included in the public LB. It can be obtained by the following method.
\nWhy is this equation derived? Please refer to the comments. I may be wrong. Please comment.
\nThe following is the above method I tried.
\n
As a result of the calculation, I estimate that Public LB contains about 70 (0.4 / 0.006 ) species.
\nBy the way, the calculation with BirdCLEF2024 included about 100 bird species.
", "rawMarkdown": "We are very interested in how many species are included in the public LB. It can be obtained by the following method.\n\n+ Submission normally (the higher the score, the better) \n + At this time, Let Public LB score be X. \n+ Next, set the species prediction score (e.g. tbsfin1) to 0, and then submit.\n + At this time, Let Public LB score be X’. \n + At this time, the AUC of tbsfin1 changes from S to 0.5.\n + S (the original AUC of tbsfin1) is here set to 0.9. \n + Because tbsfin1's calls are easy to find and can be assumed to score well. \n+ Finally, calculate how many bird species are included with the following equation.\n\n+ delta_X = X - X'\n\n\nWhy is this equation derived? Please refer to the comments. I may be wrong. Please comment.\n\nThe following is the above method I tried.\n\n\nAs a result of the calculation, I estimate that Public LB contains about **70 (0.4 / 0.006 ) species**.\n\nBy the way, the calculation with BirdCLEF2024 included about 100 bird species.", "votes": null }, { "id": "3163636", "postDate": "03/31/2025 03:49:26", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4704212%2Ffa5ba11ef43f3d3a59f444b26c9c9686%2Fmath2.png?generation=1743392962503975&alt=media\")
We can make a top estimate with this technique.
\n1) The decrease in the public score may be lower because of the score rounding. So the 0.826 score could be 0.826000, and the 0.820 score could be 0.820999 actually, so the decrease may be up to 0.005. I'm not sure about how Kaggle does the rounding, and I assume that the digits are just truncated. However, If Kaggle uses round(), the result is the same.
\n2) Actually, we are able to predict the bird with 1.0 AUC score, so the decrease in the individual bird's score may be up to 0.5.
Keeping this in mind, we get 0.5/0.005=100 species maximum from your experiment. This is an upper bound only, but it is quite strict.
", "rawMarkdown": "We can make a top estimate with this technique. \n1) The decrease in the public score may be lower because of the score rounding. So the 0.826 score could be 0.826000, and the 0.820 score could be 0.820999 actually, so the decrease may be up to 0.005. I'm not sure about how Kaggle does the rounding, and I assume that the digits are just truncated. However, If Kaggle uses `round()`, the result is the same.\n2) Actually, we are able to predict the bird with 1.0 AUC score, so the decrease in the individual bird's score may be up to 0.5.\n\nKeeping this in mind, we get **0.5/0.005=100** species maximum from your experiment. This is an upper bound only, but it is quite strict.", "votes": null }, { "id": "3164073", "postDate": "03/31/2025 12:53:31", "content": "Yes, you are right. It is true that there is uncertainty.
\nThis is why I say \"the higher the score, the better\".The higher species score, the lower the uncertainty (the larger the delta_X).
", "rawMarkdown": "Yes, you are right. It is true that there is uncertainty.\n\nThis is why I say **\"the higher the score, the better\"**.The higher species score, the lower the uncertainty (the larger the delta_X).", "votes": null }, { "id": "3164226", "postDate": "03/31/2025 14:57:05", "content": "thanks for your valuable content.
", "rawMarkdown": "thanks for your valuable content.", "votes": null }, { "id": "3204217", "postDate": "05/18/2025 00:11:18", "content": "May I ask if the competition hoster has mentioned how the public and private leaderboard are divided? We know that an audio has multiple submissions. Will there be a situation where one audio is on both the public and private leaderboard?
", "rawMarkdown": "May I ask if the competition hoster has mentioned how the public and private leaderboard are divided? We know that an audio has multiple submissions. Will there be a situation where one audio is on both the public and private leaderboard?", "votes": null }, { "id": "3204227", "postDate": "05/18/2025 01:21:45", "content": "They mentioned its random split here - https://www.kaggle.com/competitions/birdclef-2025/discussion/572159#3173630
", "rawMarkdown": "They mentioned its random split here - https://www.kaggle.com/competitions/birdclef-2025/discussion/572159#3173630", "votes": null } ], "comments": [ { "id": 3163636, "author_name": "shinmurashinmura", "author_url": "", "post_date": "03/31/2025 03:49:26", "content": "
We can make a top estimate with this technique.
\n1) The decrease in the public score may be lower because of the score rounding. So the 0.826 score could be 0.826000, and the 0.820 score could be 0.820999 actually, so the decrease may be up to 0.005. I'm not sure about how Kaggle does the rounding, and I assume that the digits are just truncated. However, If Kaggle uses round(), the result is the same.
\n2) Actually, we are able to predict the bird with 1.0 AUC score, so the decrease in the individual bird's score may be up to 0.5.
Keeping this in mind, we get 0.5/0.005=100 species maximum from your experiment. This is an upper bound only, but it is quite strict.
", "votes": null, "replies": [ { "id": 3164073, "author_name": "shinmurashinmura", "author_url": "", "post_date": "03/31/2025 12:53:31", "content": "Yes, you are right. It is true that there is uncertainty.
\nThis is why I say \"the higher the score, the better\".The higher species score, the lower the uncertainty (the larger the delta_X).
", "votes": null, "replies": [] } ] }, { "id": 3164226, "author_name": "aaryankodmalwar", "author_url": "", "post_date": "03/31/2025 14:57:05", "content": "thanks for your valuable content.
", "votes": null, "replies": [] }, { "id": 3204217, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "05/18/2025 00:11:18", "content": "May I ask if the competition hoster has mentioned how the public and private leaderboard are divided? We know that an audio has multiple submissions. Will there be a situation where one audio is on both the public and private leaderboard?
", "votes": null, "replies": [ { "id": 3204227, "author_name": "rashmibanthia", "author_url": "", "post_date": "05/18/2025 01:21:45", "content": "They mentioned its random split here - https://www.kaggle.com/competitions/birdclef-2025/discussion/572159#3173630
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3163635": "We are very interested in how many species are included in the public LB. It can be obtained by the following method.\n\n+ Submission normally (the higher the score, the better) \n + At this time, Let Public LB score be X. \n+ Next, set the species prediction score (e.g. tbsfin1) to 0, and then submit.\n + At this time, Let Public LB score be X’. \n + At this time, the AUC of tbsfin1 changes from S to 0.5.\n + S (the original AUC of tbsfin1) is here set to 0.9. \n + Because tbsfin1's calls are easy to find and can be assumed to score well. \n+ Finally, calculate how many bird species are included with the following equation.\n\n+ delta_X = X - X'\n\n\nWhy is this equation derived? Please refer to the comments. I may be wrong. Please comment.\n\nThe following is the above method I tried.\n\n\nAs a result of the calculation, I estimate that Public LB contains about **70 (0.4 / 0.006 ) species**.\n\nBy the way, the calculation with BirdCLEF2024 included about 100 bird species.", "3163636": "", "3163994": "We can make a top estimate with this technique. \n1) The decrease in the public score may be lower because of the score rounding. So the 0.826 score could be 0.826000, and the 0.820 score could be 0.820999 actually, so the decrease may be up to 0.005. I'm not sure about how Kaggle does the rounding, and I assume that the digits are just truncated. However, If Kaggle uses `round()`, the result is the same.\n2) Actually, we are able to predict the bird with 1.0 AUC score, so the decrease in the individual bird's score may be up to 0.5.\n\nKeeping this in mind, we get **0.5/0.005=100** species maximum from your experiment. This is an upper bound only, but it is quite strict.", "3164073": "Yes, you are right. It is true that there is uncertainty.\n\nThis is why I say **\"the higher the score, the better\"**.The higher species score, the lower the uncertainty (the larger the delta_X).", "3164226": "thanks for your valuable content.", "3204217": "May I ask if the competition hoster has mentioned how the public and private leaderboard are divided? We know that an audio has multiple submissions. Will there be a situation where one audio is on both the public and private leaderboard?", "3204227": "They mentioned its random split here - https://www.kaggle.com/competitions/birdclef-2025/discussion/572159#3173630" }, "source": "meta" }