{
  "id": 572155,
  "title": "Species with no observations in training datasets",
  "url": "/competitions/geolifeclef-2025/discussion/572155",
  "author_name": "",
  "post_date": "2025-04-07T22:36:02.076820400Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Dear GeoLifeCLEF25 organiser,</p>\n<p>I have been wondering about the following claim from the Overview tab:</p>\n<blockquote>\n  <p>No test sample is empty, and the test set only contains species from the train or validation set.</p>\n</blockquote>\n<ol>\n<li>First of all, it is not clear to me what is the \"validation set\" in this context.</li>\n<li>Next, I have counted only 10358 species in the PO and PA training data, but the max speciesId value is 11254, hinting that there shall be 11255 species.</li>\n<li>Finally, I made a silly submission with all 897=11255-10358 species that are not observed in the training data. Such submission shall give exact zero result on any test data if these species are not present. Yet, I received a positive result on pubic test partition. </li>\n</ol>\n<p>Hence, I am wondering what shall be correct interpretation of the claim from the Overview. Could you comment and clarify, please?</p>",
  "messages": [
    {
      "id": "3173432",
      "postDate": "04/07/2025 22:36:02",
      "content": "<p>Dear GeoLifeCLEF25 organiser,</p>\n<p>I have been wondering about the following claim from the Overview tab:</p>\n<blockquote>\n  <p>No test sample is empty, and the test set only contains species from the train or validation set.</p>\n</blockquote>\n<ol>\n<li>First of all, it is not clear to me what is the \"validation set\" in this context.</li>\n<li>Next, I have counted only 10358 species in the PO and PA training data, but the max speciesId value is 11254, hinting that there shall be 11255 species.</li>\n<li>Finally, I made a silly submission with all 897=11255-10358 species that are not observed in the training data. Such submission shall give exact zero result on any test data if these species are not present. Yet, I received a positive result on pubic test partition. </li>\n</ol>\n<p>Hence, I am wondering what shall be correct interpretation of the claim from the Overview. Could you comment and clarify, please?</p>",
      "rawMarkdown": "Dear GeoLifeCLEF25 organiser,\n\nI have been wondering about the following claim from the Overview tab:\n>No test sample is empty, and the test set only contains species from the train or validation set.\n\n1. First of all, it is not clear to me what is the \"validation set\" in this context.\n2. Next, I have counted only 10358 species in the PO and PA training data, but the max speciesId value is 11254, hinting that there shall be 11255 species.\n3. Finally, I made a silly submission with all 897=11255-10358 species that are not observed in the training data. Such submission shall give exact zero result on any test data if these species are not present. Yet, I received a positive result on pubic test partition. \n\nHence, I am wondering what shall be correct interpretation of the claim from the Overview. Could you comment and clarify, please?",
      "votes": null
    },
    {
      "id": "3173791",
      "postDate": "04/08/2025 10:51:19",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/gtikho\" target=\"_blank\">@gtikho</a>,</p>\n<p>Thank you for raising it.<br>\nThere was indeed a small issue, so thank you a lot for your investigation.<br>\nBelow, you can find some answers to your questions :)</p>\n<p><strong>About the Validation set.</strong> The <code>validation set</code> in the Overview page was just a copy-paste error that has been removed to avoid causing any confusion.</p>\n<p><strong>About the <code>speciesIds</code>.</strong> There were indeed a few samples with speciesId that were not present in the training set. Those were removed a few minutes ago, and all the submissions were re-scored. You should see that your \"dummy\" submission now scores 0. Despite this, in general, it was a minor issue, and most of the submissions' performance did not change; thank you again for raising this!</p>\n<p><strong>About the number of <code>speciesIds</code>.</strong> For clarity, yes, there are only 10358 species in the training set and there is no other species in the test set now :))</p>\n<p>Best,<br>\nLukas</p>",
      "rawMarkdown": "Dear @gtikho,\n\nThank you for raising it.\nThere was indeed a small issue, so thank you a lot for your investigation.\nBelow, you can find some answers to your questions :)\n\n**About the Validation set.** The `validation set` in the Overview page was just a copy-paste error that has been removed to avoid causing any confusion.\n\n**About the `speciesIds`.** There were indeed a few samples with speciesId that were not present in the training set. Those were removed a few minutes ago, and all the submissions were re-scored. You should see that your \"dummy\" submission now scores 0. Despite this, in general, it was a minor issue, and most of the submissions' performance did not change; thank you again for raising this!\n\n**About the number of `speciesIds`.** For clarity, yes, there are only 10358 species in the training set and there is no other species in the test set now :))\n\nBest,\nLukas",
      "votes": null
    },
    {
      "id": "3175426",
      "postDate": "04/10/2025 05:56:29",
      "content": "<p>Thank you very much for clarifications!</p>",
      "rawMarkdown": "Thank you very much for clarifications!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3173791,
      "author_name": "picekl",
      "author_url": "",
      "post_date": "04/08/2025 10:51:19",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/gtikho\" target=\"_blank\">@gtikho</a>,</p>\n<p>Thank you for raising it.<br>\nThere was indeed a small issue, so thank you a lot for your investigation.<br>\nBelow, you can find some answers to your questions :)</p>\n<p><strong>About the Validation set.</strong> The <code>validation set</code> in the Overview page was just a copy-paste error that has been removed to avoid causing any confusion.</p>\n<p><strong>About the <code>speciesIds</code>.</strong> There were indeed a few samples with speciesId that were not present in the training set. Those were removed a few minutes ago, and all the submissions were re-scored. You should see that your \"dummy\" submission now scores 0. Despite this, in general, it was a minor issue, and most of the submissions' performance did not change; thank you again for raising this!</p>\n<p><strong>About the number of <code>speciesIds</code>.</strong> For clarity, yes, there are only 10358 species in the training set and there is no other species in the test set now :))</p>\n<p>Best,<br>\nLukas</p>",
      "votes": null,
      "replies": [
        {
          "id": 3175426,
          "author_name": "gtikho",
          "author_url": "",
          "post_date": "04/10/2025 05:56:29",
          "content": "<p>Thank you very much for clarifications!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3173432": "Dear GeoLifeCLEF25 organiser,\n\nI have been wondering about the following claim from the Overview tab:\n>No test sample is empty, and the test set only contains species from the train or validation set.\n\n1. First of all, it is not clear to me what is the \"validation set\" in this context.\n2. Next, I have counted only 10358 species in the PO and PA training data, but the max speciesId value is 11254, hinting that there shall be 11255 species.\n3. Finally, I made a silly submission with all 897=11255-10358 species that are not observed in the training data. Such submission shall give exact zero result on any test data if these species are not present. Yet, I received a positive result on pubic test partition. \n\nHence, I am wondering what shall be correct interpretation of the claim from the Overview. Could you comment and clarify, please?",
    "3173791": "Dear @gtikho,\n\nThank you for raising it.\nThere was indeed a small issue, so thank you a lot for your investigation.\nBelow, you can find some answers to your questions :)\n\n**About the Validation set.** The `validation set` in the Overview page was just a copy-paste error that has been removed to avoid causing any confusion.\n\n**About the `speciesIds`.** There were indeed a few samples with speciesId that were not present in the training set. Those were removed a few minutes ago, and all the submissions were re-scored. You should see that your \"dummy\" submission now scores 0. Despite this, in general, it was a minor issue, and most of the submissions' performance did not change; thank you again for raising this!\n\n**About the number of `speciesIds`.** For clarity, yes, there are only 10358 species in the training set and there is no other species in the test set now :))\n\nBest,\nLukas",
    "3175426": "Thank you very much for clarifications!"
  },
  "source": "meta"
}