{
  "id": 232791,
  "title": "How the test data were annotated ?",
  "url": "/competitions/birdclef-2021/discussion/232791",
  "author_name": "",
  "post_date": "2021-04-15T11:44:11.366045100Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I would like to know how the test data were annotated.<br>\nWas the correct answer labeled by experts?<br>\nOr was it automatically labeled using some kind of machine learning model?<br>\nI asked this question because I thought it might be an indicator of how much noises are in the test data.<br>\nSincerely,</p>",
  "messages": [
    {
      "id": "1274535",
      "postDate": "04/15/2021 11:44:11",
      "content": "<p>I would like to know how the test data were annotated.<br>\nWas the correct answer labeled by experts?<br>\nOr was it automatically labeled using some kind of machine learning model?<br>\nI asked this question because I thought it might be an indicator of how much noises are in the test data.<br>\nSincerely,</p>",
      "rawMarkdown": "I would like to know how the test data were annotated.\nWas the correct answer labeled by experts?\nOr was it automatically labeled using some kind of machine learning model?\nI asked this question because I thought it might be an indicator of how much noises are in the test data.\nSincerely,",
      "votes": null
    },
    {
      "id": "1274537",
      "postDate": "04/15/2021 11:46:11",
      "content": "<p>The data were annotated by experts. Yet, we can't guarantee that every label is 100% correct. Annotators also skipped vocalizations that they weren't able to identify.</p>",
      "rawMarkdown": "The data were annotated by experts. Yet, we can't guarantee that every label is 100% correct. Annotators also skipped vocalizations that they weren't able to identify.",
      "votes": null
    },
    {
      "id": "1274545",
      "postDate": "04/15/2021 11:53:28",
      "content": "<p>I understood that the data was annotated by an experts. The noise about the labels in the test data seems to be small.<br>\nThank you for the quick answer!</p>",
      "rawMarkdown": "I understood that the data was annotated by an experts. The noise about the labels in the test data seems to be small.\nThank you for the quick answer!",
      "votes": null
    },
    {
      "id": "1276215",
      "postDate": "04/17/2021 08:54:18",
      "content": "<p>Is the train soundscapes also annotated by expert, same noise level as test data?</p>",
      "rawMarkdown": "Is the train soundscapes also annotated by expert, same noise level as test data?",
      "votes": null
    },
    {
      "id": "1277802",
      "postDate": "04/19/2021 08:26:29",
      "content": "<p>Yes, same experts, same annotation scheme. Yet, recording equipment and micro-habitat might differ from the hidden test set.</p>",
      "rawMarkdown": "Yes, same experts, same annotation scheme. Yet, recording equipment and micro-habitat might differ from the hidden test set.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1274537,
      "author_name": "stefankahl",
      "author_url": "",
      "post_date": "04/15/2021 11:46:11",
      "content": "<p>The data were annotated by experts. Yet, we can't guarantee that every label is 100% correct. Annotators also skipped vocalizations that they weren't able to identify.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1274545,
          "author_name": "naoism",
          "author_url": "",
          "post_date": "04/15/2021 11:53:28",
          "content": "<p>I understood that the data was annotated by an experts. The noise about the labels in the test data seems to be small.<br>\nThank you for the quick answer!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1276215,
          "author_name": "superchenhao",
          "author_url": "",
          "post_date": "04/17/2021 08:54:18",
          "content": "<p>Is the train soundscapes also annotated by expert, same noise level as test data?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1277802,
          "author_name": "stefankahl",
          "author_url": "",
          "post_date": "04/19/2021 08:26:29",
          "content": "<p>Yes, same experts, same annotation scheme. Yet, recording equipment and micro-habitat might differ from the hidden test set.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1274535": "I would like to know how the test data were annotated.\nWas the correct answer labeled by experts?\nOr was it automatically labeled using some kind of machine learning model?\nI asked this question because I thought it might be an indicator of how much noises are in the test data.\nSincerely,",
    "1274537": "The data were annotated by experts. Yet, we can't guarantee that every label is 100% correct. Annotators also skipped vocalizations that they weren't able to identify.",
    "1274545": "I understood that the data was annotated by an experts. The noise about the labels in the test data seems to be small.\nThank you for the quick answer!",
    "1276215": "Is the train soundscapes also annotated by expert, same noise level as test data?",
    "1277802": "Yes, same experts, same annotation scheme. Yet, recording equipment and micro-habitat might differ from the hidden test set."
  },
  "source": "meta"
}