{
  "id": 165067,
  "title": "Some doubts regarding evaluation",
  "url": "/competitions/birdsong-recognition/discussion/165067",
  "author_name": "",
  "post_date": "2020-07-08T12:54:37.535941200Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Suppose the ground truth for a test sample (5 sec frame) is A A (assume A, B as ebird codes) and first model predict A B , second model predict A A B, third model predict B A, fourth model predict BB. My question is:\n1. Will the score for first model is higher than fourth model because it predict one ebird code right?\n2. What should be the score for second model because ground truth is A A and i predict A A B\n3. Is the sequence of predictions matter i.e. A B (ground truth) == B A (prediction)</p>\n\n<p>Can someone clear my doubts.</p>",
  "messages": [
    {
      "id": "920224",
      "postDate": "07/08/2020 12:54:37",
      "content": "<p>Suppose the ground truth for a test sample (5 sec frame) is A A (assume A, B as ebird codes) and first model predict A B , second model predict A A B, third model predict B A, fourth model predict BB. My question is:\n1. Will the score for first model is higher than fourth model because it predict one ebird code right?\n2. What should be the score for second model because ground truth is A A and i predict A A B\n3. Is the sequence of predictions matter i.e. A B (ground truth) == B A (prediction)</p>\n\n<p>Can someone clear my doubts.</p>",
      "rawMarkdown": "Suppose the ground truth for a test sample (5 sec frame) is A A (assume A, B as ebird codes) and first model predict A B , second model predict A A B, third model predict B A, fourth model predict BB. My question is:\n1. Will the score for first model is higher than fourth model because it predict one ebird code right?\n2. What should be the score for second model because ground truth is A A and i predict A A B\n3. Is the sequence of predictions matter i.e. A B (ground truth) == B A (prediction)\n\nCan someone clear my doubts.",
      "votes": null
    },
    {
      "id": "920386",
      "postDate": "07/08/2020 14:58:59",
      "content": "<p><a href=\"/hidehisaarai1213\">@hidehisaarai1213</a> <a href=\"/shonenkov\">@shonenkov</a> It would be a great help if you guys could pitch in. The evaluation metric has me puzzled too. micro-avg F1 is all well and good, but what does this \"row-wise\" mean here? How does this fit in with the overall score of the submission file?</p>",
      "rawMarkdown": "hidehisaarai1213 @shonenkov It would be a great help if you guys could pitch in. The evaluation metric has me puzzled too. micro-avg F1 is all well and good, but what does this \"row-wise\" mean here? How does this fit in with the overall score of the submission file?",
      "votes": null
    },
    {
      "id": "920537",
      "postDate": "07/08/2020 16:54:15",
      "content": "<p>The annotations are only for presence of species ('occupancy'), not how many of a species appear in a given frame. So there's no need to duplicate tags in a given frame.</p>",
      "rawMarkdown": "The annotations are only for presence of species ('occupancy'), not how many of a species appear in a given frame. So there's no need to duplicate tags in a given frame.",
      "votes": null
    },
    {
      "id": "920581",
      "postDate": "07/08/2020 17:28:56",
      "content": "<p>Got it. Thanks for your reply</p>",
      "rawMarkdown": "Got it. Thanks for your reply",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 920386,
      "author_name": "humblediscipulus",
      "author_url": "",
      "post_date": "07/08/2020 14:58:59",
      "content": "<p><a href=\"/hidehisaarai1213\">@hidehisaarai1213</a> <a href=\"/shonenkov\">@shonenkov</a> It would be a great help if you guys could pitch in. The evaluation metric has me puzzled too. micro-avg F1 is all well and good, but what does this \"row-wise\" mean here? How does this fit in with the overall score of the submission file?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 920537,
      "author_name": "tomdenton",
      "author_url": "",
      "post_date": "07/08/2020 16:54:15",
      "content": "<p>The annotations are only for presence of species ('occupancy'), not how many of a species appear in a given frame. So there's no need to duplicate tags in a given frame.</p>",
      "votes": null,
      "replies": [
        {
          "id": 920581,
          "author_name": "rishabhdhiman",
          "author_url": "",
          "post_date": "07/08/2020 17:28:56",
          "content": "<p>Got it. Thanks for your reply</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "920224": "Suppose the ground truth for a test sample (5 sec frame) is A A (assume A, B as ebird codes) and first model predict A B , second model predict A A B, third model predict B A, fourth model predict BB. My question is:\n1. Will the score for first model is higher than fourth model because it predict one ebird code right?\n2. What should be the score for second model because ground truth is A A and i predict A A B\n3. Is the sequence of predictions matter i.e. A B (ground truth) == B A (prediction)\n\nCan someone clear my doubts.",
    "920386": "hidehisaarai1213 @shonenkov It would be a great help if you guys could pitch in. The evaluation metric has me puzzled too. micro-avg F1 is all well and good, but what does this \"row-wise\" mean here? How does this fit in with the overall score of the submission file?",
    "920537": "The annotations are only for presence of species ('occupancy'), not how many of a species appear in a given frame. So there's no need to duplicate tags in a given frame.",
    "920581": "Got it. Thanks for your reply"
  },
  "source": "meta"
}