{
  "id": 233822,
  "title": "Questions about competition metric",
  "url": "/competitions/birdclef-2021/discussion/233822",
  "author_name": "",
  "post_date": "2021-04-21T09:39:46.007822900Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>As the competition metric is described as \"row-wise micro averaged F1 score\", I am wondering if below f1_score calling is exactly same as competition metric:</p>\n<pre><code>from sklearn.metrics import f1_score\n\nscore = f1_score(y_true, y_pred, average='samples', zero_division=1)\n</code></pre>\n<p>I noticed if without zero_division=1 the returned score will always be 0, as there are some all 0 ground-truth samples in train soundscapes, so this zero_division=1 is necessary?</p>\n<p>And any typical post-process for this kind of f1 score metic?</p>",
  "messages": [
    {
      "id": "1279863",
      "postDate": "04/21/2021 09:39:46",
      "content": "<p>As the competition metric is described as \"row-wise micro averaged F1 score\", I am wondering if below f1_score calling is exactly same as competition metric:</p>\n<pre><code>from sklearn.metrics import f1_score\n\nscore = f1_score(y_true, y_pred, average='samples', zero_division=1)\n</code></pre>\n<p>I noticed if without zero_division=1 the returned score will always be 0, as there are some all 0 ground-truth samples in train soundscapes, so this zero_division=1 is necessary?</p>\n<p>And any typical post-process for this kind of f1 score metic?</p>",
      "rawMarkdown": "As the competition metric is described as \"row-wise micro averaged F1 score\", I am wondering if below f1_score calling is exactly same as competition metric:\n\n```\nfrom sklearn.metrics import f1_score\n\nscore = f1_score(y_true, y_pred, average='samples', zero_division=1)\n\n```\n\nI noticed if without zero_division=1 the returned score will always be 0, as there are some all 0 ground-truth samples in train soundscapes, so this zero_division=1 is necessary?\n\nAnd any typical post-process for this kind of f1 score metic?",
      "votes": null
    },
    {
      "id": "1283275",
      "postDate": "04/24/2021 18:54:58",
      "content": "<p>I don't think it's necessary. Check your labels to make sure they're in the expected format. If you're using label smoothing during training, you'll want to round your smoothened labels before computing F1 score.</p>",
      "rawMarkdown": "I don't think it's necessary. Check your labels to make sure they're in the expected format. If you're using label smoothing during training, you'll want to round your smoothened labels before computing F1 score.",
      "votes": null
    },
    {
      "id": "1286008",
      "postDate": "04/27/2021 13:35:43",
      "content": "<p>I think you are right.  If your model outputs probabilities, say after a softmax or a sigmoid layer, or even logits,  then you need to round them in order to use F1 score.  Once rounded then the metric is indeed f1 score with the parameters you selected IMHO.</p>",
      "rawMarkdown": "I think you are right.  If your model outputs probabilities, say after a softmax or a sigmoid layer, or even logits,  then you need to round them in order to use F1 score.  Once rounded then the metric is indeed f1 score with the parameters you selected IMHO.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1283275,
      "author_name": "adriel",
      "author_url": "",
      "post_date": "04/24/2021 18:54:58",
      "content": "<p>I don't think it's necessary. Check your labels to make sure they're in the expected format. If you're using label smoothing during training, you'll want to round your smoothened labels before computing F1 score.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1286008,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "04/27/2021 13:35:43",
      "content": "<p>I think you are right.  If your model outputs probabilities, say after a softmax or a sigmoid layer, or even logits,  then you need to round them in order to use F1 score.  Once rounded then the metric is indeed f1 score with the parameters you selected IMHO.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1279863": "As the competition metric is described as \"row-wise micro averaged F1 score\", I am wondering if below f1_score calling is exactly same as competition metric:\n\n```\nfrom sklearn.metrics import f1_score\n\nscore = f1_score(y_true, y_pred, average='samples', zero_division=1)\n\n```\n\nI noticed if without zero_division=1 the returned score will always be 0, as there are some all 0 ground-truth samples in train soundscapes, so this zero_division=1 is necessary?\n\nAnd any typical post-process for this kind of f1 score metic?",
    "1283275": "I don't think it's necessary. Check your labels to make sure they're in the expected format. If you're using label smoothing during training, you'll want to round your smoothened labels before computing F1 score.",
    "1286008": "I think you are right.  If your model outputs probabilities, say after a softmax or a sigmoid layer, or even logits,  then you need to round them in order to use F1 score.  Once rounded then the metric is indeed f1 score with the parameters you selected IMHO."
  },
  "source": "meta"
}