{
  "id": 280722,
  "title": "Eval metric question",
  "url": "/competitions/reface-fake-detection/discussion/280722",
  "author_name": "",
  "post_date": "2021-10-22T15:03:45.391531400Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi there! Does it mean that we really need to aim to micro average when dealing with binary classification? It seems that we have to treat 1 and 0 as two separate classes, thus penalizing a bit more for FP and FN in this case</p>",
  "messages": [
    {
      "id": "1553828",
      "postDate": "10/22/2021 15:03:45",
      "content": "<p>Hi there! Does it mean that we really need to aim to micro average when dealing with binary classification? It seems that we have to treat 1 and 0 as two separate classes, thus penalizing a bit more for FP and FN in this case</p>",
      "rawMarkdown": "Hi there! Does it mean that we really need to aim to micro average when dealing with binary classification? It seems that we have to treat 1 and 0 as two separate classes, thus penalizing a bit more for FP and FN in this case",
      "votes": null
    },
    {
      "id": "1554013",
      "postDate": "10/22/2021 18:39:02",
      "content": "<p>Didn't get the purpose of the question. It seems like f1-score does what you have described (<a href=\"https://deepai.org/machine-learning-glossary-and-terms/f-score)\" target=\"_blank\">https://deepai.org/machine-learning-glossary-and-terms/f-score)</a>. Alternatively, you can check out a more generalized f-beta score (<a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html\" target=\"_blank\">https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html</a>)</p>",
      "rawMarkdown": "Didn't get the purpose of the question. It seems like f1-score does what you have described (https://deepai.org/machine-learning-glossary-and-terms/f-score). Alternatively, you can check out a more generalized f-beta score (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html)",
      "votes": null
    },
    {
      "id": "1555710",
      "postDate": "10/24/2021 06:19:11",
      "content": "<p>The purpose of my question is to understand if we have to aim to f1_score(…., average=\"binary\") or the f1_score(…., average=\"micro\"). Averaging (micro/macro) the f1 score is normally designed for the multiclass problem rather than binary. But we are dealing with binary.<br>\nThere are 2 options:</p>\n<ol>\n<li>Organizers did it intentionally</li>\n<li>They just made a mistake in eval description</li>\n</ol>\n<p>It confuses</p>",
      "rawMarkdown": "The purpose of my question is to understand if we have to aim to f1_score(...., average=\"binary\") or the f1_score(...., average=\"micro\"). Averaging (micro/macro) the f1 score is normally designed for the multiclass problem rather than binary. But we are dealing with binary.\nThere are 2 options:\n1. Organizers did it intentionally\n2. They just made a mistake in eval description\n\nIt confuses",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1554013,
      "author_name": "bogdanbaraban",
      "author_url": "",
      "post_date": "10/22/2021 18:39:02",
      "content": "<p>Didn't get the purpose of the question. It seems like f1-score does what you have described (<a href=\"https://deepai.org/machine-learning-glossary-and-terms/f-score)\" target=\"_blank\">https://deepai.org/machine-learning-glossary-and-terms/f-score)</a>. Alternatively, you can check out a more generalized f-beta score (<a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html\" target=\"_blank\">https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html</a>)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1555710,
      "author_name": "codealist",
      "author_url": "",
      "post_date": "10/24/2021 06:19:11",
      "content": "<p>The purpose of my question is to understand if we have to aim to f1_score(…., average=\"binary\") or the f1_score(…., average=\"micro\"). Averaging (micro/macro) the f1 score is normally designed for the multiclass problem rather than binary. But we are dealing with binary.<br>\nThere are 2 options:</p>\n<ol>\n<li>Organizers did it intentionally</li>\n<li>They just made a mistake in eval description</li>\n</ol>\n<p>It confuses</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1553828": "Hi there! Does it mean that we really need to aim to micro average when dealing with binary classification? It seems that we have to treat 1 and 0 as two separate classes, thus penalizing a bit more for FP and FN in this case",
    "1554013": "Didn't get the purpose of the question. It seems like f1-score does what you have described (https://deepai.org/machine-learning-glossary-and-terms/f-score). Alternatively, you can check out a more generalized f-beta score (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html)",
    "1555710": "The purpose of my question is to understand if we have to aim to f1_score(...., average=\"binary\") or the f1_score(...., average=\"micro\"). Averaging (micro/macro) the f1 score is normally designed for the multiclass problem rather than binary. But we are dealing with binary.\nThere are 2 options:\n1. Organizers did it intentionally\n2. They just made a mistake in eval description\n\nIt confuses"
  },
  "source": "meta"
}