{
  "id": 197754,
  "title": "Evaluation Metric: Label ranking average precision",
  "url": "/competitions/rfcx-species-audio-detection/discussion/197754",
  "author_name": "Ultron",
  "post_date": "2020-11-17T22:50:46.191000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p><strong>Label ranking average precision (LRAP)</strong> averages over the samples the answer to the following question: <code>for each ground truth label, what fraction of higher-ranked labels were true labels?</code></p>\n<ul>\n<li>This performance measure will be higher if you are able to give a better rank to the labels associated with each sample. </li>\n</ul>\n<blockquote>\n  <p>The obtained score is always strictly greater than 0, and the best value is 1. </p>\n</blockquote>\n<ul>\n<li>If there is exactly one relevant label per sample, <strong>label ranking average precision is equivalent to the mean reciprocal rank.</strong></li>\n</ul>\n<p>Formally, given a binary indicator matrix of the ground truth labels $$y\\in \\{0, 1\\}^{n_\\text{samples} \\times n_\\text{labels}}$$ and the score associated with each label $$\\hat{f} \\in \\mathbb{R}^{n_\\text{samples} \\times n_\\text{labels}}$$ the average precision is defined as<br>\n$$LRAP(y, \\hat{f}) = \\frac{1}{n_{\\text{samples}}}\\sum_{i=0}^{n_{\\text{samples}} - 1} \\frac{1}{||y||_0}\\sum_{j:y_{ij} = 1} \\frac{|\\mathcal{L}_{ij}|}{\\text{rank}_{ij}}$$<br>\nwhere $$\\mathcal{L}_{ij} = \\{k: y_{ik} = 1, \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\}$$ and<br>\n$$\\text{rank}_{ij} = |\\{k: \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\}|$$ and $$|\\cdot|$$ computes the cardinality of the set (i.e., the number of elements in the set), and $$||\\cdot||_0$$ is the$$\\ell_0$$“norm” (which computes the number of nonzero elements in a vector).</p>\n<p>And <a href=\"https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-39940-9_488\" target=\"_blank\">What is mean reciprocal rank?</a></p>",
  "messages": [
    {
      "id": 1082451,
      "postDate": "2020-11-17T22:50:46.190Z",
      "content": "<p><strong>Label ranking average precision (LRAP)</strong> averages over the samples the answer to the following question: <code>for each ground truth label, what fraction of higher-ranked labels were true labels?</code></p>\n<ul>\n<li>This performance measure will be higher if you are able to give a better rank to the labels associated with each sample. </li>\n</ul>\n<blockquote>\n  <p>The obtained score is always strictly greater than 0, and the best value is 1. </p>\n</blockquote>\n<ul>\n<li>If there is exactly one relevant label per sample, <strong>label ranking average precision is equivalent to the mean reciprocal rank.</strong></li>\n</ul>\n<p>Formally, given a binary indicator matrix of the ground truth labels $$y\\in \\{0, 1\\}^{n_\\text{samples} \\times n_\\text{labels}}$$ and the score associated with each label $$\\hat{f} \\in \\mathbb{R}^{n_\\text{samples} \\times n_\\text{labels}}$$ the average precision is defined as<br>\n$$LRAP(y, \\hat{f}) = \\frac{1}{n_{\\text{samples}}}\\sum_{i=0}^{n_{\\text{samples}} - 1} \\frac{1}{||y||_0}\\sum_{j:y_{ij} = 1} \\frac{|\\mathcal{L}_{ij}|}{\\text{rank}_{ij}}$$<br>\nwhere $$\\mathcal{L}_{ij} = \\{k: y_{ik} = 1, \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\}$$ and<br>\n$$\\text{rank}_{ij} = |\\{k: \\hat{f}_{ik} \\geq \\hat{f}_{ij} \\}|$$ and $$|\\cdot|$$ computes the cardinality of the set (i.e., the number of elements in the set), and $$||\\cdot||_0$$ is the$$\\ell_0$$“norm” (which computes the number of nonzero elements in a vector).</p>\n<p>And <a href=\"https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-39940-9_488\" target=\"_blank\">What is mean reciprocal rank?</a></p>",
      "rawMarkdown": "**Label ranking average precision (LRAP)** averages over the samples the answer to the following question: `for each ground truth label, what fraction of higher-ranked labels were true labels?`\n\n- This performance measure will be higher if you are able to give a better rank to the labels associated with each sample. \n\n> The obtained score is always strictly greater than 0, and the best value is 1. \n\n- If there is exactly one relevant label per sample, **label ranking average precision is equivalent to the mean reciprocal rank.**\n\nFormally, given a binary indicator matrix of the ground truth labels $$y\\in \\\\{0, 1\\\\}^{n_\\text{samples} \\times n_\\text{labels}}$$ and the score associated with each label $$\\hat{f} \\in \\mathbb{R}^{n_\\text{samples} \\times n_\\text{labels}}$$ the average precision is defined as\n$$LRAP(y, \\hat{f}) = \\frac{1}{n_{\\text{samples}}}\\sum_{i=0}^{n_{\\text{samples}} - 1} \\frac{1}{||y||\\_0}\\sum_{j:y_{ij} = 1} \\frac{|\\mathcal{L}\\_{ij}|}{\\text{rank}\\_{ij}}$$\nwhere $$\\mathcal{L}\\_{ij} = \\\\{k: y\\_{ik} = 1, \\hat{f}\\_{ik} \\geq \\hat{f}\\_{ij} \\\\}$$ and\n$$\\text{rank}\\_{ij} = \\|\\\\{k: \\hat{f}\\_{ik} \\geq \\hat{f}\\_{ij} \\\\}\\|$$ and $$|\\cdot|$$ computes the cardinality of the set (i.e., the number of elements in the set), and $$||\\cdot||_0$$ is the$$\\ell_0$$“norm” (which computes the number of nonzero elements in a vector).\n\nAnd [What is mean reciprocal rank?](https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-39940-9_488)",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1082451": "**Label ranking average precision (LRAP)** averages over the samples the answer to the following question: `for each ground truth label, what fraction of higher-ranked labels were true labels?`\n\n- This performance measure will be higher if you are able to give a better rank to the labels associated with each sample. \n\n> The obtained score is always strictly greater than 0, and the best value is 1. \n\n- If there is exactly one relevant label per sample, **label ranking average precision is equivalent to the mean reciprocal rank.**\n\nFormally, given a binary indicator matrix of the ground truth labels $$y\\in \\\\{0, 1\\\\}^{n_\\text{samples} \\times n_\\text{labels}}$$ and the score associated with each label $$\\hat{f} \\in \\mathbb{R}^{n_\\text{samples} \\times n_\\text{labels}}$$ the average precision is defined as\n$$LRAP(y, \\hat{f}) = \\frac{1}{n_{\\text{samples}}}\\sum_{i=0}^{n_{\\text{samples}} - 1} \\frac{1}{||y||\\_0}\\sum_{j:y_{ij} = 1} \\frac{|\\mathcal{L}\\_{ij}|}{\\text{rank}\\_{ij}}$$\nwhere $$\\mathcal{L}\\_{ij} = \\\\{k: y\\_{ik} = 1, \\hat{f}\\_{ik} \\geq \\hat{f}\\_{ij} \\\\}$$ and\n$$\\text{rank}\\_{ij} = \\|\\\\{k: \\hat{f}\\_{ik} \\geq \\hat{f}\\_{ij} \\\\}\\|$$ and $$|\\cdot|$$ computes the cardinality of the set (i.e., the number of elements in the set), and $$||\\cdot||_0$$ is the$$\\ell_0$$“norm” (which computes the number of nonzero elements in a vector).\n\nAnd [What is mean reciprocal rank?](https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-39940-9_488)"
  }
}