{
  "id": 197834,
  "title": "[Understand] Label-weighted label-ranking average precision.",
  "url": "/competitions/rfcx-species-audio-detection/discussion/197834",
  "author_name": "",
  "post_date": "2020-11-18T08:54:28.334914600Z",
  "votes": 23,
  "comment_count": 2,
  "views": 0,
  "content": "<pre><code>How is score calculated?\n</code></pre>\n<p><img src=\"http://latex.codecogs.com/gif.latex?Score%3D%5Cfrac%7BHowManyCorrectLabelsAlongTheWay%7D%7BTookHowManyLabelsToGetToIt%7D\" alt=\"equation\"></p>\n<p>Lets pretend we are labelling an audio as [Bird, Frog, Insect].<br>\nCase 1: Audio contains Bird [1, 0, 0]<br>\nOur model gives output as [0.8, 0.2, 0.6]<br>\nBird got 1st place, Insect got 2nd place and Frog got 3rd place.<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 1st place so we don't need to check further so score=1.0</p>\n<p>Case 2: Audio contains Bird [1, 0, 0]<br>\nOur model gives output as [0.1, 0.8, 0.6]<br>\nFrog got 1st place, Insect got 2nd place and Bird got 3rd place.<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 3rd place so score= 1/3 =0.33</p>\n<h2>Multilabel</h2>\n<p>Case 3: Audio contains Bird and Frog [1, 1, 0]<br>\nOur model gives output as [0.7, 0.8, 0.1]<br>\nFrog got 1st place, Bird got 2nd place and Insect got 3rd place.<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 2nd place so score= 2/2 =1 (Check the score formula above incase any problem)<br>\nNext Frog is at 1st place so score= 1/1=1</p>\n<p>Label-weighted score= (1 bird/2 label)<em>1 + (1 frog/2 label)</em>1 =1.0</p>\n<p>Case 4: Audio contains Bird and Frog [1, 1, 0]<br>\nOur model gives output as [0.2, 0.8, 0.7]<br>\nRanking is Frog, Insect, Bird<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 3rd place so score= 2/3 =0.66 (Check the score formula above incase any problem)<br>\nNext Frog is at 1st place so score= 1/1= 1</p>\n<p>Label-weighted score= (1 bird/2 label)<em>0.66 + (1 frog/2 label)</em>1 = 0.83</p>\n<p>Hopefully this gives you a better understanding of the calculations.</p>",
  "messages": [
    {
      "id": "1082815",
      "postDate": "11/18/2020 08:54:28",
      "content": "<pre><code>How is score calculated?\n</code></pre>\n<p><img src=\"http://latex.codecogs.com/gif.latex?Score%3D%5Cfrac%7BHowManyCorrectLabelsAlongTheWay%7D%7BTookHowManyLabelsToGetToIt%7D\" alt=\"equation\"></p>\n<p>Lets pretend we are labelling an audio as [Bird, Frog, Insect].<br>\nCase 1: Audio contains Bird [1, 0, 0]<br>\nOur model gives output as [0.8, 0.2, 0.6]<br>\nBird got 1st place, Insect got 2nd place and Frog got 3rd place.<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 1st place so we don't need to check further so score=1.0</p>\n<p>Case 2: Audio contains Bird [1, 0, 0]<br>\nOur model gives output as [0.1, 0.8, 0.6]<br>\nFrog got 1st place, Insect got 2nd place and Bird got 3rd place.<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 3rd place so score= 1/3 =0.33</p>\n<h2>Multilabel</h2>\n<p>Case 3: Audio contains Bird and Frog [1, 1, 0]<br>\nOur model gives output as [0.7, 0.8, 0.1]<br>\nFrog got 1st place, Bird got 2nd place and Insect got 3rd place.<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 2nd place so score= 2/2 =1 (Check the score formula above incase any problem)<br>\nNext Frog is at 1st place so score= 1/1=1</p>\n<p>Label-weighted score= (1 bird/2 label)<em>1 + (1 frog/2 label)</em>1 =1.0</p>\n<p>Case 4: Audio contains Bird and Frog [1, 1, 0]<br>\nOur model gives output as [0.2, 0.8, 0.7]<br>\nRanking is Frog, Insect, Bird<br>\nNow we will see at which place our correct label is present.<br>\nSo Bird was at 3rd place so score= 2/3 =0.66 (Check the score formula above incase any problem)<br>\nNext Frog is at 1st place so score= 1/1= 1</p>\n<p>Label-weighted score= (1 bird/2 label)<em>0.66 + (1 frog/2 label)</em>1 = 0.83</p>\n<p>Hopefully this gives you a better understanding of the calculations.</p>",
      "rawMarkdown": "```\nHow is score calculated?\n```\n![equation](http://latex.codecogs.com/gif.latex?Score%3D%5Cfrac%7BHowManyCorrectLabelsAlongTheWay%7D%7BTookHowManyLabelsToGetToIt%7D)\n\nLets pretend we are labelling an audio as [Bird, Frog, Insect].\nCase 1: Audio contains Bird [1, 0, 0]\nOur model gives output as [0.8, 0.2, 0.6]\nBird got 1st place, Insect got 2nd place and Frog got 3rd place.\nNow we will see at which place our correct label is present.\nSo Bird was at 1st place so we don't need to check further so score=1.0\n\nCase 2: Audio contains Bird [1, 0, 0]\nOur model gives output as [0.1, 0.8, 0.6]\nFrog got 1st place, Insect got 2nd place and Bird got 3rd place.\nNow we will see at which place our correct label is present.\nSo Bird was at 3rd place so score= 1/3 =0.33\n\n\n## Multilabel\nCase 3: Audio contains Bird and Frog [1, 1, 0]\nOur model gives output as [0.7, 0.8, 0.1]\nFrog got 1st place, Bird got 2nd place and Insect got 3rd place.\nNow we will see at which place our correct label is present.\nSo Bird was at 2nd place so score= 2/2 =1 (Check the score formula above incase any problem)\nNext Frog is at 1st place so score= 1/1=1\n\nLabel-weighted score= (1 bird/2 label)*1 + (1 frog/2 label)*1 =1.0\n\nCase 4: Audio contains Bird and Frog [1, 1, 0]\nOur model gives output as [0.2, 0.8, 0.7]\nRanking is Frog, Insect, Bird\nNow we will see at which place our correct label is present.\nSo Bird was at 3rd place so score= 2/3 =0.66 (Check the score formula above incase any problem)\nNext Frog is at 1st place so score= 1/1= 1\n\nLabel-weighted score= (1 bird/2 label)*0.66 + (1 frog/2 label)*1 = 0.83\n\nHopefully this gives you a better understanding of the calculations.",
      "votes": null
    },
    {
      "id": "1137911",
      "postDate": "01/04/2021 09:24:38",
      "content": "<p>Thank you for this! Must say that this has helped me in understanding the metric a whole lot better ^_^</p>",
      "rawMarkdown": "Thank you for this! Must say that this has helped me in understanding the metric a whole lot better ^_^",
      "votes": null
    },
    {
      "id": "1138320",
      "postDate": "01/04/2021 15:30:59",
      "content": "<p>Happy to help :)</p>",
      "rawMarkdown": "Happy to help :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1137911,
      "author_name": "aritrag",
      "author_url": "",
      "post_date": "01/04/2021 09:24:38",
      "content": "<p>Thank you for this! Must say that this has helped me in understanding the metric a whole lot better ^_^</p>",
      "votes": null,
      "replies": [
        {
          "id": 1138320,
          "author_name": "jainarindam",
          "author_url": "",
          "post_date": "01/04/2021 15:30:59",
          "content": "<p>Happy to help :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1082815": "```\nHow is score calculated?\n```\n![equation](http://latex.codecogs.com/gif.latex?Score%3D%5Cfrac%7BHowManyCorrectLabelsAlongTheWay%7D%7BTookHowManyLabelsToGetToIt%7D)\n\nLets pretend we are labelling an audio as [Bird, Frog, Insect].\nCase 1: Audio contains Bird [1, 0, 0]\nOur model gives output as [0.8, 0.2, 0.6]\nBird got 1st place, Insect got 2nd place and Frog got 3rd place.\nNow we will see at which place our correct label is present.\nSo Bird was at 1st place so we don't need to check further so score=1.0\n\nCase 2: Audio contains Bird [1, 0, 0]\nOur model gives output as [0.1, 0.8, 0.6]\nFrog got 1st place, Insect got 2nd place and Bird got 3rd place.\nNow we will see at which place our correct label is present.\nSo Bird was at 3rd place so score= 1/3 =0.33\n\n\n## Multilabel\nCase 3: Audio contains Bird and Frog [1, 1, 0]\nOur model gives output as [0.7, 0.8, 0.1]\nFrog got 1st place, Bird got 2nd place and Insect got 3rd place.\nNow we will see at which place our correct label is present.\nSo Bird was at 2nd place so score= 2/2 =1 (Check the score formula above incase any problem)\nNext Frog is at 1st place so score= 1/1=1\n\nLabel-weighted score= (1 bird/2 label)*1 + (1 frog/2 label)*1 =1.0\n\nCase 4: Audio contains Bird and Frog [1, 1, 0]\nOur model gives output as [0.2, 0.8, 0.7]\nRanking is Frog, Insect, Bird\nNow we will see at which place our correct label is present.\nSo Bird was at 3rd place so score= 2/3 =0.66 (Check the score formula above incase any problem)\nNext Frog is at 1st place so score= 1/1= 1\n\nLabel-weighted score= (1 bird/2 label)*0.66 + (1 frog/2 label)*1 = 0.83\n\nHopefully this gives you a better understanding of the calculations.",
    "1137911": "Thank you for this! Must say that this has helped me in understanding the metric a whole lot better ^_^",
    "1138320": "Happy to help :)"
  },
  "source": "meta"
}