{
  "id": 92403,
  "title": "IOU Metric clarification",
  "url": "/competitions/imaterialist-fashion-2019-FGVC6/discussion/92403",
  "author_name": "",
  "post_date": "2019-05-16T08:04:51.742442200Z",
  "votes": 6,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I still have doubts about metric.</p>\n\n<p>Let's say we have in total 10 classes. \nImage 1 has true classes 1, 4 and 5 and predicted classes 1, 4, 9</p>\n\n<p>How the score for image will be calculated?</p>\n\n<ul>\n<li>Case 1: (IOU_class1 + IOU_class4 + 0.0 (class 5) + 0.0 (class 9)) / 4</li>\n<li>Case 2: (IOU_class1 + IOU_class4 + 0.0 (class 5)) / 3</li>\n<li>Case 3: (1.0 (class 0) + IOU_class1 + 1.0 (class 2) + 1.0 (class 3) + IOU_class4 + 0.0 (class 5) + 1.0 (class 6) + 1.0 (class 7) + 1.0 (class 8) + 0.0 (class 9)) / 10</li>\n</ul>\n\n<p>May be there is some kernel with metric implementation for multiclass IoU?</p>",
  "messages": [
    {
      "id": "532108",
      "postDate": "05/16/2019 08:04:51",
      "content": "<p>I still have doubts about metric.</p>\n\n<p>Let's say we have in total 10 classes. \nImage 1 has true classes 1, 4 and 5 and predicted classes 1, 4, 9</p>\n\n<p>How the score for image will be calculated?</p>\n\n<ul>\n<li>Case 1: (IOU_class1 + IOU_class4 + 0.0 (class 5) + 0.0 (class 9)) / 4</li>\n<li>Case 2: (IOU_class1 + IOU_class4 + 0.0 (class 5)) / 3</li>\n<li>Case 3: (1.0 (class 0) + IOU_class1 + 1.0 (class 2) + 1.0 (class 3) + IOU_class4 + 0.0 (class 5) + 1.0 (class 6) + 1.0 (class 7) + 1.0 (class 8) + 0.0 (class 9)) / 10</li>\n</ul>\n\n<p>May be there is some kernel with metric implementation for multiclass IoU?</p>",
      "rawMarkdown": "I still have doubts about metric.\n\n\nLet's say we have in total 10 classes. \nImage 1 has true classes 1, 4 and 5 and predicted classes 1, 4, 9\n\n\nHow the score for image will be calculated?\n\n\n- Case 1: (IOU_class1 + IOU_class4 + 0.0 (class 5) + 0.0 (class 9)) / 4\n- Case 2: (IOU_class1 + IOU_class4 + 0.0 (class 5)) / 3\n- Case 3: (1.0 (class 0) + IOU_class1 + 1.0 (class 2) + 1.0 (class 3) + IOU_class4 + 0.0 (class 5) + 1.0 (class 6) + 1.0 (class 7) + 1.0 (class 8) + 0.0 (class 9)) / 10\n\n\nMay be there is some kernel with metric implementation for multiclass IoU?",
      "votes": null
    },
    {
      "id": "532116",
      "postDate": "05/16/2019 08:21:10",
      "content": "<p>I'm not sure myself, but I think we calculate IoU for classes 1, 4, 5, 9 and average them (since IoU is not defined for all other classes), see <a href=\"https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90684#526239\">https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90684#526239</a></p>",
      "rawMarkdown": "I'm not sure myself, but I think we calculate IoU for classes 1, 4, 5, 9 and average them (since IoU is not defined for all other classes), see https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90684#526239",
      "votes": null
    },
    {
      "id": "532525",
      "postDate": "05/17/2019 07:09:49",
      "content": "<p>mIoU, following stack overflow post would help you.\n<a href=\"https://stackoverflow.com/questions/31653576/how-to-calculate-the-mean-iu-score-in-image-segmentation\">https://stackoverflow.com/questions/31653576/how-to-calculate-the-mean-iu-score-in-image-segmentation</a>\nNote that it is not jaccard score. </p>",
      "rawMarkdown": "mIoU, following stack overflow post would help you.\nhttps://stackoverflow.com/questions/31653576/how-to-calculate-the-mean-iu-score-in-image-segmentation\nNote that it is not jaccard score.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 532116,
      "author_name": "lopuhin",
      "author_url": "",
      "post_date": "05/16/2019 08:21:10",
      "content": "<p>I'm not sure myself, but I think we calculate IoU for classes 1, 4, 5, 9 and average them (since IoU is not defined for all other classes), see <a href=\"https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90684#526239\">https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90684#526239</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 532525,
      "author_name": "sunggukcha",
      "author_url": "",
      "post_date": "05/17/2019 07:09:49",
      "content": "<p>mIoU, following stack overflow post would help you.\n<a href=\"https://stackoverflow.com/questions/31653576/how-to-calculate-the-mean-iu-score-in-image-segmentation\">https://stackoverflow.com/questions/31653576/how-to-calculate-the-mean-iu-score-in-image-segmentation</a>\nNote that it is not jaccard score. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "532108": "I still have doubts about metric.\n\n\nLet's say we have in total 10 classes. \nImage 1 has true classes 1, 4 and 5 and predicted classes 1, 4, 9\n\n\nHow the score for image will be calculated?\n\n\n- Case 1: (IOU_class1 + IOU_class4 + 0.0 (class 5) + 0.0 (class 9)) / 4\n- Case 2: (IOU_class1 + IOU_class4 + 0.0 (class 5)) / 3\n- Case 3: (1.0 (class 0) + IOU_class1 + 1.0 (class 2) + 1.0 (class 3) + IOU_class4 + 0.0 (class 5) + 1.0 (class 6) + 1.0 (class 7) + 1.0 (class 8) + 0.0 (class 9)) / 10\n\n\nMay be there is some kernel with metric implementation for multiclass IoU?",
    "532116": "I'm not sure myself, but I think we calculate IoU for classes 1, 4, 5, 9 and average them (since IoU is not defined for all other classes), see https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90684#526239",
    "532525": "mIoU, following stack overflow post would help you.\nhttps://stackoverflow.com/questions/31653576/how-to-calculate-the-mean-iu-score-in-image-segmentation\nNote that it is not jaccard score."
  },
  "source": "meta"
}