{
  "id": 65334,
  "title": "Multiple matching bbox for single truth and competition metric",
  "url": "/competitions/rsna-pneumonia-detection-challenge/discussion/65334",
  "author_name": "",
  "post_date": "2018-09-09T12:37:39.663232200Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>There are situations where a model can predict multiple boxes for single ground truth, and get a precision of 0, although if we count the overall IoU of those - the result would be at least good.</p>\n\n<p>Example attached and available at <a href=\"https://imgur.com/a/AFXzd1l\">imgur</a></p>\n\n<p>I wonder if instead of calculating IoU for each bbox pair - we could calculate total IoU - would the metric be more meaningful then ?</p>",
  "messages": [
    {
      "id": "383717",
      "postDate": "09/09/2018 12:37:39",
      "content": "<p>There are situations where a model can predict multiple boxes for single ground truth, and get a precision of 0, although if we count the overall IoU of those - the result would be at least good.</p>\n\n<p>Example attached and available at <a href=\"https://imgur.com/a/AFXzd1l\">imgur</a></p>\n\n<p>I wonder if instead of calculating IoU for each bbox pair - we could calculate total IoU - would the metric be more meaningful then ?</p>",
      "rawMarkdown": "There are situations where a model can predict multiple boxes for single ground truth, and get a precision of 0, although if we count the overall IoU of those - the result would be at least good.\n\nExample attached and available at [imgur][1]\n\nI wonder if instead of calculating IoU for each bbox pair - we could calculate total IoU - would the metric be more meaningful then ?\n\n\n  [1]: https://imgur.com/a/AFXzd1l",
      "votes": null
    },
    {
      "id": "383720",
      "postDate": "09/09/2018 12:46:36",
      "content": "<p>Maybe, it would be. But competition metric fixed, so we kinda don't care about it</p>",
      "rawMarkdown": "Maybe, it would be. But competition metric fixed, so we kinda don't care about it",
      "votes": null
    },
    {
      "id": "385240",
      "postDate": "09/10/2018 15:55:19",
      "content": "<p>You could also try to solve these problems with some post-processing. For example, after predicting the bboxes have a rule to combine them given certain criteria are met.</p>",
      "rawMarkdown": "You could also try to solve these problems with some post-processing. For example, after predicting the bboxes have a rule to combine them given certain criteria are met.",
      "votes": null
    },
    {
      "id": "385321",
      "postDate": "09/10/2018 19:22:49",
      "content": "<p>Yes, but I would really not want to do that as this would be a real hand-tuning</p>",
      "rawMarkdown": "Yes, but I would really not want to do that as this would be a real hand-tuning",
      "votes": null
    },
    {
      "id": "385360",
      "postDate": "09/10/2018 20:54:37",
      "content": "<p>Or you could have a model learn to do it.</p>",
      "rawMarkdown": "Or you could have a model learn to do it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 383720,
      "author_name": "koza4ukdmitrij",
      "author_url": "",
      "post_date": "09/09/2018 12:46:36",
      "content": "<p>Maybe, it would be. But competition metric fixed, so we kinda don't care about it</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 385240,
      "author_name": "puremath86",
      "author_url": "",
      "post_date": "09/10/2018 15:55:19",
      "content": "<p>You could also try to solve these problems with some post-processing. For example, after predicting the bboxes have a rule to combine them given certain criteria are met.</p>",
      "votes": null,
      "replies": [
        {
          "id": 385321,
          "author_name": "kretes",
          "author_url": "",
          "post_date": "09/10/2018 19:22:49",
          "content": "<p>Yes, but I would really not want to do that as this would be a real hand-tuning</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 385360,
          "author_name": "puremath86",
          "author_url": "",
          "post_date": "09/10/2018 20:54:37",
          "content": "<p>Or you could have a model learn to do it.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "383717": "There are situations where a model can predict multiple boxes for single ground truth, and get a precision of 0, although if we count the overall IoU of those - the result would be at least good.\n\nExample attached and available at [imgur][1]\n\nI wonder if instead of calculating IoU for each bbox pair - we could calculate total IoU - would the metric be more meaningful then ?\n\n\n  [1]: https://imgur.com/a/AFXzd1l",
    "383720": "Maybe, it would be. But competition metric fixed, so we kinda don't care about it",
    "385240": "You could also try to solve these problems with some post-processing. For example, after predicting the bboxes have a rule to combine them given certain criteria are met.",
    "385321": "Yes, but I would really not want to do that as this would be a real hand-tuning",
    "385360": "Or you could have a model learn to do it."
  },
  "source": "meta"
}