{
  "id": 80454,
  "title": "Evaluation - if more than one prediction box overlaps reference box",
  "url": "/competitions/rsna-pneumonia-detection-challenge/discussion/80454",
  "author_name": "Keelin",
  "post_date": "2019-02-13T18:21:46.498000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I am trying to understand the evaluation process and how true-positives are decided if more than one prediction box is overlapping a reference box.   The text on evaluation states\n<em>A true positive is counted when a <strong>single</strong> predicted object matches a ground truth object with an IoU above the threshold.</em>  but I am interested in how it is decided which predicted object is chosen if more than one is overlapping with the ground truth object.\nI see also that bounding boxes are handled in order of their confidence levels, but suppose that all boxes have the same confidence...? <br>\nDo you then just pick whichever  single predicted box happens to be evaluated first?  Or do you check which has the largest IoU value with the ground truth box?  Or do you calculate the IoU for all of the overlaps and  count multiple true positives (in which case I cannot see what the confidence level is used for, in the event that they all have different confidence levels).\nThanks for any help!</p>",
  "messages": [
    {
      "id": 470895,
      "postDate": "2019-02-13T18:21:46.500Z",
      "content": "<p>I am trying to understand the evaluation process and how true-positives are decided if more than one prediction box is overlapping a reference box.   The text on evaluation states\n<em>A true positive is counted when a <strong>single</strong> predicted object matches a ground truth object with an IoU above the threshold.</em>  but I am interested in how it is decided which predicted object is chosen if more than one is overlapping with the ground truth object.\nI see also that bounding boxes are handled in order of their confidence levels, but suppose that all boxes have the same confidence...? <br>\nDo you then just pick whichever  single predicted box happens to be evaluated first?  Or do you check which has the largest IoU value with the ground truth box?  Or do you calculate the IoU for all of the overlaps and  count multiple true positives (in which case I cannot see what the confidence level is used for, in the event that they all have different confidence levels).\nThanks for any help!</p>",
      "rawMarkdown": "I am trying to understand the evaluation process and how true-positives are decided if more than one prediction box is overlapping a reference box.   The text on evaluation states\n*A true positive is counted when a **single** predicted object matches a ground truth object with an IoU above the threshold.*  but I am interested in how it is decided which predicted object is chosen if more than one is overlapping with the ground truth object.\nI see also that bounding boxes are handled in order of their confidence levels, but suppose that all boxes have the same confidence...?   \nDo you then just pick whichever  single predicted box happens to be evaluated first?  Or do you check which has the largest IoU value with the ground truth box?  Or do you calculate the IoU for all of the overlaps and  count multiple true positives (in which case I cannot see what the confidence level is used for, in the event that they all have different confidence levels).\nThanks for any help!"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "470895": "I am trying to understand the evaluation process and how true-positives are decided if more than one prediction box is overlapping a reference box.   The text on evaluation states\n*A true positive is counted when a **single** predicted object matches a ground truth object with an IoU above the threshold.*  but I am interested in how it is decided which predicted object is chosen if more than one is overlapping with the ground truth object.\nI see also that bounding boxes are handled in order of their confidence levels, but suppose that all boxes have the same confidence...?   \nDo you then just pick whichever  single predicted box happens to be evaluated first?  Or do you check which has the largest IoU value with the ground truth box?  Or do you calculate the IoU for all of the overlaps and  count multiple true positives (in which case I cannot see what the confidence level is used for, in the event that they all have different confidence levels).\nThanks for any help!"
  }
}