{
  "id": 69057,
  "title": "Image-level score if no predicted boxes and no labels",
  "url": "/competitions/rsna-pneumonia-detection-challenge/discussion/69057",
  "author_name": "",
  "post_date": "2018-10-20T02:06:28.934032100Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>From Overview/Evaluation:</p>\n\n<blockquote>\n  <p>Important note: if there are no ground truth objects at all for a\n  given image, ANY number of predictions (false positives) will result\n  in the image receiving a score of zero, and being included in the mean\n  average precision.</p>\n</blockquote>\n\n<p>What if there are no ground truth objects at all for a given image and 0 predictions: the math would get 0/0. I'm assuming we get 1.0 precision for that image at every threshold? or 0?</p>",
  "messages": [
    {
      "id": "406904",
      "postDate": "10/20/2018 02:06:28",
      "content": "<p>From Overview/Evaluation:</p>\n\n<blockquote>\n  <p>Important note: if there are no ground truth objects at all for a\n  given image, ANY number of predictions (false positives) will result\n  in the image receiving a score of zero, and being included in the mean\n  average precision.</p>\n</blockquote>\n\n<p>What if there are no ground truth objects at all for a given image and 0 predictions: the math would get 0/0. I'm assuming we get 1.0 precision for that image at every threshold? or 0?</p>",
      "rawMarkdown": "From Overview/Evaluation:\n&gt; Important note: if there are no ground truth objects at all for a\n&gt; given image, ANY number of predictions (false positives) will result\n&gt; in the image receiving a score of zero, and being included in the mean\n&gt; average precision.\n\nWhat if there are no ground truth objects at all for a given image and 0 predictions: the math would get 0/0. I'm assuming we get 1.0 precision for that image at every threshold? or 0?",
      "votes": null
    },
    {
      "id": "406905",
      "postDate": "10/20/2018 02:09:06",
      "content": "<blockquote>\n  <p><strong>Dmitrij Kozachuk wrote</strong></p>\n  \n  <blockquote>\n    <p>On this case this patient don't take into account in competition metric. I mean\n     we have three step of metric calculation:</p>\n  </blockquote>\n  \n  <ol>\n  <li><p>Founding amount of patient, who had no real boxes and had no predicted boxes. Remove these patient from our test set. Let's denote amount of remained patient by n</p></li>\n  <li><p>For each other case we calculate average precision. So, we have n values: AP_1, AP_2, ..., AP_n</p></li>\n  <li>Final score is (AP_1 + AP_2 +  AP_n) / n</li>\n  </ol>\n</blockquote>\n\n<p>The answer seems to be that if you don't have any boxes for and image AND there are no labels for that image, it is not considered in the metric.</p>",
      "rawMarkdown": "&gt; **Dmitrij Kozachuk wrote**\n&gt; \n&gt; &gt; On this case this patient don't take into account in competition metric. I mean\n&gt;  we have three step of metric calculation:\n&gt; \n&gt;  1. Founding amount of patient, who had no real boxes and had no predicted boxes. Remove these patient from our test set. Let's denote amount of remained patient by n\n&gt; \n&gt;  2. For each other case we calculate average precision. So, we have n values: AP_1, AP_2, ..., AP_n\n&gt;  3. Final score is (AP_1 + AP_2 +  AP_n) / n\n\nThe answer seems to be that if you don't have any boxes for and image AND there are no labels for that image, it is not considered in the metric.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 406905,
      "author_name": "sshleifer",
      "author_url": "",
      "post_date": "10/20/2018 02:09:06",
      "content": "<blockquote>\n  <p><strong>Dmitrij Kozachuk wrote</strong></p>\n  \n  <blockquote>\n    <p>On this case this patient don't take into account in competition metric. I mean\n     we have three step of metric calculation:</p>\n  </blockquote>\n  \n  <ol>\n  <li><p>Founding amount of patient, who had no real boxes and had no predicted boxes. Remove these patient from our test set. Let's denote amount of remained patient by n</p></li>\n  <li><p>For each other case we calculate average precision. So, we have n values: AP_1, AP_2, ..., AP_n</p></li>\n  <li>Final score is (AP_1 + AP_2 +  AP_n) / n</li>\n  </ol>\n</blockquote>\n\n<p>The answer seems to be that if you don't have any boxes for and image AND there are no labels for that image, it is not considered in the metric.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "406904": "From Overview/Evaluation:\n&gt; Important note: if there are no ground truth objects at all for a\n&gt; given image, ANY number of predictions (false positives) will result\n&gt; in the image receiving a score of zero, and being included in the mean\n&gt; average precision.\n\nWhat if there are no ground truth objects at all for a given image and 0 predictions: the math would get 0/0. I'm assuming we get 1.0 precision for that image at every threshold? or 0?",
    "406905": "&gt; **Dmitrij Kozachuk wrote**\n&gt; \n&gt; &gt; On this case this patient don't take into account in competition metric. I mean\n&gt;  we have three step of metric calculation:\n&gt; \n&gt;  1. Founding amount of patient, who had no real boxes and had no predicted boxes. Remove these patient from our test set. Let's denote amount of remained patient by n\n&gt; \n&gt;  2. For each other case we calculate average precision. So, we have n values: AP_1, AP_2, ..., AP_n\n&gt;  3. Final score is (AP_1 + AP_2 +  AP_n) / n\n\nThe answer seems to be that if you don't have any boxes for and image AND there are no labels for that image, it is not considered in the metric."
  },
  "source": "meta"
}