{
  "id": 245168,
  "title": "Is image level gt noisy?",
  "url": "/competitions/siim-covid19-detection/discussion/245168",
  "author_name": "Anii",
  "post_date": "2021-06-10T04:46:32.192000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I trained a detectron2 retinanet on image levels but seems does not converge well. I can only get validation ap around 17:</p>\n<pre><code>(AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.173\n(AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.541\n(AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.054\n(AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.026\n(AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.176\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.132\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.349\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.446\n(AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.250\n(AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.450\n[06/10 11:28:00 d2.evaluation.coco_evaluation]: Evaluation results for bbox: \n|   AP   |  AP50  |  AP75  |  APs  |  APm  |  APl   |\n| 17.324 | 54.066 | 5.373  |  nan  | 2.601 | 17.640 |\n</code></pre>\n<p>As you can see AP50 is 54, but AP75 is only 5.4, which means the model can approximately guess the position of the opacity area, but not precisely. I have no domain knowledge of radiology, is this as expected or something wrong with my model/preprocessing？</p>",
  "messages": [
    {
      "id": 1343159,
      "postDate": "2021-06-10T04:46:32.193Z",
      "content": "<p>I trained a detectron2 retinanet on image levels but seems does not converge well. I can only get validation ap around 17:</p>\n<pre><code>(AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.173\n(AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.541\n(AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.054\n(AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.026\n(AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.176\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.132\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.349\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.446\n(AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.250\n(AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.450\n[06/10 11:28:00 d2.evaluation.coco_evaluation]: Evaluation results for bbox: \n|   AP   |  AP50  |  AP75  |  APs  |  APm  |  APl   |\n| 17.324 | 54.066 | 5.373  |  nan  | 2.601 | 17.640 |\n</code></pre>\n<p>As you can see AP50 is 54, but AP75 is only 5.4, which means the model can approximately guess the position of the opacity area, but not precisely. I have no domain knowledge of radiology, is this as expected or something wrong with my model/preprocessing？</p>",
      "rawMarkdown": "I trained a detectron2 retinanet on image levels but seems does not converge well. I can only get validation ap around 17:\n```\n(AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.173\n(AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.541\n(AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.054\n(AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.026\n(AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.176\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.132\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.349\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.446\n(AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.250\n(AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.450\n[06/10 11:28:00 d2.evaluation.coco_evaluation]: Evaluation results for bbox: \n|   AP   |  AP50  |  AP75  |  APs  |  APm  |  APl   |\n| 17.324 | 54.066 | 5.373  |  nan  | 2.601 | 17.640 |\n```\n\nAs you can see AP50 is 54, but AP75 is only 5.4, which means the model can approximately guess the position of the opacity area, but not precisely. I have no domain knowledge of radiology, is this as expected or something wrong with my model/preprocessing？",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1343159": "I trained a detectron2 retinanet on image levels but seems does not converge well. I can only get validation ap around 17:\n```\n(AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.173\n(AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.541\n(AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.054\n(AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.026\n(AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.176\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.132\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.349\n(AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.446\n(AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n(AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.250\n(AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.450\n[06/10 11:28:00 d2.evaluation.coco_evaluation]: Evaluation results for bbox: \n|   AP   |  AP50  |  AP75  |  APs  |  APm  |  APl   |\n| 17.324 | 54.066 | 5.373  |  nan  | 2.601 | 17.640 |\n```\n\nAs you can see AP50 is 54, but AP75 is only 5.4, which means the model can approximately guess the position of the opacity area, but not precisely. I have no domain knowledge of radiology, is this as expected or something wrong with my model/preprocessing？"
  }
}