{
  "id": 294854,
  "title": "COCO style competition metric",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/294854",
  "author_name": "",
  "post_date": "2021-12-13T08:49:48.663380400Z",
  "votes": 24,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Hi,</p>\n<p>I am sharing with you how i achieved competition metric using COCO style:</p>\n<pre><code>IoU metric: bbox\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=  1 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets= 10 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n</code></pre>\n<p>Precission and Recall are taken for IOU 0.30 to 0.80<br>\nF2_score = 5(Precission x Recall ) / (4 x precission + recall)</p>\n<p>To achieve Cocostyle metrics you have to use COCO api showed in pytorch tutorial<br>\n<a href=\"https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html\" target=\"_blank\">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a></p>\n<p>I recommend run it on Colab. After cloning the github and installing pycocotools we have to change the coco_eval.py</p>\n<p>and add line <br>\ncoco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True) in :</p>\n<pre><code>    def update(self, predictions):\n        img_ids = list(np.unique(list(predictions.keys())))\n        self.img_ids.extend(img_ids)\n\n        for iou_type in self.iou_types:\n            results = self.prepare(predictions, iou_type)\n            with redirect_stdout(io.StringIO()):\n                coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()\n            coco_eval = self.coco_eval[iou_type]\n\n            coco_eval.cocoDt = coco_dt\n            coco_eval.params.imgIds = list(img_ids)\n            coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n            img_ids, eval_imgs = evaluate(coco_eval)\n\n            self.eval_imgs[iou_type].append(eval_imgs)\n</code></pre>\n<p>and here:</p>\n<pre><code>def create_common_coco_eval(coco_eval, img_ids, eval_imgs):\n    img_ids, eval_imgs = merge(img_ids, eval_imgs)\n    img_ids = list(img_ids)\n    eval_imgs = list(eval_imgs.flatten())\n\n    coco_eval.evalImgs = eval_imgs\n    coco_eval.params.imgIds = img_ids\n    coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n    coco_eval._paramsEval = copy.deepcopy(coco_eval.params)\n</code></pre>\n<p>now all is done in simple training loop:</p>\n<pre><code>for epoch in range(num_epochs):\n    # train for one epoch, printing every 10 iterations\n    train_one_epoch(model, optimizer, dl_train, device, epoch, print_freq=100)\n    # update the learning rate\n    lr_scheduler.step()\n    # evaluate on the test dataset\n    evaluate(model, dl_val, device=device)\n    if epoch &gt; 7:\n        torch.save(model.state_dict(), 'retina_resnet101_fpn'+str(epoch)+'.pth')\n</code></pre>\n<p>Enjoy! :)</p>",
  "messages": [
    {
      "id": "1616196",
      "postDate": "12/13/2021 08:49:48",
      "content": "<p>Hi,</p>\n<p>I am sharing with you how i achieved competition metric using COCO style:</p>\n<pre><code>IoU metric: bbox\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=  1 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets= 10 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n</code></pre>\n<p>Precission and Recall are taken for IOU 0.30 to 0.80<br>\nF2_score = 5(Precission x Recall ) / (4 x precission + recall)</p>\n<p>To achieve Cocostyle metrics you have to use COCO api showed in pytorch tutorial<br>\n<a href=\"https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html\" target=\"_blank\">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a></p>\n<p>I recommend run it on Colab. After cloning the github and installing pycocotools we have to change the coco_eval.py</p>\n<p>and add line <br>\ncoco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True) in :</p>\n<pre><code>    def update(self, predictions):\n        img_ids = list(np.unique(list(predictions.keys())))\n        self.img_ids.extend(img_ids)\n\n        for iou_type in self.iou_types:\n            results = self.prepare(predictions, iou_type)\n            with redirect_stdout(io.StringIO()):\n                coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()\n            coco_eval = self.coco_eval[iou_type]\n\n            coco_eval.cocoDt = coco_dt\n            coco_eval.params.imgIds = list(img_ids)\n            coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n            img_ids, eval_imgs = evaluate(coco_eval)\n\n            self.eval_imgs[iou_type].append(eval_imgs)\n</code></pre>\n<p>and here:</p>\n<pre><code>def create_common_coco_eval(coco_eval, img_ids, eval_imgs):\n    img_ids, eval_imgs = merge(img_ids, eval_imgs)\n    img_ids = list(img_ids)\n    eval_imgs = list(eval_imgs.flatten())\n\n    coco_eval.evalImgs = eval_imgs\n    coco_eval.params.imgIds = img_ids\n    coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n    coco_eval._paramsEval = copy.deepcopy(coco_eval.params)\n</code></pre>\n<p>now all is done in simple training loop:</p>\n<pre><code>for epoch in range(num_epochs):\n    # train for one epoch, printing every 10 iterations\n    train_one_epoch(model, optimizer, dl_train, device, epoch, print_freq=100)\n    # update the learning rate\n    lr_scheduler.step()\n    # evaluate on the test dataset\n    evaluate(model, dl_val, device=device)\n    if epoch &gt; 7:\n        torch.save(model.state_dict(), 'retina_resnet101_fpn'+str(epoch)+'.pth')\n</code></pre>\n<p>Enjoy! :)</p>",
      "rawMarkdown": "Hi,\n\nI am sharing with you how i achieved competition metric using COCO style:\n```\nIoU metric: bbox\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=  1 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets= 10 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n```\nPrecission and Recall are taken for IOU 0.30 to 0.80\nF2_score = 5(Precission x Recall ) / (4 x precission + recall)\n\nTo achieve Cocostyle metrics you have to use COCO api showed in pytorch tutorial\nhttps://pytorch.org/tutorials/intermediate/torchvision_tutorial.html\n\nI recommend run it on Colab. After cloning the github and installing pycocotools we have to change the coco_eval.py\n\nand add line \ncoco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True) in :\n\n```\n    def update(self, predictions):\n        img_ids = list(np.unique(list(predictions.keys())))\n        self.img_ids.extend(img_ids)\n\n        for iou_type in self.iou_types:\n            results = self.prepare(predictions, iou_type)\n            with redirect_stdout(io.StringIO()):\n                coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()\n            coco_eval = self.coco_eval[iou_type]\n\n            coco_eval.cocoDt = coco_dt\n            coco_eval.params.imgIds = list(img_ids)\n            coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n            img_ids, eval_imgs = evaluate(coco_eval)\n\n            self.eval_imgs[iou_type].append(eval_imgs)\n```\nand here:\n\n```\ndef create_common_coco_eval(coco_eval, img_ids, eval_imgs):\n    img_ids, eval_imgs = merge(img_ids, eval_imgs)\n    img_ids = list(img_ids)\n    eval_imgs = list(eval_imgs.flatten())\n\n    coco_eval.evalImgs = eval_imgs\n    coco_eval.params.imgIds = img_ids\n    coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n    coco_eval._paramsEval = copy.deepcopy(coco_eval.params)\n```\nnow all is done in simple training loop:\n\n\n```\nfor epoch in range(num_epochs):\n    # train for one epoch, printing every 10 iterations\n    train_one_epoch(model, optimizer, dl_train, device, epoch, print_freq=100)\n    # update the learning rate\n    lr_scheduler.step()\n    # evaluate on the test dataset\n    evaluate(model, dl_val, device=device)\n    if epoch > 7:\n        torch.save(model.state_dict(), 'retina_resnet101_fpn'+str(epoch)+'.pth')\n```\n\nEnjoy! :)",
      "votes": null
    },
    {
      "id": "1616349",
      "postDate": "12/13/2021 11:01:37",
      "content": "<p>Great! Thank you!</p>",
      "rawMarkdown": "Great! Thank you!",
      "votes": null
    },
    {
      "id": "1616352",
      "postDate": "12/13/2021 11:04:30",
      "content": "<p>:) gonna test it with last submission, i was blind watching only loss score before</p>",
      "rawMarkdown": ":) gonna test it with last submission, i was blind watching only loss score before",
      "votes": null
    },
    {
      "id": "1616429",
      "postDate": "12/13/2021 12:17:00",
      "content": "<blockquote>\n  <p>Precission and Recall are taken for IOU 0.30 to 0.80<br>\n  F2_score = 5(Precission x Recall) / (4 x precission + recall)</p>\n</blockquote>\n<p>I also modified the coco code like you.  After my experiment, the metric calculated in this way is not completely related to competition metric:<br>\nfirst calculate F2_score = 5(Precission x Recall) / (4 x precission + recall) , then average from iou 0.3 to 0.8</p>",
      "rawMarkdown": "> Precission and Recall are taken for IOU 0.30 to 0.80\nF2_score = 5(Precission x Recall) / (4 x precission + recall)\n\nI also modified the coco code like you.  After my experiment, the metric calculated in this way is not completely related to competition metric:\nfirst calculate F2_score = 5(Precission x Recall) / (4 x precission + recall) , then average from iou 0.3 to 0.8",
      "votes": null
    },
    {
      "id": "1616441",
      "postDate": "12/13/2021 12:23:38",
      "content": "<p>I know what you mean. Because it should be averaged score from F2 partial scores. Same like X^2 + Y^2 is not like (X+Y)^2 , however if you made an experiment how close was your CV with this metric ? Because it still can indicate if you are going in good direction</p>",
      "rawMarkdown": "I know what you mean. Because it should be averaged score from F2 partial scores. Same like X^2 + Y^2 is not like (X+Y)^2 , however if you made an experiment how close was your CV with this metric ? Because it still can indicate if you are going in good direction",
      "votes": null
    },
    {
      "id": "1616453",
      "postDate": "12/13/2021 12:32:38",
      "content": "<p>In terms of correlation, its value can indeed reflect good or bad (in fact, those p and r on 0.5:0.95 can also do this). <br>\nWhat worries me is that there is still a unstable gap in absolute value. Though I can know that lb will get better, I don’t know how much it will get.</p>",
      "rawMarkdown": "In terms of correlation, its value can indeed reflect good or bad (in fact, those p and r on 0.5:0.95 can also do this). \nWhat worries me is that there is still a unstable gap in absolute value. Though I can know that lb will get better, I don’t know how much it will get.",
      "votes": null
    },
    {
      "id": "1620261",
      "postDate": "12/16/2021 15:48:00",
      "content": "<p>Could you please point out where the <code>coco_eval.py</code> file is located? I have looked at the <a href=\"https://github.com/cocodataset/cocoapi\" target=\"_blank\">https://github.com/cocodataset/cocoapi</a> repository, and I couldn't find <code>coco_eval.py</code> or any <code>def update</code> function in it.</p>",
      "rawMarkdown": "Could you please point out where the `coco_eval.py` file is located? I have looked at the https://github.com/cocodataset/cocoapi repository, and I couldn't find `coco_eval.py` or any `def update` function in it.",
      "votes": null
    },
    {
      "id": "1620276",
      "postDate": "12/16/2021 16:02:15",
      "content": "<p>rerun this tutorial <a href=\"https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html\" target=\"_blank\">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a> there is git repo to clone and there are the files</p>",
      "rawMarkdown": "rerun this tutorial https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html there is git repo to clone and there are the files",
      "votes": null
    },
    {
      "id": "1620284",
      "postDate": "12/16/2021 16:08:32",
      "content": "<p>got it (<a href=\"https://github.com/pytorch/vision/tree/v0.8.2/references/detection\" target=\"_blank\">https://github.com/pytorch/vision/tree/v0.8.2/references/detection</a>)<br>\nthanks! :) </p>",
      "rawMarkdown": "got it (https://github.com/pytorch/vision/tree/v0.8.2/references/detection)\nthanks! :)",
      "votes": null
    },
    {
      "id": "1640356",
      "postDate": "01/06/2022 12:17:40",
      "content": "<p>Guys, you are in one team 👀😄.</p>",
      "rawMarkdown": "Guys, you are in one team 👀😄.",
      "votes": null
    },
    {
      "id": "1641766",
      "postDate": "01/07/2022 17:26:37",
      "content": "<p>Yeah after that :)</p>",
      "rawMarkdown": "Yeah after that :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1616349,
      "author_name": "remekkinas",
      "author_url": "",
      "post_date": "12/13/2021 11:01:37",
      "content": "<p>Great! Thank you!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1616352,
          "author_name": "lukaszborecki",
          "author_url": "",
          "post_date": "12/13/2021 11:04:30",
          "content": "<p>:) gonna test it with last submission, i was blind watching only loss score before</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1640356,
          "author_name": "vad13irt",
          "author_url": "",
          "post_date": "01/06/2022 12:17:40",
          "content": "<p>Guys, you are in one team 👀😄.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1641766,
          "author_name": "lukaszborecki",
          "author_url": "",
          "post_date": "01/07/2022 17:26:37",
          "content": "<p>Yeah after that :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1616429,
      "author_name": "zzy990106",
      "author_url": "",
      "post_date": "12/13/2021 12:17:00",
      "content": "<blockquote>\n  <p>Precission and Recall are taken for IOU 0.30 to 0.80<br>\n  F2_score = 5(Precission x Recall) / (4 x precission + recall)</p>\n</blockquote>\n<p>I also modified the coco code like you.  After my experiment, the metric calculated in this way is not completely related to competition metric:<br>\nfirst calculate F2_score = 5(Precission x Recall) / (4 x precission + recall) , then average from iou 0.3 to 0.8</p>",
      "votes": null,
      "replies": [
        {
          "id": 1616441,
          "author_name": "lukaszborecki",
          "author_url": "",
          "post_date": "12/13/2021 12:23:38",
          "content": "<p>I know what you mean. Because it should be averaged score from F2 partial scores. Same like X^2 + Y^2 is not like (X+Y)^2 , however if you made an experiment how close was your CV with this metric ? Because it still can indicate if you are going in good direction</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1616453,
          "author_name": "zzy990106",
          "author_url": "",
          "post_date": "12/13/2021 12:32:38",
          "content": "<p>In terms of correlation, its value can indeed reflect good or bad (in fact, those p and r on 0.5:0.95 can also do this). <br>\nWhat worries me is that there is still a unstable gap in absolute value. Though I can know that lb will get better, I don’t know how much it will get.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1620261,
      "author_name": "tomcspark",
      "author_url": "",
      "post_date": "12/16/2021 15:48:00",
      "content": "<p>Could you please point out where the <code>coco_eval.py</code> file is located? I have looked at the <a href=\"https://github.com/cocodataset/cocoapi\" target=\"_blank\">https://github.com/cocodataset/cocoapi</a> repository, and I couldn't find <code>coco_eval.py</code> or any <code>def update</code> function in it.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1620276,
          "author_name": "lukaszborecki",
          "author_url": "",
          "post_date": "12/16/2021 16:02:15",
          "content": "<p>rerun this tutorial <a href=\"https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html\" target=\"_blank\">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a> there is git repo to clone and there are the files</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1620284,
          "author_name": "tomcspark",
          "author_url": "",
          "post_date": "12/16/2021 16:08:32",
          "content": "<p>got it (<a href=\"https://github.com/pytorch/vision/tree/v0.8.2/references/detection\" target=\"_blank\">https://github.com/pytorch/vision/tree/v0.8.2/references/detection</a>)<br>\nthanks! :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1616196": "Hi,\n\nI am sharing with you how i achieved competition metric using COCO style:\n```\nIoU metric: bbox\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Precision  (AP) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=  1 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets= 10 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=   all | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= small | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area=medium | maxDets=100 ] = 0.000\n Average Recall     (AR) @[ IoU=0.30:0.80 | area= large | maxDets=100 ] = 0.000\n```\nPrecission and Recall are taken for IOU 0.30 to 0.80\nF2_score = 5(Precission x Recall ) / (4 x precission + recall)\n\nTo achieve Cocostyle metrics you have to use COCO api showed in pytorch tutorial\nhttps://pytorch.org/tutorials/intermediate/torchvision_tutorial.html\n\nI recommend run it on Colab. After cloning the github and installing pycocotools we have to change the coco_eval.py\n\nand add line \ncoco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True) in :\n\n```\n    def update(self, predictions):\n        img_ids = list(np.unique(list(predictions.keys())))\n        self.img_ids.extend(img_ids)\n\n        for iou_type in self.iou_types:\n            results = self.prepare(predictions, iou_type)\n            with redirect_stdout(io.StringIO()):\n                coco_dt = COCO.loadRes(self.coco_gt, results) if results else COCO()\n            coco_eval = self.coco_eval[iou_type]\n\n            coco_eval.cocoDt = coco_dt\n            coco_eval.params.imgIds = list(img_ids)\n            coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n            img_ids, eval_imgs = evaluate(coco_eval)\n\n            self.eval_imgs[iou_type].append(eval_imgs)\n```\nand here:\n\n```\ndef create_common_coco_eval(coco_eval, img_ids, eval_imgs):\n    img_ids, eval_imgs = merge(img_ids, eval_imgs)\n    img_ids = list(img_ids)\n    eval_imgs = list(eval_imgs.flatten())\n\n    coco_eval.evalImgs = eval_imgs\n    coco_eval.params.imgIds = img_ids\n    coco_eval.params.iouThrs =  np.linspace(.3, 0.8, int(np.round((0.8 - .3) / .05)) + 1, endpoint=True)\n    coco_eval._paramsEval = copy.deepcopy(coco_eval.params)\n```\nnow all is done in simple training loop:\n\n\n```\nfor epoch in range(num_epochs):\n    # train for one epoch, printing every 10 iterations\n    train_one_epoch(model, optimizer, dl_train, device, epoch, print_freq=100)\n    # update the learning rate\n    lr_scheduler.step()\n    # evaluate on the test dataset\n    evaluate(model, dl_val, device=device)\n    if epoch > 7:\n        torch.save(model.state_dict(), 'retina_resnet101_fpn'+str(epoch)+'.pth')\n```\n\nEnjoy! :)",
    "1616349": "Great! Thank you!",
    "1616352": ":) gonna test it with last submission, i was blind watching only loss score before",
    "1616429": "> Precission and Recall are taken for IOU 0.30 to 0.80\nF2_score = 5(Precission x Recall) / (4 x precission + recall)\n\nI also modified the coco code like you.  After my experiment, the metric calculated in this way is not completely related to competition metric:\nfirst calculate F2_score = 5(Precission x Recall) / (4 x precission + recall) , then average from iou 0.3 to 0.8",
    "1616441": "I know what you mean. Because it should be averaged score from F2 partial scores. Same like X^2 + Y^2 is not like (X+Y)^2 , however if you made an experiment how close was your CV with this metric ? Because it still can indicate if you are going in good direction",
    "1616453": "In terms of correlation, its value can indeed reflect good or bad (in fact, those p and r on 0.5:0.95 can also do this). \nWhat worries me is that there is still a unstable gap in absolute value. Though I can know that lb will get better, I don’t know how much it will get.",
    "1620261": "Could you please point out where the `coco_eval.py` file is located? I have looked at the https://github.com/cocodataset/cocoapi repository, and I couldn't find `coco_eval.py` or any `def update` function in it.",
    "1620276": "rerun this tutorial https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html there is git repo to clone and there are the files",
    "1620284": "got it (https://github.com/pytorch/vision/tree/v0.8.2/references/detection)\nthanks! :)",
    "1640356": "Guys, you are in one team 👀😄.",
    "1641766": "Yeah after that :)"
  },
  "source": "meta"
}