{ "id": 295862, "title": "[Guide] - How to ensemble object detection models?", "url": "/competitions/tensorflow-great-barrier-reef/discussion/295862", "author_name": "", "post_date": "2021-12-18T10:39:59.092145Z", "votes": 39, "comment_count": 3, "views": 0, "content": "
\n

TL;DR: Demo notebook is here

\n
\n

How to ensemble object detection models?

\n

Hi Guys,

\n

I've made a notebook that demonstrates how to ensemble object detection models for this competition. It uses the WBF[1] ensemble method that got popular recently on multiple competitions.

\n

\"\"
\n[1] - Weighted Boxes Fusion in comparison to other ensemble methods.

\n

Weighted Boxes Fusion Ensemble Method

\n

Two years ago ZFTurbo released a python module for ensembling boxes for Object Detection models. It includes a very promising Weighted Boxes Fusion (WBF) method, which he used instead of NMS in google open Images 2019 and google open Images 2018 competitions.

\n

https://github.com/ZFTurbo/Weighted-Boxes-Fusion

\n

The repository contains the following methods:

\n\n

Soft-NMS
\nInstead of completely removing the detection proposals with high IoU and high confidence, it reduces the confidence of the proposals proportional to the IoU value.

\n

Weighted boxes fusion (WBF)

\n
    \n

  1. Each predicted box from each model is added to a single list B. The list is sorted in decreasing order of the confidence scores C.

    2. \n

  2. Create lists: L and F for boxes clusters and fused boxes. Each position in L contains a set of boxes (or single box), which form a cluster. Each position in F contains only one box, which is the fused box from the corresponding cluster in L.

    3. \n

  3. Iterate predicted boxes in B in cycle and try to find a matching box in the list F. Match is defined as a box with a large overlap with the box under question (IoU > thresh).

    4. \n

  4. If the match is not found, add the box from the list B to the end of lists L and F as new entries; proceed to the next box in the list B.

    5. \n

  5. If the match is found, add this box to the list L at the position pos corresponding to the matching box in the list F.

    6. \n

  6. Recalculate the box coordinates and confidence score in F[pos], using all T boxes accumulated in cluster L[pos]. (see the formula in the paper)Less than 24 hours! 🤯
    \nGood luck guys!

    7. \n

  7. After all boxes in B are processed, re-scale confidence scores in F list: multiply it by a number of boxes in a cluster and divide by a number of models N. (see the formula in the paper)

    8. \n
\n

Non-maximum weighted (NMW)
\nSee: Huajun Zhou, Zechao Li, Chengcheng Ning, and Jinhui Tang. Cad: Scale invariant framework for real-time object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 760–768, 2017.

\n

Installation:
\npip install ensemble-boxes

\n

References:

\n
    \n
  1. https://www.kaggle.com/c/open-images-2019-object-detection/discussion/115086
    2. \n
  2. https://arxiv.org/pdf/1910.13302.pdf
    3. \n
\n

'

\n

Happy Kaggling!
\nCheers^^

", "messages": [ { "id": "1622082", "postDate": "12/18/2021 10:39:59", "content": "
\n

TL;DR: Demo notebook is here

\n
\n

How to ensemble object detection models?

\n

Hi Guys,

\n

I've made a notebook that demonstrates how to ensemble object detection models for this competition. It uses the WBF[1] ensemble method that got popular recently on multiple competitions.

\n

\"\"
\n[1] - Weighted Boxes Fusion in comparison to other ensemble methods.

\n

Weighted Boxes Fusion Ensemble Method

\n

Two years ago ZFTurbo released a python module for ensembling boxes for Object Detection models. It includes a very promising Weighted Boxes Fusion (WBF) method, which he used instead of NMS in google open Images 2019 and google open Images 2018 competitions.

\n

https://github.com/ZFTurbo/Weighted-Boxes-Fusion

\n

The repository contains the following methods:

\n\n

Soft-NMS
\nInstead of completely removing the detection proposals with high IoU and high confidence, it reduces the confidence of the proposals proportional to the IoU value.

\n

Weighted boxes fusion (WBF)

\n
    \n

  1. Each predicted box from each model is added to a single list B. The list is sorted in decreasing order of the confidence scores C.

    2. \n

  2. Create lists: L and F for boxes clusters and fused boxes. Each position in L contains a set of boxes (or single box), which form a cluster. Each position in F contains only one box, which is the fused box from the corresponding cluster in L.

    3. \n

  3. Iterate predicted boxes in B in cycle and try to find a matching box in the list F. Match is defined as a box with a large overlap with the box under question (IoU > thresh).

    4. \n

  4. If the match is not found, add the box from the list B to the end of lists L and F as new entries; proceed to the next box in the list B.

    5. \n

  5. If the match is found, add this box to the list L at the position pos corresponding to the matching box in the list F.

    6. \n

  6. Recalculate the box coordinates and confidence score in F[pos], using all T boxes accumulated in cluster L[pos]. (see the formula in the paper)Less than 24 hours! 🤯
    \nGood luck guys!

    7. \n

  7. After all boxes in B are processed, re-scale confidence scores in F list: multiply it by a number of boxes in a cluster and divide by a number of models N. (see the formula in the paper)

    8. \n
\n

Non-maximum weighted (NMW)
\nSee: Huajun Zhou, Zechao Li, Chengcheng Ning, and Jinhui Tang. Cad: Scale invariant framework for real-time object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 760–768, 2017.

\n

Installation:
\npip install ensemble-boxes

\n

References:

\n
    \n
  1. https://www.kaggle.com/c/open-images-2019-object-detection/discussion/115086
    2. \n
  2. https://arxiv.org/pdf/1910.13302.pdf
    3. \n
\n

'

\n

Happy Kaggling!
\nCheers^^

", "rawMarkdown": "> **TL;DR: Demo notebook is [here](https://www.kaggle.com/yamqwe/great-barrier-reef-yolox-yolov5-ensemble)**\n\n\n### How to ensemble object detection models?\n\nHi Guys,\n\nI've made a notebook that demonstrates how to ensemble object detection models for this competition. It uses the WBF[1] ensemble method that got popular recently on multiple competitions. \n\n![](https://i.ibb.co/d2P2pPL/2020-05-12-21-02-34.png)\n[1] - Weighted Boxes Fusion in comparison to other ensemble methods.\n\n#### Weighted Boxes Fusion Ensemble Method\n\nTwo years ago ZFTurbo [released a python module](https://www.kaggle.com/c/open-images-2019-object-detection/discussion/115086) for ensembling boxes for Object Detection models. It includes a very promising Weighted Boxes Fusion (WBF) method, which he used instead of NMS in google open Images 2019 and google open Images 2018 competitions.\n\nhttps://github.com/ZFTurbo/Weighted-Boxes-Fusion\n\nThe repository contains the following methods:\n\n- Non-maximum Suppression (NMS)\nDetection boxes are considered as belonging to a single object if their overlap, intersection-over union (IoU) is higher than some threshold value\n\n**Soft-NMS**\nInstead of completely removing the detection proposals with high IoU and high confidence, it reduces the confidence of the proposals proportional to the IoU value.\n\n**Weighted boxes fusion (WBF)**\n\n1. Each predicted box from each model is added to a single list B. The list is sorted in decreasing order of the confidence scores C.\n\n2. Create lists: L and F for boxes clusters and fused boxes. Each position in L contains a set of boxes (or single box), which form a cluster. Each position in F contains only one box, which is the fused box from the corresponding cluster in L. \n\n3. Iterate predicted boxes in B in cycle and try to find a matching box in the list F. Match is defined as a box with a large overlap with the box under question (IoU > thresh).\n\n4. If the match is not found, add the box from the list B to the end of lists L and F as new entries; proceed to the next box in the list B.\n\n5. If the match is found, add this box to the list L at the position pos corresponding to the matching box in the list F.\n\n6. Recalculate the box coordinates and confidence score in F[pos], using all T boxes accumulated in cluster L[pos]. (see the formula in the paper)Less than 24 hours! 🤯\nGood luck guys!\n\n\n7. After all boxes in B are processed, re-scale confidence scores in F list: multiply it by a number of boxes in a cluster and divide by a number of models N. (see the formula in the paper)\n\n**Non-maximum weighted (NMW)**\nSee: Huajun Zhou, Zechao Li, Chengcheng Ning, and Jinhui Tang. Cad: Scale invariant framework for real-time object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 760–768, 2017.\n\nInstallation: \n`pip install ensemble-boxes`\n\n**References:**\n1. https://www.kaggle.com/c/open-images-2019-object-detection/discussion/115086\n2. https://arxiv.org/pdf/1910.13302.pdf\n\n'\n\nHappy Kaggling! \nCheers^^", "votes": null }, { "id": "1622118", "postDate": "12/18/2021 11:34:09", "content": "

Thank you for sharing @yamqwe! Certainly this will be one of the score booster. We will use it when find better models …

", "rawMarkdown": "Thank you for sharing @yamqwe! Certainly this will be one of the score booster. We will use it when find better models ...", "votes": null }, { "id": "1622980", "postDate": "12/19/2021 11:07:04", "content": "

I have done wbf. It takes more time during prediction. But there is a time limit in this competition and huge frames in the test set. Do you know how to reduce this time?

", "rawMarkdown": "I have done wbf. It takes more time during prediction. But there is a time limit in this competition and huge frames in the test set. Do you know how to reduce this time?", "votes": null }, { "id": "1835755", "postDate": "06/28/2022 04:00:13", "content": "

Thank you so much! It is very helpful

", "rawMarkdown": "Thank you so much! It is very helpful", "votes": null } ], "comments": [ { "id": 1622118, "author_name": "remekkinas", "author_url": "", "post_date": "12/18/2021 11:34:09", "content": "

Thank you for sharing @yamqwe! Certainly this will be one of the score booster. We will use it when find better models …

", "votes": null, "replies": [] }, { "id": 1622980, "author_name": "faisaltfc", "author_url": "", "post_date": "12/19/2021 11:07:04", "content": "

I have done wbf. It takes more time during prediction. But there is a time limit in this competition and huge frames in the test set. Do you know how to reduce this time?

", "votes": null, "replies": [] }, { "id": 1835755, "author_name": "bhavyakumar01", "author_url": "", "post_date": "06/28/2022 04:00:13", "content": "

Thank you so much! It is very helpful

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1622082": "> **TL;DR: Demo notebook is [here](https://www.kaggle.com/yamqwe/great-barrier-reef-yolox-yolov5-ensemble)**\n\n\n### How to ensemble object detection models?\n\nHi Guys,\n\nI've made a notebook that demonstrates how to ensemble object detection models for this competition. It uses the WBF[1] ensemble method that got popular recently on multiple competitions. \n\n![](https://i.ibb.co/d2P2pPL/2020-05-12-21-02-34.png)\n[1] - Weighted Boxes Fusion in comparison to other ensemble methods.\n\n#### Weighted Boxes Fusion Ensemble Method\n\nTwo years ago ZFTurbo [released a python module](https://www.kaggle.com/c/open-images-2019-object-detection/discussion/115086) for ensembling boxes for Object Detection models. It includes a very promising Weighted Boxes Fusion (WBF) method, which he used instead of NMS in google open Images 2019 and google open Images 2018 competitions.\n\nhttps://github.com/ZFTurbo/Weighted-Boxes-Fusion\n\nThe repository contains the following methods:\n\n- Non-maximum Suppression (NMS)\nDetection boxes are considered as belonging to a single object if their overlap, intersection-over union (IoU) is higher than some threshold value\n\n**Soft-NMS**\nInstead of completely removing the detection proposals with high IoU and high confidence, it reduces the confidence of the proposals proportional to the IoU value.\n\n**Weighted boxes fusion (WBF)**\n\n1. Each predicted box from each model is added to a single list B. The list is sorted in decreasing order of the confidence scores C.\n\n2. Create lists: L and F for boxes clusters and fused boxes. Each position in L contains a set of boxes (or single box), which form a cluster. Each position in F contains only one box, which is the fused box from the corresponding cluster in L. \n\n3. Iterate predicted boxes in B in cycle and try to find a matching box in the list F. Match is defined as a box with a large overlap with the box under question (IoU > thresh).\n\n4. If the match is not found, add the box from the list B to the end of lists L and F as new entries; proceed to the next box in the list B.\n\n5. If the match is found, add this box to the list L at the position pos corresponding to the matching box in the list F.\n\n6. Recalculate the box coordinates and confidence score in F[pos], using all T boxes accumulated in cluster L[pos]. (see the formula in the paper)Less than 24 hours! 🤯\nGood luck guys!\n\n\n7. After all boxes in B are processed, re-scale confidence scores in F list: multiply it by a number of boxes in a cluster and divide by a number of models N. (see the formula in the paper)\n\n**Non-maximum weighted (NMW)**\nSee: Huajun Zhou, Zechao Li, Chengcheng Ning, and Jinhui Tang. Cad: Scale invariant framework for real-time object detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 760–768, 2017.\n\nInstallation: \n`pip install ensemble-boxes`\n\n**References:**\n1. https://www.kaggle.com/c/open-images-2019-object-detection/discussion/115086\n2. https://arxiv.org/pdf/1910.13302.pdf\n\n'\n\nHappy Kaggling! \nCheers^^", "1622118": "Thank you for sharing @yamqwe! Certainly this will be one of the score booster. We will use it when find better models ...", "1622980": "I have done wbf. It takes more time during prediction. But there is a time limit in this competition and huge frames in the test set. Do you know how to reduce this time?", "1835755": "Thank you so much! It is very helpful" }, "source": "meta" }