I would like to thank the organizers for hosting the great competition.
\nalso, I was very honored to become a teammate with @vaillant and @hwigeon.
\nI really learned a lot from our teammates. I think it will be the unforgettable good experience.
We reached LB 709 so quickly using a large-size inference trick and naive iou-tracker.
\nbut at the same time, we started to doubt the public LB since we couldn't reproduce the result clearly. we found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.
\nSo, we started to build robust models by checking CV F2 scores.
\nIn CV, only inference with originally trained size worked.
\nWe tried size threshold-based prediction, IOU tracker(i might have made a mistake), larger size than originally trained size didn't work in the local.
We found out that a large model necessarily isn't needed.
\nAlso what we found out is multi-scale ensemble boosts both cv and public LB scores.
\nInstead of investing our time in searching complex models, we prepared yolov5m trained with the sizes of [2400,2560,2688,2880,3000]
\nand yolov5s trained with the sizes of [3000,3600,3720,3840,3960,4032,4080,4244]
We changed yolo's hyperparameters slightly
\nApplying mixup 0.5, mosaic 1.0, strong hsv change helped a lot
\n10 epoch training and adam optimizer were chosen.
\nModel was chosen by weighting Recall vs Precision 4:1
We chose video-split folds since these are more realistic.
\nEnsembled CV scores for video-split were each [0.71, 0.64, 0.77]
\nAfter that, we had searched optimized WBF coefficients and confidence scores using a method of grid search.
For video fold 0-1, WBF coefficient 0.5, skip bbox 0.01, confidence score 0.1 was best
\nFor video fold 2, WBF coefficient 0.5, skip_bbox 0.1, confidence score 0.2 was best
With these values, we trained models with all the datasets and regarded the public LB as a holdout
\nthe public LB scores of each model were ranged in [0.55~0.60]
The result is private LB 0.713 and public LB 0.648.
\nWe missed the gold but we couldn't choose the best private LB since this competition is a kind of shake-up competition
Thank you for reading this!
", "messages": [ { "id": "1695523", "postDate": "02/18/2022 07:41:45", "content": "I would like to thank the organizers for hosting the great competition.
\nalso, I was very honored to become a teammate with @vaillant and @hwigeon.
\nI really learned a lot from our teammates. I think it will be the unforgettable good experience.
We reached LB 709 so quickly using a large-size inference trick and naive iou-tracker.
\nbut at the same time, we started to doubt the public LB since we couldn't reproduce the result clearly. we found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.
\nSo, we started to build robust models by checking CV F2 scores.
\nIn CV, only inference with originally trained size worked.
\nWe tried size threshold-based prediction, IOU tracker(i might have made a mistake), larger size than originally trained size didn't work in the local.
We found out that a large model necessarily isn't needed.
\nAlso what we found out is multi-scale ensemble boosts both cv and public LB scores.
\nInstead of investing our time in searching complex models, we prepared yolov5m trained with the sizes of [2400,2560,2688,2880,3000]
\nand yolov5s trained with the sizes of [3000,3600,3720,3840,3960,4032,4080,4244]
We changed yolo's hyperparameters slightly
\nApplying mixup 0.5, mosaic 1.0, strong hsv change helped a lot
\n10 epoch training and adam optimizer were chosen.
\nModel was chosen by weighting Recall vs Precision 4:1
We chose video-split folds since these are more realistic.
\nEnsembled CV scores for video-split were each [0.71, 0.64, 0.77]
\nAfter that, we had searched optimized WBF coefficients and confidence scores using a method of grid search.
For video fold 0-1, WBF coefficient 0.5, skip bbox 0.01, confidence score 0.1 was best
\nFor video fold 2, WBF coefficient 0.5, skip_bbox 0.1, confidence score 0.2 was best
With these values, we trained models with all the datasets and regarded the public LB as a holdout
\nthe public LB scores of each model were ranged in [0.55~0.60]
The result is private LB 0.713 and public LB 0.648.
\nWe missed the gold but we couldn't choose the best private LB since this competition is a kind of shake-up competition
Thank you for reading this!
", "rawMarkdown": "I would like to thank the organizers for hosting the great competition.\nalso, I was very honored to become a teammate with @vaillant and @hwigeon. \nI really learned a lot from our teammates. I think it will be the unforgettable good experience.\n\n# Summary\nWe reached LB 709 so quickly using a large-size inference trick and naive iou-tracker.\nbut at the same time, we started to doubt the public LB since we couldn't reproduce the result clearly. we found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.\nSo, we started to build robust models by checking CV F2 scores.\nIn CV, only inference with originally trained size worked.\nWe tried size threshold-based prediction, IOU tracker(i might have made a mistake), larger size than originally trained size didn't work in the local.\n\n# Models\nWe found out that a large model necessarily isn't needed.\nAlso what we found out is multi-scale ensemble boosts both cv and public LB scores.\nInstead of investing our time in searching complex models, we prepared yolov5m trained with the sizes of [2400,2560,2688,2880,3000]\nand yolov5s trained with the sizes of [3000,3600,3720,3840,3960,4032,4080,4244]\n\n# Traning Method\nWe changed yolo's hyperparameters slightly\nApplying mixup 0.5, mosaic 1.0, strong hsv change helped a lot\n10 epoch training and adam optimizer were chosen.\nModel was chosen by weighting Recall vs Precision 4:1\n\n# Validation\nWe chose video-split folds since these are more realistic.\nEnsembled CV scores for video-split were each [0.71, 0.64, 0.77]\nAfter that, we had searched optimized WBF coefficients and confidence scores using a method of grid search.\n\nFor video fold 0-1, WBF coefficient 0.5, skip bbox 0.01, confidence score 0.1 was best\nFor video fold 2, WBF coefficient 0.5, skip_bbox 0.1, confidence score 0.2 was best\n\nWith these values, we trained models with all the datasets and regarded the public LB as a holdout\nthe public LB scores of each model were ranged in [0.55~0.60]\n\nThe result is private LB 0.713 and public LB 0.648.\nWe missed the gold but we couldn't choose the best private LB since this competition is a kind of shake-up competition\n\nThank you for reading this!", "votes": null }, { "id": "1695880", "postDate": "02/18/2022 12:28:38", "content": "\n\nIn CV, only inference with originally trained size worked.
\n
This line explains why most of public inference codes didn't work well.
\nAfter competition, I found that our small models (which have training image size = inference image size) work as good as high score public codes in private dataset.
I wish I checked CV-LB correlation with image size.
\nCongrats for taking 16th!
I also checked the same situation.
\nOur yoloX (1280,720 size) 0.399 in public LB but 0.597 in private LB
\nI learned again that we should verify cv-lb very carefully!
\nThank you for reading this post!
Great to know about ensemble with multi-scale images. Thanks and congrats on silver medal!
", "rawMarkdown": "Great to know about ensemble with multi-scale images. Thanks and congrats on silver medal!", "votes": null }, { "id": "1696931", "postDate": "02/19/2022 08:17:20", "content": "Thanks! Good kaggle!
", "rawMarkdown": "Thanks! Good kaggle!", "votes": null }, { "id": "1698768", "postDate": "02/20/2022 16:34:02", "content": "@deepkim Thanks for sharing your solution and congratulations to your team on the silver!
\nI didn't understand the point shared by you here, could you kindly clarify what you mean by:
\n\n\nwe found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.
\n
TIA! :)
", "rawMarkdown": "deepkim Thanks for sharing your solution and congratulations to your team on the silver!\n\nI didn't understand the point shared by you here, could you kindly clarify what you mean by:\n\n> we found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.\n\nTIA! :)", "votes": null }, { "id": "1698806", "postDate": "02/20/2022 17:19:38", "content": "I mean increasing image size increases TRUE POSITIVES but it also increases many many FALSE POSITIVES.
\nif FALSE POSITIVES increases a lot, then It swallows the merit of increasing TRUE POSITIVES and results in a drop of F2 SCORE.
Many Thanks for clarifying! 🙏
", "rawMarkdown": "Many Thanks for clarifying! 🙏", "votes": null }, { "id": "1703217", "postDate": "02/24/2022 09:56:08", "content": "Congratulate for taking the 16th and thanks for sharing your solution. I wonder why you trained the yolov5s with large sizes when the yolov5m trained with smaller one.
", "rawMarkdown": "Congratulate for taking the 16th and thanks for sharing your solution. I wonder why you trained the yolov5s with large sizes when the yolov5m trained with smaller one.", "votes": null }, { "id": "1706639", "postDate": "02/27/2022 17:12:50", "content": "@jarviskevin It's from experiments. or to maximize variabilities. Relatively yolov5m was good at a smaller size but yolov5s was good at a larger size. also, performance was almost the same between yolov5m and yolov5s
", "rawMarkdown": "jarviskevin It's from experiments. or to maximize variabilities. Relatively yolov5m was good at a smaller size but yolov5s was good at a larger size. also, performance was almost the same between yolov5m and yolov5s", "votes": null } ], "comments": [ { "id": 1695880, "author_name": "mutsumichieda", "author_url": "", "post_date": "02/18/2022 12:28:38", "content": "\n\nIn CV, only inference with originally trained size worked.
\n
This line explains why most of public inference codes didn't work well.
\nAfter competition, I found that our small models (which have training image size = inference image size) work as good as high score public codes in private dataset.
I wish I checked CV-LB correlation with image size.
\nCongrats for taking 16th!
I also checked the same situation.
\nOur yoloX (1280,720 size) 0.399 in public LB but 0.597 in private LB
\nI learned again that we should verify cv-lb very carefully!
\nThank you for reading this post!
Great to know about ensemble with multi-scale images. Thanks and congrats on silver medal!
", "votes": null, "replies": [ { "id": 1696931, "author_name": "deepkim", "author_url": "", "post_date": "02/19/2022 08:17:20", "content": "Thanks! Good kaggle!
", "votes": null, "replies": [] } ] }, { "id": 1698768, "author_name": "init27", "author_url": "", "post_date": "02/20/2022 16:34:02", "content": "@deepkim Thanks for sharing your solution and congratulations to your team on the silver!
\nI didn't understand the point shared by you here, could you kindly clarify what you mean by:
\n\n\nwe found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.
\n
TIA! :)
", "votes": null, "replies": [ { "id": 1698806, "author_name": "deepkim", "author_url": "", "post_date": "02/20/2022 17:19:38", "content": "I mean increasing image size increases TRUE POSITIVES but it also increases many many FALSE POSITIVES.
\nif FALSE POSITIVES increases a lot, then It swallows the merit of increasing TRUE POSITIVES and results in a drop of F2 SCORE.
Many Thanks for clarifying! 🙏
", "votes": null, "replies": [] } ] }, { "id": 1703217, "author_name": "jarviskevin", "author_url": "", "post_date": "02/24/2022 09:56:08", "content": "Congratulate for taking the 16th and thanks for sharing your solution. I wonder why you trained the yolov5s with large sizes when the yolov5m trained with smaller one.
", "votes": null, "replies": [ { "id": 1706639, "author_name": "deepkim", "author_url": "", "post_date": "02/27/2022 17:12:50", "content": "@jarviskevin It's from experiments. or to maximize variabilities. Relatively yolov5m was good at a smaller size but yolov5s was good at a larger size. also, performance was almost the same between yolov5m and yolov5s
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1695523": "I would like to thank the organizers for hosting the great competition.\nalso, I was very honored to become a teammate with @vaillant and @hwigeon. \nI really learned a lot from our teammates. I think it will be the unforgettable good experience.\n\n# Summary\nWe reached LB 709 so quickly using a large-size inference trick and naive iou-tracker.\nbut at the same time, we started to doubt the public LB since we couldn't reproduce the result clearly. we found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.\nSo, we started to build robust models by checking CV F2 scores.\nIn CV, only inference with originally trained size worked.\nWe tried size threshold-based prediction, IOU tracker(i might have made a mistake), larger size than originally trained size didn't work in the local.\n\n# Models\nWe found out that a large model necessarily isn't needed.\nAlso what we found out is multi-scale ensemble boosts both cv and public LB scores.\nInstead of investing our time in searching complex models, we prepared yolov5m trained with the sizes of [2400,2560,2688,2880,3000]\nand yolov5s trained with the sizes of [3000,3600,3720,3840,3960,4032,4080,4244]\n\n# Traning Method\nWe changed yolo's hyperparameters slightly\nApplying mixup 0.5, mosaic 1.0, strong hsv change helped a lot\n10 epoch training and adam optimizer were chosen.\nModel was chosen by weighting Recall vs Precision 4:1\n\n# Validation\nWe chose video-split folds since these are more realistic.\nEnsembled CV scores for video-split were each [0.71, 0.64, 0.77]\nAfter that, we had searched optimized WBF coefficients and confidence scores using a method of grid search.\n\nFor video fold 0-1, WBF coefficient 0.5, skip bbox 0.01, confidence score 0.1 was best\nFor video fold 2, WBF coefficient 0.5, skip_bbox 0.1, confidence score 0.2 was best\n\nWith these values, we trained models with all the datasets and regarded the public LB as a holdout\nthe public LB scores of each model were ranged in [0.55~0.60]\n\nThe result is private LB 0.713 and public LB 0.648.\nWe missed the gold but we couldn't choose the best private LB since this competition is a kind of shake-up competition\n\nThank you for reading this!", "1695880": "> In CV, only inference with originally trained size worked.\n\nThis line explains why most of public inference codes didn't work well.\nAfter competition, I found that our small models (which have training image size = inference image size) work as good as high score public codes in private dataset.\n\nI wish I checked CV-LB correlation with image size.\nCongrats for taking 16th!", "1695882": "I also checked the same situation.\nOur yoloX (1280,720 size) 0.399 in public LB but 0.597 in private LB\nI learned again that we should verify cv-lb very carefully!\nThank you for reading this post!", "1696015": "Great to know about ensemble with multi-scale images. Thanks and congrats on silver medal!", "1696931": "Thanks! Good kaggle!", "1698768": "deepkim Thanks for sharing your solution and congratulations to your team on the silver!\n\nI didn't understand the point shared by you here, could you kindly clarify what you mean by:\n\n> we found out that Increasing image size increased recall but also yielded unnecessary bounding boxes.\n\nTIA! :)", "1698806": "I mean increasing image size increases **TRUE POSITIVES** but it also increases many many **FALSE POSITIVES**.\nif **FALSE POSITIVES** increases a lot, then It swallows the merit of increasing **TRUE POSITIVES** and results in a drop of F2 SCORE.", "1699184": "Many Thanks for clarifying! 🙏", "1703217": "Congratulate for taking the 16th and thanks for sharing your solution. I wonder why you trained the yolov5s with large sizes when the yolov5m trained with smaller one.", "1706639": "jarviskevin It's from experiments. or to maximize variabilities. Relatively yolov5m was good at a smaller size but yolov5s was good at a larger size. also, performance was almost the same between yolov5m and yolov5s" }, "source": "meta" }