{
  "id": 263830,
  "title": "47th place solution",
  "url": "/competitions/siim-covid19-detection/writeups/kozistr-47th-place-solution",
  "author_name": "",
  "post_date": "2021-08-19T08:43:11.313Z",
  "votes": 23,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hi everyone!</p>\n<p>First of all, congratulations to all the winners and did a great job on all participants' hard works! And Thanks to SIIM, FISABIO, RSNA, and Kaggle for hosting this competition.</p>\n<h2>TL;DR</h2>\n<p>I only got Kaggle GPU/TPU, couldn't experiment with many models with various training recipes. So, I try to implement the training codes which work on TPU as possible as I can! Usually, I trained the object-detection model on GPU, image classifier for study-level on TPU.</p>\n<p>Anyway, I hope this solution helps you in some ways :)</p>\n<h2>Study-Level</h2>\n<p>I used <code>effnet-b7</code> and <code>effnet-b6</code> models w/o the auxiliary branches (for segmentation head). The models were trained on grouped 5 folds and on different resolutions, 640, 800 respectively.</p>\n<p>And simply averaging for ensembling the models. LB score for mAP is 0.453. </p>\n<p>Additionally, I utilized <code>multi-paths dropout</code> (total 5 paths) to regularize the model &amp; found proper augmentations on my training recipes. Compared to the public notebooks, which introduce <code>effnet-b7</code> as a baseline, I guess those helps to boost CV/LB score.</p>\n<p>Lastly, I didn't apply TTA for the study-level because of the limitation of inference time.</p>\n<h2>None Classifier</h2>\n<p>I just used study-level models' negative confidence scores, and it boosts LB +0.003. Also, I tried to ensemble study-level models and opacity 2class classifier, but it dropped CV/LB scores.</p>\n<h2>Image-Level</h2>\n<p>Overall, I experimented with 3 types of models, <code>Yolov5</code>, <code>CascadeRCNN</code>, <code>VFNet</code>. In my recipes, <code>VFNet</code> achieves <code>bbox mAP</code> 0.55~, but the LB/PB score is lower than I guess.</p>\n<p>And <code>CascadeRCNN</code> didn't go well with Yolov5 models. <code>Yolov5</code> and <code>Yolov5 + CascadeRCNN</code> have comparable CV/LB scores, but only the <code>Yolov5</code> series have better LB/PB scores than the combined. (maybe the correlation of both models is high, I didn't check yet).</p>\n<p>Finally, I ensembles 3 series of Yolov5, <code>yolov5x6</code>, <code>yolov5l6</code>, <code>yolov5m6</code> respectively, and applied WBF with the same weights.</p>\n<h2>Works for me</h2>\n<ul>\n<li>label smoothing (0.05 is best on my experiments)</li>\n<li>640 ~ 800 resolutions for study-level (It's better than 512 on my experiments)</li>\n<li>512 resolution for image-level</li>\n<li>WBF (iou_threshold : 0.6, skip_box_threshold : 0.01)</li>\n<li>TTA for image-level</li>\n<li>inference higher resolution.<ul>\n<li>train 512 and inference 640 resolution for image-level (LB +0.003)</li></ul></li>\n<li>light augmentations<ul>\n<li>HorizontalFlip</li>\n<li>CutOut (huge patch, small number of patches)</li>\n<li>Brightness</li>\n<li>Scale/Shift/Rotate</li></ul></li>\n</ul>\n<h2>Not-works for me</h2>\n<ul>\n<li>train on high-resolution for study-level &amp; image-level<ul>\n<li>got higher CV score, but comparable LB, PB scores for image-level </li></ul></li>\n<li>effnetv2 series</li>\n<li>the auxiliary losses</li>\n<li>external data (w/ pseudo labeling)</li>\n<li>heavy augmentations for study-level models</li>\n<li>post-processing<ul>\n<li>calibrating the confidence score to filter <code>none</code> class</li>\n<li>modified WBF which introduced in <a href=\"https://www.kaggle.com/shonenkov/wbf-over-tta-single-model-efficientdet\" target=\"_blank\">here</a></li></ul></li>\n</ul>\n<h2>Reflections</h2>\n<ul>\n<li>One thing I regretted is the diversity of the models. Both study/image-level models of mine have a high prediction correlation because they are the same series (e.g. effnet, yolov5).</li>\n<li>trust CV</li>\n</ul>\n<h2>Source Code</h2>\n<p>You can check out my inference pipeline. <a href=\"https://www.kaggle.com/kozistr/infer-efnb6-7-yolov5m-l-x6?scriptVersionId=69448986\" target=\"_blank\">inference code</a></p>\n<p>Thank you very much!</p>",
  "messages": [
    {
      "id": "1463830",
      "postDate": "08/10/2021 10:49:37",
      "content": "<p>Hi everyone!</p>\n<p>First of all, congratulations to all the winners and did a great job on all participants' hard works! And Thanks to SIIM, FISABIO, RSNA, and Kaggle for hosting this competition.</p>\n<h2>TL;DR</h2>\n<p>I only got Kaggle GPU/TPU, couldn't experiment with many models with various training recipes. So, I try to implement the training codes which work on TPU as possible as I can! Usually, I trained the object-detection model on GPU, image classifier for study-level on TPU.</p>\n<p>Anyway, I hope this solution helps you in some ways :)</p>\n<h2>Study-Level</h2>\n<p>I used <code>effnet-b7</code> and <code>effnet-b6</code> models w/o the auxiliary branches (for segmentation head). The models were trained on grouped 5 folds and on different resolutions, 640, 800 respectively.</p>\n<p>And simply averaging for ensembling the models. LB score for mAP is 0.453. </p>\n<p>Additionally, I utilized <code>multi-paths dropout</code> (total 5 paths) to regularize the model &amp; found proper augmentations on my training recipes. Compared to the public notebooks, which introduce <code>effnet-b7</code> as a baseline, I guess those helps to boost CV/LB score.</p>\n<p>Lastly, I didn't apply TTA for the study-level because of the limitation of inference time.</p>\n<h2>None Classifier</h2>\n<p>I just used study-level models' negative confidence scores, and it boosts LB +0.003. Also, I tried to ensemble study-level models and opacity 2class classifier, but it dropped CV/LB scores.</p>\n<h2>Image-Level</h2>\n<p>Overall, I experimented with 3 types of models, <code>Yolov5</code>, <code>CascadeRCNN</code>, <code>VFNet</code>. In my recipes, <code>VFNet</code> achieves <code>bbox mAP</code> 0.55~, but the LB/PB score is lower than I guess.</p>\n<p>And <code>CascadeRCNN</code> didn't go well with Yolov5 models. <code>Yolov5</code> and <code>Yolov5 + CascadeRCNN</code> have comparable CV/LB scores, but only the <code>Yolov5</code> series have better LB/PB scores than the combined. (maybe the correlation of both models is high, I didn't check yet).</p>\n<p>Finally, I ensembles 3 series of Yolov5, <code>yolov5x6</code>, <code>yolov5l6</code>, <code>yolov5m6</code> respectively, and applied WBF with the same weights.</p>\n<h2>Works for me</h2>\n<ul>\n<li>label smoothing (0.05 is best on my experiments)</li>\n<li>640 ~ 800 resolutions for study-level (It's better than 512 on my experiments)</li>\n<li>512 resolution for image-level</li>\n<li>WBF (iou_threshold : 0.6, skip_box_threshold : 0.01)</li>\n<li>TTA for image-level</li>\n<li>inference higher resolution.<ul>\n<li>train 512 and inference 640 resolution for image-level (LB +0.003)</li></ul></li>\n<li>light augmentations<ul>\n<li>HorizontalFlip</li>\n<li>CutOut (huge patch, small number of patches)</li>\n<li>Brightness</li>\n<li>Scale/Shift/Rotate</li></ul></li>\n</ul>\n<h2>Not-works for me</h2>\n<ul>\n<li>train on high-resolution for study-level &amp; image-level<ul>\n<li>got higher CV score, but comparable LB, PB scores for image-level </li></ul></li>\n<li>effnetv2 series</li>\n<li>the auxiliary losses</li>\n<li>external data (w/ pseudo labeling)</li>\n<li>heavy augmentations for study-level models</li>\n<li>post-processing<ul>\n<li>calibrating the confidence score to filter <code>none</code> class</li>\n<li>modified WBF which introduced in <a href=\"https://www.kaggle.com/shonenkov/wbf-over-tta-single-model-efficientdet\" target=\"_blank\">here</a></li></ul></li>\n</ul>\n<h2>Reflections</h2>\n<ul>\n<li>One thing I regretted is the diversity of the models. Both study/image-level models of mine have a high prediction correlation because they are the same series (e.g. effnet, yolov5).</li>\n<li>trust CV</li>\n</ul>\n<h2>Source Code</h2>\n<p>You can check out my inference pipeline. <a href=\"https://www.kaggle.com/kozistr/infer-efnb6-7-yolov5m-l-x6?scriptVersionId=69448986\" target=\"_blank\">inference code</a></p>\n<p>Thank you very much!</p>",
      "rawMarkdown": "Hi everyone!\n\nFirst of all, congratulations to all the winners and did a great job on all participants' hard works! And Thanks to SIIM, FISABIO, RSNA, and Kaggle for hosting this competition.\n\n## TL;DR\n\nI only got Kaggle GPU/TPU, couldn't experiment with many models with various training recipes. So, I try to implement the training codes which work on TPU as possible as I can! Usually, I trained the object-detection model on GPU, image classifier for study-level on TPU.\n\nAnyway, I hope this solution helps you in some ways :)\n\n## Study-Level\n\nI used `effnet-b7` and `effnet-b6` models w/o the auxiliary branches (for segmentation head). The models were trained on grouped 5 folds and on different resolutions, 640, 800 respectively.\n\nAnd simply averaging for ensembling the models. LB score for mAP is 0.453. \n\nAdditionally, I utilized `multi-paths dropout` (total 5 paths) to regularize the model & found proper augmentations on my training recipes. Compared to the public notebooks, which introduce `effnet-b7` as a baseline, I guess those helps to boost CV/LB score.\n\nLastly, I didn't apply TTA for the study-level because of the limitation of inference time.\n\n## None Classifier\n\nI just used study-level models' negative confidence scores, and it boosts LB +0.003. Also, I tried to ensemble study-level models and opacity 2class classifier, but it dropped CV/LB scores.\n\n## Image-Level\n\nOverall, I experimented with 3 types of models, `Yolov5`, `CascadeRCNN`, `VFNet`. In my recipes, `VFNet` achieves `bbox mAP` 0.55~, but the LB/PB score is lower than I guess.\n\nAnd `CascadeRCNN` didn't go well with Yolov5 models. `Yolov5` and `Yolov5 + CascadeRCNN` have comparable CV/LB scores, but only the `Yolov5` series have better LB/PB scores than the combined. (maybe the correlation of both models is high, I didn't check yet).\n\nFinally, I ensembles 3 series of Yolov5, `yolov5x6`, `yolov5l6`, `yolov5m6` respectively, and applied WBF with the same weights.\n\n## Works for me\n\n* label smoothing (0.05 is best on my experiments)\n* 640 ~ 800 resolutions for study-level (It's better than 512 on my experiments)\n* 512 resolution for image-level\n* WBF (iou_threshold : 0.6, skip_box_threshold : 0.01)\n* TTA for image-level\n* inference higher resolution.\n    * train 512 and inference 640 resolution for image-level (LB +0.003)\n* light augmentations\n    * HorizontalFlip\n    * CutOut (huge patch, small number of patches)\n    * Brightness\n    * Scale/Shift/Rotate\n\n## Not-works for me\n\n* train on high-resolution for study-level & image-level\n    * got higher CV score, but comparable LB, PB scores for image-level \n* effnetv2 series\n* the auxiliary losses\n* external data (w/ pseudo labeling)\n* heavy augmentations for study-level models\n* post-processing\n    * calibrating the confidence score to filter `none` class\n    * modified WBF which introduced in [here](https://www.kaggle.com/shonenkov/wbf-over-tta-single-model-efficientdet)\n\n## Reflections\n\n* One thing I regretted is the diversity of the models. Both study/image-level models of mine have a high prediction correlation because they are the same series (e.g. effnet, yolov5).\n* trust CV\n\n## Source Code\n\nYou can check out my inference pipeline. [inference code](https://www.kaggle.com/kozistr/infer-efnb6-7-yolov5m-l-x6?scriptVersionId=69448986)\n\nThank you very much!",
      "votes": null
    },
    {
      "id": "1464147",
      "postDate": "08/10/2021 13:14:30",
      "content": "<p>Thanks for sharing the solution and your approach. Congratulations.</p>",
      "rawMarkdown": "Thanks for sharing the solution and your approach. Congratulations.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1464147,
      "author_name": "furcifer",
      "author_url": "",
      "post_date": "08/10/2021 13:14:30",
      "content": "<p>Thanks for sharing the solution and your approach. Congratulations.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1463830": "Hi everyone!\n\nFirst of all, congratulations to all the winners and did a great job on all participants' hard works! And Thanks to SIIM, FISABIO, RSNA, and Kaggle for hosting this competition.\n\n## TL;DR\n\nI only got Kaggle GPU/TPU, couldn't experiment with many models with various training recipes. So, I try to implement the training codes which work on TPU as possible as I can! Usually, I trained the object-detection model on GPU, image classifier for study-level on TPU.\n\nAnyway, I hope this solution helps you in some ways :)\n\n## Study-Level\n\nI used `effnet-b7` and `effnet-b6` models w/o the auxiliary branches (for segmentation head). The models were trained on grouped 5 folds and on different resolutions, 640, 800 respectively.\n\nAnd simply averaging for ensembling the models. LB score for mAP is 0.453. \n\nAdditionally, I utilized `multi-paths dropout` (total 5 paths) to regularize the model & found proper augmentations on my training recipes. Compared to the public notebooks, which introduce `effnet-b7` as a baseline, I guess those helps to boost CV/LB score.\n\nLastly, I didn't apply TTA for the study-level because of the limitation of inference time.\n\n## None Classifier\n\nI just used study-level models' negative confidence scores, and it boosts LB +0.003. Also, I tried to ensemble study-level models and opacity 2class classifier, but it dropped CV/LB scores.\n\n## Image-Level\n\nOverall, I experimented with 3 types of models, `Yolov5`, `CascadeRCNN`, `VFNet`. In my recipes, `VFNet` achieves `bbox mAP` 0.55~, but the LB/PB score is lower than I guess.\n\nAnd `CascadeRCNN` didn't go well with Yolov5 models. `Yolov5` and `Yolov5 + CascadeRCNN` have comparable CV/LB scores, but only the `Yolov5` series have better LB/PB scores than the combined. (maybe the correlation of both models is high, I didn't check yet).\n\nFinally, I ensembles 3 series of Yolov5, `yolov5x6`, `yolov5l6`, `yolov5m6` respectively, and applied WBF with the same weights.\n\n## Works for me\n\n* label smoothing (0.05 is best on my experiments)\n* 640 ~ 800 resolutions for study-level (It's better than 512 on my experiments)\n* 512 resolution for image-level\n* WBF (iou_threshold : 0.6, skip_box_threshold : 0.01)\n* TTA for image-level\n* inference higher resolution.\n    * train 512 and inference 640 resolution for image-level (LB +0.003)\n* light augmentations\n    * HorizontalFlip\n    * CutOut (huge patch, small number of patches)\n    * Brightness\n    * Scale/Shift/Rotate\n\n## Not-works for me\n\n* train on high-resolution for study-level & image-level\n    * got higher CV score, but comparable LB, PB scores for image-level \n* effnetv2 series\n* the auxiliary losses\n* external data (w/ pseudo labeling)\n* heavy augmentations for study-level models\n* post-processing\n    * calibrating the confidence score to filter `none` class\n    * modified WBF which introduced in [here](https://www.kaggle.com/shonenkov/wbf-over-tta-single-model-efficientdet)\n\n## Reflections\n\n* One thing I regretted is the diversity of the models. Both study/image-level models of mine have a high prediction correlation because they are the same series (e.g. effnet, yolov5).\n* trust CV\n\n## Source Code\n\nYou can check out my inference pipeline. [inference code](https://www.kaggle.com/kozistr/infer-efnb6-7-yolov5m-l-x6?scriptVersionId=69448986)\n\nThank you very much!",
    "1464147": "Thanks for sharing the solution and your approach. Congratulations."
  },
  "source": "meta"
}