{
  "id": 263832,
  "title": "Public 59th / Private 78th Solution",
  "url": "/competitions/siim-covid19-detection/writeups/yuki-schulta-public-59th-private-78th-solution",
  "author_name": "",
  "post_date": "2021-08-10T10:57:03.043Z",
  "votes": 11,
  "comment_count": 4,
  "views": 0,
  "content": "<h1>Acknowledgments</h1>\n<p>Thanks to Kaggle and the hosts for holding this exciting competition! I'm new to object detection tasks, and I've learned many things in this competition. Also thanks to all participants especially my teammate <a href=\"https://www.kaggle.com/schulta\" target=\"_blank\">@schulta</a>!</p>\n<h1>Detection</h1>\n<p>We used two models, EfficientDet(<a href=\"https://github.com/rwightman/efficientdet-pytorch\" target=\"_blank\">https://github.com/rwightman/efficientdet-pytorch</a>) and Yolo(<a href=\"https://github.com/ultralytics/yolov5\" target=\"_blank\">https://github.com/ultralytics/yolov5</a>)<br>\nScores of each model were as follows;</p>\n<table>\n<thead>\n<tr>\n<th>model</th>\n<th>image size</th>\n<th>AP</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>EfficientDetD7</td>\n<td>512</td>\n<td>0.2858</td>\n</tr>\n<tr>\n<td>Yolov5x</td>\n<td>512</td>\n<td>-</td>\n</tr>\n</tbody>\n</table>\n<p>What worked for us was as follows;</p>\n<ul>\n<li>Train models to predict only opacity class. (We predicted none class by classification models)</li>\n<li>Ensemble bboxes by WBF. IOU threshold was 0.65.</li>\n<li>TTA</li>\n<li>SAM optimizer</li>\n<li>Cutmix</li>\n<li>Modify the confidence of bboxes by the outputs of 2 class models like this;</li>\n</ul>\n<blockquote>\n  <p>confidence = confidence * (1 - Probability of none) ** 0.3</p>\n</blockquote>\n<p>Here, 0.3 is the arbitrary value which optimize oof score.</p>\n<h1>Classification</h1>\n<p>CV Scores and details of our models is as follows;</p>\n<table>\n<thead>\n<tr>\n<th>model</th>\n<th>image size</th>\n<th>CV</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>EfficientNet V2L</td>\n<td>512</td>\n<td>0.5613</td>\n</tr>\n<tr>\n<td>ResNet200D</td>\n<td>512</td>\n<td>0.5620</td>\n</tr>\n</tbody>\n</table>\n<p>We used two types of classification models, 4class classification model and 2class classification model. The former classifies Negative, Typical, Indeterminate and Atypical, which compose study class. The latter classifies the presence or absence of boxes in images that belong to the image class. We trained both models using 2step, which was used in the RANZCR competition (<a href=\"https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/205243\" target=\"_blank\">link</a>). </p>\n<ul>\n<li>1st step<br>\nWe created annotation masks using bboxes of image class and trained teacher models by annotated images.<br>\nThe following is an example of an annotated image.<br>\n<img alt=\"スクリーンショット 2021-08-10 19 34 12\" src=\"https://user-images.githubusercontent.com/70050531/128852437-b8c67fa5-6d13-43a7-938b-2e1236c88e42.png\"></li>\n<li>2nd step<br>\nWe trained student models without annotated images. In this step, auxiliary loss was used, which was the mean squared loss between final feature maps of student models and teacher models.</li>\n<li>3rd step<br>\nWe fine-tuned student model without auxiliary loss.</li>\n</ul>\n<p>Other methods we used were as follows;</p>\n<ul>\n<li>TTA</li>\n<li>SAM optimizer</li>\n</ul>\n<p>What didn't work was as follows;</p>\n<ul>\n<li>mixup</li>\n<li>external data</li>\n<li>Dual Attention Head(paper: <a href=\"https://arxiv.org/pdf/1809.02983.pdf\" target=\"_blank\">https://arxiv.org/pdf/1809.02983.pdf</a>)</li>\n<li>increasing the image size than 512x512</li>\n<li>increasing batch size</li>\n</ul>\n<p>If you have a question about this solution, feel free to ask!<br>\nThank you for reading!<br>\n(I'll add overview figure soon.)</p>",
  "messages": [
    {
      "id": "1463840",
      "postDate": "08/10/2021 10:54:53",
      "content": "<h1>Acknowledgments</h1>\n<p>Thanks to Kaggle and the hosts for holding this exciting competition! I'm new to object detection tasks, and I've learned many things in this competition. Also thanks to all participants especially my teammate <a href=\"https://www.kaggle.com/schulta\" target=\"_blank\">@schulta</a>!</p>\n<h1>Detection</h1>\n<p>We used two models, EfficientDet(<a href=\"https://github.com/rwightman/efficientdet-pytorch\" target=\"_blank\">https://github.com/rwightman/efficientdet-pytorch</a>) and Yolo(<a href=\"https://github.com/ultralytics/yolov5\" target=\"_blank\">https://github.com/ultralytics/yolov5</a>)<br>\nScores of each model were as follows;</p>\n<table>\n<thead>\n<tr>\n<th>model</th>\n<th>image size</th>\n<th>AP</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>EfficientDetD7</td>\n<td>512</td>\n<td>0.2858</td>\n</tr>\n<tr>\n<td>Yolov5x</td>\n<td>512</td>\n<td>-</td>\n</tr>\n</tbody>\n</table>\n<p>What worked for us was as follows;</p>\n<ul>\n<li>Train models to predict only opacity class. (We predicted none class by classification models)</li>\n<li>Ensemble bboxes by WBF. IOU threshold was 0.65.</li>\n<li>TTA</li>\n<li>SAM optimizer</li>\n<li>Cutmix</li>\n<li>Modify the confidence of bboxes by the outputs of 2 class models like this;</li>\n</ul>\n<blockquote>\n  <p>confidence = confidence * (1 - Probability of none) ** 0.3</p>\n</blockquote>\n<p>Here, 0.3 is the arbitrary value which optimize oof score.</p>\n<h1>Classification</h1>\n<p>CV Scores and details of our models is as follows;</p>\n<table>\n<thead>\n<tr>\n<th>model</th>\n<th>image size</th>\n<th>CV</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>EfficientNet V2L</td>\n<td>512</td>\n<td>0.5613</td>\n</tr>\n<tr>\n<td>ResNet200D</td>\n<td>512</td>\n<td>0.5620</td>\n</tr>\n</tbody>\n</table>\n<p>We used two types of classification models, 4class classification model and 2class classification model. The former classifies Negative, Typical, Indeterminate and Atypical, which compose study class. The latter classifies the presence or absence of boxes in images that belong to the image class. We trained both models using 2step, which was used in the RANZCR competition (<a href=\"https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/205243\" target=\"_blank\">link</a>). </p>\n<ul>\n<li>1st step<br>\nWe created annotation masks using bboxes of image class and trained teacher models by annotated images.<br>\nThe following is an example of an annotated image.<br>\n<img alt=\"スクリーンショット 2021-08-10 19 34 12\" src=\"https://user-images.githubusercontent.com/70050531/128852437-b8c67fa5-6d13-43a7-938b-2e1236c88e42.png\"></li>\n<li>2nd step<br>\nWe trained student models without annotated images. In this step, auxiliary loss was used, which was the mean squared loss between final feature maps of student models and teacher models.</li>\n<li>3rd step<br>\nWe fine-tuned student model without auxiliary loss.</li>\n</ul>\n<p>Other methods we used were as follows;</p>\n<ul>\n<li>TTA</li>\n<li>SAM optimizer</li>\n</ul>\n<p>What didn't work was as follows;</p>\n<ul>\n<li>mixup</li>\n<li>external data</li>\n<li>Dual Attention Head(paper: <a href=\"https://arxiv.org/pdf/1809.02983.pdf\" target=\"_blank\">https://arxiv.org/pdf/1809.02983.pdf</a>)</li>\n<li>increasing the image size than 512x512</li>\n<li>increasing batch size</li>\n</ul>\n<p>If you have a question about this solution, feel free to ask!<br>\nThank you for reading!<br>\n(I'll add overview figure soon.)</p>",
      "rawMarkdown": "# Acknowledgments\nThanks to Kaggle and the hosts for holding this exciting competition! I'm new to object detection tasks, and I've learned many things in this competition. Also thanks to all participants especially my teammate @schulta!\n\n# Detection\nWe used two models, EfficientDet(https://github.com/rwightman/efficientdet-pytorch) and Yolo(https://github.com/ultralytics/yolov5)\nScores of each model were as follows;\n\n| model | image size | AP | \n| :--: | :--: | :--: |\n| EfficientDetD7 | 512 | 0.2858 |\n| Yolov5x | 512 | - | <br>\n\nWhat worked for us was as follows;\n-  Train models to predict only opacity class. (We predicted none class by classification models)\n- Ensemble bboxes by WBF. IOU threshold was 0.65.\n- TTA\n- SAM optimizer\n- Cutmix\n- Modify the confidence of bboxes by the outputs of 2 class models like this;\n> confidence = confidence * (1 - Probability of none) ** 0.3\n\nHere, 0.3 is the arbitrary value which optimize oof score.\n\n# Classification\n\nCV Scores and details of our models is as follows;\n\n| model | image size | CV | \n| :--: | :--: | :--: |\n| EfficientNet V2L | 512 | 0.5613 |\n| ResNet200D | 512 | 0.5620 | <br>\n\nWe used two types of classification models, 4class classification model and 2class classification model. The former classifies Negative, Typical, Indeterminate and Atypical, which compose study class. The latter classifies the presence or absence of boxes in images that belong to the image class. We trained both models using 2step, which was used in the RANZCR competition ([link](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/205243)). \n- 1st step\nWe created annotation masks using bboxes of image class and trained teacher models by annotated images.\nThe following is an example of an annotated image.\n<img width=\"256\" alt=\"スクリーンショット 2021-08-10 19 34 12\" src=\"https://user-images.githubusercontent.com/70050531/128852437-b8c67fa5-6d13-43a7-938b-2e1236c88e42.png\">\n- 2nd step\nWe trained student models without annotated images. In this step, auxiliary loss was used, which was the mean squared loss between final feature maps of student models and teacher models.\n- 3rd step\nWe fine-tuned student model without auxiliary loss.\n\nOther methods we used were as follows;\n- TTA\n- SAM optimizer\n\nWhat didn't work was as follows;\n- mixup\n- external data\n- Dual Attention Head(paper: https://arxiv.org/pdf/1809.02983.pdf)\n- increasing the image size than 512x512\n- increasing batch size\n\nIf you have a question about this solution, feel free to ask!\nThank you for reading!\n(I'll add overview figure soon.)",
      "votes": null
    },
    {
      "id": "1463930",
      "postDate": "08/10/2021 11:42:50",
      "content": "<p>Congratulations Yuki and team!<br>\nBTW can you share your effdet detection code? I tried to work with that in beginning but failed.</p>",
      "rawMarkdown": "Congratulations Yuki and team!\nBTW can you share your effdet detection code? I tried to work with that in beginning but failed.",
      "votes": null
    },
    {
      "id": "1464046",
      "postDate": "08/10/2021 12:34:17",
      "content": "<p>Congratulations !<br>\nI have also used the SAM optimizer, it stabilized the training a lot but the final LB / CV didn't change so we didn't use it in our final submission. Did you see any improvement ? <br>\nI'm suprised that SAM is not used a lot here on kaggle, it's insanely useful to prevent overfiting in particular when the number of images is low. </p>",
      "rawMarkdown": "Congratulations !\nI have also used the SAM optimizer, it stabilized the training a lot but the final LB / CV didn't change so we didn't use it in our final submission. Did you see any improvement ? \nI'm suprised that SAM is not used a lot here on kaggle, it's insanely useful to prevent overfiting in particular when the number of images is low.",
      "votes": null
    },
    {
      "id": "1469492",
      "postDate": "08/13/2021 01:07:06",
      "content": "<p>Very sorry for late reply…<br>\nThe score of my 4 class model increased from 0.395 to 0.401 (Public score) by SAM optimizer, it's very surprised :)<br>\nYeah, I feel exactly the same way. I saw improvement in every competitions I took part in by just using SAM optimizer.</p>",
      "rawMarkdown": "Very sorry for late reply...\nThe score of my 4 class model increased from 0.395 to 0.401 (Public score) by SAM optimizer, it's very surprised :)\nYeah, I feel exactly the same way. I saw improvement in every competitions I took part in by just using SAM optimizer.",
      "votes": null
    },
    {
      "id": "1469499",
      "postDate": "08/13/2021 01:20:54",
      "content": "<p>Thank you for your reply!!!<br>\nI'm very sorry but I'm very busy after this competition, so I have no time to modify my code. (hide my personal information etc.)<br>\nI used these 2 notebooks as references.<br>\n<a href=\"https://www.kaggle.com/shonenkov/training-efficientdet\" target=\"_blank\">https://www.kaggle.com/shonenkov/training-efficientdet</a><br>\n<a href=\"https://www.kaggle.com/its7171/2class-object-detection-training\" target=\"_blank\">https://www.kaggle.com/its7171/2class-object-detection-training</a><br>\nIf you have a question, feel free to ask :)</p>",
      "rawMarkdown": "Thank you for your reply!!!\nI'm very sorry but I'm very busy after this competition, so I have no time to modify my code. (hide my personal information etc.)\nI used these 2 notebooks as references.\nhttps://www.kaggle.com/shonenkov/training-efficientdet\nhttps://www.kaggle.com/its7171/2class-object-detection-training\nIf you have a question, feel free to ask :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1463930,
      "author_name": "mrinath",
      "author_url": "",
      "post_date": "08/10/2021 11:42:50",
      "content": "<p>Congratulations Yuki and team!<br>\nBTW can you share your effdet detection code? I tried to work with that in beginning but failed.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1469499,
          "author_name": "focalor",
          "author_url": "",
          "post_date": "08/13/2021 01:20:54",
          "content": "<p>Thank you for your reply!!!<br>\nI'm very sorry but I'm very busy after this competition, so I have no time to modify my code. (hide my personal information etc.)<br>\nI used these 2 notebooks as references.<br>\n<a href=\"https://www.kaggle.com/shonenkov/training-efficientdet\" target=\"_blank\">https://www.kaggle.com/shonenkov/training-efficientdet</a><br>\n<a href=\"https://www.kaggle.com/its7171/2class-object-detection-training\" target=\"_blank\">https://www.kaggle.com/its7171/2class-object-detection-training</a><br>\nIf you have a question, feel free to ask :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1464046,
      "author_name": "tomdarmon",
      "author_url": "",
      "post_date": "08/10/2021 12:34:17",
      "content": "<p>Congratulations !<br>\nI have also used the SAM optimizer, it stabilized the training a lot but the final LB / CV didn't change so we didn't use it in our final submission. Did you see any improvement ? <br>\nI'm suprised that SAM is not used a lot here on kaggle, it's insanely useful to prevent overfiting in particular when the number of images is low. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1469492,
          "author_name": "focalor",
          "author_url": "",
          "post_date": "08/13/2021 01:07:06",
          "content": "<p>Very sorry for late reply…<br>\nThe score of my 4 class model increased from 0.395 to 0.401 (Public score) by SAM optimizer, it's very surprised :)<br>\nYeah, I feel exactly the same way. I saw improvement in every competitions I took part in by just using SAM optimizer.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1463840": "# Acknowledgments\nThanks to Kaggle and the hosts for holding this exciting competition! I'm new to object detection tasks, and I've learned many things in this competition. Also thanks to all participants especially my teammate @schulta!\n\n# Detection\nWe used two models, EfficientDet(https://github.com/rwightman/efficientdet-pytorch) and Yolo(https://github.com/ultralytics/yolov5)\nScores of each model were as follows;\n\n| model | image size | AP | \n| :--: | :--: | :--: |\n| EfficientDetD7 | 512 | 0.2858 |\n| Yolov5x | 512 | - | <br>\n\nWhat worked for us was as follows;\n-  Train models to predict only opacity class. (We predicted none class by classification models)\n- Ensemble bboxes by WBF. IOU threshold was 0.65.\n- TTA\n- SAM optimizer\n- Cutmix\n- Modify the confidence of bboxes by the outputs of 2 class models like this;\n> confidence = confidence * (1 - Probability of none) ** 0.3\n\nHere, 0.3 is the arbitrary value which optimize oof score.\n\n# Classification\n\nCV Scores and details of our models is as follows;\n\n| model | image size | CV | \n| :--: | :--: | :--: |\n| EfficientNet V2L | 512 | 0.5613 |\n| ResNet200D | 512 | 0.5620 | <br>\n\nWe used two types of classification models, 4class classification model and 2class classification model. The former classifies Negative, Typical, Indeterminate and Atypical, which compose study class. The latter classifies the presence or absence of boxes in images that belong to the image class. We trained both models using 2step, which was used in the RANZCR competition ([link](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/205243)). \n- 1st step\nWe created annotation masks using bboxes of image class and trained teacher models by annotated images.\nThe following is an example of an annotated image.\n<img width=\"256\" alt=\"スクリーンショット 2021-08-10 19 34 12\" src=\"https://user-images.githubusercontent.com/70050531/128852437-b8c67fa5-6d13-43a7-938b-2e1236c88e42.png\">\n- 2nd step\nWe trained student models without annotated images. In this step, auxiliary loss was used, which was the mean squared loss between final feature maps of student models and teacher models.\n- 3rd step\nWe fine-tuned student model without auxiliary loss.\n\nOther methods we used were as follows;\n- TTA\n- SAM optimizer\n\nWhat didn't work was as follows;\n- mixup\n- external data\n- Dual Attention Head(paper: https://arxiv.org/pdf/1809.02983.pdf)\n- increasing the image size than 512x512\n- increasing batch size\n\nIf you have a question about this solution, feel free to ask!\nThank you for reading!\n(I'll add overview figure soon.)",
    "1463930": "Congratulations Yuki and team!\nBTW can you share your effdet detection code? I tried to work with that in beginning but failed.",
    "1464046": "Congratulations !\nI have also used the SAM optimizer, it stabilized the training a lot but the final LB / CV didn't change so we didn't use it in our final submission. Did you see any improvement ? \nI'm suprised that SAM is not used a lot here on kaggle, it's insanely useful to prevent overfiting in particular when the number of images is low.",
    "1469492": "Very sorry for late reply...\nThe score of my 4 class model increased from 0.395 to 0.401 (Public score) by SAM optimizer, it's very surprised :)\nYeah, I feel exactly the same way. I saw improvement in every competitions I took part in by just using SAM optimizer.",
    "1469499": "Thank you for your reply!!!\nI'm very sorry but I'm very busy after this competition, so I have no time to modify my code. (hide my personal information etc.)\nI used these 2 notebooks as references.\nhttps://www.kaggle.com/shonenkov/training-efficientdet\nhttps://www.kaggle.com/its7171/2class-object-detection-training\nIf you have a question, feel free to ask :)"
  },
  "source": "meta"
}