{
  "id": 240015,
  "title": "💥💥 Key Takeaways of Similar Past Kaggle Competitions 🔥🔥",
  "url": "/competitions/siim-covid19-detection/discussion/240015",
  "author_name": "",
  "post_date": "2021-05-18T10:07:30.706952500Z",
  "votes": 60,
  "comment_count": 8,
  "views": 0,
  "content": "<p>In this post, I summarize some pipelines used by winners of similar past Kaggle competitions. For each competition, I summarize the processing techniques, models, and postprocessing they used. I also provide a list of <a href=\"https://www.kaggle.com/c/siim-covid19-detection/discussion/239898\" target=\"_blank\">winner solutions of similar past Kaggle competitions</a> in another post. Please check it 😃</p>\n<h2>1. VinBigData Chest X-ray Abnormalities Detection</h2>\n<p><strong>Preprocessing Techniques</strong></p>\n<ul>\n<li>Annotate part of the NIH data and trained a lung localizer for more efficient usage of GPU memory. The drawback is that diseases outside the lung area are guaranteed missing. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770\" target=\"_blank\">6th place</a></li>\n<li>Treat each image-annotation pair as independent data. So both the training and validation data were tripled. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770\" target=\"_blank\">6th place</a></li>\n</ul>\n<p><strong>Models</strong></p>\n<ul>\n<li>Detectron2 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a></li>\n<li>YOLOv5 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>ResNet101 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>ResNet152 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>HourGlass is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>EfficientDet is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229711\" target=\"_blank\">5th place</a> , <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n<li>VFNet is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n</ul>\n<p><strong>Postprocessing Techniques</strong></p>\n<ul>\n<li>Ensemble models using <a href=\"https://github.com/ZFTurbo/Weighted-Boxes-Fusion\" target=\"_blank\">https://github.com/ZFTurbo/Weighted-Boxes-Fusion</a> by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a> <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770\" target=\"_blank\">6th place</a>, <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229629\" target=\"_blank\">10th place</a></li>\n<li>No Penalty For Adding More Bbox. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n<li>Calibrate Confidence Scores. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n</ul>\n<p><strong>Important Things to Note</strong>:</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/morizin\" target=\"_blank\">@morizin</a> analyzes the predictions made for the ensemble both by submitting and also visually inspecting predicted bboxes. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a></li>\n<li>It was very difficult to make reliable Cross-Validation since the test set was generated by different labeling processes and Public LB was not trustworthy due to its small size.</li>\n<li><a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> use different image size for different models. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n</ul>\n<h2>2. RSNA Pneumonia Detection Challenge</h2>\n<p><strong>Preprocessing Techniques</strong></p>\n<ul>\n<li>Scale the original images to 512x512 resolution, with 256 resolution <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> has seen results degradation and using the full resolution was not as practical with heavier base models. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a></li>\n<li>Data Augmentation. <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a>: mild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a>. <a href=\"https://www.kaggle.com/yhirano\" target=\"_blank\">@yhirano</a> : x-flip, 90-degree rotation, zoom-in/out and random contrast changes (let k be a random int between [-10, 10], add k over all pixels) <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650\" target=\"_blank\">6th place</a></li>\n</ul>\n<p><strong>Models</strong></p>\n<ul>\n<li><a href=\"https://github.com/fizyr/keras-retinanet\" target=\"_blank\">RetinaNet</a> is used by <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a>, with <a href=\"https://arxiv.org/abs/1708.02002\" target=\"_blank\">Focal Loss</a> <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632\" target=\"_blank\">3rd place</a></li>\n<li><a href=\"https://github.com/msracver/Deformable-ConvNets\" target=\"_blank\">Deformable R-FCN</a> is used by <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li><a href=\"https://github.com/msracver/Relation-Networks-for-Object-Detection\" target=\"_blank\">Deformable Relation Networks</a>  is used by <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>Semantic Segmentation using UNet-like network architecture <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650\" target=\"_blank\">6th place</a></li>\n</ul>\n<p><strong>Postprocessing Techniques</strong></p>\n<ul>\n<li>Ensemble Boxes using <a href=\"https://github.com/ahrnbom/ensemble-objdet\" target=\"_blank\">https://github.com/ahrnbom/ensemble-objdet</a>. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>Resize the lengths and widths of the final predictions by 87.5% <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n</ul>\n<p><strong>Important Things to Note</strong></p>\n<ul>\n<li>For RetinaNet: other backbones. Only the ResNet backbones worked well for us. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>For RetinaNet: pre-training detector heads. It was a lot easier to use ImageNet pre-trained backbones and then just start training the whole network from the beginning. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>For RetinaNet: pre-training the backbone on the pneumonia dataset. No improvement. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>As classification outputs overfit much faster comparing to anchors position/size regression outputs,  <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> added dropout to anchor and the whole image class outputs. In addition to extra regularization, it helped to achieve the optimal classification and regression results around the same epoch. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a></li>\n<li><a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> added an extra output for smaller anchors (level 2 pyramid layer) to handle smaller boxes and added another classification output predicting the class of the whole image ('No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'). <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> has not used the output but even making the model predict other related functions improved the result. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a></li>\n<li><a href=\"https://www.kaggle.com/pmcheng\" target=\"_blank\">@pmcheng</a> aggressively used non-maximum suppression to eliminate any overlapping bounding boxes from each network’s output for a given image. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632\" target=\"_blank\">3rd place</a></li>\n</ul>\n<h2>3. SIIM-ACR Pneumothorax Segmentation</h2>\n<p><strong>Preprocessing Techniques</strong></p>\n<ul>\n<li>Data Augmentation. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009\" target=\"_blank\">2nd place</a>: hflip, scale, rotate, bright, blur. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a>: ShiftScaleRotate, RandomBrightnessContrast, ElasticTransform, HorizontalFlip from albumentations.</li>\n<li>Image Downsize. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981\" target=\"_blank\">3rd place</a>: 576x576 cropped images.  <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a>: training on random crops with (512, 512) size, inference on (768, 768) size.</li>\n</ul>\n<p><strong>Models</strong></p>\n<ul>\n<li>Resnet34 is used by <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981\" target=\"_blank\">3rd place</a>, <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a></li>\n<li>Resnet50 is used by <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a></li>\n<li>SeResnext50 is used by <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981\" target=\"_blank\">3rd place</a></li>\n<li>Efficientnet-B3, Efficientnet-b5 are used ny <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009\" target=\"_blank\">2nd place</a></li>\n</ul>\n<p><strong>Postprocessing Techniques</strong></p>\n<ul>\n<li><code>if pred_empty &gt; 0.4 or area(pred_mask) &lt; 800: pred_mask = empty</code><br>\nParameters are selected on the validation set. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a></li>\n</ul>\n<p><strong>Important Things to Note</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/sneddy\" target=\"_blank\">@sneddy</a> use <a href=\"https://github.com/SpaceNetChallenge/SpaceNet_Off_Nadir_Solutions/blob/master/selim_sef/training/losses.py\" target=\"_blank\">combo loss</a> → combinations of BCE, dice and focal.  <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a></li>\n</ul>",
  "messages": [
    {
      "id": "1312905",
      "postDate": "05/18/2021 10:07:30",
      "content": "<p>In this post, I summarize some pipelines used by winners of similar past Kaggle competitions. For each competition, I summarize the processing techniques, models, and postprocessing they used. I also provide a list of <a href=\"https://www.kaggle.com/c/siim-covid19-detection/discussion/239898\" target=\"_blank\">winner solutions of similar past Kaggle competitions</a> in another post. Please check it 😃</p>\n<h2>1. VinBigData Chest X-ray Abnormalities Detection</h2>\n<p><strong>Preprocessing Techniques</strong></p>\n<ul>\n<li>Annotate part of the NIH data and trained a lung localizer for more efficient usage of GPU memory. The drawback is that diseases outside the lung area are guaranteed missing. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770\" target=\"_blank\">6th place</a></li>\n<li>Treat each image-annotation pair as independent data. So both the training and validation data were tripled. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770\" target=\"_blank\">6th place</a></li>\n</ul>\n<p><strong>Models</strong></p>\n<ul>\n<li>Detectron2 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a></li>\n<li>YOLOv5 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>ResNet101 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>ResNet152 is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>HourGlass is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696\" target=\"_blank\">2nd place</a></li>\n<li>EfficientDet is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229711\" target=\"_blank\">5th place</a> , <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n<li>VFNet is used by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n</ul>\n<p><strong>Postprocessing Techniques</strong></p>\n<ul>\n<li>Ensemble models using <a href=\"https://github.com/ZFTurbo/Weighted-Boxes-Fusion\" target=\"_blank\">https://github.com/ZFTurbo/Weighted-Boxes-Fusion</a> by <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a> <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770\" target=\"_blank\">6th place</a>, <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229629\" target=\"_blank\">10th place</a></li>\n<li>No Penalty For Adding More Bbox. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n<li>Calibrate Confidence Scores. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n</ul>\n<p><strong>Important Things to Note</strong>:</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/morizin\" target=\"_blank\">@morizin</a> analyzes the predictions made for the ensemble both by submitting and also visually inspecting predicted bboxes. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511\" target=\"_blank\">1st place</a></li>\n<li>It was very difficult to make reliable Cross-Validation since the test set was generated by different labeling processes and Public LB was not trustworthy due to its small size.</li>\n<li><a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> use different image size for different models. <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637\" target=\"_blank\">7th place</a></li>\n</ul>\n<h2>2. RSNA Pneumonia Detection Challenge</h2>\n<p><strong>Preprocessing Techniques</strong></p>\n<ul>\n<li>Scale the original images to 512x512 resolution, with 256 resolution <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> has seen results degradation and using the full resolution was not as practical with heavier base models. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a></li>\n<li>Data Augmentation. <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a>: mild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a>. <a href=\"https://www.kaggle.com/yhirano\" target=\"_blank\">@yhirano</a> : x-flip, 90-degree rotation, zoom-in/out and random contrast changes (let k be a random int between [-10, 10], add k over all pixels) <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650\" target=\"_blank\">6th place</a></li>\n</ul>\n<p><strong>Models</strong></p>\n<ul>\n<li><a href=\"https://github.com/fizyr/keras-retinanet\" target=\"_blank\">RetinaNet</a> is used by <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a>, with <a href=\"https://arxiv.org/abs/1708.02002\" target=\"_blank\">Focal Loss</a> <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632\" target=\"_blank\">3rd place</a></li>\n<li><a href=\"https://github.com/msracver/Deformable-ConvNets\" target=\"_blank\">Deformable R-FCN</a> is used by <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li><a href=\"https://github.com/msracver/Relation-Networks-for-Object-Detection\" target=\"_blank\">Deformable Relation Networks</a>  is used by <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>Semantic Segmentation using UNet-like network architecture <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650\" target=\"_blank\">6th place</a></li>\n</ul>\n<p><strong>Postprocessing Techniques</strong></p>\n<ul>\n<li>Ensemble Boxes using <a href=\"https://github.com/ahrnbom/ensemble-objdet\" target=\"_blank\">https://github.com/ahrnbom/ensemble-objdet</a>. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>Resize the lengths and widths of the final predictions by 87.5% <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n</ul>\n<p><strong>Important Things to Note</strong></p>\n<ul>\n<li>For RetinaNet: other backbones. Only the ResNet backbones worked well for us. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>For RetinaNet: pre-training detector heads. It was a lot easier to use ImageNet pre-trained backbones and then just start training the whole network from the beginning. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>For RetinaNet: pre-training the backbone on the pneumonia dataset. No improvement. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421\" target=\"_blank\">1st place</a></li>\n<li>As classification outputs overfit much faster comparing to anchors position/size regression outputs,  <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> added dropout to anchor and the whole image class outputs. In addition to extra regularization, it helped to achieve the optimal classification and regression results around the same epoch. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a></li>\n<li><a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> added an extra output for smaller anchors (level 2 pyramid layer) to handle smaller boxes and added another classification output predicting the class of the whole image ('No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'). <a href=\"https://www.kaggle.com/dmytropoplavskiy\" target=\"_blank\">@dmytropoplavskiy</a> has not used the output but even making the model predict other related functions improved the result. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427\" target=\"_blank\">2nd place</a></li>\n<li><a href=\"https://www.kaggle.com/pmcheng\" target=\"_blank\">@pmcheng</a> aggressively used non-maximum suppression to eliminate any overlapping bounding boxes from each network’s output for a given image. <a href=\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632\" target=\"_blank\">3rd place</a></li>\n</ul>\n<h2>3. SIIM-ACR Pneumothorax Segmentation</h2>\n<p><strong>Preprocessing Techniques</strong></p>\n<ul>\n<li>Data Augmentation. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009\" target=\"_blank\">2nd place</a>: hflip, scale, rotate, bright, blur. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a>: ShiftScaleRotate, RandomBrightnessContrast, ElasticTransform, HorizontalFlip from albumentations.</li>\n<li>Image Downsize. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981\" target=\"_blank\">3rd place</a>: 576x576 cropped images.  <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a>: training on random crops with (512, 512) size, inference on (768, 768) size.</li>\n</ul>\n<p><strong>Models</strong></p>\n<ul>\n<li>Resnet34 is used by <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981\" target=\"_blank\">3rd place</a>, <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a></li>\n<li>Resnet50 is used by <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a></li>\n<li>SeResnext50 is used by <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a>, <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981\" target=\"_blank\">3rd place</a></li>\n<li>Efficientnet-B3, Efficientnet-b5 are used ny <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009\" target=\"_blank\">2nd place</a></li>\n</ul>\n<p><strong>Postprocessing Techniques</strong></p>\n<ul>\n<li><code>if pred_empty &gt; 0.4 or area(pred_mask) &lt; 800: pred_mask = empty</code><br>\nParameters are selected on the validation set. <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397\" target=\"_blank\">4th place</a></li>\n</ul>\n<p><strong>Important Things to Note</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/sneddy\" target=\"_blank\">@sneddy</a> use <a href=\"https://github.com/SpaceNetChallenge/SpaceNet_Off_Nadir_Solutions/blob/master/selim_sef/training/losses.py\" target=\"_blank\">combo loss</a> → combinations of BCE, dice and focal.  <a href=\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824\" target=\"_blank\">1st place</a></li>\n</ul>",
      "rawMarkdown": "In this post, I summarize some pipelines used by winners of similar past Kaggle competitions. For each competition, I summarize the processing techniques, models, and postprocessing they used. I also provide a list of [winner solutions of similar past Kaggle competitions](https://www.kaggle.com/c/siim-covid19-detection/discussion/239898) in another post. Please check it 😃\n\n## 1. VinBigData Chest X-ray Abnormalities Detection\n\n**Preprocessing Techniques**\n\n- Annotate part of the NIH data and trained a lung localizer for more efficient usage of GPU memory. The drawback is that diseases outside the lung area are guaranteed missing. [6th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770)\n- Treat each image-annotation pair as independent data. So both the training and validation data were tripled. [6th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770)\n\n**Models**\n\n- Detectron2 is used by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511)\n- YOLOv5 is used by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511), [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- ResNet101 is used by [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- ResNet152 is used by [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- HourGlass is used by [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- EfficientDet is used by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511), [5th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229711) , [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n- VFNet is used by [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n\n**Postprocessing Techniques**\n\n- Ensemble models using [https://github.com/ZFTurbo/Weighted-Boxes-Fusion](https://github.com/ZFTurbo/Weighted-Boxes-Fusion) by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511) [6th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770), [10th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229629)\n- No Penalty For Adding More Bbox. [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n- Calibrate Confidence Scores. [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n\n**Important Things to Note**:\n\n- @morizin analyzes the predictions made for the ensemble both by submitting and also visually inspecting predicted bboxes. [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511)\n- It was very difficult to make reliable Cross-Validation since the test set was generated by different labeling processes and Public LB was not trustworthy due to its small size.\n- @cdeotte use different image size for different models. [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n\n## 2. RSNA Pneumonia Detection Challenge\n\n**Preprocessing Techniques**\n\n- Scale the original images to 512x512 resolution, with 256 resolution @dmytropoplavskiy has seen results degradation and using the full resolution was not as practical with heavier base models. [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427)\n- Data Augmentation. @dmytropoplavskiy: mild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427). @yhirano : x-flip, 90-degree rotation, zoom-in/out and random contrast changes (let k be a random int between [-10, 10], add k over all pixels) [6th place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650)\n\n**Models**\n\n- [RetinaNet](https://github.com/fizyr/keras-retinanet) is used by [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421), [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427), with [Focal Loss](https://arxiv.org/abs/1708.02002) [3rd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632)\n- [Deformable R-FCN](https://github.com/msracver/Deformable-ConvNets) is used by [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- [Deformable Relation Networks](https://github.com/msracver/Relation-Networks-for-Object-Detection)  is used by [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- Semantic Segmentation using UNet-like network architecture [6th place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650)\n\n**Postprocessing Techniques**\n\n- Ensemble Boxes using [https://github.com/ahrnbom/ensemble-objdet](https://github.com/ahrnbom/ensemble-objdet). [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- Resize the lengths and widths of the final predictions by 87.5% [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n\n**Important Things to Note**\n\n- For RetinaNet: other backbones. Only the ResNet backbones worked well for us. [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- For RetinaNet: pre-training detector heads. It was a lot easier to use ImageNet pre-trained backbones and then just start training the whole network from the beginning. [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- For RetinaNet: pre-training the backbone on the pneumonia dataset. No improvement. [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- As classification outputs overfit much faster comparing to anchors position/size regression outputs,  @dmytropoplavskiy added dropout to anchor and the whole image class outputs. In addition to extra regularization, it helped to achieve the optimal classification and regression results around the same epoch. [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427)\n- @dmytropoplavskiy added an extra output for smaller anchors (level 2 pyramid layer) to handle smaller boxes and added another classification output predicting the class of the whole image ('No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'). @dmytropoplavskiy has not used the output but even making the model predict other related functions improved the result. [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427)\n- @pmcheng aggressively used non-maximum suppression to eliminate any overlapping bounding boxes from each network’s output for a given image. [3rd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632)\n\n## 3. SIIM-ACR Pneumothorax Segmentation\n\n**Preprocessing Techniques**\n\n- Data Augmentation. [2nd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009): hflip, scale, rotate, bright, blur. [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397): ShiftScaleRotate, RandomBrightnessContrast, ElasticTransform, HorizontalFlip from albumentations.\n- Image Downsize. [3rd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981): 576x576 cropped images.  [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397): training on random crops with (512, 512) size, inference on (768, 768) size.\n\n**Models**\n\n- Resnet34 is used by [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824), [3rd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981), [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397)\n- Resnet50 is used by [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824)\n- SeResnext50 is used by [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824), [3rd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981)\n- Efficientnet-B3, Efficientnet-b5 are used ny [2nd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009)\n\n**Postprocessing Techniques**\n\n- `if pred_empty > 0.4 or area(pred_mask) < 800: pred_mask = empty`\nParameters are selected on the validation set. [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397)\n\n**Important Things to Note**\n\n- @sneddy use [combo loss](https://github.com/SpaceNetChallenge/SpaceNet_Off_Nadir_Solutions/blob/master/selim_sef/training/losses.py) → combinations of BCE, dice and focal.  [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824)",
      "votes": null
    },
    {
      "id": "1313661",
      "postDate": "05/18/2021 16:39:44",
      "content": "<p>All the links in this thread are redirecting it to the competition's main discussion page. Is it for me only? </p>",
      "rawMarkdown": "All the links in this thread are redirecting it to the competition's main discussion page. Is it for me only?",
      "votes": null
    },
    {
      "id": "1313686",
      "postDate": "05/18/2021 16:52:06",
      "content": "<p>Thank you for raising this issue 🙏. I got the problem because I write on Notion first. I'm on my way to solve it 😁</p>",
      "rawMarkdown": "Thank you for raising this issue 🙏. I got the problem because I write on Notion first. I'm on my way to solve it 😁",
      "votes": null
    },
    {
      "id": "1327654",
      "postDate": "05/29/2021 13:59:15",
      "content": "<p>Great summary! </p>",
      "rawMarkdown": "Great summary!",
      "votes": null
    },
    {
      "id": "1330491",
      "postDate": "05/31/2021 21:16:09",
      "content": "<p>thank you very much. It will be very useful :) </p>",
      "rawMarkdown": "thank you very much. It will be very useful :)",
      "votes": null
    },
    {
      "id": "1337693",
      "postDate": "06/05/2021 19:38:51",
      "content": "<p>Thank you for sharing 🙌</p>",
      "rawMarkdown": "Thank you for sharing 🙌",
      "votes": null
    },
    {
      "id": "1343072",
      "postDate": "06/10/2021 02:29:46",
      "content": "<p>Thank you, I hope you get a good score too👍</p>",
      "rawMarkdown": "Thank you, I hope you get a good score too👍",
      "votes": null
    },
    {
      "id": "1355859",
      "postDate": "06/18/2021 15:27:36",
      "content": "<p>This is great summary, thanks for sharing </p>",
      "rawMarkdown": "This is great summary, thanks for sharing",
      "votes": null
    },
    {
      "id": "1362030",
      "postDate": "06/23/2021 07:15:46",
      "content": "<p>Thanks for sharing, this is super useful!</p>",
      "rawMarkdown": "Thanks for sharing, this is super useful!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1313661,
      "author_name": "hassiahk",
      "author_url": "",
      "post_date": "05/18/2021 16:39:44",
      "content": "<p>All the links in this thread are redirecting it to the competition's main discussion page. Is it for me only? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1313686,
          "author_name": "mhilmiasyrofi",
          "author_url": "",
          "post_date": "05/18/2021 16:52:06",
          "content": "<p>Thank you for raising this issue 🙏. I got the problem because I write on Notion first. I'm on my way to solve it 😁</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1327654,
      "author_name": "asmirm",
      "author_url": "",
      "post_date": "05/29/2021 13:59:15",
      "content": "<p>Great summary! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1330491,
      "author_name": "dpaluszk",
      "author_url": "",
      "post_date": "05/31/2021 21:16:09",
      "content": "<p>thank you very much. It will be very useful :) </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1337693,
      "author_name": "ladcva",
      "author_url": "",
      "post_date": "06/05/2021 19:38:51",
      "content": "<p>Thank you for sharing 🙌</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1343072,
      "author_name": "shangweichen",
      "author_url": "",
      "post_date": "06/10/2021 02:29:46",
      "content": "<p>Thank you, I hope you get a good score too👍</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1355859,
      "author_name": "hawkeat",
      "author_url": "",
      "post_date": "06/18/2021 15:27:36",
      "content": "<p>This is great summary, thanks for sharing </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1362030,
      "author_name": "marti164618",
      "author_url": "",
      "post_date": "06/23/2021 07:15:46",
      "content": "<p>Thanks for sharing, this is super useful!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1312905": "In this post, I summarize some pipelines used by winners of similar past Kaggle competitions. For each competition, I summarize the processing techniques, models, and postprocessing they used. I also provide a list of [winner solutions of similar past Kaggle competitions](https://www.kaggle.com/c/siim-covid19-detection/discussion/239898) in another post. Please check it 😃\n\n## 1. VinBigData Chest X-ray Abnormalities Detection\n\n**Preprocessing Techniques**\n\n- Annotate part of the NIH data and trained a lung localizer for more efficient usage of GPU memory. The drawback is that diseases outside the lung area are guaranteed missing. [6th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770)\n- Treat each image-annotation pair as independent data. So both the training and validation data were tripled. [6th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770)\n\n**Models**\n\n- Detectron2 is used by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511)\n- YOLOv5 is used by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511), [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- ResNet101 is used by [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- ResNet152 is used by [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- HourGlass is used by [2nd place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229696)\n- EfficientDet is used by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511), [5th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229711) , [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n- VFNet is used by [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n\n**Postprocessing Techniques**\n\n- Ensemble models using [https://github.com/ZFTurbo/Weighted-Boxes-Fusion](https://github.com/ZFTurbo/Weighted-Boxes-Fusion) by [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511) [6th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229770), [10th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229629)\n- No Penalty For Adding More Bbox. [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n- Calibrate Confidence Scores. [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n\n**Important Things to Note**:\n\n- @morizin analyzes the predictions made for the ensemble both by submitting and also visually inspecting predicted bboxes. [1st place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511)\n- It was very difficult to make reliable Cross-Validation since the test set was generated by different labeling processes and Public LB was not trustworthy due to its small size.\n- @cdeotte use different image size for different models. [7th place](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637)\n\n## 2. RSNA Pneumonia Detection Challenge\n\n**Preprocessing Techniques**\n\n- Scale the original images to 512x512 resolution, with 256 resolution @dmytropoplavskiy has seen results degradation and using the full resolution was not as practical with heavier base models. [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427)\n- Data Augmentation. @dmytropoplavskiy: mild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427). @yhirano : x-flip, 90-degree rotation, zoom-in/out and random contrast changes (let k be a random int between [-10, 10], add k over all pixels) [6th place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650)\n\n**Models**\n\n- [RetinaNet](https://github.com/fizyr/keras-retinanet) is used by [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421), [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427), with [Focal Loss](https://arxiv.org/abs/1708.02002) [3rd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632)\n- [Deformable R-FCN](https://github.com/msracver/Deformable-ConvNets) is used by [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- [Deformable Relation Networks](https://github.com/msracver/Relation-Networks-for-Object-Detection)  is used by [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- Semantic Segmentation using UNet-like network architecture [6th place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70650)\n\n**Postprocessing Techniques**\n\n- Ensemble Boxes using [https://github.com/ahrnbom/ensemble-objdet](https://github.com/ahrnbom/ensemble-objdet). [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- Resize the lengths and widths of the final predictions by 87.5% [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n\n**Important Things to Note**\n\n- For RetinaNet: other backbones. Only the ResNet backbones worked well for us. [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- For RetinaNet: pre-training detector heads. It was a lot easier to use ImageNet pre-trained backbones and then just start training the whole network from the beginning. [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- For RetinaNet: pre-training the backbone on the pneumonia dataset. No improvement. [1st place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70421)\n- As classification outputs overfit much faster comparing to anchors position/size regression outputs,  @dmytropoplavskiy added dropout to anchor and the whole image class outputs. In addition to extra regularization, it helped to achieve the optimal classification and regression results around the same epoch. [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427)\n- @dmytropoplavskiy added an extra output for smaller anchors (level 2 pyramid layer) to handle smaller boxes and added another classification output predicting the class of the whole image ('No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'). @dmytropoplavskiy has not used the output but even making the model predict other related functions improved the result. [2nd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70427)\n- @pmcheng aggressively used non-maximum suppression to eliminate any overlapping bounding boxes from each network’s output for a given image. [3rd place](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70632)\n\n## 3. SIIM-ACR Pneumothorax Segmentation\n\n**Preprocessing Techniques**\n\n- Data Augmentation. [2nd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009): hflip, scale, rotate, bright, blur. [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397): ShiftScaleRotate, RandomBrightnessContrast, ElasticTransform, HorizontalFlip from albumentations.\n- Image Downsize. [3rd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981): 576x576 cropped images.  [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397): training on random crops with (512, 512) size, inference on (768, 768) size.\n\n**Models**\n\n- Resnet34 is used by [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824), [3rd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981), [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397)\n- Resnet50 is used by [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824)\n- SeResnext50 is used by [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824), [3rd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981)\n- Efficientnet-B3, Efficientnet-b5 are used ny [2nd place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108009)\n\n**Postprocessing Techniques**\n\n- `if pred_empty > 0.4 or area(pred_mask) < 800: pred_mask = empty`\nParameters are selected on the validation set. [4th place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/108397)\n\n**Important Things to Note**\n\n- @sneddy use [combo loss](https://github.com/SpaceNetChallenge/SpaceNet_Off_Nadir_Solutions/blob/master/selim_sef/training/losses.py) → combinations of BCE, dice and focal.  [1st place](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107824)",
    "1313661": "All the links in this thread are redirecting it to the competition's main discussion page. Is it for me only?",
    "1313686": "Thank you for raising this issue 🙏. I got the problem because I write on Notion first. I'm on my way to solve it 😁",
    "1327654": "Great summary!",
    "1330491": "thank you very much. It will be very useful :)",
    "1337693": "Thank you for sharing 🙌",
    "1343072": "Thank you, I hope you get a good score too👍",
    "1355859": "This is great summary, thanks for sharing",
    "1362030": "Thanks for sharing, this is super useful!"
  },
  "source": "meta"
}