{
  "id": 226731,
  "title": "12th on Public, 28th on Private Solution",
  "url": "/competitions/ranzcr-clip-catheter-line-classification/discussion/226731",
  "author_name": "Oleg Panichev",
  "post_date": "2021-03-17T14:45:58.626000",
  "votes": 14,
  "comment_count": 0,
  "views": 0,
  "content": "<h1>1. Image pre-processing:</h1>\n<p>All our classification models in the final solution were trained in 600x600 resolution and inference in 640x640 resolution. <br>\nAll images were cropped based on black limits. We also discovered that some images are inverted, so we applied reverse inversion. <br>\nAugmentations: random resized crop, horizontal flip, random gamma, brightness, shift, scale, rotate, contrast, different blurs, elastic/optical/grid distortions, cutout.</p>\n<h1>2. Model design:</h1>\n<p>As input for most of our models we used 4 channel input: <br>\n1-channel grayscale ct-image<br>\n3-channel mask, where ETT, CVC, NGT, SwanGanz were encoded as (0, 0, 255), (0, 255, 0),  (255, 0, 0) and (255, 255, 255) respectively. To get such masks we trained a simple segmentation network Unet with EffNet-b0 backbone.</p>\n<p>Best performing classification backbones: EfficientNet-b6, EfficientNet-b7, ResNet200d, SeResNet152d</p>\n<h1>3. Training:</h1>\n<p>Group KFold 5-fold CV<br>\nBatch size: 14 with 2 batch gradient accumulation (on 4 GPUs)<br>\nAdam for 30 epochs.<br>\nLR schedule: 0.001 SGDR<br>\nThe top 5 checkpoints were used to apply SWA</p>\n<h1>4. Loss:</h1>\n<p>BCE or <a href=\"https://github.com/BloodAxe/Kaggle-2020-Alaska2/blob/3c1f5e8e564c9f04423beef69244fc74168f88ca/alaska2/loss.py#L305\" target=\"_blank\">ROCAUCLoss</a>+BCE</p>\n<h1>5. Ensembling all together:</h1>\n<p>The simple mean of the predictions appeared to work the best. We’ve tried different techniques to merge predictions but didn’t succeed.  <br>\nThe single model trained in this way (5 fold) without any post-processing gets <strong>0.971</strong> Public LB and <strong>0.970</strong> Private LB. </p>\n<p><strong>Framework:</strong> PyTorch</p>\n<h1>What didn't work:</h1>\n<ul>\n<li>Chest14</li>\n<li>Pseudo Labels</li>\n</ul>",
  "messages": [
    {
      "id": 1242327,
      "postDate": "2021-03-17T14:45:58.627Z",
      "content": "<h1>1. Image pre-processing:</h1>\n<p>All our classification models in the final solution were trained in 600x600 resolution and inference in 640x640 resolution. <br>\nAll images were cropped based on black limits. We also discovered that some images are inverted, so we applied reverse inversion. <br>\nAugmentations: random resized crop, horizontal flip, random gamma, brightness, shift, scale, rotate, contrast, different blurs, elastic/optical/grid distortions, cutout.</p>\n<h1>2. Model design:</h1>\n<p>As input for most of our models we used 4 channel input: <br>\n1-channel grayscale ct-image<br>\n3-channel mask, where ETT, CVC, NGT, SwanGanz were encoded as (0, 0, 255), (0, 255, 0),  (255, 0, 0) and (255, 255, 255) respectively. To get such masks we trained a simple segmentation network Unet with EffNet-b0 backbone.</p>\n<p>Best performing classification backbones: EfficientNet-b6, EfficientNet-b7, ResNet200d, SeResNet152d</p>\n<h1>3. Training:</h1>\n<p>Group KFold 5-fold CV<br>\nBatch size: 14 with 2 batch gradient accumulation (on 4 GPUs)<br>\nAdam for 30 epochs.<br>\nLR schedule: 0.001 SGDR<br>\nThe top 5 checkpoints were used to apply SWA</p>\n<h1>4. Loss:</h1>\n<p>BCE or <a href=\"https://github.com/BloodAxe/Kaggle-2020-Alaska2/blob/3c1f5e8e564c9f04423beef69244fc74168f88ca/alaska2/loss.py#L305\" target=\"_blank\">ROCAUCLoss</a>+BCE</p>\n<h1>5. Ensembling all together:</h1>\n<p>The simple mean of the predictions appeared to work the best. We’ve tried different techniques to merge predictions but didn’t succeed.  <br>\nThe single model trained in this way (5 fold) without any post-processing gets <strong>0.971</strong> Public LB and <strong>0.970</strong> Private LB. </p>\n<p><strong>Framework:</strong> PyTorch</p>\n<h1>What didn't work:</h1>\n<ul>\n<li>Chest14</li>\n<li>Pseudo Labels</li>\n</ul>",
      "rawMarkdown": "# 1. Image pre-processing:\n\nAll our classification models in the final solution were trained in 600x600 resolution and inference in 640x640 resolution. \nAll images were cropped based on black limits. We also discovered that some images are inverted, so we applied reverse inversion. \nAugmentations: random resized crop, horizontal flip, random gamma, brightness, shift, scale, rotate, contrast, different blurs, elastic/optical/grid distortions, cutout.\n\n# 2. Model design:\n\nAs input for most of our models we used 4 channel input: \n1-channel grayscale ct-image\n3-channel mask, where ETT, CVC, NGT, SwanGanz were encoded as (0, 0, 255), (0, 255, 0),  (255, 0, 0) and (255, 255, 255) respectively. To get such masks we trained a simple segmentation network Unet with EffNet-b0 backbone.\n\nBest performing classification backbones: EfficientNet-b6, EfficientNet-b7, ResNet200d, SeResNet152d\n\n# 3. Training:\n\nGroup KFold 5-fold CV\nBatch size: 14 with 2 batch gradient accumulation (on 4 GPUs)\nAdam for 30 epochs.\nLR schedule: 0.001 SGDR\nThe top 5 checkpoints were used to apply SWA\n\n# 4. Loss:\n\nBCE or [ROCAUCLoss](https://github.com/BloodAxe/Kaggle-2020-Alaska2/blob/3c1f5e8e564c9f04423beef69244fc74168f88ca/alaska2/loss.py#L305)+BCE\n\n# 5. Ensembling all together:\n\nThe simple mean of the predictions appeared to work the best. We’ve tried different techniques to merge predictions but didn’t succeed.  \nThe single model trained in this way (5 fold) without any post-processing gets **0.971** Public LB and **0.970** Private LB. \n\n**Framework:** PyTorch\n\n# What didn't work:\n\n- Chest14\n- Pseudo Labels\n",
      "votes": 14
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1242327": "# 1. Image pre-processing:\n\nAll our classification models in the final solution were trained in 600x600 resolution and inference in 640x640 resolution. \nAll images were cropped based on black limits. We also discovered that some images are inverted, so we applied reverse inversion. \nAugmentations: random resized crop, horizontal flip, random gamma, brightness, shift, scale, rotate, contrast, different blurs, elastic/optical/grid distortions, cutout.\n\n# 2. Model design:\n\nAs input for most of our models we used 4 channel input: \n1-channel grayscale ct-image\n3-channel mask, where ETT, CVC, NGT, SwanGanz were encoded as (0, 0, 255), (0, 255, 0),  (255, 0, 0) and (255, 255, 255) respectively. To get such masks we trained a simple segmentation network Unet with EffNet-b0 backbone.\n\nBest performing classification backbones: EfficientNet-b6, EfficientNet-b7, ResNet200d, SeResNet152d\n\n# 3. Training:\n\nGroup KFold 5-fold CV\nBatch size: 14 with 2 batch gradient accumulation (on 4 GPUs)\nAdam for 30 epochs.\nLR schedule: 0.001 SGDR\nThe top 5 checkpoints were used to apply SWA\n\n# 4. Loss:\n\nBCE or [ROCAUCLoss](https://github.com/BloodAxe/Kaggle-2020-Alaska2/blob/3c1f5e8e564c9f04423beef69244fc74168f88ca/alaska2/loss.py#L305)+BCE\n\n# 5. Ensembling all together:\n\nThe simple mean of the predictions appeared to work the best. We’ve tried different techniques to merge predictions but didn’t succeed.  \nThe single model trained in this way (5 fold) without any post-processing gets **0.971** Public LB and **0.970** Private LB. \n\n**Framework:** PyTorch\n\n# What didn't work:\n\n- Chest14\n- Pseudo Labels\n"
  }
}