{
  "id": 238046,
  "title": "54th Place Solution Pytorch Single_5fold_UNet50 private_lb:0.946",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/238046",
  "author_name": "Pratik",
  "post_date": "2021-05-11T04:03:57.944000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Model: UNet aspp with Resnext50 backbone</p>\n<p>Keys:</p>\n<ul>\n<li><strong>aspp_stride</strong>: 1024x1024 tiles rescaled to 480x480 to feed the model. With 480x480 model input, backbone output feature map is 15x15, so set aspp_stride to 2 to get dilations of 2,4,6,8 which gives the desired coverage of the 15x15 output feature map of the backbone.</li>\n<li><strong>batch_size</strong>: Chose 480x480 instead of say 512x512 for model input, so as to be able to train with a batch size of at least 32 on a P100 GPU in mixed precision, as the model has batchNorm layers. </li>\n<li><strong>edge_ignore</strong>: During inference, when making predictions for a tile, ignore the predictions at tile edges. Let overlap take care of ignored tile edges.</li>\n<li>Tuned the number of training epochs on the validation holdout.</li>\n<li>Don't overfit to the public test set. </li>\n</ul>\n<p>I went through and built on <a href=\"https://www.kaggle.com/iafoss\" target=\"_blank\">@iafoss</a> public notebook: <a href=\"https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter\" target=\"_blank\">https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter</a></p>\n<p>Generating the training dataset: <br>\n<a href=\"https://www.kaggle.com/pratikkumar/hubmapdataset-sz480-window1024\" target=\"_blank\">https://www.kaggle.com/pratikkumar/hubmapdataset-sz480-window1024</a><br>\nTile Size of 1024x1024 rescaled to 480x480 with 20% overlap.</p>\n<p>Training:<br>\n<a href=\"https://colab.research.google.com/drive/1u9NXkjRbgI9_3Mwc1FGRLciyXcMFo3Rv?usp=sharing\" target=\"_blank\">https://colab.research.google.com/drive/1u9NXkjRbgI9_3Mwc1FGRLciyXcMFo3Rv?usp=sharing</a><br>\nModel: Pytorch/Fastai UNet50 (from <a href=\"https://www.kaggle.com/iafoss\" target=\"_blank\">@iafoss</a>) with aspp_stride of 2.<br>\nData Augmentation: </p>\n<blockquote>\n  <p>HorizontalFlip(),<br>\n          VerticalFlip(),<br>\n          RandomRotate90(),<br>\n          ShiftScaleRotate(p=0.5, rotate_limit=15),<br>\n          OneOf([<br>\n              OpticalDistortion(),<br>\n              GridDistortion(),<br>\n              IAAPiecewiseAffine(scale=(0.02, 0.03)),<br>\n          ], p=0.2),<br>\n          OneOf([<br>\n              HueSaturationValue(10,15,10),<br>\n              RandomBrightnessContrast(0.1,0.1),<br>\n          ], p=0.5),</p>\n</blockquote>\n<p>Reduced the intensity of some augmentations and removed clahe.<br>\n5 fold training.</p>\n<p>Inference:<br>\n<a href=\"https://www.kaggle.com/pratikkumar/hubmap-win1024-sz480-inference-batched-subds\" target=\"_blank\">https://www.kaggle.com/pratikkumar/hubmap-win1024-sz480-inference-batched-subds</a><br>\nOverlap of 30% and Edge Ignore of slightly less than 15%.</p>",
  "messages": [
    {
      "id": 1301391,
      "postDate": "2021-05-11T04:03:57.943Z",
      "content": "<p>Model: UNet aspp with Resnext50 backbone</p>\n<p>Keys:</p>\n<ul>\n<li><strong>aspp_stride</strong>: 1024x1024 tiles rescaled to 480x480 to feed the model. With 480x480 model input, backbone output feature map is 15x15, so set aspp_stride to 2 to get dilations of 2,4,6,8 which gives the desired coverage of the 15x15 output feature map of the backbone.</li>\n<li><strong>batch_size</strong>: Chose 480x480 instead of say 512x512 for model input, so as to be able to train with a batch size of at least 32 on a P100 GPU in mixed precision, as the model has batchNorm layers. </li>\n<li><strong>edge_ignore</strong>: During inference, when making predictions for a tile, ignore the predictions at tile edges. Let overlap take care of ignored tile edges.</li>\n<li>Tuned the number of training epochs on the validation holdout.</li>\n<li>Don't overfit to the public test set. </li>\n</ul>\n<p>I went through and built on <a href=\"https://www.kaggle.com/iafoss\" target=\"_blank\">@iafoss</a> public notebook: <a href=\"https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter\" target=\"_blank\">https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter</a></p>\n<p>Generating the training dataset: <br>\n<a href=\"https://www.kaggle.com/pratikkumar/hubmapdataset-sz480-window1024\" target=\"_blank\">https://www.kaggle.com/pratikkumar/hubmapdataset-sz480-window1024</a><br>\nTile Size of 1024x1024 rescaled to 480x480 with 20% overlap.</p>\n<p>Training:<br>\n<a href=\"https://colab.research.google.com/drive/1u9NXkjRbgI9_3Mwc1FGRLciyXcMFo3Rv?usp=sharing\" target=\"_blank\">https://colab.research.google.com/drive/1u9NXkjRbgI9_3Mwc1FGRLciyXcMFo3Rv?usp=sharing</a><br>\nModel: Pytorch/Fastai UNet50 (from <a href=\"https://www.kaggle.com/iafoss\" target=\"_blank\">@iafoss</a>) with aspp_stride of 2.<br>\nData Augmentation: </p>\n<blockquote>\n  <p>HorizontalFlip(),<br>\n          VerticalFlip(),<br>\n          RandomRotate90(),<br>\n          ShiftScaleRotate(p=0.5, rotate_limit=15),<br>\n          OneOf([<br>\n              OpticalDistortion(),<br>\n              GridDistortion(),<br>\n              IAAPiecewiseAffine(scale=(0.02, 0.03)),<br>\n          ], p=0.2),<br>\n          OneOf([<br>\n              HueSaturationValue(10,15,10),<br>\n              RandomBrightnessContrast(0.1,0.1),<br>\n          ], p=0.5),</p>\n</blockquote>\n<p>Reduced the intensity of some augmentations and removed clahe.<br>\n5 fold training.</p>\n<p>Inference:<br>\n<a href=\"https://www.kaggle.com/pratikkumar/hubmap-win1024-sz480-inference-batched-subds\" target=\"_blank\">https://www.kaggle.com/pratikkumar/hubmap-win1024-sz480-inference-batched-subds</a><br>\nOverlap of 30% and Edge Ignore of slightly less than 15%.</p>",
      "rawMarkdown": "Model: UNet aspp with Resnext50 backbone\n\nKeys:\n- **aspp_stride**: 1024x1024 tiles rescaled to 480x480 to feed the model. With 480x480 model input, backbone output feature map is 15x15, so set aspp_stride to 2 to get dilations of 2,4,6,8 which gives the desired coverage of the 15x15 output feature map of the backbone.\n- **batch_size**: Chose 480x480 instead of say 512x512 for model input, so as to be able to train with a batch size of at least 32 on a P100 GPU in mixed precision, as the model has batchNorm layers. \n- **edge_ignore**: During inference, when making predictions for a tile, ignore the predictions at tile edges. Let overlap take care of ignored tile edges.\n- Tuned the number of training epochs on the validation holdout.\n- Don't overfit to the public test set. \n\nI went through and built on @iafoss public notebook: https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter\n\nGenerating the training dataset: \nhttps://www.kaggle.com/pratikkumar/hubmapdataset-sz480-window1024\nTile Size of 1024x1024 rescaled to 480x480 with 20% overlap.\n\nTraining:\nhttps://colab.research.google.com/drive/1u9NXkjRbgI9_3Mwc1FGRLciyXcMFo3Rv?usp=sharing\nModel: Pytorch/Fastai UNet50 (from @iafoss) with aspp_stride of 2.\nData Augmentation: \n> \n        HorizontalFlip(),\n        VerticalFlip(),\n        RandomRotate90(),\n        ShiftScaleRotate(p=0.5, rotate_limit=15),\n        OneOf([\n            OpticalDistortion(),\n            GridDistortion(),\n            IAAPiecewiseAffine(scale=(0.02, 0.03)),\n        ], p=0.2),\n        OneOf([\n            HueSaturationValue(10,15,10),\n            RandomBrightnessContrast(0.1,0.1),\n        ], p=0.5),\n\nReduced the intensity of some augmentations and removed clahe.\n5 fold training.\n\nInference:\nhttps://www.kaggle.com/pratikkumar/hubmap-win1024-sz480-inference-batched-subds\nOverlap of 30% and Edge Ignore of slightly less than 15%.",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1301391": "Model: UNet aspp with Resnext50 backbone\n\nKeys:\n- **aspp_stride**: 1024x1024 tiles rescaled to 480x480 to feed the model. With 480x480 model input, backbone output feature map is 15x15, so set aspp_stride to 2 to get dilations of 2,4,6,8 which gives the desired coverage of the 15x15 output feature map of the backbone.\n- **batch_size**: Chose 480x480 instead of say 512x512 for model input, so as to be able to train with a batch size of at least 32 on a P100 GPU in mixed precision, as the model has batchNorm layers. \n- **edge_ignore**: During inference, when making predictions for a tile, ignore the predictions at tile edges. Let overlap take care of ignored tile edges.\n- Tuned the number of training epochs on the validation holdout.\n- Don't overfit to the public test set. \n\nI went through and built on @iafoss public notebook: https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter\n\nGenerating the training dataset: \nhttps://www.kaggle.com/pratikkumar/hubmapdataset-sz480-window1024\nTile Size of 1024x1024 rescaled to 480x480 with 20% overlap.\n\nTraining:\nhttps://colab.research.google.com/drive/1u9NXkjRbgI9_3Mwc1FGRLciyXcMFo3Rv?usp=sharing\nModel: Pytorch/Fastai UNet50 (from @iafoss) with aspp_stride of 2.\nData Augmentation: \n> \n        HorizontalFlip(),\n        VerticalFlip(),\n        RandomRotate90(),\n        ShiftScaleRotate(p=0.5, rotate_limit=15),\n        OneOf([\n            OpticalDistortion(),\n            GridDistortion(),\n            IAAPiecewiseAffine(scale=(0.02, 0.03)),\n        ], p=0.2),\n        OneOf([\n            HueSaturationValue(10,15,10),\n            RandomBrightnessContrast(0.1,0.1),\n        ], p=0.5),\n\nReduced the intensity of some augmentations and removed clahe.\n5 fold training.\n\nInference:\nhttps://www.kaggle.com/pratikkumar/hubmap-win1024-sz480-inference-batched-subds\nOverlap of 30% and Edge Ignore of slightly less than 15%."
  }
}