Thank you kaggle and the host team for such a interesting competition and thanks for everyone to share their brilliant mind! So happy to win my first medal in my first medical segmentation competition, and i have learned a lot from this.
\nMy notebook is based on hubmap-tf-with-tpu-efficientunet-512x512-train, hubmap-tf-with-tpu-efficientunet-512x512-subm and hubmap-tf-with-tpu-efficientunet-512x512-tfrecs, and besides I've made several improvements and explorations by myself. (thank you for sharing these excellent notebooks @wrrosa)
\n Here is a brief description about my solution:
Split origin tiff images with window size of 1024x1024 and overlap size of 512, then resize to 320x320 images.
\nSplit dataset to 5-fold and make every fold have nearly the same sum of training examples.
\nUse d48-hand-labelled as external data (thanks @carnozhao!)
Note:
\nI've also tried other combinations of window size and overlap size, such as 1024-256, 1536-512 and 1560-320, but haven't get better CV and LB.
\nI haven't tried 1024-512, because training time is so long for me.
\nEqual k-fold split give slightly better CV and public LB than random split.
\nUsing d48-hand-labelled helps get better public LB on both d488c759a and other public test images (about 0.001).
\nI've also tried to add @baesiann's external dataset, but achieved low public LB.
U-Net with EfficientB3 encoder.
\nNote:
\nI've also tried LinkNet as segmentation framework and other encoders (such as EfficientB0~B2, Resnet50 and Swin-Transformer), but didn't get better CV and public LB.
cross entropy and finetune with lovasz softmax.
\nFor evaluation, compute global dice score rather than mean dice score for every tile.
\nNote:
\nFinetune with lovasz softmax gives 0.003+ inprovement in public LB.
\nI've also tried other loss functions, such as tversky loss, focal tversky loss and bce jaccard loss, but didn't get better CV and public LB.
\nWith global dice score, better CV often implies better public LB.
Adam+SAM (Sharpness-Aware Minimization)
\nNote:
\nFor me, SAM give more stable CV and public LB.
\nI've also tried Ranger as optimizer, but get better CV and worse public LB.
Shuffle train files in every fold between every epoch.
\nNote:
\nTFRecordDataset have a fixed order of input files, I believe that a more adequate shuffle in training process will benefits.
Just split test images as train dataset does. Center crop 1/2 of every test tile's predict mask and tie every tile mask together as the final mask.
\nNote:
\nCenter crop Inference is the key idea to achieve high LB score. It gives about 0.005+ improvement.
\nI have tried to crop 1/4 of predict mask, but got almost the same public LB, although has higher private LB (0.948)
Because model ensemble doesn't give better CV and LB score, I mostly use single model to submit.
\n1) inference notebook
\n2) I have modified segmentation_models.pytorch and Swin-Transformerto support Swin-Transformer encoder, and here is the modification code: (hope it helps in other competitions)
\nhttps://github.com/cersar/segmentation_models.pytorch
\nhttps://github.com/cersar/Swin-Transformer
Thank you kaggle and the host team for such a interesting competition and thanks for everyone to share their brilliant mind! So happy to win my first medal in my first medical segmentation competition, and i have learned a lot from this.
\nMy notebook is based on hubmap-tf-with-tpu-efficientunet-512x512-train, hubmap-tf-with-tpu-efficientunet-512x512-subm and hubmap-tf-with-tpu-efficientunet-512x512-tfrecs, and besides I've made several improvements and explorations by myself. (thank you for sharing these excellent notebooks @wrrosa)
\n Here is a brief description about my solution:
Split origin tiff images with window size of 1024x1024 and overlap size of 512, then resize to 320x320 images.
\nSplit dataset to 5-fold and make every fold have nearly the same sum of training examples.
\nUse d48-hand-labelled as external data (thanks @carnozhao!)
Note:
\nI've also tried other combinations of window size and overlap size, such as 1024-256, 1536-512 and 1560-320, but haven't get better CV and LB.
\nI haven't tried 1024-512, because training time is so long for me.
\nEqual k-fold split give slightly better CV and public LB than random split.
\nUsing d48-hand-labelled helps get better public LB on both d488c759a and other public test images (about 0.001).
\nI've also tried to add @baesiann's external dataset, but achieved low public LB.
U-Net with EfficientB3 encoder.
\nNote:
\nI've also tried LinkNet as segmentation framework and other encoders (such as EfficientB0~B2, Resnet50 and Swin-Transformer), but didn't get better CV and public LB.
cross entropy and finetune with lovasz softmax.
\nFor evaluation, compute global dice score rather than mean dice score for every tile.
\nNote:
\nFinetune with lovasz softmax gives 0.003+ inprovement in public LB.
\nI've also tried other loss functions, such as tversky loss, focal tversky loss and bce jaccard loss, but didn't get better CV and public LB.
\nWith global dice score, better CV often implies better public LB.
Adam+SAM (Sharpness-Aware Minimization)
\nNote:
\nFor me, SAM give more stable CV and public LB.
\nI've also tried Ranger as optimizer, but get better CV and worse public LB.
Shuffle train files in every fold between every epoch.
\nNote:
\nTFRecordDataset have a fixed order of input files, I believe that a more adequate shuffle in training process will benefits.
Just split test images as train dataset does. Center crop 1/2 of every test tile's predict mask and tie every tile mask together as the final mask.
\nNote:
\nCenter crop Inference is the key idea to achieve high LB score. It gives about 0.005+ improvement.
\nI have tried to crop 1/4 of predict mask, but got almost the same public LB, although has higher private LB (0.948)
Because model ensemble doesn't give better CV and LB score, I mostly use single model to submit.
\n1) inference notebook
\n2) I have modified segmentation_models.pytorch and Swin-Transformerto support Swin-Transformer encoder, and here is the modification code: (hope it helps in other competitions)
\nhttps://github.com/cersar/segmentation_models.pytorch
\nhttps://github.com/cersar/Swin-Transformer
Good job, and congratulation!
", "rawMarkdown": "Good job, and congratulation!", "votes": null }, { "id": "1305829", "postDate": "05/13/2021 13:50:44", "content": "@snowballball Congratulations and Thanks for sharing the approach
", "rawMarkdown": "snowballball Congratulations and Thanks for sharing the approach", "votes": null } ], "comments": [ { "id": 1304576, "author_name": "bessenyeiszilrd", "author_url": "", "post_date": "05/12/2021 17:49:49", "content": "Good job, and congratulation!
", "votes": null, "replies": [] }, { "id": 1305829, "author_name": "usharengaraju", "author_url": "", "post_date": "05/13/2021 13:50:44", "content": "@snowballball Congratulations and Thanks for sharing the approach
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1303562": "Thank you kaggle and the host team for such a interesting competition and thanks for everyone to share their brilliant mind! So happy to win my first medal in my first medical segmentation competition, and i have learned a lot from this.\nMy notebook is based on [hubmap-tf-with-tpu-efficientunet-512x512-train](https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train), [hubmap-tf-with-tpu-efficientunet-512x512-subm](https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-subm) and [hubmap-tf-with-tpu-efficientunet-512x512-tfrecs](https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-tfrecs), and besides I've made several improvements and explorations by myself. (thank you for sharing these excellent notebooks @wrrosa)\n Here is a brief description about my solution:\n# Dataset\nSplit origin tiff images with window size of 1024x1024 and overlap size of 512, then resize to 320x320 images.\nSplit dataset to 5-fold and make every fold have nearly the same sum of training examples.\nUse [d48-hand-labelled](https://www.kaggle.com/carnozhao/d48-hand-labelled) as external data (thanks [@carnozhao](https://www.kaggle.com/carnozhao)!)\n\n**Note:**\nI've also tried other combinations of window size and overlap size, such as 1024-256, 1536-512 and 1560-320, but haven't get better CV and LB.\nI haven't tried 1024-512, because training time is so long for me.\nEqual k-fold split give slightly better CV and public LB than random split.\nUsing [d48-hand-labelled](https://www.kaggle.com/carnozhao/d48-hand-labelled) helps get better public LB on both d488c759a and other public test images (about 0.001).\nI've also tried to add @baesiann's [external dataset](https://www.kaggle.com/baesiann/glomeruli-hubmap-external-1024x1024), but achieved low public LB.\n# Model\nU-Net with EfficientB3 encoder. \n**Note:**\nI've also tried LinkNet as segmentation framework and other encoders (such as EfficientB0~B2, Resnet50 and Swin-Transformer), but didn't get better CV and public LB.\n# Loss\ncross entropy and finetune with lovasz softmax.\nFor evaluation, compute global dice score rather than mean dice score for every tile.\n**Note:**\nFinetune with lovasz softmax gives 0.003+ inprovement in public LB.\nI've also tried other loss functions, such as tversky loss, focal tversky loss and bce jaccard loss, but didn't get better CV and public LB.\nWith global dice score, better CV often implies better public LB.\n# optimizer\nAdam+SAM (Sharpness-Aware Minimization)\n**Note:**\nFor me, SAM give more stable CV and public LB.\nI've also tried Ranger as optimizer, but get better CV and worse public LB.\n# Shuffle Training\nShuffle train files in every fold between every epoch.\n**Note:**\nTFRecordDataset have a fixed order of input files, I believe that a more adequate shuffle in training process will benefits.\n# Center Crop Inference\nJust split test images as train dataset does. Center crop 1/2 of every test tile's predict mask and tie every tile mask together as the final mask.\n**Note:**\nCenter crop Inference is the key idea to achieve high LB score. It gives about 0.005+ improvement.\nI have tried to crop 1/4 of predict mask, but got almost the same public LB, although has higher private LB (0.948)\n# Ensemble\nBecause model ensemble doesn't give better CV and LB score, I mostly use single model to submit.\n# Code\n1) [inference notebook](https://www.kaggle.com/snowballball/hubmap-submit1)\n2) I have modified [segmentation_models.pytorch](https://github.com/qubvel/segmentation_models.pytorch) and [Swin-Transformer](https://github.com/microsoft/Swin-Transformer)to support Swin-Transformer encoder, and here is the modification code: (hope it helps in other competitions)\nhttps://github.com/cersar/segmentation_models.pytorch\nhttps://github.com/cersar/Swin-Transformer", "1304576": "Good job, and congratulation!", "1305829": "snowballball Congratulations and Thanks for sharing the approach" }, "source": "meta" }