{ "id": 356201, "title": "1st place solution", "url": "/competitions/hubmap-organ-segmentation/discussion/356201", "author_name": "opusen", "post_date": "2022-09-29T16:07:17.843000", "votes": 48, "comment_count": 15, "views": 0, "content": "

First of all, I’d like to express my gratitude to kaggle and the competition hosts for holding such an amazing competition.
\nI’d like to thank MMSegmentation developers too. MMSegmentation is really nice and useful tool. Also great thanks to @ayuraj who incorporated a hook of Weight & Biases into MMSegmentation, which helped a lot to manage training history.

\n

In the contest, I learned a lot from @hengck23. Thank you so much for sharing knowledge with us:)

\n

Overview

\n

I used the following methods.
\n・ End-to-end MMSegmentation(with customization)
\n・ Ensemble of SegFormers
\n・ Used group normalization and trained with batch_size=1 in Colab
\n・ Hand-labeling of lung from scratch
\n・ External dataset and pseudo labeling.

\n

DataSet

\n

Used dataset:
\n・ HPA images [351 images]
\n・ HPA images stained with H&E or PAS [351 images x 4 stain patterns]
\n・ external spleen [39 images + 39 images x 4 stain patterns] *from a single image
\n・ external lung [73 images] *from a single image

\n

(Lung)When I noticed that unstable results were from lung prediction, I imagined hand labeling is better than predicting lung by low threshold, which might lead to unstable and unconfident results. So I researched how lung alveolus/alveoli look alike and hand-labeled lung images from scratch. Since there is no information on how to annotate alveolus/alveoli anywhere, I made about 7 annotation sets where potential alveolus was gradually annotated and probed which one is the best. Then I created an external dataset with pseudo labels and with the best threshold. After training with it, the best threshold for lung increased to 0.60, which gave the best result ever. I guess good modeling is better than hand labeling since there are those who got 0.11/0.22 in public/private.
\n(Spleen)Used pseudo label for an external dataset
\n(Prostate/ Largeintestine)Used pseudo label for the HPA dataset.
\nI had a feeling that my models underfit HPA dataset because these dataset didn’t put weight on HPA datasets in terms of colors. The torchstain was used for the stain tool.

\n

Models

\n

Ensemble of the following models:
\n・ 1x SegFormer mit-b3, image_size: 1024x1024
\n・ 2x SegFormer mit-b4, image_size: 960x960
\n・ 1x SegFormer mit-b5, image_size: 928x928
\n・ 2x SegFormer mit-b5, image_size: 960x960
\nDifferent pretrained models, seed and additional stain dataset were used in some models. The reason all models are SegFormer is I chose models only from MMSegmentation and it worked well.
\n*Best single model was mit-b4(private: 0.82821).

\n

Traning

\n

I used google colaboratory. Since GPU resources are limited, I trained segformer with batch_size=1 with group normalization. In my experiment, it was better than batch_size=4 with batch normalization in both CV and LB. I chose 2 class rather than 6 class segmentation because it showed better results consistently, especially in lung. In 6 classes, the area of the lung tends to shrink and even if the area was equivalent to 2 classes, LB lung score was worse.

\n

settings:
\n・ norm: group normalization(num_groups=32) for decoder
\n・ num_classes:2
\n・ loss_decode: CE:LovaszLoss=1:3
\n・ steps: 45000
\n・ optimization: AdamW with lr=0.00006

\n

augmentations:
\n・ Dataset image size: 2000x2000
\n・ Downscale for prostate
\n・ Crop/ Flip/ RandomRotate90/ HueSaturationValue/ RandomGamma/ RandomBrightness
\n・ (mmseg)Resize to (480~1600, 480~1600) * such as (480, 1600), (1024,1024), (1500,480)
\n・ (mmseg)RandomCrop(crop_size=model input size)

\n

Inference

\n

・ Whole image and ignored pixel_size
\n・ TTA: horizontal flip and different scales (approximately x0.8, x1.0, x1.2) .
\n・ Thresholds for each organ were kidney: 0.3 | large intestine: 0.2 | lung: 0.6 | prostate: 0.3 | spleen: 0.5

\n

Results

\n

all private: 0.83562 public: 0.82716
\nkidney private: 0.16940 public: 0.12175
\nlarge intestine private: 0.09005 public: 0.05730
\nlung private: 0.21586 public: 0.10963
\nprostate private: 0.17784 public: 0.14772
\nspleen private: 0.18246 public: 0.17515

\n

About license

\n

Pretrained SegFormer and its encoder “mit” are unavailable for commercial use. It seems that other winners also use this model. However, I got an answer from the host, which permits the use of SegFormer in this competiion. The reasons are:

\n
    \n
  1. Since competition hosts are non-profit, the commercial restriction isn’t applied.
    2. \n
  2. The model was Apache 2.0 license at the competition launch and the license was changed during the competition.
    3. \n
", "messages": [ { "id": 1962297, "postDate": "2022-09-29T16:07:17.843Z", "content": "

First of all, I’d like to express my gratitude to kaggle and the competition hosts for holding such an amazing competition.
\nI’d like to thank MMSegmentation developers too. MMSegmentation is really nice and useful tool. Also great thanks to @ayuraj who incorporated a hook of Weight & Biases into MMSegmentation, which helped a lot to manage training history.

\n

In the contest, I learned a lot from @hengck23. Thank you so much for sharing knowledge with us:)

\n

Overview

\n

I used the following methods.
\n・ End-to-end MMSegmentation(with customization)
\n・ Ensemble of SegFormers
\n・ Used group normalization and trained with batch_size=1 in Colab
\n・ Hand-labeling of lung from scratch
\n・ External dataset and pseudo labeling.

\n

DataSet

\n

Used dataset:
\n・ HPA images [351 images]
\n・ HPA images stained with H&E or PAS [351 images x 4 stain patterns]
\n・ external spleen [39 images + 39 images x 4 stain patterns] *from a single image
\n・ external lung [73 images] *from a single image

\n

(Lung)When I noticed that unstable results were from lung prediction, I imagined hand labeling is better than predicting lung by low threshold, which might lead to unstable and unconfident results. So I researched how lung alveolus/alveoli look alike and hand-labeled lung images from scratch. Since there is no information on how to annotate alveolus/alveoli anywhere, I made about 7 annotation sets where potential alveolus was gradually annotated and probed which one is the best. Then I created an external dataset with pseudo labels and with the best threshold. After training with it, the best threshold for lung increased to 0.60, which gave the best result ever. I guess good modeling is better than hand labeling since there are those who got 0.11/0.22 in public/private.
\n(Spleen)Used pseudo label for an external dataset
\n(Prostate/ Largeintestine)Used pseudo label for the HPA dataset.
\nI had a feeling that my models underfit HPA dataset because these dataset didn’t put weight on HPA datasets in terms of colors. The torchstain was used for the stain tool.

\n

Models

\n

Ensemble of the following models:
\n・ 1x SegFormer mit-b3, image_size: 1024x1024
\n・ 2x SegFormer mit-b4, image_size: 960x960
\n・ 1x SegFormer mit-b5, image_size: 928x928
\n・ 2x SegFormer mit-b5, image_size: 960x960
\nDifferent pretrained models, seed and additional stain dataset were used in some models. The reason all models are SegFormer is I chose models only from MMSegmentation and it worked well.
\n*Best single model was mit-b4(private: 0.82821).

\n

Traning

\n

I used google colaboratory. Since GPU resources are limited, I trained segformer with batch_size=1 with group normalization. In my experiment, it was better than batch_size=4 with batch normalization in both CV and LB. I chose 2 class rather than 6 class segmentation because it showed better results consistently, especially in lung. In 6 classes, the area of the lung tends to shrink and even if the area was equivalent to 2 classes, LB lung score was worse.

\n

settings:
\n・ norm: group normalization(num_groups=32) for decoder
\n・ num_classes:2
\n・ loss_decode: CE:LovaszLoss=1:3
\n・ steps: 45000
\n・ optimization: AdamW with lr=0.00006

\n

augmentations:
\n・ Dataset image size: 2000x2000
\n・ Downscale for prostate
\n・ Crop/ Flip/ RandomRotate90/ HueSaturationValue/ RandomGamma/ RandomBrightness
\n・ (mmseg)Resize to (480~1600, 480~1600) * such as (480, 1600), (1024,1024), (1500,480)
\n・ (mmseg)RandomCrop(crop_size=model input size)

\n

Inference

\n

・ Whole image and ignored pixel_size
\n・ TTA: horizontal flip and different scales (approximately x0.8, x1.0, x1.2) .
\n・ Thresholds for each organ were kidney: 0.3 | large intestine: 0.2 | lung: 0.6 | prostate: 0.3 | spleen: 0.5

\n

Results

\n

all private: 0.83562 public: 0.82716
\nkidney private: 0.16940 public: 0.12175
\nlarge intestine private: 0.09005 public: 0.05730
\nlung private: 0.21586 public: 0.10963
\nprostate private: 0.17784 public: 0.14772
\nspleen private: 0.18246 public: 0.17515

\n

About license

\n

Pretrained SegFormer and its encoder “mit” are unavailable for commercial use. It seems that other winners also use this model. However, I got an answer from the host, which permits the use of SegFormer in this competiion. The reasons are:

\n
    \n
  1. Since competition hosts are non-profit, the commercial restriction isn’t applied.
    2. \n
  2. The model was Apache 2.0 license at the competition launch and the license was changed during the competition.
    3. \n
", "rawMarkdown": "First of all, I’d like to express my gratitude to kaggle and the competition hosts for holding such an amazing competition.\nI’d like to thank MMSegmentation developers too. MMSegmentation is really nice and useful tool. Also great thanks to @ayuraj who incorporated a hook of Weight & Biases into MMSegmentation, which helped a lot to manage training history.\n \nIn the contest, I learned a lot from @hengck23. Thank you so much for sharing knowledge with us:)\n \n\n### **Overview**\nI used the following methods.\n・ End-to-end MMSegmentation(with customization)\n・ Ensemble of SegFormers\n・ Used group normalization and trained with batch_size=1 in Colab\n・ Hand-labeling of lung from scratch\n・ External dataset and pseudo labeling.\n \n\n### **DataSet**\nUsed dataset:\n・ HPA images [351 images]\n・ HPA images stained with H&E or PAS [351 images x 4 stain patterns]\n・ external spleen [39 images + 39 images x 4 stain patterns] *from a single image\n・ external lung [73 images] *from a single image\n \n**(Lung)**When I noticed that unstable results were from lung prediction, I imagined hand labeling is better than predicting lung by low threshold, which might lead to unstable and unconfident results. So I researched how lung alveolus/alveoli look alike and hand-labeled lung images from scratch. Since there is no information on how to annotate alveolus/alveoli anywhere, I made about 7 annotation sets where potential alveolus was gradually annotated and probed which one is the best. Then I created an external dataset with pseudo labels and with the best threshold. After training with it, the best threshold for lung increased to 0.60, which gave the best result ever. I guess good modeling is better than hand labeling since there are those who got 0.11/0.22 in public/private. \n**(Spleen)**Used pseudo label for an external dataset\n**(Prostate/ Largeintestine)**Used pseudo label for the HPA dataset.\nI had a feeling that my models underfit HPA dataset because these dataset didn’t put weight on HPA datasets in terms of colors. The torchstain was used for the stain tool.\n \n\n### **Models**\nEnsemble of the following models:\n・ 1x SegFormer mit-b3, image_size: 1024x1024\n・ 2x SegFormer mit-b4, image_size: 960x960\n・ 1x SegFormer mit-b5, image_size: 928x928\n・ 2x SegFormer mit-b5, image_size: 960x960\nDifferent pretrained models, seed and additional stain dataset were used in some models. The reason all models are SegFormer is I chose models only from MMSegmentation and it worked well.\n*Best single model was mit-b4(private: 0.82821).\n\n\n### **Traning**\nI used google colaboratory. Since GPU resources are limited, I trained segformer with batch_size=1 with group normalization. In my experiment, it was better than batch_size=4 with batch normalization in both CV and LB. I chose 2 class rather than 6 class segmentation because it showed better results consistently, especially in lung. In 6 classes, the area of the lung tends to shrink and even if the area was equivalent to 2 classes, LB lung score was worse.\n\nsettings:\n・ norm: group normalization(num_groups=32) for decoder\n・ num_classes:2\n・ loss_decode: CE:LovaszLoss=1:3\n・ steps: 45000\n・ optimization: AdamW with lr=0.00006 \n\naugmentations:\n・ Dataset image size: 2000x2000\n・ Downscale for prostate\n・ Crop/ Flip/ RandomRotate90/ HueSaturationValue/ RandomGamma/ RandomBrightness\n・ (mmseg)Resize to (480~1600, 480~1600) * such as (480, 1600), (1024,1024), (1500,480)\n・ (mmseg)RandomCrop(crop_size=model input size)\n\n\n### **Inference**\n・ Whole image and ignored pixel_size\n・ TTA: horizontal flip and different scales (approximately x0.8, x1.0, x1.2) .\n・ Thresholds for each organ were kidney: 0.3 | large intestine: 0.2 | lung: 0.6 | prostate: 0.3 | spleen: 0.5\n \n\n### **Results**\nall\t\tprivate: 0.83562\tpublic: 0.82716 \nkidney\t\tprivate: 0.16940\t\tpublic: 0.12175\nlarge intestine\tprivate: 0.09005\t\tpublic: 0.05730\nlung\t\tprivate: 0.21586\t\tpublic: 0.10963\nprostate\tprivate: 0.17784\t\tpublic: 0.14772\nspleen\t\tprivate: 0.18246\t\tpublic: 0.17515\n \n \n### **About license**\nPretrained SegFormer and its encoder “mit” are unavailable for commercial use. It seems that other winners also use this model. However, I got an answer from the host, which permits the use of SegFormer in this competiion. The reasons are:\n1. Since competition hosts are non-profit, the commercial restriction isn’t applied.\n2. The model was Apache 2.0 license at the competition launch and the license was changed during the competition.\n", "votes": 48 }, { "id": 1965406, "postDate": "2022-10-01T10:27:02.797Z", "content": "

hi can you share your mmsegmentation config file only ? i want to know how to modify input image size. i use swin,convnext,twins,segformer in mmsegmentation but i finally get 0.72 in private. i find that when i use input image size large than defult setting(like 640, 768), the performance will bad

", "rawMarkdown": "hi can you share your mmsegmentation config file only ? i want to know how to modify input image size. i use swin,convnext,twins,segformer in mmsegmentation but i finally get 0.72 in private. i find that when i use input image size large than defult setting(like 640, 768), the performance will bad ", "votes": 1, "replies": [ { "id": 1965504, "postDate": "2022-10-01T12:03:44.753Z", "content": "

Did you downscale prostate images? The image size for prostate is said to be 160x160.

\n

I used train_pipeline like below.

\n
\n

crop_size = (960, 960)
\ntrain_pipeline = [
\n dict(type='LoadImageFromFile'),
\n dict(type='LoadAnnotations'),
\n dict(type='CustomAug'), # Customized for each organs
\n dict(type='Resize', img_scale=[(480,480),(1600, 1600)], keep_ratio=False, ratio_range=None),
\n dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.5),
\n dict(type='RandomFlip', direction='horizontal', prob=0),
\n dict(type='SaveOverlay', save_root_dir='/content', save_num=500, no_overlay=True), # Customized for visualization
\n dict(type='Convert2Class1'), # Customized for converting 6 class masks to 2 class one.
\n dict(type='Normalize', **img_norm_cfg),
\n dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
\n dict(type='DefaultFormatBundle'),
\n dict(type='Collect', keys=['img', 'gt_semantic_seg']),
\n]

\n

Prostate were processed with Albumentations in CustomAug:
\n A.Downscale(scale_min=0.08, scale_max=0.4, interpolation=cv2.INTER_LINEAR, p=0.9)
\nI used 2000x2000 images. So, downscale to from 160x160 to 800x800 and upscale to 2000x2000 in this case.

", "rawMarkdown": "Did you downscale prostate images? The image size for prostate is said to be 160x160.\n\nI used train_pipeline like below.\n\n----------\ncrop_size = (960, 960)\ntrain_pipeline = [\n dict(type='LoadImageFromFile'),\n dict(type='LoadAnnotations'),\n dict(type='CustomAug'), # Customized for each organs\n dict(type='Resize', img_scale=[(480,480),(1600, 1600)], keep_ratio=False, ratio_range=None),\n dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.5), \n dict(type='RandomFlip', direction='horizontal', prob=0),\n dict(type='SaveOverlay', save_root_dir='/content', save_num=500, no_overlay=True), # Customized for visualization\n dict(type='Convert2Class1'), # Customized for converting 6 class masks to 2 class one.\n dict(type='Normalize', **img_norm_cfg),\n dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),\n dict(type='DefaultFormatBundle'),\n dict(type='Collect', keys=['img', 'gt_semantic_seg']),\n]\n\nProstate were processed with Albumentations in CustomAug:\n A.Downscale(scale_min=0.08, scale_max=0.4, interpolation=cv2.INTER_LINEAR, p=0.9)\nI used 2000x2000 images. So, downscale to from 160x160 to 800x800 and upscale to 2000x2000 in this case." } ] }, { "id": 1980298, "postDate": "2022-10-10T04:30:19.753Z", "content": "

Hi, @opusen. My evaluation mDice is low for 2-class. Loss and Acc is normal. For multi-class it work well. What is your evaluation metric?
\nI don't know whether my custom Convert2Class is correct? Is there anything else to be aware of?
\n@PIPELINES.register_module()
\nclass MaskTo2Class:

\n
def __call__(self, results):\n    for key in results.get('seg_fields', []):\n        gt_seg = (results[key]!=0).astype(np.uint8)\n        results[key] = gt_seg\n    return results\n
\n

Is there anything else to be aware of?

", "rawMarkdown": "Hi, @opusen. My evaluation mDice is low for 2-class. Loss and Acc is normal. For multi-class it work well. What is your evaluation metric?\nI don't know whether my custom Convert2Class is correct? Is there anything else to be aware of?\n@PIPELINES.register_module()\nclass MaskTo2Class:\n\n def __call__(self, results):\n for key in results.get('seg_fields', []):\n gt_seg = (results[key]!=0).astype(np.uint8)\n results[key] = gt_seg\n return results\nIs there anything else to be aware of?", "replies": [ { "id": 1980349, "postDate": "2022-10-10T05:23:37.310Z", "content": "

Hi, how degree is it low? Although I used mDice, I haven't compared the metrics of 2 class with multi class because I customized mDice calculation for 2 class, from \"calculation from total area\" to \"average of each image\".
\nI recommend you to submit the model.

\n

By the way, did you take \"class 255\" into account?
\nClass 255 means 'ignore' and would be converted to \"class 1\" in your code depending on pipeline position.
\nFor example, RandomRotate and Pad would create class 255.
\nHere is an example.

\n
@PIPELINES.register_module()\nclass MaskTo2Class:\n    def __call__(self, data):\n        mask = data['gt_semantic_seg']\n\n        mask_ignore = (mask == 255)\n        mask = np.where((mask >= 1) & (mask < 255), 1, 0)\n        mask[mask_ignore] = 255\n\n        data['gt_semantic_seg'] = mask\n\n        return data\n
\n

Anyway, creating custom pipeline for visualization would be helpful for future works:)

", "rawMarkdown": "Hi, how degree is it low? Although I used mDice, I haven't compared the metrics of 2 class with multi class because I customized mDice calculation for 2 class, from \"calculation from total area\" to \"average of each image\".\nI recommend you to submit the model.\n\nBy the way, did you take \"class 255\" into account?\nClass 255 means 'ignore' and would be converted to \"class 1\" in your code depending on pipeline position.\nFor example, RandomRotate and Pad would create class 255.\nHere is an example. \n\n```\n@PIPELINES.register_module()\nclass MaskTo2Class:\n def __call__(self, data):\n mask = data['gt_semantic_seg']\n\n mask_ignore = (mask == 255)\n mask = np.where((mask >= 1) & (mask < 255), 1, 0)\n mask[mask_ignore] = 255\n\n data['gt_semantic_seg'] = mask\n\n return data\n```\n\nAnyway, creating custom pipeline for visualization would be helpful for future works:)", "votes": 1 }, { "id": 1981034, "postDate": "2022-10-10T14:34:48.407Z", "content": "

Yes. I didn't take 255 class into account. The evaluation mDice with MMSeg is about 16. Loss is normal. I don't know how to custom mDice in MMSeg? Could you share your method?

", "rawMarkdown": "Yes. I didn't take 255 class into account. The evaluation mDice with MMSeg is about 16. Loss is normal. I don't know how to custom mDice in MMSeg? Could you share your method?" }, { "id": 1982404, "postDate": "2022-10-11T12:06:27.637Z", "content": "
\n

Loss and Acc is normal.
\n The evaluation mDice with MMSeg is about 16

\n
\n

is mDice 0.16? Do you mean Acc is decode.acc_seg?

\n
\n

Yes. I didn't take 255 class into account.

\n
\n

I feel like test accuracy is also low. Did you check it?

\n
\n

Could you share your method?

\n
\n

Actually I don't recommend to use it.
\nI couldn't modify the code to work also in multi-class and the code was written in messy way.
\nAlso, CV wasn't useful in this competition:)

", "rawMarkdown": "> Loss and Acc is normal.\nThe evaluation mDice with MMSeg is about 16\n\nis mDice 0.16? Do you mean Acc is decode.acc_seg?\n\n> Yes. I didn't take 255 class into account.\n\nI feel like test accuracy is also low. Did you check it?\n\n> Could you share your method?\n\nActually I don't recommend to use it.\nI couldn't modify the code to work also in multi-class and the code was written in messy way. \nAlso, CV wasn't useful in this competition:)", "votes": 1 } ] }, { "id": 1978706, "postDate": "2022-10-08T23:54:28.693Z", "content": "

Congratulations for your first place! I wasn't familiar with the technique of group normalization. Could you share how did you use it, specially in the context of mmsegmentation?

", "rawMarkdown": "Congratulations for your first place! I wasn't familiar with the technique of group normalization. Could you share how did you use it, specially in the context of mmsegmentation?", "replies": [ { "id": 1979778, "postDate": "2022-10-09T17:25:21.153Z", "content": "

Thanks:)

\n

norm_cfg = dict(type='GN', num_groups=32, requires_grad=True)
\nmodel = dict(
\n type='EncoderDecoder',
\n backbone=dict(
\n type='MixVisionTransformer',
\n …
\n decode_head=dict(
\n type='SegformerHead',
\n …

\n

SegFormer is composed of 'MixVisionTransformer' and 'SegformerHead'.

\n

For MixVisionTransformer, Layer Normalization was used as default.
\nhttps://github.com/open-mmlab/mmsegmentation/blob/master/mmseg/models/backbones/mit.py#L329-L330

\n

For SegformerHead, default settings is SyncBN.
\nhttps://github.com/open-mmlab/mmsegmentation/blob/master/configs/_base_/models/segformer_mit-b0.py#L2
\nSo you can change SyncBN to 'GN'.

\n

Supported normalizations would be
\nhttps://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/bricks/norm.py#L11-L21

", "rawMarkdown": "Thanks:)\n\n**norm_cfg = dict(type='GN', num_groups=32, requires_grad=True)**\nmodel = dict(\n type='EncoderDecoder',\n backbone=dict(\n type='MixVisionTransformer',\n ...\n decode_head=dict(\n type='SegformerHead',\n ...\n\nSegFormer is composed of 'MixVisionTransformer' and 'SegformerHead'.\n\nFor MixVisionTransformer, Layer Normalization was used as default.\nhttps://github.com/open-mmlab/mmsegmentation/blob/master/mmseg/models/backbones/mit.py#L329-L330\n\nFor SegformerHead, default settings is SyncBN.\nhttps://github.com/open-mmlab/mmsegmentation/blob/master/configs/_base_/models/segformer_mit-b0.py#L2\nSo you can change SyncBN to 'GN'.\n\nSupported normalizations would be\nhttps://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/bricks/norm.py#L11-L21\n", "votes": 2 } ] }, { "id": 1977924, "postDate": "2022-10-08T10:42:31.893Z", "content": "

hi, can you share your competition-dataset-only scores?

", "rawMarkdown": "hi, can you share your competition-dataset-only scores?", "replies": [ { "id": 1979751, "postDate": "2022-10-09T17:06:55.637Z", "content": "

The boost of external spleen was 0.00594 and lung was 0.00622.
\nIn simple calculation, 0.83562(final score) - (0.00594+0.00622)=0.82346

\n

It was a good opportunity to check my submission history :)

", "rawMarkdown": "The boost of external spleen was 0.00594 and lung was 0.00622.\nIn simple calculation, 0.83562(final score) - (0.00594+0.00622)=0.82346\n\nIt was a good opportunity to check my submission history :)" }, { "id": 1981816, "postDate": "2022-10-11T04:10:25.343Z", "content": "

thanks for your reply. : )

", "rawMarkdown": "thanks for your reply. : )" } ] }, { "id": 1965705, "postDate": "2022-10-01T13:36:32.817Z", "content": "

Great methods, thanks for your sharing. I have two questions. For MMSegmentation the activation function is to use argmax. How to set thresholds for each organ? Can you share your test config?

", "rawMarkdown": "Great methods, thanks for your sharing. I have two questions. For MMSegmentation the activation function is to use argmax. How to set thresholds for each organ? Can you share your test config? ", "replies": [ { "id": 1965861, "postDate": "2022-10-01T15:20:02.887Z", "content": "
\n

How to set thresholds for each organ?

\n
\n

You need to customize encorder_decoder.py
\nI created a notebook.
\nhttps://www.kaggle.com/code/opusen/customize-mmseg

\n

Then you can use sigmoid raw outputs of each class for thresholds or ensemble.
\nI ensembled 'cell class' sigmoid of several models.

\n
\n

Can you share your test config?

\n
\n

Not much different from the others.

\n

dataset_type = 'xxx'
\ndata_root = 'xxx'
\nimg_norm_cfg = dict(
\n mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
\ncrop_size = (960, 960)
\ntrain_pipeline = []
\nval_pipeline = []
\ntest_pipeline = [
\n dict(type='LoadImageFromFile'),
\n dict(
\n type='MultiScaleFlipAug',
\n img_scale=crop_size,
\n img_ratios=[0.8, 1.0, 1.2], #[0.8125, 1.0, 1.1875] for 1024x1024.
\n flip=True,
\n transforms=[
\n dict(type='Resize', keep_ratio=True),
\n dict(type='RandomFlip'),
\n dict(type='Normalize', **img_norm_cfg),
\n dict(type='ImageToTensor', keys=['img']),
\n dict(type='Collect', keys=['img']),
\n ])
\n]

", "rawMarkdown": "> How to set thresholds for each organ?\n\nYou need to customize encorder_decoder.py\nI created a notebook.\nhttps://www.kaggle.com/code/opusen/customize-mmseg\n\nThen you can use sigmoid raw outputs of each class for thresholds or ensemble.\nI ensembled 'cell class' sigmoid of several models.\n\n> Can you share your test config?\n\nNot much different from the others.\n\ndataset_type = 'xxx'\ndata_root = 'xxx'\nimg_norm_cfg = dict(\n mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)\ncrop_size = (960, 960)\ntrain_pipeline = []\nval_pipeline = []\ntest_pipeline = [\n dict(type='LoadImageFromFile'),\n dict(\n type='MultiScaleFlipAug',\n img_scale=crop_size,\n img_ratios=[0.8, 1.0, 1.2], #[0.8125, 1.0, 1.1875] for 1024x1024.\n flip=True,\n transforms=[\n dict(type='Resize', keep_ratio=True),\n dict(type='RandomFlip'),\n dict(type='Normalize', **img_norm_cfg),\n dict(type='ImageToTensor', keys=['img']),\n dict(type='Collect', keys=['img']),\n ])\n]", "votes": 2 }, { "id": 1968871, "postDate": "2022-10-03T08:43:39.527Z", "content": "

Thanks for your response.

", "rawMarkdown": "Thanks for your response." } ] }, { "id": 2095663, "postDate": "2023-01-11T14:50:22.463Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1965406, "author_name": "ananzeng", "author_url": "", "post_date": "2022-10-01T10:27:02.797000", "content": "

hi can you share your mmsegmentation config file only ? i want to know how to modify input image size. i use swin,convnext,twins,segformer in mmsegmentation but i finally get 0.72 in private. i find that when i use input image size large than defult setting(like 640, 768), the performance will bad

", "votes": 1, "replies": [ { "id": 1965504, "author_name": "opusen", "author_url": "", "post_date": "2022-10-01T12:03:44.753000", "content": "

Did you downscale prostate images? The image size for prostate is said to be 160x160.

\n

I used train_pipeline like below.

\n
\n

crop_size = (960, 960)
\ntrain_pipeline = [
\n dict(type='LoadImageFromFile'),
\n dict(type='LoadAnnotations'),
\n dict(type='CustomAug'), # Customized for each organs
\n dict(type='Resize', img_scale=[(480,480),(1600, 1600)], keep_ratio=False, ratio_range=None),
\n dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.5),
\n dict(type='RandomFlip', direction='horizontal', prob=0),
\n dict(type='SaveOverlay', save_root_dir='/content', save_num=500, no_overlay=True), # Customized for visualization
\n dict(type='Convert2Class1'), # Customized for converting 6 class masks to 2 class one.
\n dict(type='Normalize', **img_norm_cfg),
\n dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
\n dict(type='DefaultFormatBundle'),
\n dict(type='Collect', keys=['img', 'gt_semantic_seg']),
\n]

\n

Prostate were processed with Albumentations in CustomAug:
\n A.Downscale(scale_min=0.08, scale_max=0.4, interpolation=cv2.INTER_LINEAR, p=0.9)
\nI used 2000x2000 images. So, downscale to from 160x160 to 800x800 and upscale to 2000x2000 in this case.

", "votes": 0, "replies": [] } ] }, { "id": 1980298, "author_name": "ynhuhu", "author_url": "", "post_date": "2022-10-10T04:30:19.753000", "content": "

Hi, @opusen. My evaluation mDice is low for 2-class. Loss and Acc is normal. For multi-class it work well. What is your evaluation metric?
\nI don't know whether my custom Convert2Class is correct? Is there anything else to be aware of?
\n@PIPELINES.register_module()
\nclass MaskTo2Class:

\n
def __call__(self, results):\n    for key in results.get('seg_fields', []):\n        gt_seg = (results[key]!=0).astype(np.uint8)\n        results[key] = gt_seg\n    return results\n
\n

Is there anything else to be aware of?

", "votes": 0, "replies": [ { "id": 1980349, "author_name": "opusen", "author_url": "", "post_date": "2022-10-10T05:23:37.310000", "content": "

Hi, how degree is it low? Although I used mDice, I haven't compared the metrics of 2 class with multi class because I customized mDice calculation for 2 class, from \"calculation from total area\" to \"average of each image\".
\nI recommend you to submit the model.

\n

By the way, did you take \"class 255\" into account?
\nClass 255 means 'ignore' and would be converted to \"class 1\" in your code depending on pipeline position.
\nFor example, RandomRotate and Pad would create class 255.
\nHere is an example.

\n
@PIPELINES.register_module()\nclass MaskTo2Class:\n    def __call__(self, data):\n        mask = data['gt_semantic_seg']\n\n        mask_ignore = (mask == 255)\n        mask = np.where((mask >= 1) & (mask < 255), 1, 0)\n        mask[mask_ignore] = 255\n\n        data['gt_semantic_seg'] = mask\n\n        return data\n
\n

Anyway, creating custom pipeline for visualization would be helpful for future works:)

", "votes": 1, "replies": [] }, { "id": 1981034, "author_name": "ynhuhu", "author_url": "", "post_date": "2022-10-10T14:34:48.407000", "content": "

Yes. I didn't take 255 class into account. The evaluation mDice with MMSeg is about 16. Loss is normal. I don't know how to custom mDice in MMSeg? Could you share your method?

", "votes": 0, "replies": [] }, { "id": 1982404, "author_name": "opusen", "author_url": "", "post_date": "2022-10-11T12:06:27.637000", "content": "
\n

Loss and Acc is normal.
\n The evaluation mDice with MMSeg is about 16

\n
\n

is mDice 0.16? Do you mean Acc is decode.acc_seg?

\n
\n

Yes. I didn't take 255 class into account.

\n
\n

I feel like test accuracy is also low. Did you check it?

\n
\n

Could you share your method?

\n
\n

Actually I don't recommend to use it.
\nI couldn't modify the code to work also in multi-class and the code was written in messy way.
\nAlso, CV wasn't useful in this competition:)

", "votes": 1, "replies": [] } ] }, { "id": 1978706, "author_name": "CarlosRunner", "author_url": "", "post_date": "2022-10-08T23:54:28.693000", "content": "

Congratulations for your first place! I wasn't familiar with the technique of group normalization. Could you share how did you use it, specially in the context of mmsegmentation?

", "votes": 0, "replies": [ { "id": 1979778, "author_name": "opusen", "author_url": "", "post_date": "2022-10-09T17:25:21.153000", "content": "

Thanks:)

\n

norm_cfg = dict(type='GN', num_groups=32, requires_grad=True)
\nmodel = dict(
\n type='EncoderDecoder',
\n backbone=dict(
\n type='MixVisionTransformer',
\n …
\n decode_head=dict(
\n type='SegformerHead',
\n …

\n

SegFormer is composed of 'MixVisionTransformer' and 'SegformerHead'.

\n

For MixVisionTransformer, Layer Normalization was used as default.
\nhttps://github.com/open-mmlab/mmsegmentation/blob/master/mmseg/models/backbones/mit.py#L329-L330

\n

For SegformerHead, default settings is SyncBN.
\nhttps://github.com/open-mmlab/mmsegmentation/blob/master/configs/_base_/models/segformer_mit-b0.py#L2
\nSo you can change SyncBN to 'GN'.

\n

Supported normalizations would be
\nhttps://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/bricks/norm.py#L11-L21

", "votes": 2, "replies": [] } ] }, { "id": 1977924, "author_name": "Forte Lean", "author_url": "", "post_date": "2022-10-08T10:42:31.893000", "content": "

hi, can you share your competition-dataset-only scores?

", "votes": 0, "replies": [ { "id": 1979751, "author_name": "opusen", "author_url": "", "post_date": "2022-10-09T17:06:55.637000", "content": "

The boost of external spleen was 0.00594 and lung was 0.00622.
\nIn simple calculation, 0.83562(final score) - (0.00594+0.00622)=0.82346

\n

It was a good opportunity to check my submission history :)

", "votes": 0, "replies": [] }, { "id": 1981816, "author_name": "Forte Lean", "author_url": "", "post_date": "2022-10-11T04:10:25.343000", "content": "

thanks for your reply. : )

", "votes": 0, "replies": [] } ] }, { "id": 1965705, "author_name": "ynhuhu", "author_url": "", "post_date": "2022-10-01T13:36:32.817000", "content": "

Great methods, thanks for your sharing. I have two questions. For MMSegmentation the activation function is to use argmax. How to set thresholds for each organ? Can you share your test config?

", "votes": 0, "replies": [ { "id": 1965861, "author_name": "opusen", "author_url": "", "post_date": "2022-10-01T15:20:02.887000", "content": "
\n

How to set thresholds for each organ?

\n
\n

You need to customize encorder_decoder.py
\nI created a notebook.
\nhttps://www.kaggle.com/code/opusen/customize-mmseg

\n

Then you can use sigmoid raw outputs of each class for thresholds or ensemble.
\nI ensembled 'cell class' sigmoid of several models.

\n
\n

Can you share your test config?

\n
\n

Not much different from the others.

\n

dataset_type = 'xxx'
\ndata_root = 'xxx'
\nimg_norm_cfg = dict(
\n mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
\ncrop_size = (960, 960)
\ntrain_pipeline = []
\nval_pipeline = []
\ntest_pipeline = [
\n dict(type='LoadImageFromFile'),
\n dict(
\n type='MultiScaleFlipAug',
\n img_scale=crop_size,
\n img_ratios=[0.8, 1.0, 1.2], #[0.8125, 1.0, 1.1875] for 1024x1024.
\n flip=True,
\n transforms=[
\n dict(type='Resize', keep_ratio=True),
\n dict(type='RandomFlip'),
\n dict(type='Normalize', **img_norm_cfg),
\n dict(type='ImageToTensor', keys=['img']),
\n dict(type='Collect', keys=['img']),
\n ])
\n]

", "votes": 2, "replies": [] }, { "id": 1968871, "author_name": "ynhuhu", "author_url": "", "post_date": "2022-10-03T08:43:39.527000", "content": "

Thanks for your response.

", "votes": 0, "replies": [] } ] }, { "id": 2095663, "author_name": "", "author_url": "", "post_date": "2023-01-11T14:50:22.463000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1962297": "First of all, I’d like to express my gratitude to kaggle and the competition hosts for holding such an amazing competition.\nI’d like to thank MMSegmentation developers too. MMSegmentation is really nice and useful tool. Also great thanks to @ayuraj who incorporated a hook of Weight & Biases into MMSegmentation, which helped a lot to manage training history.\n \nIn the contest, I learned a lot from @hengck23. Thank you so much for sharing knowledge with us:)\n \n\n### **Overview**\nI used the following methods.\n・ End-to-end MMSegmentation(with customization)\n・ Ensemble of SegFormers\n・ Used group normalization and trained with batch_size=1 in Colab\n・ Hand-labeling of lung from scratch\n・ External dataset and pseudo labeling.\n \n\n### **DataSet**\nUsed dataset:\n・ HPA images [351 images]\n・ HPA images stained with H&E or PAS [351 images x 4 stain patterns]\n・ external spleen [39 images + 39 images x 4 stain patterns] *from a single image\n・ external lung [73 images] *from a single image\n \n**(Lung)**When I noticed that unstable results were from lung prediction, I imagined hand labeling is better than predicting lung by low threshold, which might lead to unstable and unconfident results. So I researched how lung alveolus/alveoli look alike and hand-labeled lung images from scratch. Since there is no information on how to annotate alveolus/alveoli anywhere, I made about 7 annotation sets where potential alveolus was gradually annotated and probed which one is the best. Then I created an external dataset with pseudo labels and with the best threshold. After training with it, the best threshold for lung increased to 0.60, which gave the best result ever. I guess good modeling is better than hand labeling since there are those who got 0.11/0.22 in public/private. \n**(Spleen)**Used pseudo label for an external dataset\n**(Prostate/ Largeintestine)**Used pseudo label for the HPA dataset.\nI had a feeling that my models underfit HPA dataset because these dataset didn’t put weight on HPA datasets in terms of colors. The torchstain was used for the stain tool.\n \n\n### **Models**\nEnsemble of the following models:\n・ 1x SegFormer mit-b3, image_size: 1024x1024\n・ 2x SegFormer mit-b4, image_size: 960x960\n・ 1x SegFormer mit-b5, image_size: 928x928\n・ 2x SegFormer mit-b5, image_size: 960x960\nDifferent pretrained models, seed and additional stain dataset were used in some models. The reason all models are SegFormer is I chose models only from MMSegmentation and it worked well.\n*Best single model was mit-b4(private: 0.82821).\n\n\n### **Traning**\nI used google colaboratory. Since GPU resources are limited, I trained segformer with batch_size=1 with group normalization. In my experiment, it was better than batch_size=4 with batch normalization in both CV and LB. I chose 2 class rather than 6 class segmentation because it showed better results consistently, especially in lung. In 6 classes, the area of the lung tends to shrink and even if the area was equivalent to 2 classes, LB lung score was worse.\n\nsettings:\n・ norm: group normalization(num_groups=32) for decoder\n・ num_classes:2\n・ loss_decode: CE:LovaszLoss=1:3\n・ steps: 45000\n・ optimization: AdamW with lr=0.00006 \n\naugmentations:\n・ Dataset image size: 2000x2000\n・ Downscale for prostate\n・ Crop/ Flip/ RandomRotate90/ HueSaturationValue/ RandomGamma/ RandomBrightness\n・ (mmseg)Resize to (480~1600, 480~1600) * such as (480, 1600), (1024,1024), (1500,480)\n・ (mmseg)RandomCrop(crop_size=model input size)\n\n\n### **Inference**\n・ Whole image and ignored pixel_size\n・ TTA: horizontal flip and different scales (approximately x0.8, x1.0, x1.2) .\n・ Thresholds for each organ were kidney: 0.3 | large intestine: 0.2 | lung: 0.6 | prostate: 0.3 | spleen: 0.5\n \n\n### **Results**\nall\t\tprivate: 0.83562\tpublic: 0.82716 \nkidney\t\tprivate: 0.16940\t\tpublic: 0.12175\nlarge intestine\tprivate: 0.09005\t\tpublic: 0.05730\nlung\t\tprivate: 0.21586\t\tpublic: 0.10963\nprostate\tprivate: 0.17784\t\tpublic: 0.14772\nspleen\t\tprivate: 0.18246\t\tpublic: 0.17515\n \n \n### **About license**\nPretrained SegFormer and its encoder “mit” are unavailable for commercial use. It seems that other winners also use this model. However, I got an answer from the host, which permits the use of SegFormer in this competiion. The reasons are:\n1. Since competition hosts are non-profit, the commercial restriction isn’t applied.\n2. The model was Apache 2.0 license at the competition launch and the license was changed during the competition.\n", "1965406": "hi can you share your mmsegmentation config file only ? i want to know how to modify input image size. i use swin,convnext,twins,segformer in mmsegmentation but i finally get 0.72 in private. i find that when i use input image size large than defult setting(like 640, 768), the performance will bad ", "1980298": "Hi, @opusen. My evaluation mDice is low for 2-class. Loss and Acc is normal. For multi-class it work well. What is your evaluation metric?\nI don't know whether my custom Convert2Class is correct? Is there anything else to be aware of?\n@PIPELINES.register_module()\nclass MaskTo2Class:\n\n def __call__(self, results):\n for key in results.get('seg_fields', []):\n gt_seg = (results[key]!=0).astype(np.uint8)\n results[key] = gt_seg\n return results\nIs there anything else to be aware of?", "1978706": "Congratulations for your first place! I wasn't familiar with the technique of group normalization. Could you share how did you use it, specially in the context of mmsegmentation?", "1977924": "hi, can you share your competition-dataset-only scores?", "1965705": "Great methods, thanks for your sharing. I have two questions. For MMSegmentation the activation function is to use argmax. How to set thresholds for each organ? Can you share your test config? ", "2095663": "" } }