{ "id": 354683, "title": "3d place solution ", "url": "/competitions/hubmap-organ-segmentation/writeups/human-torus-team-3d-place-solution", "author_name": "", "post_date": "2023-06-20T08:29:13.313Z", "votes": 70, "comment_count": 17, "views": 0, "content": "

First, we would like to thank the Armed Forces of Ukraine, Security Service of Ukraine, Defence Intelligence of Ukraine, State Emergency Service of Ukraine for providing safety and security to participate in this great competition, complete this work, and help science, technology, and business not stop and move forward.

\n

Also thanks to my teammates (@sakvaua , @igorkrashenyi, @alexkirnas), Kaggle team and competition organizers for this year HuBMAP + HPA event. It was a great pleasure to compete in order to bring benefit in biomedical sphere!
\nAccording to hosts the main challenge was:

\n
Adapting models to function properly  presented with  that was prepared using a different protocol will be one of the core challenges of  competition. While   expected to make the problem more difficult, developing models that generalize  a key goal of  endeavor.\n
\n


\nAnd it was really a key point to Hacking the Human Body this year :) So here is our approach

\n

Data

\n\n
, , , , , , , , ,,, ,, , \n
\n

Lately we have pseudo-labeled it and added again to training

\n\n
:  * ,\n:  * ,\n:  * ,\n:  * ,\n:  * ,\n
\n\n

Pseudo/Additional Data

\n

We used additional data from GTEX and HPA portals to complement the initial training data. The GTEX data was especially important here because it was stained, similar to HuBMAP slides, with H&E. From GTEX we downloaded prostate, large intestine, kidneys and spleen data for patients with no apparent pathologies. We ignored lungs from GTEX as we couldn’t figure out how to segment those (and neither did our model). We were progressively adding GTEX images to our pipeline ending up with around 140 at the end of the competition. And pseudo labeled them with the same ensemble, as HPA
\nFrom the HPA site we used a plethora of DAB stained slide very similar to those provided by organizers. Overall, we have added between 57-61K of additional HPA images for each organ.
\nWe pseudo labeled all the external data with our ensamples, which scored 0.59 on HubMap and 0.81 on HPA+HubMap LB.
\nFor selecting particular pseudo images for labeling we were inspired by Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks. We did not select best labels but just randomly sample the same dataset as training one from HPA pseudo and all GTEX dataset for training.
\nWe have repeated pseudo labeling twice

\n

Composed training dataset

\n

All our training dataset was composed from 3 subsets:

\n
    \n
  1. HPA train fold + HPA pseudo with Pixel Size rescale and Hist Matching - repeated X8
    2. \n
  2. HPA train fold + HPA pseudo - repeated X4
    3. \n
  3. GTEX pseudo with Pixel Size rescale - repeated X16
    4. \n
\n

Models

\n

We have used SMP. And final ensemble :

\n\n

Interesting Point is that CNNs showed better performance on public HubMap ~ 0.61 comparing to Segformers ~ 0.60, while Segformers outperformed CNNs on local validation ~0.85-0.86, while CNNs ~0.83-0.84. BUT on Private Segformers outperformed CNNs and our best Private submit (not selected) includes:

\n\n

Training

\n\n

Validation

\n\n

Inference

\n\n

Inference Kernel : https://www.kaggle.com/code/vladimirsydor/hubmap-2021-inference-v1/notebook?scriptVersionId=106283407
\nGitHub : https://github.com/VSydorskyy/hubmap_2022_htt_solution
\nPaper : https://arxiv.org/abs/2305.02148

", "messages": [ { "id": "1951890", "postDate": "09/23/2022 10:05:37", "content": "

First, we would like to thank the Armed Forces of Ukraine, Security Service of Ukraine, Defence Intelligence of Ukraine, State Emergency Service of Ukraine for providing safety and security to participate in this great competition, complete this work, and help science, technology, and business not stop and move forward.

\n

Also thanks to my teammates (@sakvaua , @igorkrashenyi, @alexkirnas), Kaggle team and competition organizers for this year HuBMAP + HPA event. It was a great pleasure to compete in order to bring benefit in biomedical sphere!
\nAccording to hosts the main challenge was:

\n
Adapting models to function properly  presented with  that was prepared using a different protocol will be one of the core challenges of  competition. While   expected to make the problem more difficult, developing models that generalize  a key goal of  endeavor.\n
\n


\nAnd it was really a key point to Hacking the Human Body this year :) So here is our approach

\n

Data

\n\n
, , , , , , , , ,,, ,, , \n
\n

Lately we have pseudo-labeled it and added again to training

\n\n
:  * ,\n:  * ,\n:  * ,\n:  * ,\n:  * ,\n
\n\n

Pseudo/Additional Data

\n

We used additional data from GTEX and HPA portals to complement the initial training data. The GTEX data was especially important here because it was stained, similar to HuBMAP slides, with H&E. From GTEX we downloaded prostate, large intestine, kidneys and spleen data for patients with no apparent pathologies. We ignored lungs from GTEX as we couldn’t figure out how to segment those (and neither did our model). We were progressively adding GTEX images to our pipeline ending up with around 140 at the end of the competition. And pseudo labeled them with the same ensemble, as HPA
\nFrom the HPA site we used a plethora of DAB stained slide very similar to those provided by organizers. Overall, we have added between 57-61K of additional HPA images for each organ.
\nWe pseudo labeled all the external data with our ensamples, which scored 0.59 on HubMap and 0.81 on HPA+HubMap LB.
\nFor selecting particular pseudo images for labeling we were inspired by Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks. We did not select best labels but just randomly sample the same dataset as training one from HPA pseudo and all GTEX dataset for training.
\nWe have repeated pseudo labeling twice

\n

Composed training dataset

\n

All our training dataset was composed from 3 subsets:

\n
    \n
  1. HPA train fold + HPA pseudo with Pixel Size rescale and Hist Matching - repeated X8
    2. \n
  2. HPA train fold + HPA pseudo - repeated X4
    3. \n
  3. GTEX pseudo with Pixel Size rescale - repeated X16
    4. \n
\n

Models

\n

We have used SMP. And final ensemble :

\n\n

Interesting Point is that CNNs showed better performance on public HubMap ~ 0.61 comparing to Segformers ~ 0.60, while Segformers outperformed CNNs on local validation ~0.85-0.86, while CNNs ~0.83-0.84. BUT on Private Segformers outperformed CNNs and our best Private submit (not selected) includes:

\n\n

Training

\n\n

Validation

\n\n

Inference

\n\n

Inference Kernel : https://www.kaggle.com/code/vladimirsydor/hubmap-2021-inference-v1/notebook?scriptVersionId=106283407
\nGitHub : https://github.com/VSydorskyy/hubmap_2022_htt_solution
\nPaper : https://arxiv.org/abs/2305.02148

", "rawMarkdown": "*First, we would like to thank the Armed Forces of Ukraine, Security Service of Ukraine, Defence Intelligence of Ukraine, State Emergency Service of Ukraine for providing safety and security to participate in this great competition, complete this work, and help science, technology, and business not stop and move forward.*\n\nAlso thanks to my teammates (@sakvaua , @igorkrashenyi, @alexkirnas), Kaggle team and competition organizers for this year HuBMAP + HPA event. It was a great pleasure to compete in order to bring benefit in biomedical sphere!\nAccording to hosts the main challenge was:\n```\nAdapting models to function properly when presented with data that was prepared using a different protocol will be one of the core challenges of this competition. While this is expected to make the problem more difficult, developing models that generalize is a key goal of this endeavor.\n``` \nAnd it was really a key point to Hacking the Human Body this year :) So here is our approach \n# Data \n- Drop some completely (to our point of view) mislabeled lungs. IDS. Later the organizers clarified that those ids are not mislabeled but rather represent a different way to section alveoli. Anyway there were too few of them to be useful and they confused our models without providing any meaningful new data. So we decided to focus on the horizontally sectioned alveoli that look like bubbles.\n```\n12476, 127, 13189, 15124, 16564, 23252, 25516, 25945, 29610,30084,30500, 31139,31571, 7359, 8151\n```\nLately we have pseudo-labeled it and added again to training \n- One of key points was adapting to wildly varying pixel sizes. The images scales ranged from 6.3um for prostate and down to 0.2um/pixel for large intestine. We tackled this issue by rescaling our train dataset to the target HuBMAP resolution. Though to increase the model’s receptive field we applied additional downscalers for larger images and upscalers for the prostate. \nWe used one dataset rescaled to HuBMAP scales and another dataset with the original HPA scales. The latter one was not only important for HPA predictions (absent in the private LB) but also to provide some additional scaling information to the model. \nHere is our additional rescaling dict (original scale / additional scale). \n\n```\n\"prostate\": 0.15 * 2,\n\"spleen\": 1 * 2,\n\"lung\": 0.5 * 2,\n\"kidney\": 1 * 2,\n\"largeintestine\": 1 * 2,\n```\n- One important aug was CutMix. We have used pretty aggressive CutMix (\"prob\": 0.5, \"alpha\": 1.0) though the trick here was to apply CutMix only within the single organ class.\n- We have trained CNN based models on 512 crops and segformer on 1024 crop. As for segformer bigger training crop it was pretty important, because results on 512 crop were much worse. While increasing training crop size for CNN - increased local validation but decreased HubMap LB score. Also we have sampled Non Empty Mask with probability 0.5\n- Also we have used pretty heavy augs - Geometric, Color, Distortions, Scales. The overall pipeline is really long :). And it might be the key point to model performance and stability on. The main idea behind the color augmentation was to suggest the model that the color is not important and it had to look for the other clues\n- To deal with the color shift between the DAB and H&E stains we used Histogram Matching of training pictures to H&E stained GTEX and HubMap images. We have used Historgram Matchin for all the images.\n\n## Pseudo/Additional Data\nWe used additional data from GTEX and HPA portals to complement the initial training data. The GTEX data was especially important here because it was stained, similar to HuBMAP slides, with H&E. From GTEX we downloaded prostate, large intestine, kidneys and spleen data for patients with no apparent pathologies. We ignored lungs from GTEX as we couldn’t figure out how to segment those (and neither did our model). We were progressively adding GTEX images to our pipeline ending up with around 140 at the end of the competition. And pseudo labeled them with the same ensemble, as HPA\nFrom the HPA site we used a plethora of DAB stained slide very similar to those provided by organizers. Overall, we have added between 57-61K of additional HPA images for each organ. \nWe pseudo labeled all the external data with our ensamples, which scored 0.59 on HubMap and 0.81 on HPA+HubMap LB.\nFor selecting particular pseudo images for labeling we were inspired by [Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks](https://arxiv.org/pdf/1904.04445.pdf%3C/p%3E). We did not select best labels but just randomly sample the same dataset as training one from HPA pseudo and all GTEX dataset for training.\nWe have repeated pseudo labeling twice \n\n## Composed training dataset\nAll our training dataset was composed from 3 subsets:\n1. HPA train fold + HPA pseudo with Pixel Size rescale and Hist Matching - repeated X8\n2. HPA train fold + HPA pseudo - repeated X4\n3. GTEX pseudo with Pixel Size rescale - repeated X16\n# Models\nWe have used [SMP](https://github.com/qubvel/segmentation_models.pytorch). And final ensemble :\n- Unet ++. Encoder Effnet B7 (noisy student pretrain). First HPA pseudo iteration. X5 - folds\n- Unet. Encoder Mit B5 (imagenet pretrain). Second HPA pseudo iteration. X5 - folds\n- Unet . Encoder Effnet B7 (noisy student pretrain) + Point Rand only for training. First HPA pseudo iteration. X5 - folds \n- Unet . Encoder Effnet B7 (noisy student pretrain) + Point Rand only for training. Second HPA pseudo iteration. X5 - folds \nPublic Hubmap: 0.61453\nPublic Hubmap+HPA: 0.83983\nPrivate: 0.83266\nLocal OOF: 0.8493729371691843\nLocal organ scores:\nkidney: 0.954695\nlargeintestine: 0.913645\nlung: 0.488490\nprostate: 0.833822\nspleen: 0.834292\nAs for [PointRand](https://arxiv.org/abs/1912.08193) we have used it only as additional loss as regularization while training \n\nInteresting Point is that CNNs showed better performance on public HubMap ~ 0.61 comparing to Segformers ~ 0.60, while Segformers outperformed CNNs on local validation ~0.85-0.86, while CNNs ~0.83-0.84. BUT on Private Segformers outperformed CNNs and our best Private submit (not selected) includes:\n- Unet ++. Encoder Effnet B7 (noisy student pretrain). First HPA pseudo iteration. X5 - folds\n- Unet. Encoder Mit B5 (imagenet pretrain). Second HPA pseudo iteration. X5 - folds\n- Unet. Encoder Mit B3 (imagenet pretrain). First HPA pseudo iteration. X5 - folds\nPublic Hubmap: 0.60931\nPublic Hubmap+HPA: ???\nPrivate: 0.83419\nLocal OOF: 0.8542834156960896\nLocal organ scores:\nkidney: 0.956776\nlargeintestine: 0.917554\nlung: 0.488331\nprostate: 0.838744\nspleen: 0.848721\nIt may granted us second place :) \n\n# Training\n- We used one channel output\n- Adam 1e-3\n- ReduceLROnPlateau : patience=3; factor=0.5; min_lr=1e-7 by valid dice\n- losses: softbce + tversky + focal + jaccard + (for some models) point rand loss * 2\n- fp16\n\n# Validation \n- 5 folds CV with stratification by organs \n- Predict image by image (batch size 1) in full scale with rescale to HubMap scale\n\n# Inference\n- Average 3 best checkpoints for each fold - just average model weights\n- Mean Fold prediction\n- 4 Flips TTA\n- Remove small regions - compute relative area on HPA data and take as threshold - <0.5 quantile\n- Predict image in full scale. We have only used sliding window for Segformers (1024 and 0.75 overlap) just because of cuda out of memory problem\n\n**Inference Kernel** : https://www.kaggle.com/code/vladimirsydor/hubmap-2021-inference-v1/notebook?scriptVersionId=106283407\n**GitHub** : https://github.com/VSydorskyy/hubmap_2022_htt_solution \n**Paper** : https://arxiv.org/abs/2305.02148", "votes": null }, { "id": "1952007", "postDate": "09/23/2022 11:37:01", "content": "

The greatest beginning words I have ever seen!

\n

Great solution!

", "rawMarkdown": "The greatest beginning words I have ever seen! \n\nGreat solution!", "votes": null }, { "id": "1952086", "postDate": "09/23/2022 12:24:59", "content": "

A few words on what else we tried:

\n", "rawMarkdown": "A few words on what else we tried:\n- Self-supervised learning. The idea was to train a model using both supervised and self-supervised approaches mixed in the same step. For self-supervised we tried consistency loss for color augmented external GTEX images. Didn't help. A purely supervised model always yielded better results.\n- 5 output channels instead of one. The idea was to give our model an additional signal in a form of an image class. It was worse than the single output channel model.\n- Additional classification head. Didn't work either. Higher weights to the classification head cannibalized classification scores.\n- Additional input (contour, class) and output channels (contours, centroids). Didn't help.\n- individual sets of augmentations for each organ. Didn't help. In the end, I couldn't find a set of augmentations that would give better results compared to our default one.\n- We had high hopes for stain normalization. The idea was to normalize GTEX H&E pseudolabeled images and all of the test set HuBMAP images so as to bring them into a common color space. Didn't help. Probably our aggressive color augmentations solved this problem.\n- Progressive pre-training with first epochs trained on 128-256-384-512 crops. Didn't help.\nA myriad of other smaller ideas which I have trouble recalling :)", "votes": null }, { "id": "1952089", "postDate": "09/23/2022 12:28:31", "content": "

\"Progressive pre-training with first epochs trained on 128-256-384-512 crops.\"

\n

this is because of overfitting in my experiment. i end up training only the target size.
\n(you get the same results for other Progressive-xxx experiments, where xxx = augmentation, more network layers, etc)

", "rawMarkdown": "\"Progressive pre-training with first epochs trained on 128-256-384-512 crops.\"\n\nthis is because of overfitting in my experiment. i end up training only the target size.\n(you get the same results for other Progressive-xxx experiments, where xxx = augmentation, more network layers, etc)", "votes": null }, { "id": "1952103", "postDate": "09/23/2022 12:41:28", "content": "

The results weren't worse. Just not better. And required an additional step. So we decided to skip it.
\nBTW We used crops and not scaled-down images. We also increased batch size proportionally. So effectively the NN was seeing the same number of pixels in a batch but split among a larger number of samples.

", "rawMarkdown": "The results weren't worse. Just not better. And required an additional step. So we decided to skip it.\nBTW We used crops and not scaled-down images. We also increased batch size proportionally. So effectively the NN was seeing the same number of pixels in a batch but split among a larger number of samples.", "votes": null }, { "id": "1952189", "postDate": "09/23/2022 13:42:33", "content": "

Do you use cutmix like this? Or just randomly select a part from another example?
\n\"\"

", "rawMarkdown": "Do you use cutmix like this? Or just randomly select a part from another example?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8569374%2F8f2f99a9af1303c1e0ee3a8958018ef4%2Foutput.png?generation=1663907208170410&alt=media)", "votes": null }, { "id": "1952215", "postDate": "09/23/2022 14:02:14", "content": "

just randomly select a part from another example

", "rawMarkdown": "just randomly select a part from another example", "votes": null }, { "id": "1952224", "postDate": "09/23/2022 14:05:45", "content": "

Thanks for sharing. We have done a lot of expriments on this but it performs very poorly.

", "rawMarkdown": "Thanks for sharing. We have done a lot of expriments on this but it performs very poorly.", "votes": null }, { "id": "1953479", "postDate": "09/24/2022 14:05:54", "content": "

The words are awesome. Loved it and thanks for the solutions

", "rawMarkdown": "The words are awesome. Loved it and thanks for the solutions", "votes": null }, { "id": "1953854", "postDate": "09/24/2022 19:19:15", "content": "

nice solution

", "rawMarkdown": "nice solution", "votes": null }, { "id": "1954105", "postDate": "09/25/2022 03:09:45", "content": "

Thanks for sharing your great solution!
\nBut I still have a dumb question.😜 Do you train on whole images or tiled ones?

", "rawMarkdown": "Thanks for sharing your great solution! \nBut I still have a dumb question.😜 Do you train on whole images or tiled ones?", "votes": null }, { "id": "1954647", "postDate": "09/25/2022 11:18:39", "content": "
We have trained CNN based models on 512 crops and segformer on 1024 crop. As for segformer bigger training crop it was pretty important, because results on 512 crop were much worse. While increasing training crop size for CNN - increased local validation but decreased HubMap LB score. Also we have sampled Non Empty Mask with probability 0.5\n
", "rawMarkdown": "```\nWe have trained CNN based models on 512 crops and segformer on 1024 crop. As for segformer bigger training crop it was pretty important, because results on 512 crop were much worse. While increasing training crop size for CNN - increased local validation but decreased HubMap LB score. Also we have sampled Non Empty Mask with probability 0.5\n```", "votes": null }, { "id": "1954702", "postDate": "09/25/2022 11:54:56", "content": "

Congrats Brother

", "rawMarkdown": "Congrats Brother", "votes": null }, { "id": "1954837", "postDate": "09/25/2022 13:52:40", "content": "

thanks a lot

", "rawMarkdown": "thanks a lot", "votes": null }, { "id": "1957427", "postDate": "09/27/2022 01:07:40", "content": "

Congratulations, and thanks for sharing your great solution!

\n

I have a question about PointRend.
\nWhy did you use it for training only?
\nDoes using it for inference degrade performance?

", "rawMarkdown": "Congratulations, and thanks for sharing your great solution!\n\nI have a question about PointRend.\nWhy did you use it for training only?\nDoes using it for inference degrade performance?", "votes": null }, { "id": "1958128", "postDate": "09/27/2022 09:35:19", "content": "

Loved it, thanks for the solution

", "rawMarkdown": "Loved it, thanks for the solution", "votes": null }, { "id": "1964323", "postDate": "09/30/2022 17:45:48", "content": "

thx for sharing

", "rawMarkdown": "thx for sharing", "votes": null }, { "id": "1971974", "postDate": "10/04/2022 21:15:32", "content": "

Slava Ukraini!!
\nAnd congratulation on the big win!

", "rawMarkdown": "Slava Ukraini!!\nAnd congratulation on the big win!", "votes": null } ], "comments": [ { "id": 1952007, "author_name": "vad13irt", "author_url": "", "post_date": "09/23/2022 11:37:01", "content": "

The greatest beginning words I have ever seen!

\n

Great solution!

", "votes": null, "replies": [] }, { "id": 1952086, "author_name": "sakvaua", "author_url": "", "post_date": "09/23/2022 12:24:59", "content": "

A few words on what else we tried:

\n", "votes": null, "replies": [ { "id": 1952089, "author_name": "hengck23", "author_url": "", "post_date": "09/23/2022 12:28:31", "content": "

\"Progressive pre-training with first epochs trained on 128-256-384-512 crops.\"

\n

this is because of overfitting in my experiment. i end up training only the target size.
\n(you get the same results for other Progressive-xxx experiments, where xxx = augmentation, more network layers, etc)

", "votes": null, "replies": [] }, { "id": 1952103, "author_name": "sakvaua", "author_url": "", "post_date": "09/23/2022 12:41:28", "content": "

The results weren't worse. Just not better. And required an additional step. So we decided to skip it.
\nBTW We used crops and not scaled-down images. We also increased batch size proportionally. So effectively the NN was seeing the same number of pixels in a batch but split among a larger number of samples.

", "votes": null, "replies": [] } ] }, { "id": 1952189, "author_name": "chinartist", "author_url": "", "post_date": "09/23/2022 13:42:33", "content": "

Do you use cutmix like this? Or just randomly select a part from another example?
\n\"\"

", "votes": null, "replies": [ { "id": 1952215, "author_name": "vladimirsydor", "author_url": "", "post_date": "09/23/2022 14:02:14", "content": "

just randomly select a part from another example

", "votes": null, "replies": [] }, { "id": 1952224, "author_name": "chinartist", "author_url": "", "post_date": "09/23/2022 14:05:45", "content": "

Thanks for sharing. We have done a lot of expriments on this but it performs very poorly.

", "votes": null, "replies": [] } ] }, { "id": 1953479, "author_name": "ekrajghimire", "author_url": "", "post_date": "09/24/2022 14:05:54", "content": "

The words are awesome. Loved it and thanks for the solutions

", "votes": null, "replies": [] }, { "id": 1953854, "author_name": "himanshublack", "author_url": "", "post_date": "09/24/2022 19:19:15", "content": "

nice solution

", "votes": null, "replies": [] }, { "id": 1954105, "author_name": "aldebaranhd", "author_url": "", "post_date": "09/25/2022 03:09:45", "content": "

Thanks for sharing your great solution!
\nBut I still have a dumb question.😜 Do you train on whole images or tiled ones?

", "votes": null, "replies": [ { "id": 1954647, "author_name": "vladimirsydor", "author_url": "", "post_date": "09/25/2022 11:18:39", "content": "
We have trained CNN based models on 512 crops and segformer on 1024 crop. As for segformer bigger training crop it was pretty important, because results on 512 crop were much worse. While increasing training crop size for CNN - increased local validation but decreased HubMap LB score. Also we have sampled Non Empty Mask with probability 0.5\n
", "votes": null, "replies": [] }, { "id": 1954837, "author_name": "aldebaranhd", "author_url": "", "post_date": "09/25/2022 13:52:40", "content": "

thanks a lot

", "votes": null, "replies": [] } ] }, { "id": 1954702, "author_name": "dhinakarank", "author_url": "", "post_date": "09/25/2022 11:54:56", "content": "

Congrats Brother

", "votes": null, "replies": [] }, { "id": 1957427, "author_name": "tattaka", "author_url": "", "post_date": "09/27/2022 01:07:40", "content": "

Congratulations, and thanks for sharing your great solution!

\n

I have a question about PointRend.
\nWhy did you use it for training only?
\nDoes using it for inference degrade performance?

", "votes": null, "replies": [] }, { "id": 1958128, "author_name": "bishu5c6", "author_url": "", "post_date": "09/27/2022 09:35:19", "content": "

Loved it, thanks for the solution

", "votes": null, "replies": [] }, { "id": 1964323, "author_name": "dragonzhang", "author_url": "", "post_date": "09/30/2022 17:45:48", "content": "

thx for sharing

", "votes": null, "replies": [] }, { "id": 1971974, "author_name": "azureblue83", "author_url": "", "post_date": "10/04/2022 21:15:32", "content": "

Slava Ukraini!!
\nAnd congratulation on the big win!

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1951890": "*First, we would like to thank the Armed Forces of Ukraine, Security Service of Ukraine, Defence Intelligence of Ukraine, State Emergency Service of Ukraine for providing safety and security to participate in this great competition, complete this work, and help science, technology, and business not stop and move forward.*\n\nAlso thanks to my teammates (@sakvaua , @igorkrashenyi, @alexkirnas), Kaggle team and competition organizers for this year HuBMAP + HPA event. It was a great pleasure to compete in order to bring benefit in biomedical sphere!\nAccording to hosts the main challenge was:\n```\nAdapting models to function properly when presented with data that was prepared using a different protocol will be one of the core challenges of this competition. While this is expected to make the problem more difficult, developing models that generalize is a key goal of this endeavor.\n``` \nAnd it was really a key point to Hacking the Human Body this year :) So here is our approach \n# Data \n- Drop some completely (to our point of view) mislabeled lungs. IDS. Later the organizers clarified that those ids are not mislabeled but rather represent a different way to section alveoli. Anyway there were too few of them to be useful and they confused our models without providing any meaningful new data. So we decided to focus on the horizontally sectioned alveoli that look like bubbles.\n```\n12476, 127, 13189, 15124, 16564, 23252, 25516, 25945, 29610,30084,30500, 31139,31571, 7359, 8151\n```\nLately we have pseudo-labeled it and added again to training \n- One of key points was adapting to wildly varying pixel sizes. The images scales ranged from 6.3um for prostate and down to 0.2um/pixel for large intestine. We tackled this issue by rescaling our train dataset to the target HuBMAP resolution. Though to increase the model’s receptive field we applied additional downscalers for larger images and upscalers for the prostate. \nWe used one dataset rescaled to HuBMAP scales and another dataset with the original HPA scales. The latter one was not only important for HPA predictions (absent in the private LB) but also to provide some additional scaling information to the model. \nHere is our additional rescaling dict (original scale / additional scale). \n\n```\n\"prostate\": 0.15 * 2,\n\"spleen\": 1 * 2,\n\"lung\": 0.5 * 2,\n\"kidney\": 1 * 2,\n\"largeintestine\": 1 * 2,\n```\n- One important aug was CutMix. We have used pretty aggressive CutMix (\"prob\": 0.5, \"alpha\": 1.0) though the trick here was to apply CutMix only within the single organ class.\n- We have trained CNN based models on 512 crops and segformer on 1024 crop. As for segformer bigger training crop it was pretty important, because results on 512 crop were much worse. While increasing training crop size for CNN - increased local validation but decreased HubMap LB score. Also we have sampled Non Empty Mask with probability 0.5\n- Also we have used pretty heavy augs - Geometric, Color, Distortions, Scales. The overall pipeline is really long :). And it might be the key point to model performance and stability on. The main idea behind the color augmentation was to suggest the model that the color is not important and it had to look for the other clues\n- To deal with the color shift between the DAB and H&E stains we used Histogram Matching of training pictures to H&E stained GTEX and HubMap images. We have used Historgram Matchin for all the images.\n\n## Pseudo/Additional Data\nWe used additional data from GTEX and HPA portals to complement the initial training data. The GTEX data was especially important here because it was stained, similar to HuBMAP slides, with H&E. From GTEX we downloaded prostate, large intestine, kidneys and spleen data for patients with no apparent pathologies. We ignored lungs from GTEX as we couldn’t figure out how to segment those (and neither did our model). We were progressively adding GTEX images to our pipeline ending up with around 140 at the end of the competition. And pseudo labeled them with the same ensemble, as HPA\nFrom the HPA site we used a plethora of DAB stained slide very similar to those provided by organizers. Overall, we have added between 57-61K of additional HPA images for each organ. \nWe pseudo labeled all the external data with our ensamples, which scored 0.59 on HubMap and 0.81 on HPA+HubMap LB.\nFor selecting particular pseudo images for labeling we were inspired by [Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks](https://arxiv.org/pdf/1904.04445.pdf%3C/p%3E). We did not select best labels but just randomly sample the same dataset as training one from HPA pseudo and all GTEX dataset for training.\nWe have repeated pseudo labeling twice \n\n## Composed training dataset\nAll our training dataset was composed from 3 subsets:\n1. HPA train fold + HPA pseudo with Pixel Size rescale and Hist Matching - repeated X8\n2. HPA train fold + HPA pseudo - repeated X4\n3. GTEX pseudo with Pixel Size rescale - repeated X16\n# Models\nWe have used [SMP](https://github.com/qubvel/segmentation_models.pytorch). And final ensemble :\n- Unet ++. Encoder Effnet B7 (noisy student pretrain). First HPA pseudo iteration. X5 - folds\n- Unet. Encoder Mit B5 (imagenet pretrain). Second HPA pseudo iteration. X5 - folds\n- Unet . Encoder Effnet B7 (noisy student pretrain) + Point Rand only for training. First HPA pseudo iteration. X5 - folds \n- Unet . Encoder Effnet B7 (noisy student pretrain) + Point Rand only for training. Second HPA pseudo iteration. X5 - folds \nPublic Hubmap: 0.61453\nPublic Hubmap+HPA: 0.83983\nPrivate: 0.83266\nLocal OOF: 0.8493729371691843\nLocal organ scores:\nkidney: 0.954695\nlargeintestine: 0.913645\nlung: 0.488490\nprostate: 0.833822\nspleen: 0.834292\nAs for [PointRand](https://arxiv.org/abs/1912.08193) we have used it only as additional loss as regularization while training \n\nInteresting Point is that CNNs showed better performance on public HubMap ~ 0.61 comparing to Segformers ~ 0.60, while Segformers outperformed CNNs on local validation ~0.85-0.86, while CNNs ~0.83-0.84. BUT on Private Segformers outperformed CNNs and our best Private submit (not selected) includes:\n- Unet ++. Encoder Effnet B7 (noisy student pretrain). First HPA pseudo iteration. X5 - folds\n- Unet. Encoder Mit B5 (imagenet pretrain). Second HPA pseudo iteration. X5 - folds\n- Unet. Encoder Mit B3 (imagenet pretrain). First HPA pseudo iteration. X5 - folds\nPublic Hubmap: 0.60931\nPublic Hubmap+HPA: ???\nPrivate: 0.83419\nLocal OOF: 0.8542834156960896\nLocal organ scores:\nkidney: 0.956776\nlargeintestine: 0.917554\nlung: 0.488331\nprostate: 0.838744\nspleen: 0.848721\nIt may granted us second place :) \n\n# Training\n- We used one channel output\n- Adam 1e-3\n- ReduceLROnPlateau : patience=3; factor=0.5; min_lr=1e-7 by valid dice\n- losses: softbce + tversky + focal + jaccard + (for some models) point rand loss * 2\n- fp16\n\n# Validation \n- 5 folds CV with stratification by organs \n- Predict image by image (batch size 1) in full scale with rescale to HubMap scale\n\n# Inference\n- Average 3 best checkpoints for each fold - just average model weights\n- Mean Fold prediction\n- 4 Flips TTA\n- Remove small regions - compute relative area on HPA data and take as threshold - <0.5 quantile\n- Predict image in full scale. We have only used sliding window for Segformers (1024 and 0.75 overlap) just because of cuda out of memory problem\n\n**Inference Kernel** : https://www.kaggle.com/code/vladimirsydor/hubmap-2021-inference-v1/notebook?scriptVersionId=106283407\n**GitHub** : https://github.com/VSydorskyy/hubmap_2022_htt_solution \n**Paper** : https://arxiv.org/abs/2305.02148", "1952007": "The greatest beginning words I have ever seen! \n\nGreat solution!", "1952086": "A few words on what else we tried:\n- Self-supervised learning. The idea was to train a model using both supervised and self-supervised approaches mixed in the same step. For self-supervised we tried consistency loss for color augmented external GTEX images. Didn't help. A purely supervised model always yielded better results.\n- 5 output channels instead of one. The idea was to give our model an additional signal in a form of an image class. It was worse than the single output channel model.\n- Additional classification head. Didn't work either. Higher weights to the classification head cannibalized classification scores.\n- Additional input (contour, class) and output channels (contours, centroids). Didn't help.\n- individual sets of augmentations for each organ. Didn't help. In the end, I couldn't find a set of augmentations that would give better results compared to our default one.\n- We had high hopes for stain normalization. The idea was to normalize GTEX H&E pseudolabeled images and all of the test set HuBMAP images so as to bring them into a common color space. Didn't help. Probably our aggressive color augmentations solved this problem.\n- Progressive pre-training with first epochs trained on 128-256-384-512 crops. Didn't help.\nA myriad of other smaller ideas which I have trouble recalling :)", "1952089": "\"Progressive pre-training with first epochs trained on 128-256-384-512 crops.\"\n\nthis is because of overfitting in my experiment. i end up training only the target size.\n(you get the same results for other Progressive-xxx experiments, where xxx = augmentation, more network layers, etc)", "1952103": "The results weren't worse. Just not better. And required an additional step. So we decided to skip it.\nBTW We used crops and not scaled-down images. We also increased batch size proportionally. So effectively the NN was seeing the same number of pixels in a batch but split among a larger number of samples.", "1952189": "Do you use cutmix like this? Or just randomly select a part from another example?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8569374%2F8f2f99a9af1303c1e0ee3a8958018ef4%2Foutput.png?generation=1663907208170410&alt=media)", "1952215": "just randomly select a part from another example", "1952224": "Thanks for sharing. We have done a lot of expriments on this but it performs very poorly.", "1953479": "The words are awesome. Loved it and thanks for the solutions", "1953854": "nice solution", "1954105": "Thanks for sharing your great solution! \nBut I still have a dumb question.😜 Do you train on whole images or tiled ones?", "1954647": "```\nWe have trained CNN based models on 512 crops and segformer on 1024 crop. As for segformer bigger training crop it was pretty important, because results on 512 crop were much worse. While increasing training crop size for CNN - increased local validation but decreased HubMap LB score. Also we have sampled Non Empty Mask with probability 0.5\n```", "1954702": "Congrats Brother", "1954837": "thanks a lot", "1957427": "Congratulations, and thanks for sharing your great solution!\n\nI have a question about PointRend.\nWhy did you use it for training only?\nDoes using it for inference degrade performance?", "1958128": "Loved it, thanks for the solution", "1964323": "thx for sharing", "1971974": "Slava Ukraini!!\nAnd congratulation on the big win!" }, "source": "meta" }