![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2F8a4a0c4ea0e4edbedc48368eb8c66c0a%2Fimg_10.png?generation=1663923823988013&alt=media\")
Dedicated to the Armed Forces of Ukraine!
\nGlory to the Armed Forces of Ukraine for the opportunity to provide science and education for the benefit of Ukraine!
Hello everyone from the complex team!
\nFirst of all, I would like to give my huge thank you to the Kaggle team for hosting such an amazing competition.
\nEvery time, I feel honored to be a part of such outstanding research.
\nI want to give a big thank you from our team to @hengck23. We have learned so much from your notebooks :)
\nIt is always a pleasure to have this great of a competitor from whom you can learn!
Our submission is a result of a few stages. We don't mainly do any pre-processing of the data. Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]
\nAfter many experiments with training all organs with one model, we gave up this idea. The reason was we didn't see any improvement with it after 0.77.
\nThus, we decided that the model needed contextual information about the organ type. The first experiments were set up with additional aux classification head for classifying each organ type. However, this became a false path after all; the models couldn't converge, and scores were not getting better.
Then, we decided to train separate models for the lung and the rest of the organs together. We adopted this strategy and proceeded with training the models in different conditions:
\nWe ensemble 6 models with different weights applied (5 folds per model) based on the best local cv scores. Plus, only one model for the lungs.
\n
Before training, we resized tissue slices to 768 and trained using strong augmentation.
\nWe find that intense color shifts and dropout noise produce good results.
_transforms = A.Compose([\n A.VerticalFlip(p=0.5),\n A.HorizontalFlip(p=0.5),\n A.RandomRotate90(p=0.5),\n\n A.RandomBrightnessContrast(brightness_limit=(-0.02, 0.02), contrast_limit=(-0.02, 0.02), p=0.25),\n A.RGBShift(r_shift_limit=(-25, 25), g_shift_limit=(-20, 20), b_shift_limit=(-20, 20), p=0.5),\n A.HueSaturationValue(hue_shift_limit=(-15, 15), sat_shift_limit=(-15, 15), val_shift_limit=(-15, 15), p=0.5),\n A.CoarseDropout(max_holes=20, min_holes=7, p=0.5),\n A.ShiftScaleRotate(shift_limit=0, scale_limit=(-0.2, 0.2), rotate_limit=0, p=0.5),\n A.OneOf([\n A.GridDistortion(num_steps=5, distort_limit=0.05, p=1.0),\n A.ElasticTransform(alpha=1, sigma=50, alpha_affine=50, p=1.0)\n ], p=0.5),\n])\nThe resulting pipeline is a model zoo :)
\nWe use pretrained ConvNext and HorNet backbone architectures. For the decoder block, we only use UPerNet.
\nBest HorNet Single Model Submissions
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2F5e2fb6e9d794947a77280a957f79a427%2Fimg_1.png?generation=1663921043762859&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2Fe32e4bd32acecc9868ebf968843561cf%2Fimg_2.png?generation=1663921060337152&alt=media\")
Best ConvNext Single Model Submissions
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2F2f57c0eda913e6ce17a655fb6a06161f%2Fimg_4.png?generation=1663921074450105&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2Fe795dcd4552bf563af59928437e28706%2Fimg_3.png?generation=1663921086356023&alt=media\")
We used the tiling approach for lung training, which we found to be very robust. The images were first resized to 1024 size. After, the image batch got tiled the convolution way with 0.5 overlaps between neighboring tiles.
\nWe also try randomly sampling 6 out of 9 tiles for data variability per mini-batch. Thus, we result in a bigger batch size with smaller amount of tiles per tissue slice. However, the LB standing didn't change. Indeed, it stayed the same.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2F889525186f8cb99f1b7cfe2b4f6ab13a%2Fimg_7.png?generation=1663921988920443&alt=media\")
For the inference, we rescale the images normalized to tissue slice thickness of size 1024 and use full images for predictions.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2F006a4459b0c93626ddaa4078da8550f5%2Fimg_8.png?generation=1663921968542680&alt=media\")
After reviewing all the lung masks and corresponding images, we concluded that several annotators might have done annotations.
\nThus, we decide to cluster the lung set into two groups with similar annotations styles. We find the following:
\ntype1_anns = [10488, 11064, 11629, 1220, 12827, 15067, 15329, 1731, 1878,
\n 20563, 24782, 26480, 27232, 2793, 28052, 28189, 28429, 30084,
\n 31800, 32151, 4301, 4412, 4776, 5086, 5552, 8231, 8343, 9387,
\n 9450, 12452, 14388, 25516, 30394, 5777, 686]
type2_anns = [12476, 15124, 23252, 25945, 30500, 31571,
\n 7359, 8151, 127, 13189, 16564, 29610, 31139]
type3_anomaly_anns = [30084, 31800]
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2Fda5003ee2d2c568ead11f7da1fbbe0f0%2Fimg.png?generation=1663920807141538&alt=media\")
Here we tried fixing the morphological structure for kidney masks. Instances of kidney masks are pretty separate from each other. Thus, we expected not to have many problems with watershed segmentation. If the instances result in touching points at the inference stage, we apply binary opening with disk size 5 and opening with disk size 10 operations to remove the extra connected parts if present.
\nThis is one of the essential parts of the prediction side. We notice that hand-picking thresholds might result in some inconsistencies for some predictions.
\nThus, we present the adaptive way for thresholding a probability map. We find that models indeed can produce an accuracy measure:
Private/Public: 0.81/0.81 (many models ensemble + thresholding strategy introduce good metric stability :))
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6468176%2Ff57edc4ec940697aed0303a5dd7a1570%2Fimg_6.png?generation=1663921344885286&alt=media\")
Dedicated to the Armed Forces of Ukraine!
\nGlory to the Armed Forces of Ukraine for the opportunity to provide science and education for the benefit of Ukraine!
Hello everyone from the complex team!
\nFirst of all, I would like to give my huge thank you to the Kaggle team for hosting such an amazing competition.
\nEvery time, I feel honored to be a part of such outstanding research.
\nI want to give a big thank you from our team to @hengck23. We have learned so much from your notebooks :)
\nIt is always a pleasure to have this great of a competitor from whom you can learn!
Our submission is a result of a few stages. We don't mainly do any pre-processing of the data. Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]
\nAfter many experiments with training all organs with one model, we gave up this idea. The reason was we didn't see any improvement with it after 0.77.
\nThus, we decided that the model needed contextual information about the organ type. The first experiments were set up with additional aux classification head for classifying each organ type. However, this became a false path after all; the models couldn't converge, and scores were not getting better.
Then, we decided to train separate models for the lung and the rest of the organs together. We adopted this strategy and proceeded with training the models in different conditions:
\nWe ensemble 6 models with different weights applied (5 folds per model) based on the best local cv scores. Plus, only one model for the lungs.
\n
Before training, we resized tissue slices to 768 and trained using strong augmentation.
\nWe find that intense color shifts and dropout noise produce good results.
_transforms = A.Compose([\n A.VerticalFlip(p=0.5),\n A.HorizontalFlip(p=0.5),\n A.RandomRotate90(p=0.5),\n\n A.RandomBrightnessContrast(brightness_limit=(-0.02, 0.02), contrast_limit=(-0.02, 0.02), p=0.25),\n A.RGBShift(r_shift_limit=(-25, 25), g_shift_limit=(-20, 20), b_shift_limit=(-20, 20), p=0.5),\n A.HueSaturationValue(hue_shift_limit=(-15, 15), sat_shift_limit=(-15, 15), val_shift_limit=(-15, 15), p=0.5),\n A.CoarseDropout(max_holes=20, min_holes=7, p=0.5),\n A.ShiftScaleRotate(shift_limit=0, scale_limit=(-0.2, 0.2), rotate_limit=0, p=0.5),\n A.OneOf([\n A.GridDistortion(num_steps=5, distort_limit=0.05, p=1.0),\n A.ElasticTransform(alpha=1, sigma=50, alpha_affine=50, p=1.0)\n ], p=0.5),\n])\nThe resulting pipeline is a model zoo :)
\nWe use pretrained ConvNext and HorNet backbone architectures. For the decoder block, we only use UPerNet.
\nBest HorNet Single Model Submissions
\n
\n
Best ConvNext Single Model Submissions
\n
\n
We used the tiling approach for lung training, which we found to be very robust. The images were first resized to 1024 size. After, the image batch got tiled the convolution way with 0.5 overlaps between neighboring tiles.
\nWe also try randomly sampling 6 out of 9 tiles for data variability per mini-batch. Thus, we result in a bigger batch size with smaller amount of tiles per tissue slice. However, the LB standing didn't change. Indeed, it stayed the same.
For the inference, we rescale the images normalized to tissue slice thickness of size 1024 and use full images for predictions.
\n
After reviewing all the lung masks and corresponding images, we concluded that several annotators might have done annotations.
\nThus, we decide to cluster the lung set into two groups with similar annotations styles. We find the following:
\ntype1_anns = [10488, 11064, 11629, 1220, 12827, 15067, 15329, 1731, 1878,
\n 20563, 24782, 26480, 27232, 2793, 28052, 28189, 28429, 30084,
\n 31800, 32151, 4301, 4412, 4776, 5086, 5552, 8231, 8343, 9387,
\n 9450, 12452, 14388, 25516, 30394, 5777, 686]
type2_anns = [12476, 15124, 23252, 25945, 30500, 31571,
\n 7359, 8151, 127, 13189, 16564, 29610, 31139]
type3_anomaly_anns = [30084, 31800]
\n
Here we tried fixing the morphological structure for kidney masks. Instances of kidney masks are pretty separate from each other. Thus, we expected not to have many problems with watershed segmentation. If the instances result in touching points at the inference stage, we apply binary opening with disk size 5 and opening with disk size 10 operations to remove the extra connected parts if present.
\nThis is one of the essential parts of the prediction side. We notice that hand-picking thresholds might result in some inconsistencies for some predictions.
\nThus, we present the adaptive way for thresholding a probability map. We find that models indeed can produce an accuracy measure:
Private/Public: 0.81/0.81 (many models ensemble + thresholding strategy introduce good metric stability :))
\n
Congratulations, really good summary! can't wait to see the code
", "rawMarkdown": "Congratulations, really good summary! can't wait to see the code", "votes": null }, { "id": "1972522", "postDate": "10/05/2022 07:37:45", "content": "Thank you sharing your great solutions!!
\n\n\nOur submission is a result of a few stages. We don't mainly do any pre-processing of the data. Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]
\n
Is this mean that you don't care too much about domain gap between HPA and HubMAP (for example resolution and color)?
", "rawMarkdown": "Thank you sharing your great solutions!!\n\n> Our submission is a result of a few stages. **We don't mainly do any pre-processing of the data.** Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]\n\nIs this mean that you don't care too much about domain gap between HPA and HubMAP (for example resolution and color)?", "votes": null } ], "comments": [ { "id": 1951995, "author_name": "mohamedaminesaighi", "author_url": "", "post_date": "09/23/2022 11:32:51", "content": "Congratulations, really good summary! can't wait to see the code
", "votes": null, "replies": [] }, { "id": 1972522, "author_name": "yoshoo", "author_url": "", "post_date": "10/05/2022 07:37:45", "content": "Thank you sharing your great solutions!!
\n\n\nOur submission is a result of a few stages. We don't mainly do any pre-processing of the data. Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]
\n
Is this mean that you don't care too much about domain gap between HPA and HubMAP (for example resolution and color)?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1951835": "*Dedicated to the Armed Forces of Ukraine!*\n*Glory to the Armed Forces of Ukraine for the opportunity to provide science and education for the benefit of Ukraine!*\n\n----\n\nHello everyone from the complex team! \nFirst of all, I would like to give my huge thank you to the Kaggle team for hosting such an amazing competition. \nEvery time, I feel honored to be a part of such outstanding research.\nI want to give a big thank you from our team to @hengck23. We have learned so much from your notebooks :)\nIt is always a pleasure to have this great of a competitor from whom you can learn!\n\n\nOur submission is a result of a few stages. We don't mainly do any pre-processing of the data. Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]\n\n \n# Overview\n\nAfter many experiments with training all organs with one model, we gave up this idea. The reason was we didn't see any improvement with it after 0.77. \nThus, we decided that the model needed contextual information about the organ type. The first experiments were set up with additional aux classification head for classifying each organ type. However, this became a false path after all; the models couldn't converge, and scores were not getting better.\n\nThen, we decided to train separate models for the lung and the rest of the organs together. We adopted this strategy and proceeded with training the models in different conditions:\n- we use full masks resized to 768 for ['kidney', 'largeintestine', 'lungs', 'prostate', 'spleen'], + some models were trained with lung excluded from the train set\n- for lungs, we chose only type1 masked lungs [Lung Segmentation Section] and pre-processed the mask batch by resizing it first to 1024 and then tiling with kernel size 512 and stride 2 \n\nWe ensemble 6 models with different weights applied (5 folds per model) based on the best local cv scores. Plus, only one model for the lungs.\n\n\n\n# Segmentation \n\n\n\nBefore training, we resized tissue slices to 768 and trained using strong augmentation. \nWe find that intense color shifts and dropout noise produce good results. \n\n```python\n_transforms = A.Compose([\n A.VerticalFlip(p=0.5),\n A.HorizontalFlip(p=0.5),\n A.RandomRotate90(p=0.5),\n\n A.RandomBrightnessContrast(brightness_limit=(-0.02, 0.02), contrast_limit=(-0.02, 0.02), p=0.25),\n A.RGBShift(r_shift_limit=(-25, 25), g_shift_limit=(-20, 20), b_shift_limit=(-20, 20), p=0.5),\n A.HueSaturationValue(hue_shift_limit=(-15, 15), sat_shift_limit=(-15, 15), val_shift_limit=(-15, 15), p=0.5),\n A.CoarseDropout(max_holes=20, min_holes=7, p=0.5),\n A.ShiftScaleRotate(shift_limit=0, scale_limit=(-0.2, 0.2), rotate_limit=0, p=0.5),\n A.OneOf([\n A.GridDistortion(num_steps=5, distort_limit=0.05, p=1.0),\n A.ElasticTransform(alpha=1, sigma=50, alpha_affine=50, p=1.0)\n ], p=0.5),\n])\n```\n----\n**The resulting pipeline is a model zoo :)**\n\nWe use pretrained ConvNext and HorNet backbone architectures. For the decoder block, we only use UPerNet.\n\n- *Inside: After the competition, we find that HorNet with recursive gated convolutions for high-order spatial interactions and large receptive field (the 7x7 conv and global filter) gave significantly stable results for all organs!*\n\n### ConvNext vs HorNet\nBest HorNet Single Model Submissions\n\n\n\nBest ConvNext Single Model Submissions\n\n\n\n# Lung Segmentation | Tiles\n\nWe used the tiling approach for lung training, which we found to be very robust. The images were first resized to 1024 size. After, the image batch got tiled the convolution way with 0.5 overlaps between neighboring tiles. \nWe also try randomly sampling 6 out of 9 tiles for data variability per mini-batch. Thus, we result in a bigger batch size with smaller amount of tiles per tissue slice. However, the LB standing didn't change. Indeed, it stayed the same.\n\n\n\nFor the inference, we rescale the images normalized to tissue slice thickness of size 1024 and use full images for predictions.\n\n\n\n### type1 masked lungs\nAfter reviewing all the lung masks and corresponding images, we concluded that several annotators might have done annotations. \n\nThus, we decide to cluster the lung set into two groups with similar annotations styles. We find the following:\n\n\n***type1_anns** = [10488, 11064, 11629, 1220, 12827, 15067, 15329, 1731, 1878,\n 20563, 24782, 26480, 27232, 2793, 28052, 28189, 28429, 30084,\n 31800, 32151, 4301, 4412, 4776, 5086, 5552, 8231, 8343, 9387,\n 9450, 12452, 14388, 25516, 30394, 5777, 686]*\n\n***type2_anns** = [12476, 15124, 23252, 25945, 30500, 31571,\n 7359, 8151, 127, 13189, 16564, 29610, 31139]*\n\n***type3_anomaly_anns** = [30084, 31800]*\n\n\n\n\n# Post-Processing | Refine Mask\n\nHere we tried fixing the morphological structure for kidney masks. Instances of kidney masks are pretty separate from each other. Thus, we expected not to have many problems with watershed segmentation. If the instances result in touching points at the inference stage, we apply binary opening with disk size 5 and opening with disk size 10 operations to remove the extra connected parts if present. \n\n\n# Adaptive Thresholding\n\nThis is one of the essential parts of the prediction side. We notice that hand-picking thresholds might result in some inconsistencies for some predictions.\nThus, we present the adaptive way for thresholding a probability map. We find that models indeed can produce an accuracy measure: \n- We calculate the mean of the models' prediction mask and set it as our threshold. We additionally divide it by 1.75 constant to lower the threshold step. We find that this constant is as good for kidney predictions as it is suitable for lungs.\n- Additionally, assuming every mask can't be empty, we utilize a linear descent in the threshold by the factor of 0.2 if any segmentation is found to be empty.\n\n# LB Scores\n\nPrivate/Public: 0.81/0.81 (many models ensemble + thresholding strategy introduce good metric stability :))\n\n### Ideas we didn't try:\n- Pseudo-labeling/SSL (unfortunately, we didn't have time to try this approach; it seems to be a game changer for many teams)\n- Cleaning masks before training (hole filling)\n\n\n\n# **Glory to Ukraine! Glory to Heroes! πΊπ¦**", "1951995": "Congratulations, really good summary! can't wait to see the code", "1972522": "Thank you sharing your great solutions!!\n\n> Our submission is a result of a few stages. **We don't mainly do any pre-processing of the data.** Although, at the very end of the competition, we found a lack in annotations that could have caused a slight weakening of masks and added noise to the training [read about it at the end of the overview]\n\nIs this mean that you don't care too much about domain gap between HPA and HubMAP (for example resolution and color)?" }, "source": "meta" }