This was my first for computer vision project, and I joined the competition quite late (two weeks before the deadline!). While my highest ranking was 122, it unfortunately dropped to 302 due to inconsistencies between private and public scores. Despite the rank drop, I gained valuable insights thanks to the shared kernels (Special thanks to @misakimatsutomo). Congratulations to everyone who achieved great results!
\nI've shared my notebook for anyone interested:
\n[Infer] Ensembles of three models with weights
\n[Train] Segmentation mask model
I'd be happy to receive any feedback!
\npatterning of blood vessels in human tissue.This notebook loads a pre-trained CNN segmentation model and predicts the run-length encoded (RLE) masks for each test image. RLE masks represent segmentation results for 3D images of human kidneys.
\nThis notenook leverages an ensemble consisting of a large 1024x1024 model and two smaller 512x512 models. The large model, trained on larger 1024x1024 images, receives a higher weight of 0.75. Conversely, the smaller models, trained on single-channel 512x512 images, have a lower weight of 0.25. This weighting scheme adjusts the individual predictions from each model as follows:
\npredictions = 0.75 * large-model + 0.25 * (0.8 * small-model + 0.2 * small-model2)
The predictions are subsequently adjusted using three color channels: channel 0 with weight 0.3288, channel 1 with weight 0.3366, and channel 2 with weight 0.3346.
\nThe binary prediction threshold is set based on the 0.0014109 percentile of predicted results, as mentioned in forum discussions due to its sensitive nature and potential impact on scoring. This choice of threshold has been identified as a potential factor contributing to the unexpected shakeup between private and public scores.
\nThis notebook utilizes one large (1024x1024) and two smaller (512x512) segmentation models. Two of these models (the large one and one of the smaller ones) were adapted from publicly shared models found in the discussions. The third, smaller model was specifically trained using this notebook.: [Train] Segmentation Mask
All three models employed the Unet architecture with se_resnext50_32x4d backbone, as discussed in the forum. Prior to training, model weights were initialized using ImageNet. The dataset underwent diverse data augmentation (rotation, scaling, cropping, and other techniques like GaussianBlur and adding noise), and splitted into training (kidney_1_dense) and validation (kidney_3_sparse + kidney_3_dense) sets.
Training loops lasted for 30 epochs using a OneCycleLR scheduler with an initial learning rate of 6e-5. Model performance was evaluated using the surface Dice metric, and the model with the lowest loss was saved for submission.
\nI tried out three different models with varying weights to figure out which one worked best.
\n| Predictions | \nPublic Score | \nPrivate Score | \n
|---|---|---|
0.75* large-model + 0.25 * (0.8 * small-model + 0.2 * small-model2) | \n0.86 | \n0.458 | \n
0.75* large-model + 0.25 * (0.9 * small-model + 0.1 * small-model) | \n0.86 | \n0.457 | \n
0.75* large-model + 0.25 * small-model | \n0.86 | \n0.455 | \n
0.75* large-model + 0.25 * small-model2 | \n0.86 | \n0.455 | \n
The weighted ensemble of three models achieved the best performance!
\nData argumentation:
\nMy solution utilized random data rotation for augmentation. However, the top-performing solution within this task suggest that 3D rotations generating non-axis-parallel slices yield greater effectiveness.
Loss function:
\nMy approach used a pure Dice loss function, while a proposed custom loss function incorporates 1.0 focal loss, 1.0 dice loss, 0.01 boundary loss, and a custom loss component (weight = 1.0). This revised loss function addresses several issues observed during training:
\n1) Fluctuations in the surface Dice metric.
\n2) Poor performance in 3D and 2.5D.
\n3) Poor confidence estimates for smaller vessels.
Model architecure:
\nWhile I selected the U-Net architecture with SE-ResNeXt50_32x4d as the backbone, utilizing only one channel may have contributed to overestimated scores. Notably, the top-performing solution from SMP employed an ensemble of two 2.5D ConvNeXt Tiny U-Nets with 3 channels, demonstrating consistent performance across public and private leaderboards.
Reference
\nThanks for sharing your solution. I also tried ensemble of three models. But the notebook exceeded the 9hr time limit and I could not succeed. Can you share how you managed to run 3 models in the 9hr limit/
", "rawMarkdown": "Thanks for sharing your solution. I also tried ensemble of three models. But the notebook exceeded the 9hr time limit and I could not succeed. Can you share how you managed to run 3 models in the 9hr limit/", "votes": 1, "replies": [ { "id": 2645449, "postDate": "2024-02-10T08:35:56.767Z", "content": "Indeed the primary challenge of ensemble models was the lengthy runtime.
\nTo address this, I minimized memory usage, restricted the weighted ensemble to a single axis, and leveraged GPUs for performance gains. Despite optimizations, processing all test data still required over 8 hours.
", "rawMarkdown": "Indeed the primary challenge of ensemble models was the lengthy runtime.\n\nTo address this, I minimized memory usage, restricted the weighted ensemble to a single axis, and leveraged GPUs for performance gains. Despite optimizations, processing all test data still required over 8 hours." } ] }, { "id": 2643493, "postDate": "2024-02-08T21:24:45.903Z", "content": "This was my first for computer vision project, and I joined the competition quite late (two weeks before the deadline!). While my highest ranking was 122, it unfortunately dropped to 302 due to inconsistencies between private and public scores. Despite the rank drop, I gained valuable insights thanks to the shared kernels (Special thanks to @misakimatsutomo). Congratulations to everyone who achieved great results!
\nI've shared my notebook for anyone interested:
\n[Infer] Ensembles of three models with weights
\n[Train] Segmentation mask model
I'd be happy to receive any feedback!
\npatterning of blood vessels in human tissue.This notebook loads a pre-trained CNN segmentation model and predicts the run-length encoded (RLE) masks for each test image. RLE masks represent segmentation results for 3D images of human kidneys.
\nThis notenook leverages an ensemble consisting of a large 1024x1024 model and two smaller 512x512 models. The large model, trained on larger 1024x1024 images, receives a higher weight of 0.75. Conversely, the smaller models, trained on single-channel 512x512 images, have a lower weight of 0.25. This weighting scheme adjusts the individual predictions from each model as follows:
\npredictions = 0.75 * large-model + 0.25 * (0.8 * small-model + 0.2 * small-model2)
The predictions are subsequently adjusted using three color channels: channel 0 with weight 0.3288, channel 1 with weight 0.3366, and channel 2 with weight 0.3346.
\nThe binary prediction threshold is set based on the 0.0014109 percentile of predicted results, as mentioned in forum discussions due to its sensitive nature and potential impact on scoring. This choice of threshold has been identified as a potential factor contributing to the unexpected shakeup between private and public scores.
\nThis notebook utilizes one large (1024x1024) and two smaller (512x512) segmentation models. Two of these models (the large one and one of the smaller ones) were adapted from publicly shared models found in the discussions. The third, smaller model was specifically trained using this notebook.: [Train] Segmentation Mask
All three models employed the Unet architecture with se_resnext50_32x4d backbone, as discussed in the forum. Prior to training, model weights were initialized using ImageNet. The dataset underwent diverse data augmentation (rotation, scaling, cropping, and other techniques like GaussianBlur and adding noise), and splitted into training (kidney_1_dense) and validation (kidney_3_sparse + kidney_3_dense) sets.
Training loops lasted for 30 epochs using a OneCycleLR scheduler with an initial learning rate of 6e-5. Model performance was evaluated using the surface Dice metric, and the model with the lowest loss was saved for submission.
\nI tried out three different models with varying weights to figure out which one worked best.
\n| Predictions | \nPublic Score | \nPrivate Score | \n
|---|---|---|
0.75* large-model + 0.25 * (0.8 * small-model + 0.2 * small-model2) | \n0.86 | \n0.458 | \n
0.75* large-model + 0.25 * (0.9 * small-model + 0.1 * small-model) | \n0.86 | \n0.457 | \n
0.75* large-model + 0.25 * small-model | \n0.86 | \n0.455 | \n
0.75* large-model + 0.25 * small-model2 | \n0.86 | \n0.455 | \n
The weighted ensemble of three models achieved the best performance!
\nData argumentation:
\nMy solution utilized random data rotation for augmentation. However, the top-performing solution within this task suggest that 3D rotations generating non-axis-parallel slices yield greater effectiveness.
Loss function:
\nMy approach used a pure Dice loss function, while a proposed custom loss function incorporates 1.0 focal loss, 1.0 dice loss, 0.01 boundary loss, and a custom loss component (weight = 1.0). This revised loss function addresses several issues observed during training:
\n1) Fluctuations in the surface Dice metric.
\n2) Poor performance in 3D and 2.5D.
\n3) Poor confidence estimates for smaller vessels.
Model architecure:
\nWhile I selected the U-Net architecture with SE-ResNeXt50_32x4d as the backbone, utilizing only one channel may have contributed to overestimated scores. Notably, the top-performing solution from SMP employed an ensemble of two 2.5D ConvNeXt Tiny U-Nets with 3 channels, demonstrating consistent performance across public and private leaderboards.
Reference
\nThanks for sharing your solution. I also tried ensemble of three models. But the notebook exceeded the 9hr time limit and I could not succeed. Can you share how you managed to run 3 models in the 9hr limit/
", "votes": 1, "replies": [ { "id": 2645449, "author_name": "Min-Hsien Weng", "author_url": "", "post_date": "2024-02-10T08:35:56.767000", "content": "Indeed the primary challenge of ensemble models was the lengthy runtime.
\nTo address this, I minimized memory usage, restricted the weighted ensemble to a single axis, and leveraged GPUs for performance gains. Despite optimizations, processing all test data still required over 8 hours.
", "votes": 0, "replies": [] } ] }, { "id": 2645448, "author_name": "", "author_url": "", "post_date": "2024-02-10T08:35:37.247000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2645364": "Thanks for sharing your solution. I also tried ensemble of three models. But the notebook exceeded the 9hr time limit and I could not succeed. Can you share how you managed to run 3 models in the 9hr limit/", "2643493": "# Introduction\nThis was my first for computer vision project, and I joined the competition quite late (two weeks before the deadline!). While my highest ranking was 122, it unfortunately dropped to 302 due to inconsistencies between private and public scores. Despite the rank drop, I gained valuable insights thanks to the shared kernels (Special thanks to @misakimatsutomo). Congratulations to everyone who achieved great results!\n\nI've shared my notebook for anyone interested: \n[[Infer] Ensembles of three models with weights](https://www.kaggle.com/code/minhsienweng/infer-ensembles-of-three-models-with-weights)\n[[Train] Segmentation mask model](https://www.kaggle.com/minhsienweng/train-segmentation-mask)\n\nI'd be happy to receive any feedback!\n\n## Competition \n- **Goal:** Develop a model to segment blood vessels in 3D scans of human kidneys (HiP-CT data).\n- **Impact:** Help researchers understand the size, shape, branching angles, and `patterning` of blood vessels in human tissue.\n- **Competition link**: [SenNet + HOA - Hacking the Human Vasculature in 3D](https://www.kaggle.com/competitions/blood-vessel-segmentation)\n\n## Solution summary\nThis notebook loads a pre-trained CNN segmentation model and predicts the run-length encoded (RLE) masks for each test image. RLE masks represent segmentation results for 3D images of human kidneys.\n\nThis notenook leverages an ensemble consisting of a large 1024x1024 model and two smaller 512x512 models. The large model, trained on larger 1024x1024 images, receives a higher weight of 0.75. Conversely, the smaller models, trained on single-channel 512x512 images, have a lower weight of 0.25. This weighting scheme adjusts the individual predictions from each model as follows:\n\n`predictions = 0.75 * large-model + 0.25 * (0.8 * small-model + 0.2 * small-model2)`\n\nThe predictions are subsequently adjusted using three color channels: channel 0 with weight 0.3288, channel 1 with weight 0.3366, and channel 2 with weight 0.3346.\n\nThe **binary prediction threshold** is set based on the 0.0014109 percentile of predicted results, as mentioned in forum discussions due to its sensitive nature and potential impact on scoring. This choice of threshold has been identified as a potential factor contributing to the unexpected shakeup between private and public scores.\n\n## Models\nThis notebook utilizes one `large` (1024x1024) and two `smaller` (512x512) segmentation models. Two of these models (the large one and one of the smaller ones) were adapted from publicly shared models found in the discussions. The third, smaller model was specifically trained using this notebook.: [[Train] Segmentation Mask](https://www.kaggle.com/minhsienweng/train-segmentation-mask)\n\nAll three models employed the `Unet` architecture with `se_resnext50_32x4d` backbone, as discussed in the forum. Prior to training, model weights were initialized using ImageNet. The dataset underwent diverse data augmentation (rotation, scaling, cropping, and other techniques like GaussianBlur and adding noise), and splitted into training (`kidney_1_dense`) and validation (`kidney_3_sparse` + `kidney_3_dense`) sets.\n\nTraining loops lasted for 30 epochs using a OneCycleLR scheduler with an initial learning rate of 6e-5. Model performance was evaluated using the surface Dice metric, and the model with the lowest loss was saved for submission.\n\n## Experiement results\nI tried out three different models with varying weights to figure out which one worked best.\n\n| Predictions | Public Score | Private Score |\n| --- | --- | --- |\n| `0.75* large-model + 0.25 * (0.8 * small-model + 0.2 * small-model2)` | 0.86 | **0.458** | \n| `0.75* large-model + 0.25 * (0.9 * small-model + 0.1 * small-model)` | 0.86 | 0.457 | \n| `0.75* large-model + 0.25 * small-model` | 0.86 | 0.455 |\n| `0.75* large-model + 0.25 * small-model2` | 0.86 | 0.455 |\n\nThe weighted ensemble of three models achieved the best performance!\n\n# Ways to level up\n- Data argumentation: \nMy solution utilized random data rotation for augmentation. However, the top-performing solution within this task suggest that 3D rotations generating **non-axis-parallel slices** yield greater effectiveness.\n\n- Loss function: \nMy approach used a pure Dice loss function, while a proposed custom loss function incorporates 1.0 focal loss, 1.0 dice loss, 0.01 boundary loss, and a custom loss component (weight = 1.0). This revised loss function addresses several issues observed during training:\n1) Fluctuations in the surface Dice metric.\n2) Poor performance in 3D and 2.5D.\n3) Poor confidence estimates for smaller vessels.\n\n- Model architecure: \nWhile I selected the U-Net architecture with SE-ResNeXt50_32x4d as the backbone, utilizing only one channel may have contributed to overestimated scores. Notably, the top-performing solution from SMP employed an ensemble of two 2.5D ConvNeXt Tiny U-Nets with 3 channels, demonstrating consistent performance across public and private leaderboards.\n\n\n**Reference**\n- [1st place solution](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/475522)\n- [Inference 1024 should have a percentile of 0.00149](https://www.kaggle.com/code/misakimatsutomo/inference-1024-should-have-a-percentile-of-0-00149)\n- [Clean Code 📚| Weighted Ensemble [Inference]](https://www.kaggle.com/code/bhavyadhingra00020/clean-code-weighted-ensemble-inference)\n", "2645448": "" } }