![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4210346%2Fa083b64e1f0a236f4847a75decccc4c4%2FBiological%20Context%20Feature-2026-07-30-155424.png?generation=1785426902397067&alt=media\")
My day job is bioimage analysis — building segmentation pipelines for microscopy data. When I saw this competition's task (detecting papyrus surfaces in 3D CT scans), it looked a lot like what I do at work: find structures in noisy volumetric data. So I borrowed a tool from that world.
\nBest submission: 0.582 private LB, 113th place
\n
Postprocessing was deliberately simple: hysteresis thresholding (T_low 0.30, T_high 0.90), z-closing (radius 1), dust removal (min_size 100) — applied to the averaged probability maps. It beat every tuned alternative I tried, including an Optuna-optimised version and the winning team's pipeline.
\nIn figures above, you can see that WNet3D (100 epoch) predicts:
\nIn this competition I tuned WNet to find sheet-like and larger structures, but with more classes and careful tuning of the SoftNCuts loss parameters (intensity sigma, spatial sigma, radius), I believe WNet could learn to separate ink from sheet from background. The ink channel could then be fed into nnUNet as an additional input, giving the segmentation model explicit ink-awareness — potentially helping in regions where ink and sheet are hard to distinguish.
\nOther ideas I didn't have time to explore:
\nIf anyone is interested in exploring the WNet preprocessing direction, happy to chat. @seanjohnsonsp @giorgioangelotti
\nHere's every submission I made during the competition, roughly in order. The story is one of diminishing returns from model tweaks and hard-won lessons about what actually moves the needle.
\n| Submission | \nDescription | \nPublic LB | \nPrivate LB | \n
|---|---|---|---|
| Dataset321 | \nnnUNet on first half of training data, WNet20 preprocessor | \n0.553 | \n0.566 | \n
| Dataset322 | \nnnUNet on second half of training data, WNet20 preprocessor | \n0.549 | \n0.564 | \n
| 321+322 ensemble | \n50/50 average of the two half-data models | \n0.556 | \n0.576 | \n
| Dataset323 | \nnnUNet on full data, WNet100 preprocessor, DC+CE loss | \n0.563 | \n0.579 | \n
| Dataset324 | \nnnUNet on full data, WNet20 preprocessor, DC+CE loss | \n— | \n0.579 | \n
| Dataset325 | \nnnUNet on full data, WNet100 preprocessor, BoundaryDoULoss, cleaned GT | \n0.553 | \n0.575 | \n
| 323 + TransUNet | \n75/25 nnUNet/TransUNet ensemble | \n0.558 | \n0.579 | \n
| 323 + Optuna PP | \nOptuna-optimised postprocessing on best solo model | \n— | \n0.577 | \n
| 324+325 ensemble | \n0.5/0.5 ensemble, different WNet checkpoints + different GT | \n0.557 | \n0.582 | \n
| 325 + winning PP | \nDataset325 with 1st place postprocessing (post-competition test) | \n— | \n0.582 | \n
A few things stand out from this table. The half-data models (321, 322) are surprisingly competitive, only ~0.013 behind the full-data model (323), which suggests the training set has a lot of redundancy. The best ensemble (324+325) only edges out the best solo by 0.003, but it required genuine diversity: different WNet checkpoints (20 vs 100 epochs) and different ground truth versions. Models that were too similar (e.g. 323+TransUNet) just averaged their shared mistakes.
\nThe public/private LB gap is also notable. The best public score (0.563, Dataset323) and best private score (0.582, 324+325 ensemble) came from different submissions entirely. If I'd been selecting on public LB alone, I'd have picked the wrong model.
\nIt's worth looking at where the gap came from:
\n| \n | Ours (113th) | \n1st Place | \n
|---|---|---|
| Backbone | \nnnUNet ResEncUNetXL | \nnnUNet (4 models) | \n
| Patch sizes | \nSingle | \n128, 160, 192, 224, 256 | \n
| Preprocessing | \nWNet3D (self-supervised) | \nRaw CT | \n
| Postprocessing | \nSimple hysteresis | \nHeight-map patching, LUT hole repair, oriented closing | \n
| GT handling | \nManual cleaning | \nStandard | \n
| Private LB | \n0.582 | \n0.627 | \n
The ~0.045 gap is substantial, and it came primarily from their topology-aware postprocessing and multi-scale ensemble diversity (five different patch sizes). Both solutions validated nnUNet as the right backbone we invested in preprocessing, they invested in postprocessing and model diversity. Crucially, when I applied their postprocessing to our model post-competition, it scored 0.582, which is identical to our simple approach. Their postprocessing was co-optimised with their specific ensemble's error profile and didn't transfer.
\nnnUNet configuration: ResEncUNetXL backbone, 3d_fullres, trained with the standard nnUNet pipeline. Dataset324 used the default Dice + Cross-Entropy loss. Dataset325 used BoundaryDoULoss3D, adapted from Igor Krashenyi's 4th place solution in the SenNet+HOA blood vessel segmentation competition (original repo). This boundary-aware loss was designed to improve segmentation at tissue boundaries — a natural fit for sheet detection.
\nWNet3D: From CellSeg3D (Achard et al., eLife 2025). Trained two checkpoints — 20 epochs (patch 64) and 100 epochs (patch 128) — on all raw CT training data in a fully self-supervised way (no labels required). The 20 epoch was my first attempt, and then when I got access to more compute, I trained one with larger patch size for more epochs. The encoder's intermediate feature maps (3 channels) were concatenated with the raw CT to form a 4-channel input for nnUNet. Using two different WNet checkpoints for the two nnUNet models was key to ensemble diversity.
\nInference: Sliding window with TTA (mirror axes 0, 1), tile_step_size=0.5. Initially used 1xP100 GPU, but I've learned to run both models run in parallel on Kaggle's 2×T4 GPUs.
\nPostprocessing: Hysteresis thresholding (T_low=0.30, T_high=0.90) to get connected high-confidence regions and grow them into lower-confidence voxels. Morphological z-closing (radius=1) to fill single-slice gaps. Dust removal (min connected component size=100) to clean up small fragments. Simple, conservative, and it generalised better than anything tuned.
\nThis was my first Kaggle competition and I learned a huge deal. My background in bioimage analysis turned out to be more transferable than I expected, i.e., volumetric segmentation is volumetric segmentation whether it's cells or scrolls, and tools like nnUNet and WNet3D crossed domains surprisingly well. At the same time, I learned just as much from reading other people's discussions and solutions. The community here is incredibly generous with sharing ideas, and I picked up techniques I'd never have found on my own.
\nNext time I'd love to work in a team. It seems like a great way to connect with people, learn faster, and push further than you can solo. I'd also aim to contribute more to the discussion forums during the competition rather than just lurking, it would be a great way to pay forward what others shared with me + can get input from many.
\nComing from biological and microscopy data, I'm especially appreciative of the effort that went into curating this dataset. Good ground truth is expensive and time-consuming to produce, and it's what makes competitions like this possible. Thanks to the organisers for putting it all together.
", "messages": [ { "id": 3416229, "postDate": "2026-03-02T10:51:44.720Z", "content": "My day job is bioimage analysis — building segmentation pipelines for microscopy data. When I saw this competition's task (detecting papyrus surfaces in 3D CT scans), it looked a lot like what I do at work: find structures in noisy volumetric data. So I borrowed a tool from that world.
\nBest submission: 0.582 private LB, 113th place
\n
Postprocessing was deliberately simple: hysteresis thresholding (T_low 0.30, T_high 0.90), z-closing (radius 1), dust removal (min_size 100) — applied to the averaged probability maps. It beat every tuned alternative I tried, including an Optuna-optimised version and the winning team's pipeline.
\nIn figures above, you can see that WNet3D (100 epoch) predicts:
\nIn this competition I tuned WNet to find sheet-like and larger structures, but with more classes and careful tuning of the SoftNCuts loss parameters (intensity sigma, spatial sigma, radius), I believe WNet could learn to separate ink from sheet from background. The ink channel could then be fed into nnUNet as an additional input, giving the segmentation model explicit ink-awareness — potentially helping in regions where ink and sheet are hard to distinguish.
\nOther ideas I didn't have time to explore:
\nIf anyone is interested in exploring the WNet preprocessing direction, happy to chat. @seanjohnsonsp @giorgioangelotti
\nHere's every submission I made during the competition, roughly in order. The story is one of diminishing returns from model tweaks and hard-won lessons about what actually moves the needle.
\n| Submission | \nDescription | \nPublic LB | \nPrivate LB | \n
|---|---|---|---|
| Dataset321 | \nnnUNet on first half of training data, WNet20 preprocessor | \n0.553 | \n0.566 | \n
| Dataset322 | \nnnUNet on second half of training data, WNet20 preprocessor | \n0.549 | \n0.564 | \n
| 321+322 ensemble | \n50/50 average of the two half-data models | \n0.556 | \n0.576 | \n
| Dataset323 | \nnnUNet on full data, WNet100 preprocessor, DC+CE loss | \n0.563 | \n0.579 | \n
| Dataset324 | \nnnUNet on full data, WNet20 preprocessor, DC+CE loss | \n— | \n0.579 | \n
| Dataset325 | \nnnUNet on full data, WNet100 preprocessor, BoundaryDoULoss, cleaned GT | \n0.553 | \n0.575 | \n
| 323 + TransUNet | \n75/25 nnUNet/TransUNet ensemble | \n0.558 | \n0.579 | \n
| 323 + Optuna PP | \nOptuna-optimised postprocessing on best solo model | \n— | \n0.577 | \n
| 324+325 ensemble | \n0.5/0.5 ensemble, different WNet checkpoints + different GT | \n0.557 | \n0.582 | \n
| 325 + winning PP | \nDataset325 with 1st place postprocessing (post-competition test) | \n— | \n0.582 | \n
A few things stand out from this table. The half-data models (321, 322) are surprisingly competitive, only ~0.013 behind the full-data model (323), which suggests the training set has a lot of redundancy. The best ensemble (324+325) only edges out the best solo by 0.003, but it required genuine diversity: different WNet checkpoints (20 vs 100 epochs) and different ground truth versions. Models that were too similar (e.g. 323+TransUNet) just averaged their shared mistakes.
\nThe public/private LB gap is also notable. The best public score (0.563, Dataset323) and best private score (0.582, 324+325 ensemble) came from different submissions entirely. If I'd been selecting on public LB alone, I'd have picked the wrong model.
\nIt's worth looking at where the gap came from:
\n| \n | Ours (113th) | \n1st Place | \n
|---|---|---|
| Backbone | \nnnUNet ResEncUNetXL | \nnnUNet (4 models) | \n
| Patch sizes | \nSingle | \n128, 160, 192, 224, 256 | \n
| Preprocessing | \nWNet3D (self-supervised) | \nRaw CT | \n
| Postprocessing | \nSimple hysteresis | \nHeight-map patching, LUT hole repair, oriented closing | \n
| GT handling | \nManual cleaning | \nStandard | \n
| Private LB | \n0.582 | \n0.627 | \n
The ~0.045 gap is substantial, and it came primarily from their topology-aware postprocessing and multi-scale ensemble diversity (five different patch sizes). Both solutions validated nnUNet as the right backbone we invested in preprocessing, they invested in postprocessing and model diversity. Crucially, when I applied their postprocessing to our model post-competition, it scored 0.582, which is identical to our simple approach. Their postprocessing was co-optimised with their specific ensemble's error profile and didn't transfer.
\nnnUNet configuration: ResEncUNetXL backbone, 3d_fullres, trained with the standard nnUNet pipeline. Dataset324 used the default Dice + Cross-Entropy loss. Dataset325 used BoundaryDoULoss3D, adapted from Igor Krashenyi's 4th place solution in the SenNet+HOA blood vessel segmentation competition (original repo). This boundary-aware loss was designed to improve segmentation at tissue boundaries — a natural fit for sheet detection.
\nWNet3D: From CellSeg3D (Achard et al., eLife 2025). Trained two checkpoints — 20 epochs (patch 64) and 100 epochs (patch 128) — on all raw CT training data in a fully self-supervised way (no labels required). The 20 epoch was my first attempt, and then when I got access to more compute, I trained one with larger patch size for more epochs. The encoder's intermediate feature maps (3 channels) were concatenated with the raw CT to form a 4-channel input for nnUNet. Using two different WNet checkpoints for the two nnUNet models was key to ensemble diversity.
\nInference: Sliding window with TTA (mirror axes 0, 1), tile_step_size=0.5. Initially used 1xP100 GPU, but I've learned to run both models run in parallel on Kaggle's 2×T4 GPUs.
\nPostprocessing: Hysteresis thresholding (T_low=0.30, T_high=0.90) to get connected high-confidence regions and grow them into lower-confidence voxels. Morphological z-closing (radius=1) to fill single-slice gaps. Dust removal (min connected component size=100) to clean up small fragments. Simple, conservative, and it generalised better than anything tuned.
\nThis was my first Kaggle competition and I learned a huge deal. My background in bioimage analysis turned out to be more transferable than I expected, i.e., volumetric segmentation is volumetric segmentation whether it's cells or scrolls, and tools like nnUNet and WNet3D crossed domains surprisingly well. At the same time, I learned just as much from reading other people's discussions and solutions. The community here is incredibly generous with sharing ideas, and I picked up techniques I'd never have found on my own.
\nNext time I'd love to work in a team. It seems like a great way to connect with people, learn faster, and push further than you can solo. I'd also aim to contribute more to the discussion forums during the competition rather than just lurking, it would be a great way to pay forward what others shared with me + can get input from many.
\nComing from biological and microscopy data, I'm especially appreciative of the effort that went into curating this dataset. Good ground truth is expensive and time-consuming to produce, and it's what makes competitions like this possible. Thanks to the organisers for putting it all together.
", "rawMarkdown": "My day job is bioimage analysis — building segmentation pipelines for microscopy data. When I saw this competition's task (detecting papyrus surfaces in 3D CT scans), it looked a lot like what I do at work: find structures in noisy volumetric data. So I borrowed a tool from that world. \n\nBest submission: **0.582 private LB, 113th place**\n\n## Pipeline\n\n\n\n[***>NOTEBOOK LINK***](https://kaggle.com/code/pr4deepr/wnet3d-nnunet-ensemble-125th-place) \n\nPostprocessing was deliberately simple: hysteresis thresholding (T_low 0.30, T_high 0.90), z-closing (radius 1), dust removal (min_size 100) — applied to the averaged probability maps. It beat every tuned alternative I tried, including an Optuna-optimised version and the winning team's pipeline.\n\n## The WNet Idea\n\n- Raw CT intensity alone is a limited input — subtle contrast between sheet and background, multiple overlapping layers\n- [WNet3D](https://elifesciences.org/articles/99848) is a dual 3D U-Net trained without labels via a self-supervised reconstruction objective\n- Originally designed for cell segmentation in cleared neural tissue (light-sheet microscopy)\n- We repurposed it as a learned feature extractor for scroll CT. WNet3D was configured to predict 3 output classes for larger structures with the hope that it would capture ink, background and a third class focusing on edges, ink or other structural features.\n- One channel focused on ink, whereas other channels focused on background, specifically big and small gaps between sheets.\n- All 3 feature maps were concatenated with raw CT → 4-channel input for nnUNet\n- Observed faster nnUNet training convergence with WNet input vs raw CT (preliminary tests only — no controlled LB ablation)\n\nIn figures above, you can see that WNet3D (100 epoch) predicts:\n- 1st channel: small gaps between sheets\n- 2nd channel: Sheets with bright spots which I believe are ink?\n- 3rd channel: Large gaps between sheet\n\n## What Worked\n\n- **WNet preprocessing** — faster convergence during nnUNet training. All submitted models used WNet-enriched input.\n- **GT cleaning** — manually corrected misalignments/noise in ground truth (Dataset325). Solo: 0.575. But critical ingredient in best ensemble (0.582).\n- **Sheet separation** — Sheet merging didn't seem to be a big problem (Visual QC), possibly due to WNet3D channels focusing on big and small gaps.\n- **Half the data is enough** — nnUNet models trained on 50% of training data scored 0.564–0.566. Full data: 0.579. Ensembling two half-data models: 0.576. The training set has substantial redundancy.\n- **Simple postprocessing** — hysteresis thresholding (T_low=0.30, T_high=0.90), z-closing (radius=1), dust removal (min_size=100). Beat every \"optimised\" alternative.\n- **Ensemble diversity via data** — best result (0.582) came from ensembling models with different preprocessor checkpoints (WNet20 vs WNet100) AND different GT versions. Half-data ensembles (321+322) also showed clear gains (+0.010 over best single). Models sharing the same preprocessor but differing only in GT gave less benefit.\n\n\n## What Didn't Work\n\n- **Optuna postprocessing** — Used Optuna to find best postprocessing parameters, but overfit to local validation. 0.577 vs 0.579 for hand-picked params on same model.\n- **Fragmentation problem** Fragmentation appeared to be the main issue and postprocessing needs to be optimized for this.\n- **[TransUNet](https://www.kaggle.com/models/ipythonx/vsd-model/Keras/transunet/4) ensemble** — Tried TransUNet model by @ipythonx at 25% weight with nnUNet 323: 0.579, identical to 323 solo. Correlated errors.\n- **Winning solution postprocessing on our model** — After the competition, I applied the winning postprocessing as-is and scored 0.582, same as our simple postprocessing. Their pipeline was co-optimised with their 4-model ensemble. Postprocessing is model-specific.\n- **Public LB for model selection** — best public (0.563) and best private (0.582) came from different submissions. Would have picked the wrong model.\n- **Local metrics** — best local topometric scores consistently corresponded to worse LB results. Train/test distribution mismatch.\n\n## Key Takeaways\n\n1. **Self-supervised preprocessing is underexplored** — WNet from microscopy transferred to scroll CT. No other team tried learned preprocessing.\n2. **Data quality > model complexity** — GT cleaning gave comparable gains to architectural changes.\n3. **You don't need all the data** — 50% data used with nnUNet got within 0.013 of full-data model. (Caveat: WNet was trained on all data.)\n4. **Simple postprocessing beats tuned parameters** — train/test distribution gap made local optimisation unreliable.\n5. **Postprocessing is model-specific** — winning team's PP didn't help our model.\n\n\n## Future Directions\n\n In this competition I tuned WNet to find sheet-like and larger structures, but with more classes and careful tuning of the SoftNCuts loss parameters (intensity sigma, spatial sigma, radius), I believe WNet could learn to separate ink from sheet from background. The ink channel could then be fed into nnUNet as an additional input, giving the segmentation model explicit ink-awareness — potentially helping in regions where ink and sheet are hard to distinguish.\n\nOther ideas I didn't have time to explore:\n\n- **Controlled WNet ablation** - train identical nnUNet models with and without WNet features to properly quantify the preprocessing contribution on the leaderboard, not just training convergence\n- **Multi-scale WNet features** - the winning solution gained a lot from multiple patch sizes. Extracting WNet features at different scales and stacking them could capture both fine ink texture and coarse sheet structure\n- **WNet-guided postprocessing** —-use WNet's class maps to inform topology repair, e.g. only fill holes in regions WNet classifies as sheet, or preserve gaps where WNet detects ink. It seems to detect large and small gaps between sheets\n- **More WNet output classes** — going beyond 3 classes to see if finer structural categories emerge (sheet interior, sheet boundary, ink, voids, noise)\n\nIf anyone is interested in exploring the WNet preprocessing direction, happy to chat. @seanjohnsonsp @giorgioangelotti \n\n\n## All Results\n\nHere's every submission I made during the competition, roughly in order. The story is one of diminishing returns from model tweaks and hard-won lessons about what actually moves the needle.\n\n| Submission | Description | Public LB | Private LB |\n|---|---|---|---|\n| Dataset321 | nnUNet on first half of training data, WNet20 preprocessor | 0.553 | 0.566 |\n| Dataset322 | nnUNet on second half of training data, WNet20 preprocessor | 0.549 | 0.564 |\n| 321+322 ensemble | 50/50 average of the two half-data models | 0.556 | 0.576 |\n| Dataset323 | nnUNet on full data, WNet100 preprocessor, DC+CE loss | 0.563 | 0.579 |\n| Dataset324 | nnUNet on full data, WNet20 preprocessor, DC+CE loss | — | 0.579 |\n| Dataset325 | nnUNet on full data, WNet100 preprocessor, BoundaryDoULoss, cleaned GT | 0.553 | 0.575 |\n| 323 + TransUNet | 75/25 nnUNet/TransUNet ensemble | 0.558 | 0.579 |\n| 323 + Optuna PP | Optuna-optimised postprocessing on best solo model | — | 0.577 |\n| **324+325 ensemble** | **0.5/0.5 ensemble, different WNet checkpoints + different GT** | **0.557** | **0.582** |\n| 325 + winning PP | Dataset325 with 1st place postprocessing (post-competition test) | — | 0.582 |\n\nA few things stand out from this table. The half-data models (321, 322) are surprisingly competitive, only ~0.013 behind the full-data model (323), which suggests the training set has a lot of redundancy. The best ensemble (324+325) only edges out the best solo by 0.003, but it required genuine diversity: different WNet checkpoints (20 vs 100 epochs) and different ground truth versions. Models that were too similar (e.g. 323+TransUNet) just averaged their shared mistakes.\n\nThe public/private LB gap is also notable. The best public score (0.563, Dataset323) and best private score (0.582, 324+325 ensemble) came from different submissions entirely. If I'd been selecting on public LB alone, I'd have picked the wrong model.\n\n## Comparison to Winning Solution (0.627)\n\nIt's worth looking at where the gap came from:\n\n| | Ours (113th) | 1st Place |\n|---|---|---|\n| Backbone | nnUNet ResEncUNetXL | nnUNet (4 models) |\n| Patch sizes | Single | 128, 160, 192, 224, 256 |\n| Preprocessing | WNet3D (self-supervised) | Raw CT |\n| Postprocessing | Simple hysteresis | Height-map patching, LUT hole repair, oriented closing |\n| GT handling | Manual cleaning | Standard |\n| Private LB | 0.582 | 0.627 |\n\nThe ~0.045 gap is substantial, and it came primarily from their topology-aware postprocessing and multi-scale ensemble diversity (five different patch sizes). Both solutions validated nnUNet as the right backbone we invested in preprocessing, they invested in postprocessing and model diversity. Crucially, when I applied their postprocessing to our model post-competition, it scored 0.582, which is identical to our simple approach. Their postprocessing was co-optimised with their specific ensemble's error profile and didn't transfer.\n\n## Technical Details\n\n**nnUNet configuration:** ResEncUNetXL backbone, 3d_fullres, trained with the standard nnUNet pipeline. Dataset324 used the default Dice + Cross-Entropy loss. Dataset325 used [BoundaryDoULoss3D](https://arxiv.org/pdf/2308.00220.pdf), adapted from [Igor Krashenyi's 4th place solution](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/igor-krashenyi-4th-place-solution-boundary-dou-los) in the SenNet+HOA blood vessel segmentation competition ([original repo](https://github.com/sunfan-bvb/BoundaryDoULoss)). This boundary-aware loss was designed to improve segmentation at tissue boundaries — a natural fit for sheet detection.\n\n**WNet3D:** From [CellSeg3D](https://elifesciences.org/articles/99848) (Achard et al., eLife 2025). Trained two checkpoints — 20 epochs (patch 64) and 100 epochs (patch 128) — on all raw CT training data in a fully self-supervised way (no labels required). The 20 epoch was my first attempt, and then when I got access to more compute, I trained one with larger patch size for more epochs. The encoder's intermediate feature maps (3 channels) were concatenated with the raw CT to form a 4-channel input for nnUNet. Using two different WNet checkpoints for the two nnUNet models was key to ensemble diversity.\n\n**Inference:** Sliding window with TTA (mirror axes 0, 1), tile_step_size=0.5. Initially used 1xP100 GPU, but I've learned to run both models run in parallel on Kaggle's 2×T4 GPUs. \n\n**Postprocessing:** Hysteresis thresholding (T_low=0.30, T_high=0.90) to get connected high-confidence regions and grow them into lower-confidence voxels. Morphological z-closing (radius=1) to fill single-slice gaps. Dust removal (min connected component size=100) to clean up small fragments. Simple, conservative, and it generalised better than anything tuned.\n\n\n## Reflections\n\nThis was my first Kaggle competition and I learned a huge deal. My background in bioimage analysis turned out to be more transferable than I expected, i.e., volumetric segmentation is volumetric segmentation whether it's cells or scrolls, and tools like nnUNet and WNet3D crossed domains surprisingly well. At the same time, I learned just as much from reading other people's discussions and solutions. The community here is incredibly generous with sharing ideas, and I picked up techniques I'd never have found on my own.\n\nNext time I'd love to work in a team. It seems like a great way to connect with people, learn faster, and push further than you can solo. I'd also aim to contribute more to the discussion forums during the competition rather than just lurking, it would be a great way to pay forward what others shared with me + can get input from many.\n\nComing from biological and microscopy data, I'm especially appreciative of the effort that went into curating this dataset. Good ground truth is expensive and time-consuming to produce, and it's what makes competitions like this possible. Thanks to the organisers for putting it all together.\n", "votes": 2 } ], "comments": [], "raw_markdown_by_id": { "3416229": "My day job is bioimage analysis — building segmentation pipelines for microscopy data. When I saw this competition's task (detecting papyrus surfaces in 3D CT scans), it looked a lot like what I do at work: find structures in noisy volumetric data. So I borrowed a tool from that world. \n\nBest submission: **0.582 private LB, 113th place**\n\n## Pipeline\n\n\n\n[***>NOTEBOOK LINK***](https://kaggle.com/code/pr4deepr/wnet3d-nnunet-ensemble-125th-place) \n\nPostprocessing was deliberately simple: hysteresis thresholding (T_low 0.30, T_high 0.90), z-closing (radius 1), dust removal (min_size 100) — applied to the averaged probability maps. It beat every tuned alternative I tried, including an Optuna-optimised version and the winning team's pipeline.\n\n## The WNet Idea\n\n- Raw CT intensity alone is a limited input — subtle contrast between sheet and background, multiple overlapping layers\n- [WNet3D](https://elifesciences.org/articles/99848) is a dual 3D U-Net trained without labels via a self-supervised reconstruction objective\n- Originally designed for cell segmentation in cleared neural tissue (light-sheet microscopy)\n- We repurposed it as a learned feature extractor for scroll CT. WNet3D was configured to predict 3 output classes for larger structures with the hope that it would capture ink, background and a third class focusing on edges, ink or other structural features.\n- One channel focused on ink, whereas other channels focused on background, specifically big and small gaps between sheets.\n- All 3 feature maps were concatenated with raw CT → 4-channel input for nnUNet\n- Observed faster nnUNet training convergence with WNet input vs raw CT (preliminary tests only — no controlled LB ablation)\n\nIn figures above, you can see that WNet3D (100 epoch) predicts:\n- 1st channel: small gaps between sheets\n- 2nd channel: Sheets with bright spots which I believe are ink?\n- 3rd channel: Large gaps between sheet\n\n## What Worked\n\n- **WNet preprocessing** — faster convergence during nnUNet training. All submitted models used WNet-enriched input.\n- **GT cleaning** — manually corrected misalignments/noise in ground truth (Dataset325). Solo: 0.575. But critical ingredient in best ensemble (0.582).\n- **Sheet separation** — Sheet merging didn't seem to be a big problem (Visual QC), possibly due to WNet3D channels focusing on big and small gaps.\n- **Half the data is enough** — nnUNet models trained on 50% of training data scored 0.564–0.566. Full data: 0.579. Ensembling two half-data models: 0.576. The training set has substantial redundancy.\n- **Simple postprocessing** — hysteresis thresholding (T_low=0.30, T_high=0.90), z-closing (radius=1), dust removal (min_size=100). Beat every \"optimised\" alternative.\n- **Ensemble diversity via data** — best result (0.582) came from ensembling models with different preprocessor checkpoints (WNet20 vs WNet100) AND different GT versions. Half-data ensembles (321+322) also showed clear gains (+0.010 over best single). Models sharing the same preprocessor but differing only in GT gave less benefit.\n\n\n## What Didn't Work\n\n- **Optuna postprocessing** — Used Optuna to find best postprocessing parameters, but overfit to local validation. 0.577 vs 0.579 for hand-picked params on same model.\n- **Fragmentation problem** Fragmentation appeared to be the main issue and postprocessing needs to be optimized for this.\n- **[TransUNet](https://www.kaggle.com/models/ipythonx/vsd-model/Keras/transunet/4) ensemble** — Tried TransUNet model by @ipythonx at 25% weight with nnUNet 323: 0.579, identical to 323 solo. Correlated errors.\n- **Winning solution postprocessing on our model** — After the competition, I applied the winning postprocessing as-is and scored 0.582, same as our simple postprocessing. Their pipeline was co-optimised with their 4-model ensemble. Postprocessing is model-specific.\n- **Public LB for model selection** — best public (0.563) and best private (0.582) came from different submissions. Would have picked the wrong model.\n- **Local metrics** — best local topometric scores consistently corresponded to worse LB results. Train/test distribution mismatch.\n\n## Key Takeaways\n\n1. **Self-supervised preprocessing is underexplored** — WNet from microscopy transferred to scroll CT. No other team tried learned preprocessing.\n2. **Data quality > model complexity** — GT cleaning gave comparable gains to architectural changes.\n3. **You don't need all the data** — 50% data used with nnUNet got within 0.013 of full-data model. (Caveat: WNet was trained on all data.)\n4. **Simple postprocessing beats tuned parameters** — train/test distribution gap made local optimisation unreliable.\n5. **Postprocessing is model-specific** — winning team's PP didn't help our model.\n\n\n## Future Directions\n\n In this competition I tuned WNet to find sheet-like and larger structures, but with more classes and careful tuning of the SoftNCuts loss parameters (intensity sigma, spatial sigma, radius), I believe WNet could learn to separate ink from sheet from background. The ink channel could then be fed into nnUNet as an additional input, giving the segmentation model explicit ink-awareness — potentially helping in regions where ink and sheet are hard to distinguish.\n\nOther ideas I didn't have time to explore:\n\n- **Controlled WNet ablation** - train identical nnUNet models with and without WNet features to properly quantify the preprocessing contribution on the leaderboard, not just training convergence\n- **Multi-scale WNet features** - the winning solution gained a lot from multiple patch sizes. Extracting WNet features at different scales and stacking them could capture both fine ink texture and coarse sheet structure\n- **WNet-guided postprocessing** —-use WNet's class maps to inform topology repair, e.g. only fill holes in regions WNet classifies as sheet, or preserve gaps where WNet detects ink. It seems to detect large and small gaps between sheets\n- **More WNet output classes** — going beyond 3 classes to see if finer structural categories emerge (sheet interior, sheet boundary, ink, voids, noise)\n\nIf anyone is interested in exploring the WNet preprocessing direction, happy to chat. @seanjohnsonsp @giorgioangelotti \n\n\n## All Results\n\nHere's every submission I made during the competition, roughly in order. The story is one of diminishing returns from model tweaks and hard-won lessons about what actually moves the needle.\n\n| Submission | Description | Public LB | Private LB |\n|---|---|---|---|\n| Dataset321 | nnUNet on first half of training data, WNet20 preprocessor | 0.553 | 0.566 |\n| Dataset322 | nnUNet on second half of training data, WNet20 preprocessor | 0.549 | 0.564 |\n| 321+322 ensemble | 50/50 average of the two half-data models | 0.556 | 0.576 |\n| Dataset323 | nnUNet on full data, WNet100 preprocessor, DC+CE loss | 0.563 | 0.579 |\n| Dataset324 | nnUNet on full data, WNet20 preprocessor, DC+CE loss | — | 0.579 |\n| Dataset325 | nnUNet on full data, WNet100 preprocessor, BoundaryDoULoss, cleaned GT | 0.553 | 0.575 |\n| 323 + TransUNet | 75/25 nnUNet/TransUNet ensemble | 0.558 | 0.579 |\n| 323 + Optuna PP | Optuna-optimised postprocessing on best solo model | — | 0.577 |\n| **324+325 ensemble** | **0.5/0.5 ensemble, different WNet checkpoints + different GT** | **0.557** | **0.582** |\n| 325 + winning PP | Dataset325 with 1st place postprocessing (post-competition test) | — | 0.582 |\n\nA few things stand out from this table. The half-data models (321, 322) are surprisingly competitive, only ~0.013 behind the full-data model (323), which suggests the training set has a lot of redundancy. The best ensemble (324+325) only edges out the best solo by 0.003, but it required genuine diversity: different WNet checkpoints (20 vs 100 epochs) and different ground truth versions. Models that were too similar (e.g. 323+TransUNet) just averaged their shared mistakes.\n\nThe public/private LB gap is also notable. The best public score (0.563, Dataset323) and best private score (0.582, 324+325 ensemble) came from different submissions entirely. If I'd been selecting on public LB alone, I'd have picked the wrong model.\n\n## Comparison to Winning Solution (0.627)\n\nIt's worth looking at where the gap came from:\n\n| | Ours (113th) | 1st Place |\n|---|---|---|\n| Backbone | nnUNet ResEncUNetXL | nnUNet (4 models) |\n| Patch sizes | Single | 128, 160, 192, 224, 256 |\n| Preprocessing | WNet3D (self-supervised) | Raw CT |\n| Postprocessing | Simple hysteresis | Height-map patching, LUT hole repair, oriented closing |\n| GT handling | Manual cleaning | Standard |\n| Private LB | 0.582 | 0.627 |\n\nThe ~0.045 gap is substantial, and it came primarily from their topology-aware postprocessing and multi-scale ensemble diversity (five different patch sizes). Both solutions validated nnUNet as the right backbone we invested in preprocessing, they invested in postprocessing and model diversity. Crucially, when I applied their postprocessing to our model post-competition, it scored 0.582, which is identical to our simple approach. Their postprocessing was co-optimised with their specific ensemble's error profile and didn't transfer.\n\n## Technical Details\n\n**nnUNet configuration:** ResEncUNetXL backbone, 3d_fullres, trained with the standard nnUNet pipeline. Dataset324 used the default Dice + Cross-Entropy loss. Dataset325 used [BoundaryDoULoss3D](https://arxiv.org/pdf/2308.00220.pdf), adapted from [Igor Krashenyi's 4th place solution](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/igor-krashenyi-4th-place-solution-boundary-dou-los) in the SenNet+HOA blood vessel segmentation competition ([original repo](https://github.com/sunfan-bvb/BoundaryDoULoss)). This boundary-aware loss was designed to improve segmentation at tissue boundaries — a natural fit for sheet detection.\n\n**WNet3D:** From [CellSeg3D](https://elifesciences.org/articles/99848) (Achard et al., eLife 2025). Trained two checkpoints — 20 epochs (patch 64) and 100 epochs (patch 128) — on all raw CT training data in a fully self-supervised way (no labels required). The 20 epoch was my first attempt, and then when I got access to more compute, I trained one with larger patch size for more epochs. The encoder's intermediate feature maps (3 channels) were concatenated with the raw CT to form a 4-channel input for nnUNet. Using two different WNet checkpoints for the two nnUNet models was key to ensemble diversity.\n\n**Inference:** Sliding window with TTA (mirror axes 0, 1), tile_step_size=0.5. Initially used 1xP100 GPU, but I've learned to run both models run in parallel on Kaggle's 2×T4 GPUs. \n\n**Postprocessing:** Hysteresis thresholding (T_low=0.30, T_high=0.90) to get connected high-confidence regions and grow them into lower-confidence voxels. Morphological z-closing (radius=1) to fill single-slice gaps. Dust removal (min connected component size=100) to clean up small fragments. Simple, conservative, and it generalised better than anything tuned.\n\n\n## Reflections\n\nThis was my first Kaggle competition and I learned a huge deal. My background in bioimage analysis turned out to be more transferable than I expected, i.e., volumetric segmentation is volumetric segmentation whether it's cells or scrolls, and tools like nnUNet and WNet3D crossed domains surprisingly well. At the same time, I learned just as much from reading other people's discussions and solutions. The community here is incredibly generous with sharing ideas, and I picked up techniques I'd never have found on my own.\n\nNext time I'd love to work in a team. It seems like a great way to connect with people, learn faster, and push further than you can solo. I'd also aim to contribute more to the discussion forums during the competition rather than just lurking, it would be a great way to pay forward what others shared with me + can get input from many.\n\nComing from biological and microscopy data, I'm especially appreciative of the effort that went into curating this dataset. Good ground truth is expensive and time-consuming to produce, and it's what makes competitions like this possible. Thanks to the organisers for putting it all together.\n" } }