In my solution, I used Z compression to speed things up.\nI also ran some basic experiments to confirm that using Z compression wouldn’t cause any issues, so I’m sharing the results here.
\n3D training with nnU-Net is extremely computationally expensive. The Host Baseline (160x160x160 patch, ResidualEncoderUNet) takes approximately 165-200 seconds per epoch, making it difficult to compete with a single RTX 5090.
\nWe considered an approach of compressing the Z-axis by 1/2 for training and inference, then restoring to the original resolution. To assess feasibility, we first quantitatively evaluated how much information is lost by compression and restoration alone (without any model), using the official metrics.
\nWe compressed and restored GT labels (3D binary masks), then compared them against the original GT. Since no model prediction is involved, the results represent the theoretical upper bound of what a compression-based approach can achieve.
\nGT (320³) ──→ Z-axis 1/2 compression (160x320x320) ──→ Restore to original size (320³) ──→ Compare with GT\nWe used scipy.ndimage.zoom for compression, progressively improving the interpolation method, label format, and post-processing.
We used the official metrics (Combined Score = 0.30 × TopoScore + 0.35 × SurfaceDice + 0.35 × VOI Score). Perfect reversibility would yield Combined = 1.0.
\nWe sampled 5 volumes per scroll ID from the training data (three volume sizes exist: 320³: 738 samples, 256³: 47 samples, 384³: 1 sample).
\nAs the simplest approach, we performed compression and restoration using Nearest Neighbor interpolation (order=0).
| Combined | \nSDice | \nVOI | \nTopo | \n
|---|---|---|---|
| 0.435 | \n0.962 | \n0.265 | \n0.020 | \n
SDice was a reasonable 0.96, but TopoScore was nearly zero at 0.02. Nearest Neighbor discards alternating slices during compression and duplicates them during restoration, significantly altering the 3D connectivity structure.
\nTo address the Nearest Neighbor issues, we switched to Linear interpolation (order=1). Here, a critical design decision arose:
Should we binarize after compression (Hard), or preserve probability values (Soft)?
\nHard: GT → Linear compress → Binarize at threshold 0.5 → Linear restore → Threshold 0.5\nSoft: GT → Linear compress → Keep as probabilities (0.0-1.0) → Linear restore → Threshold 0.5\nThe Hard approach discards information at the compression stage and degrades again through thresholding during restoration. The Soft approach preserves the intermediate representation and only binarizes at the final stage.
\n| Method | \nCombined | \nSDice | \nVOI | \nTopo | \n
|---|---|---|---|---|
| Nearest Neighbor | \n0.435 | \n0.962 | \n0.265 | \n0.020 | \n
| Linear (Hard) | \n0.435 | \n0.962 | \n0.265 | \n0.020 | \n
| Linear (Soft) | \n0.810 | \n1.000 | \n0.844 | \n0.550 | \n
Linear (Hard) produced exactly the same scores as Nearest Neighbor. In contrast, simply preserving soft labels jumped Combined from 0.44 to 0.81. SDice reached a perfect 1.000, and VOI was a strong 0.84. Soft label preservation was the single biggest breakthrough in this experiment.
\nHowever, TopoScore remained at 0.55, leaving room for improvement.
\nscipy.ndimage.zoom has a known issue where boundary frames collapse (the last frame becomes 0). In our data, the first and last 3 pixels along the Z-axis are ignore regions, so we overwrote these boundary frames with the original data after compression.
comp[0] = orig[0] # ignore start\ncomp[1] = orig[3] # valid region boundary (preserve binary)\ncomp[-2] = orig[-4] # valid region boundary (preserve binary)\ncomp[-1] = orig[-1] # ignore end\nThe results in Step 2's table (Combined 0.810) already include this boundary preservation.
\nWe investigated why TopoScore remained at 0.55 after Steps 2-3. Linear interpolation generates intermediate values (0.3-0.7), and thresholding at 0.5 creates tiny holes. In 3D, these holes form loop structures (B1 tunnels) that are invisible in 2D slices but change the 3D topology.
\nWe applied dilation followed by erosion (Closing) to repair these artifacts.
\nfrom scipy.ndimage import binary_dilation, binary_erosion\npred_fixed = binary_erosion(binary_dilation(pred, iterations=1), iterations=1)\nThe effect varied dramatically by data size.
\nSize 320 (738 samples, GT tunnel count B1=0, simple structure):
\n| Processing | \nCombined | \nTopo | \nB1 Change | \n
|---|---|---|---|
| Soft + Boundary | \n0.798 | \n0.550 | \n0→3 | \n
| + Closing | \n0.798 | \n0.550 | \n0→3 | \n
No effect on size 320. The 3 B1 tunnels introduced by compression-restoration could not be removed by closing.
\nSize 256 (47 samples, GT tunnel count B1≈40, complex branching structure):
\n| Processing | \nCombined | \nTopo | \nB1 Change | \n
|---|---|---|---|
| Soft + Boundary | \n0.752 | \n0.474 | \n40→2919 | \n
| + Closing | \n0.888 | \n0.899 | \n40→50 | \n
For size 256, B1 had exploded from 40 to 2,919, but closing suppressed it to 50, resulting in a massive TopoScore improvement from 0.47 to 0.90.
\nWe also tested 1/3 compression for further speedup.
\nSize 320:
\n| Ratio | \nCompressed Z | \nCombined | \nVOI | \nTopo | \n
|---|---|---|---|---|
| 1/2 | \n160 | \n0.798 | \n0.809 | \n0.550 | \n
| 1/3 | \n106 | \n0.685 | \n0.566 | \n0.456 | \n
Size 256 (after closing):
\n| Ratio | \nCompressed Z | \nCombined | \nVOI | \nTopo | \n
|---|---|---|---|---|
| 1/2 | \n128 | \n0.888 | \n0.765 | \n0.899 | \n
| 1/3 | \n85 | \n0.614 | \n0.485 | \n0.314 | \n
1/3 compression caused significant Combined drops: -0.11 for size 320 and -0.27 for size 256. 1/2 is the practical limit.
\nWe investigated the remaining TopoScore = 0.55 issue for size 320. Examining the Betti Matching breakdown:
\n| Betti Number | \nGT | \nRestored | \nMatched | \nF1 | \n
|---|---|---|---|---|
| B0 (connected components) | \n17 | \n19 | \n17 | \n0.94 | \n
| B1 (tunnels) | \n0 | \n3 | \n0 | \n0.14 | \n
B0 (component count) is nearly perfectly preserved (F1=0.94). The problem lies in B1: 3 tunnels appear where there were originally 0. Since TopoScore averages the F1 of B0 and B1, the B1 F1 of 0.14 drags the overall score down to 0.55.
\nThese 3 B1 tunnels are tiny loops created by the threshold processing during compression-restoration, and could not be removed by morphological closing. However, this is an error of \"3 tunnels added to a perfectly clean GT\" --- in practice, model predictions contain far more topological errors, so this is not a concern for real-world use.
\nZ-axis 1/2 compression is nearly reversible when combining soft label preservation + boundary preservation + morphological closing.
\n| Data | \nCombined | \nSDice | \nVOI | \nTopo | \n
|---|---|---|---|---|
| Size 320 (738 samples) | \n0.810 | \n1.000 | \n0.844 | \n0.550 | \n
| Size 256 (47 samples) + post-processing | \n0.888 | \n1.000 | \n0.765 | \n0.899 | \n
| Improvement Step | \nEffect | \n
|---|---|
| Nearest Neighbor → Linear (Soft) | \nCombined +0.37 (largest gain) | \n
| + Boundary preservation | \nBoundary artifact elimination | \n
| + Morphological closing | \nSize 256 Topo +0.43 | \n
SDice and VOI are sufficiently high, indicating minimal geometric degradation from compression. The information loss from compression-restoration is judged to be small relative to model prediction errors, confirming the feasibility of compressed training. However, since top solutions achieved their scores with full-resolution training, further validation is needed to determine whether this method can reach gold-medal territory.
", "messages": [ { "id": "3415527", "postDate": "03/01/2026 01:16:32", "content": "In my solution, I used Z compression to speed things up.\nI also ran some basic experiments to confirm that using Z compression wouldn’t cause any issues, so I’m sharing the results here.
\n3D training with nnU-Net is extremely computationally expensive. The Host Baseline (160x160x160 patch, ResidualEncoderUNet) takes approximately 165-200 seconds per epoch, making it difficult to compete with a single RTX 5090.
\nWe considered an approach of compressing the Z-axis by 1/2 for training and inference, then restoring to the original resolution. To assess feasibility, we first quantitatively evaluated how much information is lost by compression and restoration alone (without any model), using the official metrics.
\nWe compressed and restored GT labels (3D binary masks), then compared them against the original GT. Since no model prediction is involved, the results represent the theoretical upper bound of what a compression-based approach can achieve.
\nGT (320³) ──→ Z-axis 1/2 compression (160x320x320) ──→ Restore to original size (320³) ──→ Compare with GT\nWe used scipy.ndimage.zoom for compression, progressively improving the interpolation method, label format, and post-processing.
We used the official metrics (Combined Score = 0.30 × TopoScore + 0.35 × SurfaceDice + 0.35 × VOI Score). Perfect reversibility would yield Combined = 1.0.
\nWe sampled 5 volumes per scroll ID from the training data (three volume sizes exist: 320³: 738 samples, 256³: 47 samples, 384³: 1 sample).
\nAs the simplest approach, we performed compression and restoration using Nearest Neighbor interpolation (order=0).
| Combined | \nSDice | \nVOI | \nTopo | \n
|---|---|---|---|
| 0.435 | \n0.962 | \n0.265 | \n0.020 | \n
SDice was a reasonable 0.96, but TopoScore was nearly zero at 0.02. Nearest Neighbor discards alternating slices during compression and duplicates them during restoration, significantly altering the 3D connectivity structure.
\nTo address the Nearest Neighbor issues, we switched to Linear interpolation (order=1). Here, a critical design decision arose:
Should we binarize after compression (Hard), or preserve probability values (Soft)?
\nHard: GT → Linear compress → Binarize at threshold 0.5 → Linear restore → Threshold 0.5\nSoft: GT → Linear compress → Keep as probabilities (0.0-1.0) → Linear restore → Threshold 0.5\nThe Hard approach discards information at the compression stage and degrades again through thresholding during restoration. The Soft approach preserves the intermediate representation and only binarizes at the final stage.
\n| Method | \nCombined | \nSDice | \nVOI | \nTopo | \n
|---|---|---|---|---|
| Nearest Neighbor | \n0.435 | \n0.962 | \n0.265 | \n0.020 | \n
| Linear (Hard) | \n0.435 | \n0.962 | \n0.265 | \n0.020 | \n
| Linear (Soft) | \n0.810 | \n1.000 | \n0.844 | \n0.550 | \n
Linear (Hard) produced exactly the same scores as Nearest Neighbor. In contrast, simply preserving soft labels jumped Combined from 0.44 to 0.81. SDice reached a perfect 1.000, and VOI was a strong 0.84. Soft label preservation was the single biggest breakthrough in this experiment.
\nHowever, TopoScore remained at 0.55, leaving room for improvement.
\nscipy.ndimage.zoom has a known issue where boundary frames collapse (the last frame becomes 0). In our data, the first and last 3 pixels along the Z-axis are ignore regions, so we overwrote these boundary frames with the original data after compression.
comp[0] = orig[0] # ignore start\ncomp[1] = orig[3] # valid region boundary (preserve binary)\ncomp[-2] = orig[-4] # valid region boundary (preserve binary)\ncomp[-1] = orig[-1] # ignore end\nThe results in Step 2's table (Combined 0.810) already include this boundary preservation.
\nWe investigated why TopoScore remained at 0.55 after Steps 2-3. Linear interpolation generates intermediate values (0.3-0.7), and thresholding at 0.5 creates tiny holes. In 3D, these holes form loop structures (B1 tunnels) that are invisible in 2D slices but change the 3D topology.
\nWe applied dilation followed by erosion (Closing) to repair these artifacts.
\nfrom scipy.ndimage import binary_dilation, binary_erosion\npred_fixed = binary_erosion(binary_dilation(pred, iterations=1), iterations=1)\nThe effect varied dramatically by data size.
\nSize 320 (738 samples, GT tunnel count B1=0, simple structure):
\n| Processing | \nCombined | \nTopo | \nB1 Change | \n
|---|---|---|---|
| Soft + Boundary | \n0.798 | \n0.550 | \n0→3 | \n
| + Closing | \n0.798 | \n0.550 | \n0→3 | \n
No effect on size 320. The 3 B1 tunnels introduced by compression-restoration could not be removed by closing.
\nSize 256 (47 samples, GT tunnel count B1≈40, complex branching structure):
\n| Processing | \nCombined | \nTopo | \nB1 Change | \n
|---|---|---|---|
| Soft + Boundary | \n0.752 | \n0.474 | \n40→2919 | \n
| + Closing | \n0.888 | \n0.899 | \n40→50 | \n
For size 256, B1 had exploded from 40 to 2,919, but closing suppressed it to 50, resulting in a massive TopoScore improvement from 0.47 to 0.90.
\nWe also tested 1/3 compression for further speedup.
\nSize 320:
\n| Ratio | \nCompressed Z | \nCombined | \nVOI | \nTopo | \n
|---|---|---|---|---|
| 1/2 | \n160 | \n0.798 | \n0.809 | \n0.550 | \n
| 1/3 | \n106 | \n0.685 | \n0.566 | \n0.456 | \n
Size 256 (after closing):
\n| Ratio | \nCompressed Z | \nCombined | \nVOI | \nTopo | \n
|---|---|---|---|---|
| 1/2 | \n128 | \n0.888 | \n0.765 | \n0.899 | \n
| 1/3 | \n85 | \n0.614 | \n0.485 | \n0.314 | \n
1/3 compression caused significant Combined drops: -0.11 for size 320 and -0.27 for size 256. 1/2 is the practical limit.
\nWe investigated the remaining TopoScore = 0.55 issue for size 320. Examining the Betti Matching breakdown:
\n| Betti Number | \nGT | \nRestored | \nMatched | \nF1 | \n
|---|---|---|---|---|
| B0 (connected components) | \n17 | \n19 | \n17 | \n0.94 | \n
| B1 (tunnels) | \n0 | \n3 | \n0 | \n0.14 | \n
B0 (component count) is nearly perfectly preserved (F1=0.94). The problem lies in B1: 3 tunnels appear where there were originally 0. Since TopoScore averages the F1 of B0 and B1, the B1 F1 of 0.14 drags the overall score down to 0.55.
\nThese 3 B1 tunnels are tiny loops created by the threshold processing during compression-restoration, and could not be removed by morphological closing. However, this is an error of \"3 tunnels added to a perfectly clean GT\" --- in practice, model predictions contain far more topological errors, so this is not a concern for real-world use.
\nZ-axis 1/2 compression is nearly reversible when combining soft label preservation + boundary preservation + morphological closing.
\n| Data | \nCombined | \nSDice | \nVOI | \nTopo | \n
|---|---|---|---|---|
| Size 320 (738 samples) | \n0.810 | \n1.000 | \n0.844 | \n0.550 | \n
| Size 256 (47 samples) + post-processing | \n0.888 | \n1.000 | \n0.765 | \n0.899 | \n
| Improvement Step | \nEffect | \n
|---|---|
| Nearest Neighbor → Linear (Soft) | \nCombined +0.37 (largest gain) | \n
| + Boundary preservation | \nBoundary artifact elimination | \n
| + Morphological closing | \nSize 256 Topo +0.43 | \n
SDice and VOI are sufficiently high, indicating minimal geometric degradation from compression. The information loss from compression-restoration is judged to be small relative to model prediction errors, confirming the feasibility of compressed training. However, since top solutions achieved their scores with full-resolution training, further validation is needed to determine whether this method can reach gold-medal territory.
", "rawMarkdown": "# Reversibility of Z-axis Compression\n\n## Background\nIn my [solution](https://www.kaggle.com/competitions/vesuvius-challenge-surface-detection/writeups/18th-median-filter-x-7-post-processing-is-very-s), I used Z compression to speed things up.\nI also ran some basic experiments to confirm that using Z compression wouldn’t cause any issues, so I’m sharing the results here.\n\n\n3D training with nnU-Net is extremely computationally expensive. The Host Baseline (160x160x160 patch, ResidualEncoderUNet) takes approximately 165-200 seconds per epoch, making it difficult to compete with a single RTX 5090.\n\nWe considered an approach of **compressing the Z-axis by 1/2 for training and inference, then restoring to the original resolution**. To assess feasibility, we first quantitatively evaluated how much information is lost by compression and restoration alone (without any model), using the official metrics.\n\n## Experimental Design\n\nWe compressed and restored GT labels (3D binary masks), then compared them against the original GT. Since no model prediction is involved, the results represent the **theoretical upper bound** of what a compression-based approach can achieve.\n\n```\nGT (320³) ──→ Z-axis 1/2 compression (160x320x320) ──→ Restore to original size (320³) ──→ Compare with GT\n```\n\nWe used `scipy.ndimage.zoom` for compression, progressively improving the interpolation method, label format, and post-processing.\n\n### Evaluation Metrics\n\nWe used the official metrics (Combined Score = 0.30 × TopoScore + 0.35 × SurfaceDice + 0.35 × VOI Score). Perfect reversibility would yield Combined = 1.0.\n\n### Data\n\nWe sampled 5 volumes per scroll ID from the training data (three volume sizes exist: 320³: 738 samples, 256³: 47 samples, 384³: 1 sample).\n\n## Results\n\n### Step 1: Nearest Neighbor (Baseline)\n\nAs the simplest approach, we performed compression and restoration using Nearest Neighbor interpolation (`order=0`).\n\n| Combined | SDice | VOI | Topo |\n|----------|-------|-----|------|\n| 0.435 | 0.962 | 0.265 | 0.020 |\n\nSDice was a reasonable 0.96, but **TopoScore was nearly zero at 0.02**. Nearest Neighbor discards alternating slices during compression and duplicates them during restoration, significantly altering the 3D connectivity structure.\n\n### Step 2: Linear Interpolation + Soft Labels\n\nTo address the Nearest Neighbor issues, we switched to Linear interpolation (`order=1`). Here, a critical design decision arose:\n\n**Should we binarize after compression (Hard), or preserve probability values (Soft)?**\n\n```\nHard: GT → Linear compress → Binarize at threshold 0.5 → Linear restore → Threshold 0.5\nSoft: GT → Linear compress → Keep as probabilities (0.0-1.0) → Linear restore → Threshold 0.5\n```\n\nThe Hard approach discards information at the compression stage and degrades again through thresholding during restoration. The Soft approach preserves the intermediate representation and only binarizes at the final stage.\n\n| Method | Combined | SDice | VOI | Topo |\n|--------|----------|-------|-----|------|\n| Nearest Neighbor | 0.435 | 0.962 | 0.265 | 0.020 |\n| Linear (Hard) | 0.435 | 0.962 | 0.265 | 0.020 |\n| **Linear (Soft)** | **0.810** | **1.000** | **0.844** | **0.550** |\n\nLinear (Hard) produced exactly the same scores as Nearest Neighbor. In contrast, **simply preserving soft labels jumped Combined from 0.44 to 0.81**. SDice reached a perfect 1.000, and VOI was a strong 0.84. Soft label preservation was the single biggest breakthrough in this experiment.\n\nHowever, TopoScore remained at 0.55, leaving room for improvement.\n\n### Step 3: Boundary Frame Preservation\n\n`scipy.ndimage.zoom` has a known issue where boundary frames collapse (the last frame becomes 0). In our data, the first and last 3 pixels along the Z-axis are ignore regions, so we overwrote these boundary frames with the original data after compression.\n\n```python\ncomp[0] = orig[0] # ignore start\ncomp[1] = orig[3] # valid region boundary (preserve binary)\ncomp[-2] = orig[-4] # valid region boundary (preserve binary)\ncomp[-1] = orig[-1] # ignore end\n```\n\nThe results in Step 2's table (Combined 0.810) already include this boundary preservation.\n\n### Step 4: Morphological Topology Repair\n\nWe investigated why TopoScore remained at 0.55 after Steps 2-3. Linear interpolation generates intermediate values (0.3-0.7), and thresholding at 0.5 creates tiny holes. In 3D, these holes form **loop structures (B1 tunnels)** that are invisible in 2D slices but change the 3D topology.\n\nWe applied **dilation followed by erosion (Closing)** to repair these artifacts.\n\n```python\nfrom scipy.ndimage import binary_dilation, binary_erosion\npred_fixed = binary_erosion(binary_dilation(pred, iterations=1), iterations=1)\n```\n\nThe effect varied dramatically by data size.\n\n**Size 320 (738 samples, GT tunnel count B1=0, simple structure):**\n\n| Processing | Combined | Topo | B1 Change |\n|------------|----------|------|-----------|\n| Soft + Boundary | 0.798 | 0.550 | 0→3 |\n| + Closing | 0.798 | 0.550 | 0→3 |\n\nNo effect on size 320. The 3 B1 tunnels introduced by compression-restoration could not be removed by closing.\n\n**Size 256 (47 samples, GT tunnel count B1≈40, complex branching structure):**\n\n| Processing | Combined | Topo | B1 Change |\n|------------|----------|------|-----------|\n| Soft + Boundary | 0.752 | 0.474 | 40→2919 |\n| **+ Closing** | **0.888** | **0.899** | **40→50** |\n\nFor size 256, B1 had exploded from 40 to 2,919, but closing suppressed it to 50, resulting in a **massive TopoScore improvement from 0.47 to 0.90**.\n\n### Step 5: Compression Ratio Comparison (1/2 vs 1/3)\n\nWe also tested 1/3 compression for further speedup.\n\n**Size 320:**\n\n| Ratio | Compressed Z | Combined | VOI | Topo |\n|-------|-------------|----------|-----|------|\n| 1/2 | 160 | 0.798 | 0.809 | 0.550 |\n| 1/3 | 106 | 0.685 | 0.566 | 0.456 |\n\n**Size 256 (after closing):**\n\n| Ratio | Compressed Z | Combined | VOI | Topo |\n|-------|-------------|----------|-----|------|\n| 1/2 | 128 | **0.888** | 0.765 | **0.899** |\n| 1/3 | 85 | 0.614 | 0.485 | 0.314 |\n\n1/3 compression caused significant Combined drops: -0.11 for size 320 and -0.27 for size 256. **1/2 is the practical limit**.\n\n## Analysis of Remaining TopoScore Degradation\n\nWe investigated the remaining TopoScore = 0.55 issue for size 320. Examining the Betti Matching breakdown:\n\n| Betti Number | GT | Restored | Matched | F1 |\n|-------------|---:|--------:|--------:|---:|\n| B0 (connected components) | 17 | 19 | 17 | 0.94 |\n| B1 (tunnels) | 0 | 3 | 0 | 0.14 |\n\nB0 (component count) is nearly perfectly preserved (F1=0.94). The problem lies in B1: 3 tunnels appear where there were originally 0. Since TopoScore averages the F1 of B0 and B1, the B1 F1 of 0.14 drags the overall score down to 0.55.\n\nThese 3 B1 tunnels are tiny loops created by the threshold processing during compression-restoration, and could not be removed by morphological closing. However, this is an error of \"3 tunnels added to a perfectly clean GT\" --- in practice, model predictions contain far more topological errors, so this is not a concern for real-world use.\n\n## Summary\n\nZ-axis 1/2 compression is **nearly reversible** when combining soft label preservation + boundary preservation + morphological closing.\n\n| Data | Combined | SDice | VOI | Topo |\n|------|----------|-------|-----|------|\n| Size 320 (738 samples) | 0.810 | 1.000 | 0.844 | 0.550 |\n| Size 256 (47 samples) + post-processing | 0.888 | 1.000 | 0.765 | 0.899 |\n\n| Improvement Step | Effect |\n|-----------------|--------|\n| Nearest Neighbor → Linear (Soft) | Combined +0.37 (largest gain) |\n| + Boundary preservation | Boundary artifact elimination |\n| + Morphological closing | Size 256 Topo +0.43 |\n\nSDice and VOI are sufficiently high, indicating minimal geometric degradation from compression. The information loss from compression-restoration is judged to be small relative to model prediction errors, confirming the feasibility of compressed training. However, since top solutions achieved their scores with full-resolution training, further validation is needed to determine whether this method can reach gold-medal territory.", "votes": null }, { "id": "3415764", "postDate": "03/01/2026 09:26:59", "content": "This is very interesting, thank you!
\nMy question regarding an analysis was also related to a preference in the axis (Z rather than other ones) because, even if we scan the scrolls vertically, some regions can heavily fold!
", "rawMarkdown": "This is very interesting, thank you!\n\nMy question regarding an analysis was also related to a preference in the axis (Z rather than other ones) because, even if we scan the scrolls vertically, some regions can heavily fold!", "votes": null }, { "id": "3416070", "postDate": "03/02/2026 02:10:54", "content": "Thank you for your comment!!!\nSorry—I may not have fully understood the intent of your question.
\nI tried testing compression along the XY, XZ, and YZ axes. When using XY as the vertical/horizontal directions, I often found that the XZ/YZ plane intersects a wide surface area in certain slices, so I judged that compression along the other axes could be risky.
\nThat said, given that my model is not currently in the gold range, it’s possible there are issues in the area you pointed out.
", "rawMarkdown": "Thank you for your comment!!!\nSorry—I may not have fully understood the intent of your question.\n\nI tried testing compression along the XY, XZ, and YZ axes. When using XY as the vertical/horizontal directions, I often found that the XZ/YZ plane intersects a wide surface area in certain slices, so I judged that compression along the other axes could be risky.\n\nThat said, given that my model is not currently in the gold range, it’s possible there are issues in the area you pointed out.", "votes": null } ], "comments": [ { "id": 3415764, "author_name": "giorgioangelotti", "author_url": "", "post_date": "03/01/2026 09:26:59", "content": "This is very interesting, thank you!
\nMy question regarding an analysis was also related to a preference in the axis (Z rather than other ones) because, even if we scan the scrolls vertically, some regions can heavily fold!
", "votes": null, "replies": [ { "id": 3416070, "author_name": "sugupoko", "author_url": "", "post_date": "03/02/2026 02:10:54", "content": "Thank you for your comment!!!\nSorry—I may not have fully understood the intent of your question.
\nI tried testing compression along the XY, XZ, and YZ axes. When using XY as the vertical/horizontal directions, I often found that the XZ/YZ plane intersects a wide surface area in certain slices, so I judged that compression along the other axes could be risky.
\nThat said, given that my model is not currently in the gold range, it’s possible there are issues in the area you pointed out.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3415527": "# Reversibility of Z-axis Compression\n\n## Background\nIn my [solution](https://www.kaggle.com/competitions/vesuvius-challenge-surface-detection/writeups/18th-median-filter-x-7-post-processing-is-very-s), I used Z compression to speed things up.\nI also ran some basic experiments to confirm that using Z compression wouldn’t cause any issues, so I’m sharing the results here.\n\n\n3D training with nnU-Net is extremely computationally expensive. The Host Baseline (160x160x160 patch, ResidualEncoderUNet) takes approximately 165-200 seconds per epoch, making it difficult to compete with a single RTX 5090.\n\nWe considered an approach of **compressing the Z-axis by 1/2 for training and inference, then restoring to the original resolution**. To assess feasibility, we first quantitatively evaluated how much information is lost by compression and restoration alone (without any model), using the official metrics.\n\n## Experimental Design\n\nWe compressed and restored GT labels (3D binary masks), then compared them against the original GT. Since no model prediction is involved, the results represent the **theoretical upper bound** of what a compression-based approach can achieve.\n\n```\nGT (320³) ──→ Z-axis 1/2 compression (160x320x320) ──→ Restore to original size (320³) ──→ Compare with GT\n```\n\nWe used `scipy.ndimage.zoom` for compression, progressively improving the interpolation method, label format, and post-processing.\n\n### Evaluation Metrics\n\nWe used the official metrics (Combined Score = 0.30 × TopoScore + 0.35 × SurfaceDice + 0.35 × VOI Score). Perfect reversibility would yield Combined = 1.0.\n\n### Data\n\nWe sampled 5 volumes per scroll ID from the training data (three volume sizes exist: 320³: 738 samples, 256³: 47 samples, 384³: 1 sample).\n\n## Results\n\n### Step 1: Nearest Neighbor (Baseline)\n\nAs the simplest approach, we performed compression and restoration using Nearest Neighbor interpolation (`order=0`).\n\n| Combined | SDice | VOI | Topo |\n|----------|-------|-----|------|\n| 0.435 | 0.962 | 0.265 | 0.020 |\n\nSDice was a reasonable 0.96, but **TopoScore was nearly zero at 0.02**. Nearest Neighbor discards alternating slices during compression and duplicates them during restoration, significantly altering the 3D connectivity structure.\n\n### Step 2: Linear Interpolation + Soft Labels\n\nTo address the Nearest Neighbor issues, we switched to Linear interpolation (`order=1`). Here, a critical design decision arose:\n\n**Should we binarize after compression (Hard), or preserve probability values (Soft)?**\n\n```\nHard: GT → Linear compress → Binarize at threshold 0.5 → Linear restore → Threshold 0.5\nSoft: GT → Linear compress → Keep as probabilities (0.0-1.0) → Linear restore → Threshold 0.5\n```\n\nThe Hard approach discards information at the compression stage and degrades again through thresholding during restoration. The Soft approach preserves the intermediate representation and only binarizes at the final stage.\n\n| Method | Combined | SDice | VOI | Topo |\n|--------|----------|-------|-----|------|\n| Nearest Neighbor | 0.435 | 0.962 | 0.265 | 0.020 |\n| Linear (Hard) | 0.435 | 0.962 | 0.265 | 0.020 |\n| **Linear (Soft)** | **0.810** | **1.000** | **0.844** | **0.550** |\n\nLinear (Hard) produced exactly the same scores as Nearest Neighbor. In contrast, **simply preserving soft labels jumped Combined from 0.44 to 0.81**. SDice reached a perfect 1.000, and VOI was a strong 0.84. Soft label preservation was the single biggest breakthrough in this experiment.\n\nHowever, TopoScore remained at 0.55, leaving room for improvement.\n\n### Step 3: Boundary Frame Preservation\n\n`scipy.ndimage.zoom` has a known issue where boundary frames collapse (the last frame becomes 0). In our data, the first and last 3 pixels along the Z-axis are ignore regions, so we overwrote these boundary frames with the original data after compression.\n\n```python\ncomp[0] = orig[0] # ignore start\ncomp[1] = orig[3] # valid region boundary (preserve binary)\ncomp[-2] = orig[-4] # valid region boundary (preserve binary)\ncomp[-1] = orig[-1] # ignore end\n```\n\nThe results in Step 2's table (Combined 0.810) already include this boundary preservation.\n\n### Step 4: Morphological Topology Repair\n\nWe investigated why TopoScore remained at 0.55 after Steps 2-3. Linear interpolation generates intermediate values (0.3-0.7), and thresholding at 0.5 creates tiny holes. In 3D, these holes form **loop structures (B1 tunnels)** that are invisible in 2D slices but change the 3D topology.\n\nWe applied **dilation followed by erosion (Closing)** to repair these artifacts.\n\n```python\nfrom scipy.ndimage import binary_dilation, binary_erosion\npred_fixed = binary_erosion(binary_dilation(pred, iterations=1), iterations=1)\n```\n\nThe effect varied dramatically by data size.\n\n**Size 320 (738 samples, GT tunnel count B1=0, simple structure):**\n\n| Processing | Combined | Topo | B1 Change |\n|------------|----------|------|-----------|\n| Soft + Boundary | 0.798 | 0.550 | 0→3 |\n| + Closing | 0.798 | 0.550 | 0→3 |\n\nNo effect on size 320. The 3 B1 tunnels introduced by compression-restoration could not be removed by closing.\n\n**Size 256 (47 samples, GT tunnel count B1≈40, complex branching structure):**\n\n| Processing | Combined | Topo | B1 Change |\n|------------|----------|------|-----------|\n| Soft + Boundary | 0.752 | 0.474 | 40→2919 |\n| **+ Closing** | **0.888** | **0.899** | **40→50** |\n\nFor size 256, B1 had exploded from 40 to 2,919, but closing suppressed it to 50, resulting in a **massive TopoScore improvement from 0.47 to 0.90**.\n\n### Step 5: Compression Ratio Comparison (1/2 vs 1/3)\n\nWe also tested 1/3 compression for further speedup.\n\n**Size 320:**\n\n| Ratio | Compressed Z | Combined | VOI | Topo |\n|-------|-------------|----------|-----|------|\n| 1/2 | 160 | 0.798 | 0.809 | 0.550 |\n| 1/3 | 106 | 0.685 | 0.566 | 0.456 |\n\n**Size 256 (after closing):**\n\n| Ratio | Compressed Z | Combined | VOI | Topo |\n|-------|-------------|----------|-----|------|\n| 1/2 | 128 | **0.888** | 0.765 | **0.899** |\n| 1/3 | 85 | 0.614 | 0.485 | 0.314 |\n\n1/3 compression caused significant Combined drops: -0.11 for size 320 and -0.27 for size 256. **1/2 is the practical limit**.\n\n## Analysis of Remaining TopoScore Degradation\n\nWe investigated the remaining TopoScore = 0.55 issue for size 320. Examining the Betti Matching breakdown:\n\n| Betti Number | GT | Restored | Matched | F1 |\n|-------------|---:|--------:|--------:|---:|\n| B0 (connected components) | 17 | 19 | 17 | 0.94 |\n| B1 (tunnels) | 0 | 3 | 0 | 0.14 |\n\nB0 (component count) is nearly perfectly preserved (F1=0.94). The problem lies in B1: 3 tunnels appear where there were originally 0. Since TopoScore averages the F1 of B0 and B1, the B1 F1 of 0.14 drags the overall score down to 0.55.\n\nThese 3 B1 tunnels are tiny loops created by the threshold processing during compression-restoration, and could not be removed by morphological closing. However, this is an error of \"3 tunnels added to a perfectly clean GT\" --- in practice, model predictions contain far more topological errors, so this is not a concern for real-world use.\n\n## Summary\n\nZ-axis 1/2 compression is **nearly reversible** when combining soft label preservation + boundary preservation + morphological closing.\n\n| Data | Combined | SDice | VOI | Topo |\n|------|----------|-------|-----|------|\n| Size 320 (738 samples) | 0.810 | 1.000 | 0.844 | 0.550 |\n| Size 256 (47 samples) + post-processing | 0.888 | 1.000 | 0.765 | 0.899 |\n\n| Improvement Step | Effect |\n|-----------------|--------|\n| Nearest Neighbor → Linear (Soft) | Combined +0.37 (largest gain) |\n| + Boundary preservation | Boundary artifact elimination |\n| + Morphological closing | Size 256 Topo +0.43 |\n\nSDice and VOI are sufficiently high, indicating minimal geometric degradation from compression. The information loss from compression-restoration is judged to be small relative to model prediction errors, confirming the feasibility of compressed training. However, since top solutions achieved their scores with full-resolution training, further validation is needed to determine whether this method can reach gold-medal territory.", "3415764": "This is very interesting, thank you!\n\nMy question regarding an analysis was also related to a preference in the axis (Z rather than other ones) because, even if we scan the scrolls vertically, some regions can heavily fold!", "3416070": "Thank you for your comment!!!\nSorry—I may not have fully understood the intent of your question.\n\nI tried testing compression along the XY, XZ, and YZ axes. When using XY as the vertical/horizontal directions, I often found that the XZ/YZ plane intersects a wide surface area in certain slices, so I judged that compression along the other axes could be risky.\n\nThat said, given that my model is not currently in the gold range, it’s possible there are issues in the area you pointed out." }, "source": "meta" }