Thank you for the organizers and Kaggle team for this great competition! I also thank all open-source contributors for sharing high-quality code and models.
\nMy model for continued training is based on Innat's pre-trained TransUNet model (https://www.kaggle.com/models/ipythonx/vsd-model/Keras/transunet/3?select=transunet.seresnext50.160px.comboloss.weights.h5), and the training code is adapted from Innat's notebook (https://www.kaggle.com/code/ipythonx/train-vesuvius-surface-3d-detection-on-tpu?scriptVersionId=294535277).
\nUltimateStableSWICallback to replace the original solution:skip_mismatch=True)warmup_steps=0 for continued trainingModified training notebooks (only epoch number adjusted):
\nTraining was halted at 830 epochs due to the competition deadline. Validation results by phase:
\n| Epochs | \nValidation Dice | \nValidation Loss | \nBest Epoch | \n
|---|---|---|---|
| 0-50 | \n0.8075 | \n0.9753 | \n48 | \n
| 50-250 | \n0.8140 | \n0.9396 | \n249 | \n
| 250-450 | \n0.8176 | \n0.9256 | \n444 | \n
| 450-600 | \n0.8193 | \n0.9200 | \n598 | \n
| 600-830 | \n0.8223 | \n0.8937 | \n816 | \n
The inference code is based on Tony Li · PaulG · Yiheng Wang's notebook (https://www.kaggle.com/code/tonylica/vesuvius-0-552), with 3-level overlap sliding window and 7-fold TTA.
\nBest Submission (Code 1): https://www.kaggle.com/code/tonyai007/vesuvius-seresnext50-comboloss-830ft
\nAblation on Inference Params:
\nAblation on Training Epochs (Code 1 params):
\nSubmission (Code 2): https://www.kaggle.com/code/tonyai007/inference-vesuvius-surface-3d-detection-epoch-x5?scriptVersionId=300146063
\nI thank all upvoters for their support, which helped me earn two Notebook Gold Medals.
", "messages": [ { "id": "3415184", "postDate": "02/28/2026 11:15:54", "content": "Thank you for the organizers and Kaggle team for this great competition! I also thank all open-source contributors for sharing high-quality code and models.
\nMy model for continued training is based on Innat's pre-trained TransUNet model (https://www.kaggle.com/models/ipythonx/vsd-model/Keras/transunet/3?select=transunet.seresnext50.160px.comboloss.weights.h5), and the training code is adapted from Innat's notebook (https://www.kaggle.com/code/ipythonx/train-vesuvius-surface-3d-detection-on-tpu?scriptVersionId=294535277).
\nUltimateStableSWICallback to replace the original solution:skip_mismatch=True)warmup_steps=0 for continued trainingModified training notebooks (only epoch number adjusted):
\nTraining was halted at 830 epochs due to the competition deadline. Validation results by phase:
\n| Epochs | \nValidation Dice | \nValidation Loss | \nBest Epoch | \n
|---|---|---|---|
| 0-50 | \n0.8075 | \n0.9753 | \n48 | \n
| 50-250 | \n0.8140 | \n0.9396 | \n249 | \n
| 250-450 | \n0.8176 | \n0.9256 | \n444 | \n
| 450-600 | \n0.8193 | \n0.9200 | \n598 | \n
| 600-830 | \n0.8223 | \n0.8937 | \n816 | \n
The inference code is based on Tony Li · PaulG · Yiheng Wang's notebook (https://www.kaggle.com/code/tonylica/vesuvius-0-552), with 3-level overlap sliding window and 7-fold TTA.
\nBest Submission (Code 1): https://www.kaggle.com/code/tonyai007/vesuvius-seresnext50-comboloss-830ft
\nAblation on Inference Params:
\nAblation on Training Epochs (Code 1 params):
\nSubmission (Code 2): https://www.kaggle.com/code/tonyai007/inference-vesuvius-surface-3d-detection-epoch-x5?scriptVersionId=300146063
\nI thank all upvoters for their support, which helped me earn two Notebook Gold Medals.
", "rawMarkdown": "Thank you for the organizers and Kaggle team for this great competition! I also thank all open-source contributors for sharing high-quality code and models.\n\n## 1. Model and Training Foundation\nMy model for continued training is based on Innat's pre-trained TransUNet model (https://www.kaggle.com/models/ipythonx/vsd-model/Keras/transunet/3?select=transunet.seresnext50.160px.comboloss.weights.h5), and the training code is adapted from Innat's notebook (https://www.kaggle.com/code/ipythonx/train-vesuvius-surface-3d-detection-on-tpu?scriptVersionId=294535277).\n\n## 2. Key Modifications\n### 2.1 Model Architecture\n- Replaced SegFormer (mit_b0) with TransUNet (seresnext50)\n- Increased input size from 128³ to 160³\n### 2.2 TPU Optimization & Distributed Computing\n- Added JAX/TPU memory environment variables to enhance stability across 8 TPU cores\n- Adapted distribution strategy for compatibility with different Keras versions\n- Separated training and inference (TPU for training, CPU for inference) to avoid memory overflow\n### 2.3 Sliding Window Inference Callback\n- Implemented a custom `UltimateStableSWICallback` to replace the original solution:\n - Automatic switching between CPU/TPU environments (inference failures do not interrupt training)\n - Proactive garbage collection to free up memory\n### 2.4 Weight Management\n- Supported loading pre-trained weights for continued training (with `skip_mismatch=True`)\n- Implemented a triple weight-saving mechanism: temporary weights, best model (by validation dice), and final model weights\n### 2.5 Training Parameters\n- Expanded validation set from 1 to 2 files to improve robustness\n- Set `warmup_steps=0` for continued training\n- Increased shuffle buffer size from 100 to 400\n\n## 3. Training Process & Results\nModified training notebooks (only epoch number adjusted):\n- 50 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss\n- 250 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-2\n- 450 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-3\n- 600 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-4\n- 830 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-5\n\nTraining was halted at 830 epochs due to the competition deadline. Validation results by phase:\n\n| Epochs | Validation Dice | Validation Loss | Best Epoch |\n|--------|-----------------|-----------------|------------|\n| 0-50 | 0.8075 | 0.9753 | 48 |\n| 50-250 | 0.8140 | 0.9396 | 249 |\n| 250-450| 0.8176 | 0.9256 | 444 |\n| 450-600| 0.8193 | 0.9200 | 598 |\n| 600-830| 0.8223 | 0.8937 | 816 |\n\n## 4. Inference & Results\n### 4.1 Single-Model Inference\nThe inference code is based on Tony Li · PaulG · Yiheng Wang's notebook (https://www.kaggle.com/code/tonylica/vesuvius-0-552), with 3-level overlap sliding window and 7-fold TTA.\n\n**Best Submission (Code 1):** https://www.kaggle.com/code/tonyai007/vesuvius-seresnext50-comboloss-830ft\n- Parameters: overlap_public=0.42, overlap_base=0.48, overlap_hi=0.6, T_low=0.5, T_high=0.9, z_radius=0, xy_radius=2, dust_min_size=200\n- Scores: Public LB 0.565, Private LB 0.582\n\n**Ablation on Inference Params:**\n- Adjusting T_low (0.35) and T_high (0.85) led to lower [scores](https://www.kaggle.com/code/tonyai007/inference-vesuvius-surface-3d-detection-test?scriptVersionId=300377344): Public LB 0.558, Private LB 0.570\n- z_radius impact: 0 > 1 > 2 > 3 > 4 (z_radius=0 achieved the best performance)\n\n**Ablation on Training Epochs (Code 1 params):**\n- 50 epochs: Public LB 0.558, Private LB 0.571 (z_radius=1)\n- 250 epochs: Public LB 0.563, Private LB 0.573 (z_radius=0)\n- 450 epochs: Public LB 0.556, Private LB 0.577 (z_radius=0)\n- 600 epochs: Public LB 0.562, Private LB 0.574 (z_radius=0)\n- 830 epochs: Public LB 0.565, Private LB 0.582 (z_radius=0)\n\n### 4.2 Multi-Model Inference\nSubmission (Code 2): https://www.kaggle.com/code/tonyai007/inference-vesuvius-surface-3d-detection-epoch-x5?scriptVersionId=300146063\n- Scores: Public LB 0.566, Private LB 0.577\nKey settings:\nCombination of 3 epoch + 7TTA: Adjustments to **overlap**, **T_low/T_high**, and **z_radius** affected public LB scores, but private LB remained stable at ≈0.577\nCombination of 5 epoch + 3/4TTA: Changes to the same hyperparameters impacted public LB, with private LB consistently around 0.571\nNote: Multi-model fusion improved public LB score but underperformed compared to the single 830-epoch model on private LB (0.577 vs. 0.582), which may have led to misinterpretation.\n\n### 4.3 Baseline Comparisons\n- [SegFormer](https://www.kaggle.com/code/ipythonx/train-vesuvius-surface-3d-detection-on-tpu) (128³, 1200 epochs on TPU, ≈25 hours, 1overlap + 7TTA): Public/Private LB 0.524\n- [nnUNet](https://www.kaggle.com/code/jirkaborovec/surface-nnunet-training-inference-with-2xt4) (3d_lowres/fold_all, 500 epochs on T4 GPU, ≈40 hours): Public/Private LB 0.53 (200ep: 0.513 / 300ep: 0.521 / 400ep: 0.523)\n\n## Acknowledgements\nI thank all upvoters for their support, which helped me earn two Notebook Gold Medals.", "votes": null }, { "id": "3415191", "postDate": "02/28/2026 11:59:07", "content": "@tonyai007 Congrats. Did you perform all your experiment in kaggle only?
", "rawMarkdown": "tonyai007 Congrats. Did you perform all your experiment in kaggle only?", "votes": null }, { "id": "3415225", "postDate": "02/28/2026 13:00:12", "content": "@Innat Yes, I only used Kaggle's graphics card. Thank you for providing the training and high score prediction code. I have learned a lot
", "rawMarkdown": "Innat Yes, I only used Kaggle's graphics card. Thank you for providing the training and high score prediction code. I have learned a lot", "votes": null } ], "comments": [ { "id": 3415191, "author_name": "ipythonx", "author_url": "", "post_date": "02/28/2026 11:59:07", "content": "@tonyai007 Congrats. Did you perform all your experiment in kaggle only?
", "votes": null, "replies": [ { "id": 3415225, "author_name": "tonyai007", "author_url": "", "post_date": "02/28/2026 13:00:12", "content": "@Innat Yes, I only used Kaggle's graphics card. Thank you for providing the training and high score prediction code. I have learned a lot
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3415184": "Thank you for the organizers and Kaggle team for this great competition! I also thank all open-source contributors for sharing high-quality code and models.\n\n## 1. Model and Training Foundation\nMy model for continued training is based on Innat's pre-trained TransUNet model (https://www.kaggle.com/models/ipythonx/vsd-model/Keras/transunet/3?select=transunet.seresnext50.160px.comboloss.weights.h5), and the training code is adapted from Innat's notebook (https://www.kaggle.com/code/ipythonx/train-vesuvius-surface-3d-detection-on-tpu?scriptVersionId=294535277).\n\n## 2. Key Modifications\n### 2.1 Model Architecture\n- Replaced SegFormer (mit_b0) with TransUNet (seresnext50)\n- Increased input size from 128³ to 160³\n### 2.2 TPU Optimization & Distributed Computing\n- Added JAX/TPU memory environment variables to enhance stability across 8 TPU cores\n- Adapted distribution strategy for compatibility with different Keras versions\n- Separated training and inference (TPU for training, CPU for inference) to avoid memory overflow\n### 2.3 Sliding Window Inference Callback\n- Implemented a custom `UltimateStableSWICallback` to replace the original solution:\n - Automatic switching between CPU/TPU environments (inference failures do not interrupt training)\n - Proactive garbage collection to free up memory\n### 2.4 Weight Management\n- Supported loading pre-trained weights for continued training (with `skip_mismatch=True`)\n- Implemented a triple weight-saving mechanism: temporary weights, best model (by validation dice), and final model weights\n### 2.5 Training Parameters\n- Expanded validation set from 1 to 2 files to improve robustness\n- Set `warmup_steps=0` for continued training\n- Increased shuffle buffer size from 100 to 400\n\n## 3. Training Process & Results\nModified training notebooks (only epoch number adjusted):\n- 50 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss\n- 250 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-2\n- 450 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-3\n- 600 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-4\n- 830 epochs: https://www.kaggle.com/code/tonyai007/train-vesuvius-seresnext50-comboloss-5\n\nTraining was halted at 830 epochs due to the competition deadline. Validation results by phase:\n\n| Epochs | Validation Dice | Validation Loss | Best Epoch |\n|--------|-----------------|-----------------|------------|\n| 0-50 | 0.8075 | 0.9753 | 48 |\n| 50-250 | 0.8140 | 0.9396 | 249 |\n| 250-450| 0.8176 | 0.9256 | 444 |\n| 450-600| 0.8193 | 0.9200 | 598 |\n| 600-830| 0.8223 | 0.8937 | 816 |\n\n## 4. Inference & Results\n### 4.1 Single-Model Inference\nThe inference code is based on Tony Li · PaulG · Yiheng Wang's notebook (https://www.kaggle.com/code/tonylica/vesuvius-0-552), with 3-level overlap sliding window and 7-fold TTA.\n\n**Best Submission (Code 1):** https://www.kaggle.com/code/tonyai007/vesuvius-seresnext50-comboloss-830ft\n- Parameters: overlap_public=0.42, overlap_base=0.48, overlap_hi=0.6, T_low=0.5, T_high=0.9, z_radius=0, xy_radius=2, dust_min_size=200\n- Scores: Public LB 0.565, Private LB 0.582\n\n**Ablation on Inference Params:**\n- Adjusting T_low (0.35) and T_high (0.85) led to lower [scores](https://www.kaggle.com/code/tonyai007/inference-vesuvius-surface-3d-detection-test?scriptVersionId=300377344): Public LB 0.558, Private LB 0.570\n- z_radius impact: 0 > 1 > 2 > 3 > 4 (z_radius=0 achieved the best performance)\n\n**Ablation on Training Epochs (Code 1 params):**\n- 50 epochs: Public LB 0.558, Private LB 0.571 (z_radius=1)\n- 250 epochs: Public LB 0.563, Private LB 0.573 (z_radius=0)\n- 450 epochs: Public LB 0.556, Private LB 0.577 (z_radius=0)\n- 600 epochs: Public LB 0.562, Private LB 0.574 (z_radius=0)\n- 830 epochs: Public LB 0.565, Private LB 0.582 (z_radius=0)\n\n### 4.2 Multi-Model Inference\nSubmission (Code 2): https://www.kaggle.com/code/tonyai007/inference-vesuvius-surface-3d-detection-epoch-x5?scriptVersionId=300146063\n- Scores: Public LB 0.566, Private LB 0.577\nKey settings:\nCombination of 3 epoch + 7TTA: Adjustments to **overlap**, **T_low/T_high**, and **z_radius** affected public LB scores, but private LB remained stable at ≈0.577\nCombination of 5 epoch + 3/4TTA: Changes to the same hyperparameters impacted public LB, with private LB consistently around 0.571\nNote: Multi-model fusion improved public LB score but underperformed compared to the single 830-epoch model on private LB (0.577 vs. 0.582), which may have led to misinterpretation.\n\n### 4.3 Baseline Comparisons\n- [SegFormer](https://www.kaggle.com/code/ipythonx/train-vesuvius-surface-3d-detection-on-tpu) (128³, 1200 epochs on TPU, ≈25 hours, 1overlap + 7TTA): Public/Private LB 0.524\n- [nnUNet](https://www.kaggle.com/code/jirkaborovec/surface-nnunet-training-inference-with-2xt4) (3d_lowres/fold_all, 500 epochs on T4 GPU, ≈40 hours): Public/Private LB 0.53 (200ep: 0.513 / 300ep: 0.521 / 400ep: 0.523)\n\n## Acknowledgements\nI thank all upvoters for their support, which helped me earn two Notebook Gold Medals.", "3415191": "tonyai007 Congrats. Did you perform all your experiment in kaggle only?", "3415225": "Innat Yes, I only used Kaggle's graphics card. Thank you for providing the training and high score prediction code. I have learned a lot" }, "source": "meta" }