First, I’d like to thank the competition organizers and the open-source community—your work and shared code were invaluable in helping me learn and improve.
\n1. Synthetic Training Data Generation
\nTo augment the dataset, I generated approximately 10 million additional training samples. To optimize storage efficiency, inputs were downsampled to dimensions (5, 500, 70) and stored in np.float16 format. The synthetic data consisted of:
\n40% Gaussian noise/random rotations/scaling/shifts
\n40% Mixup-augmented samples
\n20% Samples generated via Denoising Diffusion Implicit Models (DDIM)
\nCode references:
\nImproved VEL to SEIS
\nWaveform Inversion (VEL to SEIS)
2. Model Architecture
\nConvNeXtV2-Base (convnextv2_base.fcmae_ft_in22k_in1k_384)
\nCaFormer-B36 (caformer_b36.sail_in22k_ft_in1k_384)
\nModifications were adapted from:
\nCaFormer Full-Resolution Improved
\nConvNeXt Full-Resolution Baseline
3. Training Strategy
\nStage 1 (Primary Training)
Trained for 60 epochs on the synthetic dataset.
\nCross-validation (CV) scores:
\nCaFormer: 9.2/ConvNeXtV2: 10.2/Ensemble: 8.1
\nLeaderboard (LB) / Privateboard (PB): 9.9 / 10.0
\nStage 2 (Test-Time Fine-Tuning)
\nFurther fine-tuned for 2 epochs on synthetic test-derived data.
\nGenerated additional training samples via:
\nForward modeling predictions on test data (y → x).
\nApplied horizontal flipping (HFlip) for augmentation.
\nTotal synthetic data: 300k samples (from 75,818 test/validation samples × 2 models × 2 flips).
\nResulting CV scores:
\nCaFormer: 8.6/ConvNeXtV2: 9.0/Ensemble: 7.6
\nLeaderboard (LB) / Privateboard (PB): 8.8 / 8.9
4. Late-Stage Insights
\nDiscovered Stage 2’s significant impact in the final hours (~1.1 LB improvement).
\nTraining was halted 30 minutes before the deadline—further data generation and training could likely have yielded additional gains.
", "messages": [ { "id": "3237533", "postDate": "07/01/2025 06:41:22", "content": "First, I’d like to thank the competition organizers and the open-source community—your work and shared code were invaluable in helping me learn and improve.
\n1. Synthetic Training Data Generation
\nTo augment the dataset, I generated approximately 10 million additional training samples. To optimize storage efficiency, inputs were downsampled to dimensions (5, 500, 70) and stored in np.float16 format. The synthetic data consisted of:
\n40% Gaussian noise/random rotations/scaling/shifts
\n40% Mixup-augmented samples
\n20% Samples generated via Denoising Diffusion Implicit Models (DDIM)
\nCode references:
\nImproved VEL to SEIS
\nWaveform Inversion (VEL to SEIS)
2. Model Architecture
\nConvNeXtV2-Base (convnextv2_base.fcmae_ft_in22k_in1k_384)
\nCaFormer-B36 (caformer_b36.sail_in22k_ft_in1k_384)
\nModifications were adapted from:
\nCaFormer Full-Resolution Improved
\nConvNeXt Full-Resolution Baseline
3. Training Strategy
\nStage 1 (Primary Training)
Trained for 60 epochs on the synthetic dataset.
\nCross-validation (CV) scores:
\nCaFormer: 9.2/ConvNeXtV2: 10.2/Ensemble: 8.1
\nLeaderboard (LB) / Privateboard (PB): 9.9 / 10.0
\nStage 2 (Test-Time Fine-Tuning)
\nFurther fine-tuned for 2 epochs on synthetic test-derived data.
\nGenerated additional training samples via:
\nForward modeling predictions on test data (y → x).
\nApplied horizontal flipping (HFlip) for augmentation.
\nTotal synthetic data: 300k samples (from 75,818 test/validation samples × 2 models × 2 flips).
\nResulting CV scores:
\nCaFormer: 8.6/ConvNeXtV2: 9.0/Ensemble: 7.6
\nLeaderboard (LB) / Privateboard (PB): 8.8 / 8.9
4. Late-Stage Insights
\nDiscovered Stage 2’s significant impact in the final hours (~1.1 LB improvement).
\nTraining was halted 30 minutes before the deadline—further data generation and training could likely have yielded additional gains.
", "rawMarkdown": "First, I’d like to thank the competition organizers and the open-source community—your work and shared code were invaluable in helping me learn and improve.\n\n\n**1. Synthetic Training Data Generation**\n\nTo augment the dataset, I generated approximately 10 million additional training samples. To optimize storage efficiency, inputs were downsampled to dimensions (5, 500, 70) and stored in np.float16 format. The synthetic data consisted of:\n\n40% Gaussian noise/random rotations/scaling/shifts\n\n40% Mixup-augmented samples\n\n20% Samples generated via Denoising Diffusion Implicit Models (DDIM)\n\nCode references:\n[Improved VEL to SEIS]( https://www.kaggle.com/code/manatoyo/improved-vel-to-seis/notebook)\n[Waveform Inversion (VEL to SEIS)](https://www.kaggle.com/code/jaewook704/waveform-inversion-vel-to-seis)\n\n**2. Model Architecture**\n\nConvNeXtV2-Base (convnextv2_base.fcmae_ft_in22k_in1k_384)\n\nCaFormer-B36 (caformer_b36.sail_in22k_ft_in1k_384)\n\nModifications were adapted from:\n\n[CaFormer Full-Resolution Improved](https://www.kaggle.com/code/brendanartley/caformer-full-resolution-improved)\n[ConvNeXt Full-Resolution Baseline]( https://www.kaggle.com/code/brendanartley/convnext-full-resolution-baseline)\n\n**3. Training Strategy**\n*Stage 1 (Primary Training)*\n\nTrained for 60 epochs on the synthetic dataset.\n\nCross-validation (CV) scores:\n\nCaFormer: 9.2/ConvNeXtV2: 10.2/Ensemble: 8.1\n\nLeaderboard (LB) / Privateboard (PB): 9.9 / 10.0\n\n\n*Stage 2 (Test-Time Fine-Tuning)*\n\nFurther fine-tuned for 2 epochs on synthetic test-derived data.\n\nGenerated additional training samples via:\n\nForward modeling predictions on test data (y → x).\n\nApplied horizontal flipping (HFlip) for augmentation.\n\nTotal synthetic data: 300k samples (from 75,818 test/validation samples × 2 models × 2 flips).\n\nResulting CV scores:\n\nCaFormer: 8.6/ConvNeXtV2: 9.0/Ensemble: 7.6\nLeaderboard (LB) / Privateboard (PB): 8.8 / 8.9\n\n**4. Late-Stage Insights**\n\nDiscovered Stage 2’s significant impact in the final hours (~1.1 LB improvement).\n\nTraining was halted 30 minutes before the deadline—further data generation and training could likely have yielded additional gains.", "votes": null }, { "id": "3237544", "postDate": "07/01/2025 06:49:06", "content": "I'm so lucky you didn't jump higher! 🤣
\nTraining on test-predict-generation data was also a key for 1st place! Good job for finding it.
Great write up, appreciate you sharing this!
", "rawMarkdown": "Great write up, appreciate you sharing this!", "votes": null }, { "id": "3237679", "postDate": "07/01/2025 08:24:35", "content": "What an elegant solution and an elegant write-up! Huge congrats and thx for sharing 🙂 🙏
", "rawMarkdown": "What an elegant solution and an elegant write-up! Huge congrats and thx for sharing 🙂 🙏", "votes": null }, { "id": "3237709", "postDate": "07/01/2025 08:52:17", "content": "Fantastic job! thx for sharing! 😀
", "rawMarkdown": "Fantastic job! thx for sharing! 😀", "votes": null }, { "id": "3237867", "postDate": "07/01/2025 10:58:20", "content": "Hi! Great work.
\nHow have you got the Leaderboard (LB) / Privateboard (PB): 9.9 / 10.0 and metrics in other case, if you have only one submission on LB?
Late Submission.
", "rawMarkdown": "Late Submission.", "votes": null }, { "id": "3238399", "postDate": "07/01/2025 19:57:37", "content": "I am curious what hardware did you use for training and about how long did this take?
", "rawMarkdown": "I am curious what hardware did you use for training and about how long did this take?", "votes": null }, { "id": "3238603", "postDate": "07/02/2025 02:45:02", "content": "2 8*4090 trained for 13-14 days, but maybe 8-9 days is enough, there was almost no improvement in the last few days.
", "rawMarkdown": "2 8*4090 trained for 13-14 days, but maybe 8-9 days is enough, there was almost no improvement in the last few days.", "votes": null }, { "id": "3239757", "postDate": "07/03/2025 05:24:28", "content": "Just one submission!😳
", "rawMarkdown": "Just one submission!😳", "votes": null }, { "id": "3239813", "postDate": "07/03/2025 07:12:57", "content": "I couldn't wrap my head around generating test-derived data. If I have an auto-encoder I could imagine training only the encoder on test input. But there is no VEL for VEL to SEIS for \"Forward modeling predictions on test data (y → x).\" Could you please share in more details what you did?
", "rawMarkdown": "I couldn't wrap my head around generating test-derived data. If I have an auto-encoder I could imagine training only the encoder on test input. But there is no VEL for VEL to SEIS for \"Forward modeling predictions on test data (y → x).\" Could you please share in more details what you did?", "votes": null }, { "id": "3239817", "postDate": "07/03/2025 07:19:32", "content": "VEL is your trained model predicitions on test data, then we can use Forward modeling to get the SEIS, and fine-tuning on the generating test-derived data.
", "rawMarkdown": "VEL is your trained model predicitions on test data, then we can use Forward modeling to get the SEIS, and fine-tuning on the generating test-derived data.", "votes": null } ], "comments": [ { "id": 3237544, "author_name": "shlomoron", "author_url": "", "post_date": "07/01/2025 06:49:06", "content": "I'm so lucky you didn't jump higher! 🤣
\nTraining on test-predict-generation data was also a key for 1st place! Good job for finding it.
Great write up, appreciate you sharing this!
", "votes": null, "replies": [] }, { "id": 3237679, "author_name": "radek1", "author_url": "", "post_date": "07/01/2025 08:24:35", "content": "What an elegant solution and an elegant write-up! Huge congrats and thx for sharing 🙂 🙏
", "votes": null, "replies": [] }, { "id": 3237709, "author_name": "", "author_url": "", "post_date": "07/01/2025 08:52:17", "content": "Fantastic job! thx for sharing! 😀
", "votes": null, "replies": [] }, { "id": 3237867, "author_name": "ivanblch", "author_url": "", "post_date": "07/01/2025 10:58:20", "content": "Hi! Great work.
\nHow have you got the Leaderboard (LB) / Privateboard (PB): 9.9 / 10.0 and metrics in other case, if you have only one submission on LB?
Late Submission.
", "votes": null, "replies": [] } ] }, { "id": 3238399, "author_name": "cody11null", "author_url": "", "post_date": "07/01/2025 19:57:37", "content": "I am curious what hardware did you use for training and about how long did this take?
", "votes": null, "replies": [ { "id": 3238603, "author_name": "hydantess", "author_url": "", "post_date": "07/02/2025 02:45:02", "content": "2 8*4090 trained for 13-14 days, but maybe 8-9 days is enough, there was almost no improvement in the last few days.
", "votes": null, "replies": [] } ] }, { "id": 3239757, "author_name": "graceqianshun", "author_url": "", "post_date": "07/03/2025 05:24:28", "content": "Just one submission!😳
", "votes": null, "replies": [] }, { "id": 3239813, "author_name": "tinkei", "author_url": "", "post_date": "07/03/2025 07:12:57", "content": "I couldn't wrap my head around generating test-derived data. If I have an auto-encoder I could imagine training only the encoder on test input. But there is no VEL for VEL to SEIS for \"Forward modeling predictions on test data (y → x).\" Could you please share in more details what you did?
", "votes": null, "replies": [ { "id": 3239817, "author_name": "hydantess", "author_url": "", "post_date": "07/03/2025 07:19:32", "content": "VEL is your trained model predicitions on test data, then we can use Forward modeling to get the SEIS, and fine-tuning on the generating test-derived data.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3237533": "First, I’d like to thank the competition organizers and the open-source community—your work and shared code were invaluable in helping me learn and improve.\n\n\n**1. Synthetic Training Data Generation**\n\nTo augment the dataset, I generated approximately 10 million additional training samples. To optimize storage efficiency, inputs were downsampled to dimensions (5, 500, 70) and stored in np.float16 format. The synthetic data consisted of:\n\n40% Gaussian noise/random rotations/scaling/shifts\n\n40% Mixup-augmented samples\n\n20% Samples generated via Denoising Diffusion Implicit Models (DDIM)\n\nCode references:\n[Improved VEL to SEIS]( https://www.kaggle.com/code/manatoyo/improved-vel-to-seis/notebook)\n[Waveform Inversion (VEL to SEIS)](https://www.kaggle.com/code/jaewook704/waveform-inversion-vel-to-seis)\n\n**2. Model Architecture**\n\nConvNeXtV2-Base (convnextv2_base.fcmae_ft_in22k_in1k_384)\n\nCaFormer-B36 (caformer_b36.sail_in22k_ft_in1k_384)\n\nModifications were adapted from:\n\n[CaFormer Full-Resolution Improved](https://www.kaggle.com/code/brendanartley/caformer-full-resolution-improved)\n[ConvNeXt Full-Resolution Baseline]( https://www.kaggle.com/code/brendanartley/convnext-full-resolution-baseline)\n\n**3. Training Strategy**\n*Stage 1 (Primary Training)*\n\nTrained for 60 epochs on the synthetic dataset.\n\nCross-validation (CV) scores:\n\nCaFormer: 9.2/ConvNeXtV2: 10.2/Ensemble: 8.1\n\nLeaderboard (LB) / Privateboard (PB): 9.9 / 10.0\n\n\n*Stage 2 (Test-Time Fine-Tuning)*\n\nFurther fine-tuned for 2 epochs on synthetic test-derived data.\n\nGenerated additional training samples via:\n\nForward modeling predictions on test data (y → x).\n\nApplied horizontal flipping (HFlip) for augmentation.\n\nTotal synthetic data: 300k samples (from 75,818 test/validation samples × 2 models × 2 flips).\n\nResulting CV scores:\n\nCaFormer: 8.6/ConvNeXtV2: 9.0/Ensemble: 7.6\nLeaderboard (LB) / Privateboard (PB): 8.8 / 8.9\n\n**4. Late-Stage Insights**\n\nDiscovered Stage 2’s significant impact in the final hours (~1.1 LB improvement).\n\nTraining was halted 30 minutes before the deadline—further data generation and training could likely have yielded additional gains.", "3237544": "I'm so lucky you didn't jump higher! 🤣 \nTraining on test-predict-generation data was also a key for 1st place! Good job for finding it.", "3237551": "Great write up, appreciate you sharing this!", "3237679": "What an elegant solution and an elegant write-up! Huge congrats and thx for sharing 🙂 🙏", "3237709": "Fantastic job! thx for sharing! 😀", "3237867": "Hi! Great work.\nHow have you got the Leaderboard (LB) / Privateboard (PB): 9.9 / 10.0 and metrics in other case, if you have only one submission on LB?", "3237964": "Late Submission.", "3238399": "I am curious what hardware did you use for training and about how long did this take?", "3238603": "2 8*4090 trained for 13-14 days, but maybe 8-9 days is enough, there was almost no improvement in the last few days.", "3239757": "Just one submission!😳", "3239813": "I couldn't wrap my head around generating test-derived data. If I have an auto-encoder I could imagine training only the encoder on test input. But there is no VEL for VEL to SEIS for \"Forward modeling predictions on test data (y → x).\" Could you please share in more details what you did?", "3239817": "VEL is your trained model predicitions on test data, then we can use Forward modeling to get the SEIS, and fine-tuning on the generating test-derived data." }, "source": "meta" }