Thanks to the organizers and everyone who shared insights. Here is a short writeup of our 4th-place solution (final Public LB 0.4611, submission submission_r2_full_b10.0_yr3.0.csv).
The repository is published as-is — daily reports, session notes, and exploratory dead-ends are all there, so you can see the actual day-to-day process, not a polished retrospective.
\nThe core ideas:
\nViTD2PC24All (DINOv2 ViT-B/14) on test-tile pseudo-labels — head-only, FixMatch-style.The single biggest jump came from (1)+(2). Post-processing then gave the last few points.
\nReproduced the 2025 winner's pipeline:
\n| step | \ndata | \nmodel | \nLB | \n
|---|---|---|---|
| baseline | \n— | \nofficial ViTD2PC24All | \n0.405 | \n
| Round 1 | \n17,676 test tiles / 340 spp (top1>0.7, top2<0.2), hard label | \nhead-only, 5 ep | \n0.460 | \n
| Round 2 | \nre-inferred with R1 model → new pseudo-labels | \nhead-only, 5 ep | \n0.461 | \n
Choices that mattered:
\nApplied on top of the per-tile species scores:
\nquadrat_id across different years (this is the \"temporal\" prior).The interesting observation: after domain adaptation, the optimal year boost moved from b=2 → b=3, while consensus and cluster stayed the same. Adapting to the plot-image domain made cross-year detections of the same species more consistent, so year boost stops generating false positives and can be pushed harder.
\n| post-proc | \nLB | \n
|---|---|
| raw max-pool | \n0.4496 | \n
| + cluster + geo | \n0.4514 | \n
| + consensus (b=8) | \n0.4588 | \n
| + year (b=2) | \n0.4601 | \n
| + year (b=3) | \n0.4611 | \n
Thanks again — congrats to the top teams!
", "messages": [ { "id": "3455033", "postDate": "05/08/2026 13:03:39", "content": "Thanks to the organizers and everyone who shared insights. Here is a short writeup of our 4th-place solution (final Public LB 0.4611, submission submission_r2_full_b10.0_yr3.0.csv).
The repository is published as-is — daily reports, session notes, and exploratory dead-ends are all there, so you can see the actual day-to-day process, not a polished retrospective.
\nThe core ideas:
\nViTD2PC24All (DINOv2 ViT-B/14) on test-tile pseudo-labels — head-only, FixMatch-style.The single biggest jump came from (1)+(2). Post-processing then gave the last few points.
\nReproduced the 2025 winner's pipeline:
\n| step | \ndata | \nmodel | \nLB | \n
|---|---|---|---|
| baseline | \n— | \nofficial ViTD2PC24All | \n0.405 | \n
| Round 1 | \n17,676 test tiles / 340 spp (top1>0.7, top2<0.2), hard label | \nhead-only, 5 ep | \n0.460 | \n
| Round 2 | \nre-inferred with R1 model → new pseudo-labels | \nhead-only, 5 ep | \n0.461 | \n
Choices that mattered:
\nApplied on top of the per-tile species scores:
\nquadrat_id across different years (this is the \"temporal\" prior).The interesting observation: after domain adaptation, the optimal year boost moved from b=2 → b=3, while consensus and cluster stayed the same. Adapting to the plot-image domain made cross-year detections of the same species more consistent, so year boost stops generating false positives and can be pushed harder.
\n| post-proc | \nLB | \n
|---|---|
| raw max-pool | \n0.4496 | \n
| + cluster + geo | \n0.4514 | \n
| + consensus (b=8) | \n0.4588 | \n
| + year (b=2) | \n0.4601 | \n
| + year (b=3) | \n0.4611 | \n
Thanks again — congrats to the top teams!
", "rawMarkdown": "# 4th place solution — Self-training on test tiles + temporal/cluster post-processing\n\nThanks to the organizers and everyone who shared insights. Here is a short writeup of our 4th-place solution (final Public LB **0.4611**, submission `submission_r2_full_b10.0_yr3.0.csv`).\n\n- **Code & full experiment log**: https://github.com/sugupoko/2026_kaggle_PlantCLEF2026\n- **Worked entirely with [Claude Code](https://www.anthropic.com/claude-code)**, on top of my own Kaggle starter template: https://github.com/sugupoko/xxxx_kaggle_starterRepository\n\nThe repository is published as-is — daily reports, session notes, and exploratory dead-ends are all there, so you can see the actual day-to-day process, not a polished retrospective.\n\n## Overview\n\nThe core ideas:\n\n1. **Domain-adapt the official `ViTD2PC24All` (DINOv2 ViT-B/14) on test-tile pseudo-labels** — head-only, FixMatch-style.\n2. **Iterate the pseudo-labeling once** (Round 2).\n3. **Stack four post-processing priors at inference** — consensus / year / cluster / geo.\n\nThe single biggest jump came from (1)+(2). Post-processing then gave the last few points.\n\n## Pre-processing & tiling\n\nReproduced the 2025 winner's pipeline:\n\n- Lanczos resize + JPEG re-compression (q=85, YCbCr 4:2:2)\n- Multi-scale tiling, scales 1..6 → 91 tiles / image, all at 518 px (matching ViT input)\n- Tile aggregation: max-pool over species\n\n## Self-training (the main contribution)\n\n| step | data | model | LB |\n|---|---|---|---|\n| baseline | — | official ViTD2PC24All | 0.405 |\n| Round 1 | 17,676 test tiles / 340 spp (top1>0.7, top2<0.2), hard label | head-only, 5 ep | 0.460 |\n| **Round 2** | re-inferred with R1 model → new pseudo-labels | head-only, 5 ep | **0.461** |\n\nChoices that mattered:\n\n- **Hard labels beat soft labels** by +0.015–0.018. With teacher = student, soft uncertainty carries no information; hard + high threshold = entropy minimization (FixMatch).\n- **30% pseudo-label sampler**. Without oversampling (the natural ratio is ~2%) the adaptation does nothing.\n- **Use the *last* checkpoint, not the *best***. Val accuracy on PlantCLEF-style single-plant images keeps decreasing as the model adapts to plot-style images, but LB keeps going up. Best-on-val ≠ best-on-test under domain shift.\n- Lowering the confidence threshold (top1>0.6) added more pseudo-labels but no gain — extra noise canceled extra coverage.\n\n## Post-processing stack\n\nApplied on top of the per-tile species scores:\n\n1. **Consensus boost (b=10)** — boost species seen across many quadrats inside the same cluster.\n2. **Year boost (yr=3)** — boost species detected at the same `quadrat_id` across different years (this is the \"temporal\" prior).\n3. **Cluster prior (α=3)** — per-cluster species prior.\n4. **Geo filter** — drop species impossible at the location.\n\nThe interesting observation: after domain adaptation, the optimal **year boost moved from b=2 → b=3**, while consensus and cluster stayed the same. Adapting to the plot-image domain made cross-year detections of the same species more consistent, so year boost stops generating false positives and can be pushed harder.\n\n## Ablation (post-processing on the R2 model)\n\n| post-proc | LB |\n|---|---|\n| raw max-pool | 0.4496 |\n| + cluster + geo | 0.4514 |\n| + consensus (b=8) | 0.4588 |\n| + year (b=2) | 0.4601 |\n| + year (b=3) | **0.4611** |\n\n## What didn't work / didn't help\n\n- Soft pseudo-labels\n- Lower confidence threshold (top1 > 0.6)\n- Best-on-val checkpoint\n- Adding more pseudo-label rounds beyond Round 2 (no time to fully verify, but Round 2 was already at noise floor)\n\n## Things I wish I had tried\n\n- Using the 212K LUCAS unlabeled images for pseudo-labeling (largest unused signal in this competition)\n- Unfreezing more than the head in later rounds\n- Ensembling R1 and R2 score tensors instead of replacing\n\nThanks again — congrats to the top teams!", "votes": null }, { "id": "3455246", "postDate": "05/08/2026 22:36:21", "content": "Cool solution, thanks for sharing! As a note, we experimented with semi-supervised learning (FixMatch-style) on the LUCAS dataset, but the image distribution seems to differ significantly from the test set. As a result, the pseudo-labels were too noisy to provide a useful learning signal. Would be interesting to see if others can make it work.
", "rawMarkdown": "Cool solution, thanks for sharing! As a note, we experimented with semi-supervised learning (FixMatch-style) on the LUCAS dataset, but the image distribution seems to differ significantly from the test set. As a result, the pseudo-labels were too noisy to provide a useful learning signal. Would be interesting to see if others can make it work.", "votes": null }, { "id": "3456211", "postDate": "05/11/2026 13:24:57", "content": "Thank you for sharing this detailed writeup and for your efforts throughout the competition. Congratulations on achieving 4th place! Your observations regarding the self-training approach, the domain shift, and the post-processing rules are highly relevant. We hope you will consider writing these findings and sharing them by submitting a working note for the LifeCLEF evaluation campaign :)
", "rawMarkdown": "Thank you for sharing this detailed writeup and for your efforts throughout the competition. Congratulations on achieving 4th place! Your observations regarding the self-training approach, the domain shift, and the post-processing rules are highly relevant. We hope you will consider writing these findings and sharing them by submitting a working note for the LifeCLEF evaluation campaign :)", "votes": null }, { "id": "3456669", "postDate": "05/12/2026 11:49:12", "content": "Wow, this is a really strong solution. I especially like the self-training on test tiles and the temporal/cluster post-processing. Congratulations on 4th place and thank you for sharing
", "rawMarkdown": "Wow, this is a really strong solution. I especially like the self-training on test tiles and the temporal/cluster post-processing. Congratulations on 4th place and thank you for sharing", "votes": null }, { "id": "3456969", "postDate": "05/12/2026 20:40:44", "content": "Thanks for sharing with such important knowledge and experience. Congratulations on 4th place, great job!
", "rawMarkdown": "Thanks for sharing with such important knowledge and experience. Congratulations on 4th place, great job!", "votes": null } ], "comments": [ { "id": 3455246, "author_name": "alpere", "author_url": "", "post_date": "05/08/2026 22:36:21", "content": "Cool solution, thanks for sharing! As a note, we experimented with semi-supervised learning (FixMatch-style) on the LUCAS dataset, but the image distribution seems to differ significantly from the test set. As a result, the pseudo-labels were too noisy to provide a useful learning signal. Would be interesting to see if others can make it work.
", "votes": null, "replies": [] }, { "id": 3456211, "author_name": "hgoeau", "author_url": "", "post_date": "05/11/2026 13:24:57", "content": "Thank you for sharing this detailed writeup and for your efforts throughout the competition. Congratulations on achieving 4th place! Your observations regarding the self-training approach, the domain shift, and the post-processing rules are highly relevant. We hope you will consider writing these findings and sharing them by submitting a working note for the LifeCLEF evaluation campaign :)
", "votes": null, "replies": [] }, { "id": 3456669, "author_name": "stevefffff", "author_url": "", "post_date": "05/12/2026 11:49:12", "content": "Wow, this is a really strong solution. I especially like the self-training on test tiles and the temporal/cluster post-processing. Congratulations on 4th place and thank you for sharing
", "votes": null, "replies": [ { "id": 3456969, "author_name": "sardorrazikov", "author_url": "", "post_date": "05/12/2026 20:40:44", "content": "Thanks for sharing with such important knowledge and experience. Congratulations on 4th place, great job!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3455033": "# 4th place solution — Self-training on test tiles + temporal/cluster post-processing\n\nThanks to the organizers and everyone who shared insights. Here is a short writeup of our 4th-place solution (final Public LB **0.4611**, submission `submission_r2_full_b10.0_yr3.0.csv`).\n\n- **Code & full experiment log**: https://github.com/sugupoko/2026_kaggle_PlantCLEF2026\n- **Worked entirely with [Claude Code](https://www.anthropic.com/claude-code)**, on top of my own Kaggle starter template: https://github.com/sugupoko/xxxx_kaggle_starterRepository\n\nThe repository is published as-is — daily reports, session notes, and exploratory dead-ends are all there, so you can see the actual day-to-day process, not a polished retrospective.\n\n## Overview\n\nThe core ideas:\n\n1. **Domain-adapt the official `ViTD2PC24All` (DINOv2 ViT-B/14) on test-tile pseudo-labels** — head-only, FixMatch-style.\n2. **Iterate the pseudo-labeling once** (Round 2).\n3. **Stack four post-processing priors at inference** — consensus / year / cluster / geo.\n\nThe single biggest jump came from (1)+(2). Post-processing then gave the last few points.\n\n## Pre-processing & tiling\n\nReproduced the 2025 winner's pipeline:\n\n- Lanczos resize + JPEG re-compression (q=85, YCbCr 4:2:2)\n- Multi-scale tiling, scales 1..6 → 91 tiles / image, all at 518 px (matching ViT input)\n- Tile aggregation: max-pool over species\n\n## Self-training (the main contribution)\n\n| step | data | model | LB |\n|---|---|---|---|\n| baseline | — | official ViTD2PC24All | 0.405 |\n| Round 1 | 17,676 test tiles / 340 spp (top1>0.7, top2<0.2), hard label | head-only, 5 ep | 0.460 |\n| **Round 2** | re-inferred with R1 model → new pseudo-labels | head-only, 5 ep | **0.461** |\n\nChoices that mattered:\n\n- **Hard labels beat soft labels** by +0.015–0.018. With teacher = student, soft uncertainty carries no information; hard + high threshold = entropy minimization (FixMatch).\n- **30% pseudo-label sampler**. Without oversampling (the natural ratio is ~2%) the adaptation does nothing.\n- **Use the *last* checkpoint, not the *best***. Val accuracy on PlantCLEF-style single-plant images keeps decreasing as the model adapts to plot-style images, but LB keeps going up. Best-on-val ≠ best-on-test under domain shift.\n- Lowering the confidence threshold (top1>0.6) added more pseudo-labels but no gain — extra noise canceled extra coverage.\n\n## Post-processing stack\n\nApplied on top of the per-tile species scores:\n\n1. **Consensus boost (b=10)** — boost species seen across many quadrats inside the same cluster.\n2. **Year boost (yr=3)** — boost species detected at the same `quadrat_id` across different years (this is the \"temporal\" prior).\n3. **Cluster prior (α=3)** — per-cluster species prior.\n4. **Geo filter** — drop species impossible at the location.\n\nThe interesting observation: after domain adaptation, the optimal **year boost moved from b=2 → b=3**, while consensus and cluster stayed the same. Adapting to the plot-image domain made cross-year detections of the same species more consistent, so year boost stops generating false positives and can be pushed harder.\n\n## Ablation (post-processing on the R2 model)\n\n| post-proc | LB |\n|---|---|\n| raw max-pool | 0.4496 |\n| + cluster + geo | 0.4514 |\n| + consensus (b=8) | 0.4588 |\n| + year (b=2) | 0.4601 |\n| + year (b=3) | **0.4611** |\n\n## What didn't work / didn't help\n\n- Soft pseudo-labels\n- Lower confidence threshold (top1 > 0.6)\n- Best-on-val checkpoint\n- Adding more pseudo-label rounds beyond Round 2 (no time to fully verify, but Round 2 was already at noise floor)\n\n## Things I wish I had tried\n\n- Using the 212K LUCAS unlabeled images for pseudo-labeling (largest unused signal in this competition)\n- Unfreezing more than the head in later rounds\n- Ensembling R1 and R2 score tensors instead of replacing\n\nThanks again — congrats to the top teams!", "3455246": "Cool solution, thanks for sharing! As a note, we experimented with semi-supervised learning (FixMatch-style) on the LUCAS dataset, but the image distribution seems to differ significantly from the test set. As a result, the pseudo-labels were too noisy to provide a useful learning signal. Would be interesting to see if others can make it work.", "3456211": "Thank you for sharing this detailed writeup and for your efforts throughout the competition. Congratulations on achieving 4th place! Your observations regarding the self-training approach, the domain shift, and the post-processing rules are highly relevant. We hope you will consider writing these findings and sharing them by submitting a working note for the LifeCLEF evaluation campaign :)", "3456669": "Wow, this is a really strong solution. I especially like the self-training on test tiles and the temporal/cluster post-processing. Congratulations on 4th place and thank you for sharing", "3456969": "Thanks for sharing with such important knowledge and experience. Congratulations on 4th place, great job!" }, "source": "meta" }