![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3460252%2F999f9fc8a0b32bfc93a6e3490803859d%2Fsifd-inference-pipeline.png?generation=1778090730128978&alt=media\")
Scientific figures come in different types, simple single-panel images and compound multi-panel figures. Many images in this competition were compound figures containing multiple panels -- microscopy images, gel blots, graphs, labels, and annotations arranged together.
\nOur approach: first understand the structure of the figure, then run reuse detection at the panel level. This kept the matching focused on actual scientific content rather than figure layout, labels, or borders.
\nOur solution focused on inter-panel reuse detection , comparing panels against each other within the same figure. We also planned intra-panel copy-move detection (forgery within a single panel), but this module was partially implemented and not completed in time for the final submission.
\n
Figure 1. Inference pipeline. Panel detection runs first, then a figure type decision determines whether matching is needed. Compound multi-panel figures go through the full pipeline. All other figure types are predicted as authentic without running matching.
\nCode: https://github.com/Nivratti/recodai-luc-sifd-4th-place-solution
\nThe repository contains a cleaned and easier to setup version of our final inference pipeline. The public version focuses on the approach used for the competition submission and avoids unrelated post-competition experiments.
\nWe downloaded millions of scientific figures from the PubMed Central (PMC) Open Access subset for analysis purposes.
\nRaw PMC packages were downloaded paper by paper, figures were extracted from each package, and the raw package was then deleted to keep disk usage manageable.
\nAfter extraction, we ran the YOLOv5 panel detector on figures to split them into three buckets: figures containing at least one Blot or Microscopy panel, figures containing only other panel types such as Flow Cytometry or Graphs, and figures where no panels were detected. Keeping these buckets separate made it easier to browse and analyze the distribution of real-world scientific figure layouts, panel sizes, aspect ratios, and arrangement patterns.
\nA subset of the downloaded PMC figures is publicly available on Kaggle:
\n\nAfter reviewing the test set image and the article links provided by the competition host, it was clear that this was not a simple image forgery detection problem. Scientific figures are compound images with multiple panels, each representing a different experiment or condition. We decided early on to build a panel-first approach.
\nWe used a YOLOv5 panel detector (adapted from ResearchIntegrity/panel-extractor) to locate panels inside each figure. The model classifies panels into four types: Blots, Microscopy, Flow Cytometry, and Graphs .
\nAfter analyzing the dataset, we focused only on Blots and Microscopy panels. These were the panel types where forgery actually appeared in the competition data. Flow Cytometry and graph panels were skipped
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3460252%2Fdb38a03251e9ca6c4e0f1aa3df0ec91f%2Fpanel-detection-example.png?generation=1778090779416396&alt=media\")
Figure 2. Panel detection on two compound scientific figures. Red boxes show detected Blot panels (left), blue boxes show detected Microscopy panels (right), with confidence scores shown for each. These panel crops are used as the main units for pairwise reuse detection.
\nBefore running any matching, we classified every figure into one of four types based on the panel detection output:
\n| Type | \nCondition | \nAction | \n
|---|---|---|
COMPOUND_MULTI | \n≥ 2 Blot/Micro panels detected | \nRun full matching pipeline | \n
SIMPLE | \n1 panel detected touching image edge | \nPredict authentic, skip | \n
COMPOUND_SINGLE | \n1 panel detected but not touching edge | \nPredict authentic, skip | \n
UNKNOWN | \n0 panels detected | \nPredict authentic, skip | \n
Only COMPOUND_MULTI figures went through the full pipeline. All other types were predicted as authentic.
Intra-panel copy-move detection (forgery within a single panel) was planned for COMPOUND_SINGLE , SIMPLE and UNKNOWN figures. The module was partially implemented but not completed in time for the final submission, so these figures were predicted as authentic.
For compound multi-panel figures, we extracted all detected panel crops. Comparing every possible panel pair is expensive. We added a three-level pruning step before running the keypoint matcher:
\nLevel 1 -- Broad Grouping\nPanels were grouped by class (blot, microscopy, other). Matching was only performed within groups
Level 2 -- Geometry Filter\nPairs were rejected if aspect ratios or areas were too different to be copies.
\nLevel 3 -- CBIR Shortlisting\nAll panels were embedded using ResNet-50. For each panel, only the top-K most similar panels proceeded to keypoint matching. We used cbir_topk=3 for the final submission.
For each candidate panel pair, we ran keypoint-based matching to detect reused regions. The matcher is based on the ResearchIntegrity/copy-move-detection-keypoint project, extended with a Python API for pairwise cross-image matching and feature caching.
\nFeature Extraction\nWe used RootSIFT, which applies Hellinger normalization to SIFT descriptors. This improves matching quality for images with similar local textures, which is common in gel blots. Panels smaller than 300px on any side were upscaled before extraction to ensure enough keypoints.
\nFlip Detection\nFeatures were extracted from both the original and horizontally mirrored panel. If the flipped match produced a larger shared region, that result was used instead.
\nGeometric Verification\nMatched keypoints were verified using MAGSAC++, which is more robust than standard RANSAC against outliers, important for panels with repeated or noisy textures.
\nMatch Decision\nA pair was declared a match if it had ≥ 20 geometrically verified inlier keypoints. Shared area was estimated via convex hull over the matched keypoints.
\nThe reuse detector produced masks in panel-crop coordinates. The competition required masks in original image coordinates.
\nFor every matched crop mask, we projected it back to the full figure using the detected panel bounding box. This was a small but critical implementation detail, a reused region detected correctly at crop level will still score poorly if the final mask is misaligned in the original image.
\nAfter grouping and projection, we applied a final filtering step before encoding the submission.
\nWe removed masks smaller than 10 pixels, merged strongly overlapping masks, and dropped anything outside the image boundary. If no valid mask remained after filtering, the figure was submitted as authentic.
Otherwise, the remaining instance masks were RLE encoded into the competition submission format.
\nPanel-first approach\nProcessing figures at the panel level was the most important decision. Detecting panels first kept the matching focused on actual scientific content and avoided false matches from labels, borders, and figure layout.
\nSkipping figures that didn't need matching\nWe only ran the full pipeline on compound multi-panel figures. Simple, single-panel, and unknown figures were predicted as authentic without running any matching, as module to handle that figure types was not fully implemented. This significantly reduced false positives.
\nThree-level candidate pruning\nGrouping panels by class, filtering geometrically incompatible pairs, and using CBIR to shortlist similar candidates before keypoint matching reduced computation.
\nMAGSAC++ for geometric verification\nMAGSAC++ was more robust than standard RANSAC for panels with repeated or noisy textures, which is common in gel blots.
\nConnected component grouping\nGrouping matched pairs into connected components avoided duplicate instance masks when the same reused content appeared across multiple panels.
\nModular architecture\nKeeping panel detection, keypoint matching, and CBIR as separate installable packages made it easy to benchmark and swap each component independently.
\nDirectly running general image forgery detection models on the full image did not work well.
\nWe tried reducing the maximum side length of panel images in reuse detection.
\nIt reduced accuracy when the image size became too small. Keypoint-based matching depends on local visual details, and aggressive resizing can remove small structures that are useful for matching.
\nBecause of this, we avoided resizing too aggressively in the reuse detection step.
\nUsing the same pipeline for every image was less reliable. The figure type decision step helped us avoid unnecessary predictions.
\nAggressive mask filtering or merging was risky.
\nIf filtering was too strong, it could remove useful small masks. If merging was too aggressive, it will combine separate reused regions into one incorrect instance.
\nSeveral things were planned but not fully completed before the submission deadline.
\nA model to detect and segment copy-move forgery within a single panel image.
\nWe explored VisMatch to evaluate multiple matchers including MatchAnything-RoMa, MatchAnything-EfficientLoFTR, and others.
\nA model to segment annotation elements inside panels — text labels, arrows, scale bars, callouts, and bounding boxes etc.. The purpose was to pass the resulting annotation mask as context to the intra-panel forgery segmnetation model, allowing it to focus on actual scientific content rather than figure annotations.
\nThanks to Recod.ai, LUC, and the competition organizers for hosting this interesting and challenging competition.
\nThis solution builds on two open source projects by ResearchIntegrity:
\nThanks to the Kaggle community for useful discussions.
", "messages": [ { "id": 3454207, "postDate": "2026-05-06T18:08:33.103Z", "content": "Scientific figures come in different types, simple single-panel images and compound multi-panel figures. Many images in this competition were compound figures containing multiple panels -- microscopy images, gel blots, graphs, labels, and annotations arranged together.
\nOur approach: first understand the structure of the figure, then run reuse detection at the panel level. This kept the matching focused on actual scientific content rather than figure layout, labels, or borders.
\nOur solution focused on inter-panel reuse detection , comparing panels against each other within the same figure. We also planned intra-panel copy-move detection (forgery within a single panel), but this module was partially implemented and not completed in time for the final submission.
\n
Figure 1. Inference pipeline. Panel detection runs first, then a figure type decision determines whether matching is needed. Compound multi-panel figures go through the full pipeline. All other figure types are predicted as authentic without running matching.
\nCode: https://github.com/Nivratti/recodai-luc-sifd-4th-place-solution
\nThe repository contains a cleaned and easier to setup version of our final inference pipeline. The public version focuses on the approach used for the competition submission and avoids unrelated post-competition experiments.
\nWe downloaded millions of scientific figures from the PubMed Central (PMC) Open Access subset for analysis purposes.
\nRaw PMC packages were downloaded paper by paper, figures were extracted from each package, and the raw package was then deleted to keep disk usage manageable.
\nAfter extraction, we ran the YOLOv5 panel detector on figures to split them into three buckets: figures containing at least one Blot or Microscopy panel, figures containing only other panel types such as Flow Cytometry or Graphs, and figures where no panels were detected. Keeping these buckets separate made it easier to browse and analyze the distribution of real-world scientific figure layouts, panel sizes, aspect ratios, and arrangement patterns.
\nA subset of the downloaded PMC figures is publicly available on Kaggle:
\n\nAfter reviewing the test set image and the article links provided by the competition host, it was clear that this was not a simple image forgery detection problem. Scientific figures are compound images with multiple panels, each representing a different experiment or condition. We decided early on to build a panel-first approach.
\nWe used a YOLOv5 panel detector (adapted from ResearchIntegrity/panel-extractor) to locate panels inside each figure. The model classifies panels into four types: Blots, Microscopy, Flow Cytometry, and Graphs .
\nAfter analyzing the dataset, we focused only on Blots and Microscopy panels. These were the panel types where forgery actually appeared in the competition data. Flow Cytometry and graph panels were skipped
\n
Figure 2. Panel detection on two compound scientific figures. Red boxes show detected Blot panels (left), blue boxes show detected Microscopy panels (right), with confidence scores shown for each. These panel crops are used as the main units for pairwise reuse detection.
\nBefore running any matching, we classified every figure into one of four types based on the panel detection output:
\n| Type | \nCondition | \nAction | \n
|---|---|---|
COMPOUND_MULTI | \n≥ 2 Blot/Micro panels detected | \nRun full matching pipeline | \n
SIMPLE | \n1 panel detected touching image edge | \nPredict authentic, skip | \n
COMPOUND_SINGLE | \n1 panel detected but not touching edge | \nPredict authentic, skip | \n
UNKNOWN | \n0 panels detected | \nPredict authentic, skip | \n
Only COMPOUND_MULTI figures went through the full pipeline. All other types were predicted as authentic.
Intra-panel copy-move detection (forgery within a single panel) was planned for COMPOUND_SINGLE , SIMPLE and UNKNOWN figures. The module was partially implemented but not completed in time for the final submission, so these figures were predicted as authentic.
For compound multi-panel figures, we extracted all detected panel crops. Comparing every possible panel pair is expensive. We added a three-level pruning step before running the keypoint matcher:
\nLevel 1 -- Broad Grouping\nPanels were grouped by class (blot, microscopy, other). Matching was only performed within groups
Level 2 -- Geometry Filter\nPairs were rejected if aspect ratios or areas were too different to be copies.
\nLevel 3 -- CBIR Shortlisting\nAll panels were embedded using ResNet-50. For each panel, only the top-K most similar panels proceeded to keypoint matching. We used cbir_topk=3 for the final submission.
For each candidate panel pair, we ran keypoint-based matching to detect reused regions. The matcher is based on the ResearchIntegrity/copy-move-detection-keypoint project, extended with a Python API for pairwise cross-image matching and feature caching.
\nFeature Extraction\nWe used RootSIFT, which applies Hellinger normalization to SIFT descriptors. This improves matching quality for images with similar local textures, which is common in gel blots. Panels smaller than 300px on any side were upscaled before extraction to ensure enough keypoints.
\nFlip Detection\nFeatures were extracted from both the original and horizontally mirrored panel. If the flipped match produced a larger shared region, that result was used instead.
\nGeometric Verification\nMatched keypoints were verified using MAGSAC++, which is more robust than standard RANSAC against outliers, important for panels with repeated or noisy textures.
\nMatch Decision\nA pair was declared a match if it had ≥ 20 geometrically verified inlier keypoints. Shared area was estimated via convex hull over the matched keypoints.
\nThe reuse detector produced masks in panel-crop coordinates. The competition required masks in original image coordinates.
\nFor every matched crop mask, we projected it back to the full figure using the detected panel bounding box. This was a small but critical implementation detail, a reused region detected correctly at crop level will still score poorly if the final mask is misaligned in the original image.
\nAfter grouping and projection, we applied a final filtering step before encoding the submission.
\nWe removed masks smaller than 10 pixels, merged strongly overlapping masks, and dropped anything outside the image boundary. If no valid mask remained after filtering, the figure was submitted as authentic.
Otherwise, the remaining instance masks were RLE encoded into the competition submission format.
\nPanel-first approach\nProcessing figures at the panel level was the most important decision. Detecting panels first kept the matching focused on actual scientific content and avoided false matches from labels, borders, and figure layout.
\nSkipping figures that didn't need matching\nWe only ran the full pipeline on compound multi-panel figures. Simple, single-panel, and unknown figures were predicted as authentic without running any matching, as module to handle that figure types was not fully implemented. This significantly reduced false positives.
\nThree-level candidate pruning\nGrouping panels by class, filtering geometrically incompatible pairs, and using CBIR to shortlist similar candidates before keypoint matching reduced computation.
\nMAGSAC++ for geometric verification\nMAGSAC++ was more robust than standard RANSAC for panels with repeated or noisy textures, which is common in gel blots.
\nConnected component grouping\nGrouping matched pairs into connected components avoided duplicate instance masks when the same reused content appeared across multiple panels.
\nModular architecture\nKeeping panel detection, keypoint matching, and CBIR as separate installable packages made it easy to benchmark and swap each component independently.
\nDirectly running general image forgery detection models on the full image did not work well.
\nWe tried reducing the maximum side length of panel images in reuse detection.
\nIt reduced accuracy when the image size became too small. Keypoint-based matching depends on local visual details, and aggressive resizing can remove small structures that are useful for matching.
\nBecause of this, we avoided resizing too aggressively in the reuse detection step.
\nUsing the same pipeline for every image was less reliable. The figure type decision step helped us avoid unnecessary predictions.
\nAggressive mask filtering or merging was risky.
\nIf filtering was too strong, it could remove useful small masks. If merging was too aggressive, it will combine separate reused regions into one incorrect instance.
\nSeveral things were planned but not fully completed before the submission deadline.
\nA model to detect and segment copy-move forgery within a single panel image.
\nWe explored VisMatch to evaluate multiple matchers including MatchAnything-RoMa, MatchAnything-EfficientLoFTR, and others.
\nA model to segment annotation elements inside panels — text labels, arrows, scale bars, callouts, and bounding boxes etc.. The purpose was to pass the resulting annotation mask as context to the intra-panel forgery segmnetation model, allowing it to focus on actual scientific content rather than figure annotations.
\nThanks to Recod.ai, LUC, and the competition organizers for hosting this interesting and challenging competition.
\nThis solution builds on two open source projects by ResearchIntegrity:
\nThanks to the Kaggle community for useful discussions.
", "rawMarkdown": "## Overview\n\nScientific figures come in different types, simple single-panel images and compound multi-panel figures. Many images in this competition were **compound figures** containing multiple panels -- microscopy images, gel blots, graphs, labels, and annotations arranged together.\n\nOur approach: first understand the **structure** of the figure, then run reuse detection at the **panel level**. This kept the matching focused on actual scientific content rather than figure layout, labels, or borders.\n\nOur solution focused on **inter-panel reuse detection** , comparing panels against each other within the same figure. We also planned **intra-panel copy-move detection** (forgery within a single panel), but this module was partially implemented and not completed in time for the final submission.\n\n\n\n**Figure 1.** Inference pipeline. Panel detection runs first, then a figure type decision determines whether matching is needed. Compound multi-panel figures go through the full pipeline. All other figure types are predicted as authentic without running matching.\n\n---\n\n## Code\n\nCode: [https://github.com/Nivratti/recodai-luc-sifd-4th-place-solution](https://github.com/Nivratti/recodai-luc-sifd-4th-place-solution)\n\nThe repository contains a cleaned and easier to setup version of our final inference pipeline. The public version focuses on the approach used for the competition submission and avoids unrelated post-competition experiments.\n\n---\n\n## Data\n\n### External Data -- PubMed Central (PMC)\n\nWe downloaded millions of scientific figures from the [PubMed Central (PMC)](https://www.ncbi.nlm.nih.gov/pmc/) Open Access subset for analysis purposes.\n\nRaw PMC packages were downloaded paper by paper, figures were extracted from each package, and the raw package was then deleted to keep disk usage manageable.\n\nAfter extraction, we ran the YOLOv5 panel detector on figures to split them into three buckets: figures containing at least one Blot or Microscopy panel, figures containing only other panel types such as Flow Cytometry or Graphs, and figures where no panels were detected. Keeping these buckets separate made it easier to browse and analyze the distribution of real-world scientific figure layouts, panel sizes, aspect ratios, and arrangement patterns.\n\nA subset of the downloaded PMC figures is publicly available on Kaggle:\n\n- [PMC OA Biomedical Figures — Dec 2025 (weekly)](https://www.kaggle.com/datasets/chatrapati/pmc-oa-biomed-figures-20251201-20251208-base)\n- [PMC OA Biomedical Figures — Nov 2025 (monthly)](https://www.kaggle.com/datasets/chatrapati/pmc-oa-biomed-figures-2025-nov-month-base)\n\n---\n\n## Solution Details\n\n### 1. Panel Detection\n\nAfter reviewing the test set image and the article links provided by the competition host, it was clear that this was not a simple image forgery detection problem. Scientific figures are compound images with multiple panels, each representing a different experiment or condition. We decided early on to build a panel-first approach.\n\nWe used a **YOLOv5** panel detector (adapted from [ResearchIntegrity/panel-extractor](https://github.com/researchintegrity/panel-extractor)) to locate panels inside each figure. The model classifies panels into four types: **Blots**, **Microscopy**, **Flow Cytometry**, and **Graphs** .\n\nAfter analyzing the dataset, we focused only on **Blots** and **Microscopy** panels. These were the panel types where forgery actually appeared in the competition data. Flow Cytometry and graph panels were skipped\n\n\n\n**Figure 2.** Panel detection on two compound scientific figures. Red boxes show detected **Blot** panels (left), blue boxes show detected **Microscopy** panels (right), with confidence scores shown for each. These panel crops are used as the main units for pairwise reuse detection.\n\n---\n\n### 2. Figure Kind Classification\n\nBefore running any matching, we classified every figure into one of four types based on the panel detection output:\n\n| Type | Condition | Action |\n| ------------------- | -------------------------------------- | ---------------------------- |\n| `COMPOUND_MULTI` | ≥ 2 Blot/Micro panels detected | Run full matching pipeline |\n| `SIMPLE` | 1 panel detected touching image edge | Predict `authentic`, skip |\n| `COMPOUND_SINGLE` | 1 panel detected but not touching edge | Predict `authentic`, skip |\n| `UNKNOWN` | 0 panels detected | Predict `authentic`, skip |\n\n