{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":113558,"databundleVersionId":14878066,"isSourceIdPinned":false,"sourceType":"competition"},{"sourceId":90860,"sourceType":"modelInstanceVersion","isSourceIdPinned":false,"modelInstanceId":76172,"modelId":100857}],"dockerImageVersionId":31192,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Experiment Motivation: Using SAM2 for Forgery Candidate Generation\n\n**Goal:** Evaluate whether the \"Segment Anything Model 2\" (SAM2) can effectively identify and isolate potential forgery regions in scientific images without prior training on this specific task.\n\n**Hypothesis:** \nSAM2 is designed to generate masks for all discernible objects and regions in an image. If a scientific image contains a forgery (such as a copy-move or splicing manipulation), the manipulated region often constitutes a distinct visual entity. Therefore, SAM2 should ideally segment the forged region as a standalone mask.\n\n**Why this matters:**\nIf SAM2 successfully segments these manipulated regions, it can serve as a powerful, unsupervised **\"candidate generator\"** for forgery detection pipelines. Instead of scanning the entire image with a sliding window, we could:\n1. Generate masks for all objects using SAM2.\n2. Extract features from these masked regions.\n3. Perform **matching** (e.g., comparing feature vectors of different masks) to find duplicated or anomalous regions.\n\nThis simple experiment visualizes the raw segmentation output on forged images to qualitatively assess if the forged artifacts are being captured as individual segments.","metadata":{}},{"cell_type":"code","source":"!pip list | grep torch","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-15T13:12:37.281779Z","iopub.execute_input":"2026-01-15T13:12:37.282358Z","iopub.status.idle":"2026-01-15T13:12:40.628688Z","shell.execute_reply.started":"2026-01-15T13:12:37.282331Z","shell.execute_reply":"2026-01-15T13:12:40.627761Z"},"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport glob\n\nPATH_DATASET = \"/kaggle/input/recodai-luc-scientific-image-forgery-detection\"\nauthentic_images = glob.glob(os.path.join(PATH_DATASET, 'train_images', 'authentic', '*.png'))\nforged_images = glob.glob(os.path.join(PATH_DATASET, 'train_images', 'forged', '*.png'))\nforged_images += glob.glob(os.path.join(PATH_DATASET, 'supplemental_images', '*.png'))\n\nprint(f\"Found {len(authentic_images)} authentic images.\")\nprint(f\"Found {len(forged_images)} forged images.\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2026-01-15T13:12:40.630356Z","iopub.execute_input":"2026-01-15T13:12:40.630865Z","iopub.status.idle":"2026-01-15T13:12:40.698083Z","shell.execute_reply.started":"2026-01-15T13:12:40.630837Z","shell.execute_reply":"2026-01-15T13:12:40.697539Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 📦✈️ Generate package for offline","metadata":{}},{"cell_type":"code","source":"!pip uninstall -y torchaudio\n!pip wheel git+https://github.com/facebookresearch/segment-anything-2.git -q -w packages --extra-index-url=\"https://download.pytorch.org/whl/cu124\"\n!pip download \"torch<2.9.0\" \"torchvision<0.24.0\" -q -d packages --index-url=\"https://download.pytorch.org/whl/cu124\"\n# !pip download \"libraft-cu12==25.6.*\" \"pylibraft-cu12==25.6.*\" \"pylibcugraph-cu12==25.6.0\" \"rmm-cu12==25.6.*\" -q -d packages --extra-index-url=\"https://download.pytorch.org/whl/cu124\"","metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2026-01-15T13:12:40.698783Z","iopub.execute_input":"2026-01-15T13:12:40.699021Z","iopub.status.idle":"2026-01-15T13:18:17.083772Z","shell.execute_reply.started":"2026-01-15T13:12:40.699003Z","shell.execute_reply":"2026-01-15T13:18:17.082701Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!pip install -q sam_2 \"torch<2.9.0\" \"torchvision<0.24.0\" -f packages/ --no-index\n!pip list | grep torch","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2026-01-15T13:18:17.086134Z","iopub.execute_input":"2026-01-15T13:18:17.086424Z","iopub.status.idle":"2026-01-15T13:19:04.184689Z","shell.execute_reply.started":"2026-01-15T13:18:17.086399Z","shell.execute_reply":"2026-01-15T13:19:04.183949Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🏗️🧠 Build the SAM model","metadata":{}},{"cell_type":"code","source":"import torch\nfrom sam2.build_sam import build_sam2\nfrom sam2.automatic_mask_generator import SAM2AutomaticMaskGenerator\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nprint(f\"Using device: {device}\")\n\n# Specify the path to the pre-trained model weights file.\n# This corresponds to the \"brain\" of the AI.\ncheckpoint_path = \"/kaggle/input/segment-anything-2/pytorch/sam2-hiera-base-plus/1/sam2_hiera_base_plus.pt\"\n\n# Specify the configuration file, which acts as the model's blueprint.\nmodel_config = \"sam2_hiera_b+.yaml\"\n\n# Build the SAM2 model using the blueprint and the weight file.\n# apply_postprocessing=False is set to obtain the raw output from the model.\nsam2_model = build_sam2(model_config, checkpoint_path, device=device, apply_postprocessing=False)\n\n# Create a \"Mask Generator\" to automatically detect all objects within the image.\nmask_generator = SAM2AutomaticMaskGenerator(sam2_model)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-15T13:19:04.186165Z","iopub.execute_input":"2026-01-15T13:19:04.186387Z","iopub.status.idle":"2026-01-15T13:19:15.744696Z","shell.execute_reply.started":"2026-01-15T13:19:04.186367Z","shell.execute_reply":"2026-01-15T13:19:15.744094Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## ▶️🖼️ Run sample segmentations","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\n\ndef display_image_with_annotations(image, annotations, figsize=(12, 12)):\n    \"\"\"Overlays masks on an image and returns the Matplotlib Figure object.\"\"\"\n    # 1. Create the Figure and Axes objects explicitly\n    fig, ax = plt.subplots(figsize=figsize)\n    # 2. Display the base image on the axes\n    ax.imshow(image)\n    ax.axis('off') # Hide the axis ruler/numbers\n    # 3. If no annotations, return the figure with just the base image\n    if not annotations:\n        return fig\n    # 4. Sort masks: Largest first\n    annotations.sort(key=lambda x: x['area'], reverse=True)\n    # 5. Create the RGBA overlay layer\n    h, w = image.shape[:2]\n    overlay_rgba = np.zeros((h, w, 4), dtype=np.float32)\n    # 6. Draw masks onto the overlay layer\n    for ann in annotations:\n        mask = ann['segmentation']\n        rgb = np.random.random(3) # Random RGB color\n        overlay_rgba[mask, :3] = rgb # Color\n        overlay_rgba[mask, 3] = 0.5  # Alpha (Transparency)\n    # 7. Add the overlay to the axes\n    ax.imshow(overlay_rgba)\n    # 8. Return the figure object instead of showing it immediately\n    return fig","metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2026-01-15T13:19:15.745414Z","iopub.execute_input":"2026-01-15T13:19:15.745703Z","iopub.status.idle":"2026-01-15T13:19:15.751568Z","shell.execute_reply.started":"2026-01-15T13:19:15.745685Z","shell.execute_reply":"2026-01-15T13:19:15.750734Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import random\nimport numpy as np\nimport torch\nimport gc\n\n# Reduce batch size to save memory during generation\n# Default is usually 64; reducing it lowers peak GPU memory usage.\n# if hasattr(mask_generator, 'points_per_batch'):\n#     mask_generator.points_per_batch = 32\n\ndef move_masks_to_cpu(masks):\n    \"\"\"Moves mask data to CPU/numpy to free GPU memory.\"\"\"\n    for mask in masks:\n        for k, v in mask.items():\n            if isinstance(v, torch.Tensor):\n                mask[k] = v.cpu().numpy()\n    return masks\n\n\ndef generate_masks_for_image(image):\n    if image.ndim == 2: # Grayscale image\n        image = np.stack([image, image, image], axis=-1)\n    else:\n        image = image[..., :3] # Ensure RGB or RGBA becomes RGB\n\n    # Run in inference mode and with mixed precision (float16) to drastically reduce memory usage\n    with torch.inference_mode(), torch.autocast(\"cuda\", dtype=torch.float16):\n        try:\n            masks = mask_generator.generate(image)\n        except Exception:  # OutOfMemoryError\n            print(f\"image with dim {image.shape} is too large <- '{img_path}'\")\n            torch.cuda.empty_cache()\n            return None\n    # Move masks data to CPU before saving and clear GPU memory\n    return move_masks_to_cpu(masks)\n\n\nfor img_path in random.sample(forged_images, 10):\n    # Explicitly clear cache before each large allocation\n    gc.collect(); torch.cuda.empty_cache()\n\n    image = plt.imread(img_path)\n    print(f\"Starting segmentation for '{os.path.basename(img_path)}'...\")\n    masks = generate_masks_for_image(image)\n    if masks:\n        display_image_with_annotations(image, masks, figsize=None)\n\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-15T13:27:45.394567Z","iopub.execute_input":"2026-01-15T13:27:45.395259Z","iopub.status.idle":"2026-01-15T13:28:28.364847Z","shell.execute_reply.started":"2026-01-15T13:27:45.395235Z","shell.execute_reply":"2026-01-15T13:28:28.363991Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🧱🤯 Generate mask for all","metadata":{}},{"cell_type":"code","source":"from tqdm.auto import tqdm\nimport gc\n\n# Create output directory\noutput_dir = 'SAM2_masks_dataset'\nos.makedirs(output_dir, exist_ok=True)\n\n# run only on GPU\nif not torch.cuda.is_available():\n    forged_images = []\n\nfor img_path in tqdm(forged_images, desc=\"Processing forged images\"):\n    image = plt.imread(img_path)\n\n    # Run in inference mode and with ...\n    masks = generate_masks_for_image(image)\n    if not masks:\n        gc.collect()\n        continue\n    del image # Delete image from memory\n\n    # Create a unique filename for the mask file based on the image name\n    base_name = os.path.basename(img_path)\n    save_name = os.path.splitext(base_name)[0] + '.npz'\n    save_path = os.path.join(output_dir, save_name)\n    # Save the list of masks to a compressed .npz file\n    # We convert the list of dicts to an object array to store it in the npz archive\n    np.savez_compressed(save_path, masks=np.array(masks, dtype=object))\n\n    del masks # Delete masks from memory\n    gc.collect(); torch.cuda.empty_cache()\n\nprint(f\"Saved individual masks to directory: {output_dir}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-15T13:28:28.365888Z","iopub.execute_input":"2026-01-15T13:28:28.366098Z","execution_failed":"2026-01-15T13:31:42.271Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🗃️📤 Export as dataset","metadata":{}},{"cell_type":"code","source":"# !echo '{\"username\":\"your-name\",\"key\":\"KEY-HASH-HERE\"}' > /root/.config/kaggle/kaggle.json","metadata":{"trusted":true,"execution":{"execution_failed":"2026-01-15T13:31:42.271Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import json\nimport shutil\nimport warnings\n\n# Define directories\nsource_dir = 'SAM2_masks_dataset'\nupload_dir = 'dataset_upload'\nos.makedirs(upload_dir, exist_ok=True)\n\n# 1. Create a zip archive of the dataset content\nprint(\"Creating zip archive...\")\n# This creates 'SAM2_masks.zip' inside 'dataset_upload' containing files from 'source_dir'\nshutil.make_archive(os.path.join(upload_dir, 'SAM2_masks'), 'zip', source_dir)\n!rm -rf {source_dir}\n\n# 2. Fetch Kaggle username\ntry:\n    from kaggle.api.kaggle_api_extended import KaggleApi\n    # Fetch Kaggle username for the metadata ID\n    api = KaggleApi()\n    api.authenticate()\nexcept OSError as err:\n    warnings.warn(str(err))\n    api = None\nusername = api.config_values['username'] if api else \"\"\n\n# 3. Create dataset-metadata.json inside the upload directory\nmetadata = {\n    \"title\": \"Forgery Detection: Segment Anything with SAM2\",\n    \"id\": f\"{username}/forgery-segment-sam2\",\n    \"licenses\": [{\"name\": \"CC0-1.0\"}]\n}\n\nmetadata_path = os.path.join(upload_dir, \"dataset-metadata.json\")\nwith open(metadata_path, \"w\") as f:\n    json.dump(metadata, f, indent=4)\n\nprint(f\"Metadata and archive prepared in: {upload_dir}\")\n\n# 4. Create the dataset by pointing to the upload directory\n# The directory contains: dataset-metadata.json and SAM2_masks.zip\n# !kaggle datasets create -p {upload_dir}","metadata":{"trusted":true,"execution":{"execution_failed":"2026-01-15T13:31:42.271Z"}},"outputs":[],"execution_count":null}]}