{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"competition","sourceId":113558,"databundleVersionId":14878066},{"sourceType":"datasetVersion","sourceId":14386957,"datasetId":9153851,"databundleVersionId":15200055},{"sourceType":"modelInstanceVersion","sourceId":4534,"databundleVersionId":6346558,"modelInstanceId":3326}],"dockerImageVersionId":31236,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Hybrid DINOv2-UNet Model for Image Forgery Detection\n## Thanks to Hossam Hamouda and Pankaj Gupta for the ideas and the dataset. (links after submission)\n## If you have any ideas or suggestions for improving this solution, feel free to share them in the comments.\n### Model Architecture\n\nOur hybrid model combines a DINOv2 encoder with a UNet-style decoder for semantic segmentation of forged regions in images.\n\n**Encoder (DINOv2-based):**\nGiven an input image $I \\in \\mathbb{R}^{H \\times W \\times 3}$, we first preprocess it to size $S \\times S$ (where $S = 720$):\n\n$$I' = \\text{Resize}(I, (S, S))$$\n\nThe DINOv2 encoder extracts features:\n\n$$\\mathbf{F}_{\\text{dino}} = \\text{DINOv2}(I') \\in \\mathbb{R}^{B \\times N \\times 768}$$\n\nwhere $B$ is batch size, $N$ is the number of patches, and 768 is the DINOv2 base dimension. We reshape to spatial feature maps:\n\n$$\\mathbf{F}_{\\text{spatial}} = \\text{Reshape}(\\mathbf{F}_{\\text{dino}}[:, 1:, :]) \\in \\mathbb{R}^{B \\times 768 \\times \\sqrt{N-1} \\times \\sqrt{N-1}}$$\n\nA projection layer maps to decoder dimensions:\n\n$$\\mathbf{E}_4 = \\text{Proj}(\\mathbf{F}_{\\text{spatial}}) = \\text{BN}(\\text{ReLU}(\\text{Conv}_{1 \\times 1}(\\mathbf{F}_{\\text{spatial}}))) \\in \\mathbb{R}^{B \\times 512 \\times H' \\times W'}$$\n\n**Decoder (UNet-style):**\n\nThe bottleneck processes the encoder output:\n\n$$\\mathbf{B} = \\text{Bottleneck}(\\text{Dropout}(\\mathbf{E}_4)) \\in \\mathbb{R}^{B \\times 1024 \\times H'/2 \\times W'/2}$$\n\nThe decoder uses transposed convolutions with skip connections:\n\n$$\\mathbf{D}_4 = \\text{Dec}_4(\\text{Concat}(\\text{Up}_4(\\mathbf{B}), \\mathbf{E}_4))$$\n$$\\mathbf{D}_3 = \\text{Dec}_3(\\text{Concat}(\\text{Up}_3(\\mathbf{D}_4), \\mathbf{E}_3))$$\n$$\\mathbf{D}_2 = \\text{Dec}_2(\\text{Concat}(\\text{Up}_2(\\mathbf{D}_3), \\mathbf{E}_2))$$\n$$\\mathbf{D}_1 = \\text{Dec}_1(\\text{Concat}(\\text{Up}_1(\\mathbf{D}_2), \\mathbf{E}_1))$$\n\nwhere $\\text{Up}_i$ denotes transposed convolution upsampling and $\\text{Dec}_i$ are decoder blocks with SE attention.\n\n**Output:**\nThe final segmentation probability map:\n\n$$P = \\sigma(\\text{Conv}_{1 \\times 1}(\\mathbf{D}_1)) \\in [0, 1]^{B \\times 1 \\times S \\times S}$$\n\nwhere $\\sigma$ is the sigmoid function.\n\n### Loss Function\n\nWe use a combination of Binary Cross-Entropy and Dice loss:\n\n$$\\mathcal{L}_{\\text{BCE}} = -\\frac{1}{N}\\sum_{i=1}^{N} [y_i \\log(p_i) + (1-y_i) \\log(1-p_i)]$$\n\n$$\\mathcal{L}_{\\text{Dice}} = 1 - \\frac{2\\sum_{i} p_i y_i + \\epsilon}{\\sum_{i} p_i + \\sum_{i} y_i + \\epsilon}$$\n\n$$\\mathcal{L}_{\\text{total}} = \\mathcal{L}_{\\text{BCE}} + \\mathcal{L}_{\\text{Dice}}$$\n\nwhere $y_i$ are ground truth labels, $p_i$ are predicted probabilities, and $\\epsilon = 10^{-6}$ is a smoothing term.\n\n### Post-processing Pipeline\n\n**Test Time Augmentation (TTA):**\nWe average predictions from original and flipped images:\n\n$$P_{\\text{final}} = \\frac{1}{3}\\left[P_{\\text{orig}} + \\text{Flip}_H(P_{\\text{hflip}}) + \\text{Flip}_V(P_{\\text{vflip}})\\right]$$\n\n**Adaptive Mask Enhancement:**\nWe compute gradient magnitude using Sobel operators:\n\n$$G_x = \\text{Sobel}_x(P_{\\text{final}}), \\quad G_y = \\text{Sobel}_y(P_{\\text{final}})$$\n$$|\\nabla P| = \\sqrt{G_x^2 + G_y^2}, \\quad |\\nabla P|_{\\text{norm}} = \\frac{|\\nabla P|}{\\max(|\\nabla P|) + \\epsilon}$$\n\nThe enhanced probability map combines original probabilities with gradient information:\n\n$$P_{\\text{enhanced}} = (1-\\alpha) \\cdot P_{\\text{final}} + \\alpha \\cdot |\\nabla P|_{\\text{norm}}$$\n\nwhere $\\alpha = 0.35$ is the gradient weight. After Gaussian blur:\n\n$$P_{\\text{blur}} = \\text{GaussianBlur}(P_{\\text{enhanced}}, \\sigma=1)$$\n\n**Adaptive Thresholding:**\nThe threshold is computed adaptively:\n\n$$\\tau = \\mu(P_{\\text{blur}}) + 0.3 \\cdot \\sigma(P_{\\text{blur}})$$\n\nwhere $\\mu$ and $\\sigma$ are mean and standard deviation. The binary mask:\n\n$$M = \\mathbb{1}[P_{\\text{blur}} > \\tau]$$\n\n**Morphological Operations:**\nWe apply closing and opening to clean the mask:\n\n$$M' = \\text{Open}(\\text{Close}(M, k_1=5), k_2=3)$$\n\n**Filtering:**\nFinal decision based on area and mean probability:\n\n$$\\text{Label} = \\begin{cases}\n\\text{\"forged\"} & \\text{if } \\text{Area}(M') \\geq 300 \\text{ and } \\bar{P}_{\\text{inside}} \\geq 0.25 \\\\\n\\text{\"authentic\"} & \\text{otherwise}\n\\end{cases}$$\n\nwhere $\\bar{P}_{\\text{inside}} = \\frac{1}{|\\Omega|}\\sum_{(i,j) \\in \\Omega} P_{\\text{final}}(i,j)$ and $\\Omega = \\{(i,j) : M'(i,j) = 1\\}$.\n","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:44.785145Z","iopub.execute_input":"2026-01-04T10:08:44.785449Z","iopub.status.idle":"2026-01-04T10:08:44.789344Z","shell.execute_reply.started":"2026-01-04T10:08:44.785421Z","shell.execute_reply":"2026-01-04T10:08:44.788721Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import transforms\nfrom PIL import Image\nimport cv2\nfrom pathlib import Path\nimport os\nimport json\nimport math\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as mpatches\nimport seaborn as sns\n\n\n# DINOv2\ntry:\n    from transformers import AutoImageProcessor, AutoModel\n    HAS_TRANSFORMERS = True\nexcept ImportError as e:\n    HAS_TRANSFORMERS = False\n    print(e)\n\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nprint(f'Device: {device}')\n\nplt.style.use('seaborn-v0_8-darkgrid')\nsns.set_palette(\"husl\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:44.790449Z","iopub.execute_input":"2026-01-04T10:08:44.790669Z","iopub.status.idle":"2026-01-04T10:08:44.808240Z","shell.execute_reply.started":"2026-01-04T10:08:44.790651Z","shell.execute_reply":"2026-01-04T10:08:44.807523Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def rle_encode(mask: np.ndarray, fg_val: int = 1) -> str:\n    pixels = mask.T.flatten()\n    dots = np.where(pixels == fg_val)[0]\n    if len(dots) == 0:\n        return \"authentic\"\n    run_lengths = []\n    prev = -2\n    for b in dots:\n        if b > prev + 1:\n            run_lengths.extend((b + 1, 0))\n        run_lengths[-1] += 1\n        prev = b\n    return json.dumps([int(x) for x in run_lengths])\n\ndef load_mask(path):\n    if os.path.exists(path):\n        mask = np.load(path)\n        if mask.ndim == 3:\n            mask = mask.max(axis=0)\n        return (mask > 0).astype(np.uint8)\n    return None\n\ndef combine_masks(mask_paths):\n    masks = []\n    for path in mask_paths:\n        mask = load_mask(path)\n        if mask is not None:\n            masks.append(mask)\n    if masks:\n        combined = np.zeros_like(masks[0])\n        for mask in masks:\n            combined = np.maximum(combined, mask)\n        return combined\n    return None","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:44.809135Z","iopub.execute_input":"2026-01-04T10:08:44.809572Z","iopub.status.idle":"2026-01-04T10:08:44.821669Z","shell.execute_reply.started":"2026-01-04T10:08:44.809551Z","shell.execute_reply":"2026-01-04T10:08:44.821138Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class ImgDataset(Dataset):\n    def __init__(self, img_paths, mask_paths=None, train=True, sz=720):\n        self.img_paths = img_paths\n        self.mask_paths = mask_paths\n        self.train = train\n        self.sz = sz\n        \n        if train:\n            self.tfms = transforms.Compose([\n                transforms.RandomHorizontalFlip(0.5),\n                transforms.RandomVerticalFlip(0.5),\n                transforms.RandomRotation(90),\n                transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2, hue=0.1)\n            ])\n        \n    def __len__(self):\n        return len(self.img_paths)\n    \n    def __getitem__(self, idx):\n        img_path = self.img_paths[idx]\n        \n        try:\n            img = Image.open(img_path).convert('RGB')\n        except:\n            img = Image.new('RGB', (self.sz, self.sz), (128, 128, 128))\n        \n        img = img.resize((self.sz, self.sz), Image.LANCZOS)\n        img = np.array(img).astype(np.float32) / 255.0\n        \n        if self.mask_paths:\n            mask_paths = self.mask_paths[idx]\n            mask = combine_masks(mask_paths) if isinstance(mask_paths, list) else load_mask(mask_paths)\n            if mask is None:\n                mask = np.zeros((img.shape[0], img.shape[1]), dtype=np.uint8)\n            else:\n                mask = cv2.resize(mask, (self.sz, self.sz), interpolation=cv2.INTER_NEAREST)\n            \n            img_tensor = torch.from_numpy(img).permute(2, 0, 1)\n            mask_tensor = torch.from_numpy(mask).unsqueeze(0).float()\n            \n            if self.train:\n                seed = np.random.randint(2147483647)\n                torch.manual_seed(seed)\n                img_tensor = self.tfms(img_tensor)\n                torch.manual_seed(seed)\n                mask_tensor = self.tfms(mask_tensor)\n            \n            return img_tensor, mask_tensor\n        else:\n            img_tensor = torch.from_numpy(img).permute(2, 0, 1)\n            return img_tensor, str(img_path)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:44.874892Z","iopub.execute_input":"2026-01-04T10:08:44.875103Z","iopub.status.idle":"2026-01-04T10:08:44.883274Z","shell.execute_reply.started":"2026-01-04T10:08:44.875086Z","shell.execute_reply":"2026-01-04T10:08:44.882839Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class SEBlock(nn.Module):\n    def __init__(self, ch, r=16):\n        super().__init__()\n        self.se = nn.Sequential(\n            nn.AdaptiveAvgPool2d(1),\n            nn.Conv2d(ch, ch // r, 1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(ch // r, ch, 1),\n            nn.Sigmoid()\n        )\n    \n    def forward(self, x):\n        return x * self.se(x)\n\nclass ConvBlock(nn.Module):\n    def __init__(self, in_ch, out_ch, use_se=True):\n        super().__init__()\n        self.conv = nn.Sequential(\n            nn.Conv2d(in_ch, out_ch, 3, padding=1, bias=False),\n            nn.BatchNorm2d(out_ch),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(out_ch, out_ch, 3, padding=1, bias=False),\n            nn.BatchNorm2d(out_ch),\n            nn.ReLU(inplace=True)\n        )\n        self.se = SEBlock(out_ch) if use_se else nn.Identity()\n    \n    def forward(self, x):\n        return self.se(self.conv(x))\n\n# UNet (fallback)\nclass UNet(nn.Module):\n    def __init__(self, n_channels=3, n_classes=1):\n        super().__init__()\n        self.enc1 = ConvBlock(n_channels, 64)\n        self.enc2 = ConvBlock(64, 128)\n        self.enc3 = ConvBlock(128, 256)\n        self.enc4 = ConvBlock(256, 512)\n        \n        self.pool = nn.MaxPool2d(2)\n        self.drop = nn.Dropout2d(0.1)\n        \n        self.bottleneck = ConvBlock(512, 1024)\n        \n        self.up4 = nn.ConvTranspose2d(1024, 512, 2, stride=2)\n        self.dec4 = ConvBlock(1024, 512)\n        \n        self.up3 = nn.ConvTranspose2d(512, 256, 2, stride=2)\n        self.dec3 = ConvBlock(512, 256)\n        \n        self.up2 = nn.ConvTranspose2d(256, 128, 2, stride=2)\n        self.dec2 = ConvBlock(256, 128)\n        \n        self.up1 = nn.ConvTranspose2d(128, 64, 2, stride=2)\n        self.dec1 = ConvBlock(128, 64)\n        \n        self.final = nn.Conv2d(64, n_classes, 1)\n    \n    def crop(self, x, target):\n        _, _, h, w = target.size()\n        _, _, xh, xw = x.size()\n        diff_h = (xh - h) // 2\n        diff_w = (xw - w) // 2\n        return x[:, :, diff_h:diff_h+h, diff_w:diff_w+w]\n    \n    def forward(self, x):\n        e1 = self.enc1(x)\n        e2 = self.enc2(self.pool(e1))\n        e3 = self.enc3(self.pool(e2))\n        e4 = self.enc4(self.pool(e3))\n        \n        b = self.bottleneck(self.drop(self.pool(e4)))\n        \n        d4 = self.up4(b)\n        d4 = self.crop(d4, e4)\n        d4 = torch.cat([d4, e4], dim=1)\n        d4 = self.dec4(d4)\n        \n        d3 = self.up3(d4)\n        d3 = self.crop(d3, e3)\n        d3 = torch.cat([d3, e3], dim=1)\n        d3 = self.dec3(d3)\n        \n        d2 = self.up2(d3)\n        d2 = self.crop(d2, e2)\n        d2 = torch.cat([d2, e2], dim=1)\n        d2 = self.dec2(d2)\n        \n        d1 = self.up1(d2)\n        d1 = self.crop(d1, e1)\n        d1 = torch.cat([d1, e1], dim=1)\n        d1 = self.dec1(d1)\n        \n        return torch.sigmoid(self.final(d1))\n\n# Hybrid model (DINO's encoder and UNet decoder)\nclass DINOv2UNet(nn.Module):\n    def __init__(self, dino_path=None, n_classes=1, img_size=720, use_dino=True):\n        super().__init__()\n        self.use_dino = use_dino and HAS_TRANSFORMERS\n        self.img_size = img_size\n        \n        if self.use_dino and dino_path:\n            print(f\"Loading DINOv2 from {dino_path}\")\n            try:\n                self.processor = AutoImageProcessor.from_pretrained(\n                    dino_path, local_files_only=True, use_fast=False\n                )\n                self.dino_encoder = AutoModel.from_pretrained(\n                    dino_path, local_files_only=True\n                ).to('cpu')\n                for p in self.dino_encoder.parameters():\n                    p.requires_grad = False\n                self.dino_encoder.eval()\n                dino_dim = 768\n                print(\"DINOv2 loaded on CPU\")\n            except Exception as e:\n                print(e)\n                self.use_dino = False\n                dino_dim = 0\n        else:\n            self.use_dino = False\n            dino_dim = 0\n\n        self.drop = nn.Dropout2d(0.1)\n\n        if self.use_dino:\n            self.dino_proj = nn.Sequential(\n                nn.Conv2d(dino_dim, 512, 1),\n                nn.BatchNorm2d(512),\n                nn.ReLU(inplace=True)\n            )\n            self.dino_proj_e3 = nn.Sequential(\n                nn.Conv2d(512, 256, 1),\n                nn.BatchNorm2d(256),\n                nn.ReLU(inplace=True)\n            )\n            self.dino_proj_e2 = nn.Sequential(\n                nn.Conv2d(512, 128, 1),\n                nn.BatchNorm2d(128),\n                nn.ReLU(inplace=True)\n            )\n            self.dino_proj_e1 = nn.Sequential(\n                nn.Conv2d(512, 64, 1),\n                nn.BatchNorm2d(64),\n                nn.ReLU(inplace=True)\n            )\n            bottleneck_in = 512\n        else:\n            self.enc1 = ConvBlock(3, 64)\n            self.enc2 = ConvBlock(64, 128)\n            self.enc3 = ConvBlock(128, 256)\n            self.enc4 = ConvBlock(256, 512)\n            self.pool = nn.MaxPool2d(2)\n            bottleneck_in = 512\n        \n        self.bottleneck = ConvBlock(bottleneck_in, 1024)\n        \n        self.up4 = nn.ConvTranspose2d(1024, 512, 2, stride=2)\n        self.dec4 = ConvBlock(1024, 512)\n        \n        self.up3 = nn.ConvTranspose2d(512, 256, 2, stride=2)\n        self.dec3 = ConvBlock(512, 256)\n        \n        self.up2 = nn.ConvTranspose2d(256, 128, 2, stride=2)\n        self.dec2 = ConvBlock(256, 128)\n        \n        self.up1 = nn.ConvTranspose2d(128, 64, 2, stride=2)\n        self.dec1 = ConvBlock(128, 64)\n        \n        self.final = nn.Conv2d(64, n_classes, 1)\n    \n    def forward_features_dino(self, x):\n        imgs = (x * 255).clamp(0, 255).byte().permute(0, 2, 3, 1).cpu().numpy()\n        inputs = self.processor(images=list(imgs), return_tensors=\"pt\").to('cpu')\n        \n        if self.training:\n            feats = self.dino_encoder(**inputs).last_hidden_state\n        else:\n            with torch.no_grad():\n                feats = self.dino_encoder(**inputs).last_hidden_state\n        \n        B, N, C = feats.shape\n        fmap = feats[:, 1:, :].permute(0, 2, 1)\n        s = int(math.sqrt(N - 1))\n        fmap = fmap.reshape(B, C, s, s)\n        \n        return fmap.to(x.device)\n    \n    def forward(self, x):\n        if self.use_dino:\n            dino_feats = self.forward_features_dino(x)\n            e4 = self.dino_proj(dino_feats)\n            target_size = x.shape[2] // 16\n            e4 = F.interpolate(e4, size=(target_size, target_size), \n                             mode='bilinear', align_corners=False)\n            e3_interp = F.interpolate(e4, size=(x.shape[2]//4, x.shape[3]//4), \n                                     mode='bilinear', align_corners=False)\n            e3 = self.dino_proj_e3(e3_interp)\n            \n            e2_interp = F.interpolate(e4, size=(x.shape[2]//2, x.shape[3]//2), \n                                     mode='bilinear', align_corners=False)\n            e2 = self.dino_proj_e2(e2_interp)\n            \n            e1_interp = F.interpolate(e4, size=(x.shape[2], x.shape[3]), \n                                     mode='bilinear', align_corners=False)\n            e1 = self.dino_proj_e1(e1_interp)\n            \n            b = self.bottleneck(self.drop(e4))\n        else:\n            e1 = self.enc1(x)\n            e2 = self.enc2(self.pool(e1))\n            e3 = self.enc3(self.pool(e2))\n            e4 = self.enc4(self.pool(e3))\n            b = self.bottleneck(self.drop(self.pool(e4)))\n        \n        d4 = self.up4(b)\n        if d4.shape[2:] != e4.shape[2:]:\n            e4 = F.interpolate(e4, size=d4.shape[2:], mode='bilinear', align_corners=False)\n        d4 = torch.cat([d4, e4], dim=1)\n        d4 = self.dec4(d4)\n        \n        d3 = self.up3(d4)\n        if d3.shape[2:] != e3.shape[2:]:\n            e3 = F.interpolate(e3, size=d3.shape[2:], mode='bilinear', align_corners=False)\n        d3 = torch.cat([d3, e3], dim=1)\n        d3 = self.dec3(d3)\n        \n        d2 = self.up2(d3)\n        if d2.shape[2:] != e2.shape[2:]:\n            e2 = F.interpolate(e2, size=d2.shape[2:], mode='bilinear', align_corners=False)\n        d2 = torch.cat([d2, e2], dim=1)\n        d2 = self.dec2(d2)\n        \n        d1 = self.up1(d2)\n        if d1.shape[2:] != e1.shape[2:]:\n            e1 = F.interpolate(e1, size=d1.shape[2:], mode='bilinear', align_corners=False)\n        d1 = torch.cat([d1, e1], dim=1)\n        d1 = self.dec1(d1)\n        \n        return torch.sigmoid(self.final(d1))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:45.010706Z","iopub.execute_input":"2026-01-04T10:08:45.011096Z","iopub.status.idle":"2026-01-04T10:08:45.037147Z","shell.execute_reply.started":"2026-01-04T10:08:45.011078Z","shell.execute_reply":"2026-01-04T10:08:45.036538Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def dice_loss(pred, target, smooth=1e-6):\n    pred_flat = pred.view(-1)\n    target_flat = target.view(-1)\n    intersection = (pred_flat * target_flat).sum()\n    return 1 - (2. * intersection + smooth) / (pred_flat.sum() + target_flat.sum() + smooth)\n\ndef bce_dice_loss(pred, target):\n    bce = nn.functional.binary_cross_entropy(pred, target)\n    dice = dice_loss(pred, target)\n    return bce + dice\n\ndef train_epoch(model, loader, optimizer, criterion, device):\n    model.train()\n    total_loss = 0\n    for imgs, masks in tqdm(loader, desc='Training'):\n        imgs = imgs.to(device)\n        masks = masks.to(device)\n        \n        optimizer.zero_grad()\n        preds = model(imgs)\n        loss = criterion(preds, masks)\n        loss.backward()\n        optimizer.step()\n        \n        total_loss += loss.item()\n    return total_loss / len(loader)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:45.038259Z","iopub.execute_input":"2026-01-04T10:08:45.038509Z","iopub.status.idle":"2026-01-04T10:08:45.049895Z","shell.execute_reply.started":"2026-01-04T10:08:45.038491Z","shell.execute_reply":"2026-01-04T10:08:45.049257Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"BASE_PATH = '/kaggle/input/recodai-luc-scientific-image-forgery-detection'\n\ntrain_img_dir = Path(BASE_PATH) / 'train_images'\ntrain_mask_dir = Path(BASE_PATH) / 'train_masks'\nsupp_img_dir = Path(BASE_PATH) / 'supplemental_images'\nsupp_mask_dir = Path(BASE_PATH) / 'supplemental_masks'\ntest_img_dir = Path(BASE_PATH) / 'test_images'\n\ntrain_imgs = sorted(list(train_img_dir.glob('*.png')))\nsupp_imgs = sorted(list(supp_img_dir.glob('*.png'))) if supp_img_dir.exists() else []\ntest_imgs = sorted(list(test_img_dir.glob('*.png')))\n\nprint(f'Train images: {len(train_imgs)}')\nprint(f'Supplemental images: {len(supp_imgs)}')\nprint(f'Test images: {len(test_imgs)}')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:45.071958Z","iopub.execute_input":"2026-01-04T10:08:45.072167Z","iopub.status.idle":"2026-01-04T10:08:45.087927Z","shell.execute_reply.started":"2026-01-04T10:08:45.072150Z","shell.execute_reply":"2026-01-04T10:08:45.087367Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def get_mask_paths(img_path, train_dir, mask_dir):\n    img_id = img_path.stem\n    mask_path = mask_dir / f'{img_id}.npy'\n    if mask_path.exists():\n        return mask_path\n    return None\n\ntrain_mask_paths = []\ntrain_valid_imgs = []\n\nfor img_path in train_imgs:\n    mask_path = get_mask_paths(img_path, train_img_dir, train_mask_dir)\n    if mask_path:\n        train_valid_imgs.append(img_path)\n        train_mask_paths.append(mask_path)\n\nsupp_mask_paths = []\nsupp_valid_imgs = []\n\nif supp_mask_dir.exists():\n    for img_path in supp_imgs:\n        mask_path = get_mask_paths(img_path, supp_img_dir, supp_mask_dir)\n        if mask_path:\n            supp_valid_imgs.append(img_path)\n            supp_mask_paths.append(mask_path)\n\nall_train_imgs = train_valid_imgs + supp_valid_imgs\nall_train_masks = train_mask_paths + supp_mask_paths\n\nprint(f'Valid train images with masks: {len(all_train_imgs)}')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:45.089094Z","iopub.execute_input":"2026-01-04T10:08:45.089356Z","iopub.status.idle":"2026-01-04T10:08:45.134501Z","shell.execute_reply.started":"2026-01-04T10:08:45.089339Z","shell.execute_reply":"2026-01-04T10:08:45.134011Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\ntrain_imgs_split, val_imgs_split, train_masks_split, val_masks_split = train_test_split(\n    all_train_imgs, all_train_masks, test_size=0.1, random_state=42\n)\n\ntrain_ds = ImgDataset(train_imgs_split, train_masks_split, train=True, sz=720)\nval_ds = ImgDataset(val_imgs_split, val_masks_split, train=False, sz=720)\n\ntrain_loader = DataLoader(train_ds, batch_size=2, shuffle=True, num_workers=2, pin_memory=True, persistent_workers=True)\nval_loader = DataLoader(val_ds, batch_size=2, shuffle=False, num_workers=2, pin_memory=True, persistent_workers=True)\n\nprint(f'Train batches: {len(train_loader)}')\nprint(f'Val batches: {len(val_loader)}')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:45.135127Z","iopub.execute_input":"2026-01-04T10:08:45.135290Z","iopub.status.idle":"2026-01-04T10:08:45.142348Z","shell.execute_reply.started":"2026-01-04T10:08:45.135274Z","shell.execute_reply":"2026-01-04T10:08:45.141739Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# DINOv2UNet or UNet \n","metadata":{}},{"cell_type":"code","source":"torch.cuda.empty_cache()\nimport gc\ngc.collect()\n\nDINO_PATH = \"/kaggle/input/dinov2/pytorch/base/1\"\nUSE_ENSEMBLE = True\n\nprint(\"Creating ensemble models:\")\n\nprint(\"Creating UNet model\")\nmodel_unet = UNet(n_channels=3, n_classes=1).to(device)\nprint(f\"UNet created - Parameters: {sum(p.numel() for p in model_unet.parameters()):,}\")\n\nif USE_ENSEMBLE and HAS_TRANSFORMERS and os.path.exists(DINO_PATH):\n    print(\"Creating DINOv2UNet model\")\n    model_dino = DINOv2UNet(\n        dino_path=DINO_PATH,\n        n_classes=1,\n        img_size=720,\n        use_dino=True\n    )\n    if model_dino.use_dino:\n        model_dino.dino_proj = model_dino.dino_proj.to(device)\n        model_dino.dino_proj_e3 = model_dino.dino_proj_e3.to(device)\n        model_dino.dino_proj_e2 = model_dino.dino_proj_e2.to(device)\n        model_dino.dino_proj_e1 = model_dino.dino_proj_e1.to(device)\n    model_dino.bottleneck = model_dino.bottleneck.to(device)\n    model_dino.up4 = model_dino.up4.to(device)\n    model_dino.dec4 = model_dino.dec4.to(device)\n    model_dino.up3 = model_dino.up3.to(device)\n    model_dino.dec3 = model_dino.dec3.to(device)\n    model_dino.up2 = model_dino.up2.to(device)\n    model_dino.dec2 = model_dino.dec2.to(device)\n    model_dino.up1 = model_dino.up1.to(device)\n    model_dino.dec1 = model_dino.dec1.to(device)\n    model_dino.final = model_dino.final.to(device)\n    model_dino.drop = model_dino.drop.to(device)\n    print(f\"DINOv2UNet created - Parameters: {sum(p.numel() for p in model_dino.parameters()):,}\")\n    print(f\"DINOv2 encoder: on CPU\")\n    print(f\"Decoder: on {device}\")\nelse:\n    model_dino = None\n    print(\"DINOv2UNet not available, using UNet only\")\n\nENSEMBLE_WEIGHTS = [0.7, 0.3]\nprint(f\"Ensemble weights: UNet={ENSEMBLE_WEIGHTS[0]}, DINOv2UNet={ENSEMBLE_WEIGHTS[1]}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:45.143508Z","iopub.execute_input":"2026-01-04T10:08:45.143923Z","iopub.status.idle":"2026-01-04T10:08:46.143981Z","shell.execute_reply.started":"2026-01-04T10:08:45.143897Z","shell.execute_reply":"2026-01-04T10:08:46.143338Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Training UNet model\")\noptimizer_unet = optim.AdamW(model_unet.parameters(), lr=2e-4, weight_decay=1e-5)\nscheduler_unet = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer_unet, T_0=5, T_mult=2, eta_min=1e-6)\ncriterion = bce_dice_loss\n\nbest_val_loss_unet = float('inf')\nepochs = 20\npatience = 5\npatience_counter = 0\n\nfor epoch in range(epochs):\n    train_loss = train_epoch(model_unet, train_loader, optimizer_unet, criterion, device)\n    \n    model_unet.eval()\n    val_loss = 0\n    with torch.no_grad():\n        for imgs, masks in val_loader:\n            imgs = imgs.to(device)\n            masks = masks.to(device)\n            preds = model_unet(imgs)\n            loss = criterion(preds, masks)\n            val_loss += loss.item()\n    val_loss /= len(val_loader)\n    \n    scheduler_unet.step()\n    \n    print(f'UNet - Epoch {epoch+1}/{epochs} - Train Loss: {train_loss:.4f}, Val Loss: {val_loss:.4f}, LR: {optimizer_unet.param_groups[0][\"lr\"]:.6f}')\n    \n    if val_loss < best_val_loss_unet:\n        best_val_loss_unet = val_loss\n        patience_counter = 0\n        torch.save(model_unet.state_dict(), 'best_model_unet.pth')\n        print(f'UNet model saved!')\n    else:\n        patience_counter += 1\n        if patience_counter >= patience:\n            print(f'Early stopping at epoch {epoch+1}')\n            break\n\nmodel_unet.load_state_dict(torch.load('best_model_unet.pth'))\nprint(\"UNet training completed!\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:08:46.145350Z","iopub.execute_input":"2026-01-04T10:08:46.145867Z","iopub.status.idle":"2026-01-04T10:16:14.675927Z","shell.execute_reply.started":"2026-01-04T10:08:46.145845Z","shell.execute_reply":"2026-01-04T10:16:14.675029Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"if model_dino is not None:\n    print(\"Training DINOv2UNet model...\")\n    optimizer_dino = optim.AdamW(model_dino.parameters(), lr=2e-4, weight_decay=1e-5)\n    scheduler_dino = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer_dino, T_0=5, T_mult=2, eta_min=1e-6)\n    \n    best_val_loss_dino = float('inf')\n    patience_counter = 0\n    \n    for epoch in range(epochs):\n        train_loss = train_epoch(model_dino, train_loader, optimizer_dino, criterion, device)\n        \n        model_dino.eval()\n        val_loss = 0\n        with torch.no_grad():\n            for imgs, masks in val_loader:\n                imgs = imgs.to(device)\n                masks = masks.to(device)\n                preds = model_dino(imgs)\n                loss = criterion(preds, masks)\n                val_loss += loss.item()\n        val_loss /= len(val_loader)\n        \n        scheduler_dino.step()\n        \n        print(f'DINOv2UNet - Epoch {epoch+1}/{epochs} - Train Loss: {train_loss:.4f}, Val Loss: {val_loss:.4f}, LR: {optimizer_dino.param_groups[0][\"lr\"]:.6f}')\n        \n        if val_loss < best_val_loss_dino:\n            best_val_loss_dino = val_loss\n            patience_counter = 0\n            torch.save(model_dino.state_dict(), 'best_model_dino.pth')\n            print(f'DINOv2UNet model saved!')\n        else:\n            patience_counter += 1\n            if patience_counter >= patience:\n                print(f'Early stopping at epoch {epoch+1}')\n                break\n    \n    model_dino.load_state_dict(torch.load('best_model_dino.pth'))\n    print(\"DINOv2UNet training completed!\")\nelse:\n    print(\"Skipping DINOv2UNet training (model not available)\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:16:14.677724Z","iopub.execute_input":"2026-01-04T10:16:14.678434Z","iopub.status.idle":"2026-01-04T10:29:51.526019Z","shell.execute_reply.started":"2026-01-04T10:16:14.678394Z","shell.execute_reply":"2026-01-04T10:29:51.525340Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"AREA_THR = 100\nMEAN_THR = 0.15\nUSE_TTA = True\n\ndef enhanced_adaptive_mask(prob, alpha_grad=0.35):\n    gx = cv2.Sobel(prob, cv2.CV_32F, 1, 0, ksize=3)\n    gy = cv2.Sobel(prob, cv2.CV_32F, 0, 1, ksize=3)\n    grad_mag = np.sqrt(gx**2 + gy**2)\n    grad_norm = grad_mag / (grad_mag.max() + 1e-6)\n    \n    enhanced = (1 - alpha_grad) * prob + alpha_grad * grad_norm\n    enhanced = cv2.GaussianBlur(enhanced, (3, 3), 0)\n    \n    thr = np.mean(enhanced) + 0.3 * np.std(enhanced)\n    mask = (enhanced > thr).astype(np.uint8)\n    \n    mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, np.ones((5, 5), np.uint8))\n    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8))\n    \n    return mask, thr\n\ndef finalize_mask(prob, orig_size, img_size=720):\n    mask, thr = enhanced_adaptive_mask(prob)\n    mask = cv2.resize(mask, orig_size, interpolation=cv2.INTER_NEAREST)\n    return mask, thr\n\ndef postprocess_mask(mask, min_area=100):\n    mask = mask.astype(np.uint8)\n    num_labels, labels, stats, _ = cv2.connectedComponentsWithStats(mask, connectivity=8)\n    for i in range(1, num_labels):\n        if stats[i, cv2.CC_STAT_AREA] < min_area:\n            mask[labels == i] = 0\n    return mask\n\ndef predict_tta(model, img_tensor, device):\n    model.eval()\n    preds = []\n    \n    with torch.no_grad():\n        img = img_tensor.unsqueeze(0).to(device)\n        \n        pred = model(img).cpu().numpy()[0, 0]\n        preds.append(pred)\n        \n        img_hflip = torch.flip(img, [3])\n        pred_hflip = model(img_hflip).cpu().numpy()[0, 0]\n        preds.append(np.fliplr(pred_hflip))\n        \n        img_vflip = torch.flip(img, [2])\n        pred_vflip = model(img_vflip).cpu().numpy()[0, 0]\n        preds.append(np.flipud(pred_vflip))\n    \n    return np.mean(preds, axis=0)\n\ndef predict_ensemble(models, weights, img_tensor, device, use_tta=True):\n    probs = []\n    for model in models:\n        if model is None:\n            continue\n        if use_tta:\n            prob = predict_tta(model, img_tensor, device)\n        else:\n            with torch.no_grad():\n                model.eval()\n                img = img_tensor.unsqueeze(0).to(device)\n                prob = model(img).cpu().numpy()[0, 0]\n        probs.append(prob)\n    \n    if len(probs) == 0:\n        return None\n    \n    if len(probs) == 1:\n        return probs[0]\n    \n    weights_norm = np.array(weights[:len(probs)])\n    weights_norm = weights_norm / weights_norm.sum()\n    \n    ensemble_prob = np.zeros_like(probs[0])\n    for prob, w in zip(probs, weights_norm):\n        ensemble_prob += w * prob\n    \n    return ensemble_prob\n\ndef pipeline_final(models, weights, pil_img, device, img_size=720):\n    img_tensor = torch.from_numpy(\n        np.array(pil_img.resize((img_size, img_size)), np.float32) / 255.\n    ).permute(2, 0, 1)\n    \n    prob = predict_ensemble(models, weights, img_tensor, device, use_tta=USE_TTA)\n    \n    if prob is None:\n        return \"authentic\", None, {\"area\": 0, \"mean_inside\": 0.0, \"thr\": 0.0}\n    \n    mask, thr = finalize_mask(prob, pil_img.size, img_size)\n    \n    area = int(mask.sum())\n    prob_resized = cv2.resize(prob, pil_img.size, interpolation=cv2.INTER_LINEAR)\n    mean_inside = float(prob_resized[mask == 1].mean()) if area > 0 else 0.0\n    \n    if area < AREA_THR or mean_inside < MEAN_THR:\n        return \"authentic\", None, {\"area\": area, \"mean_inside\": mean_inside, \"thr\": thr}\n    \n    return \"forged\", mask, {\"area\": area, \"mean_inside\": mean_inside, \"thr\": thr}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:29:51.527859Z","iopub.execute_input":"2026-01-04T10:29:51.528083Z","iopub.status.idle":"2026-01-04T10:29:51.542581Z","shell.execute_reply.started":"2026-01-04T10:29:51.528064Z","shell.execute_reply":"2026-01-04T10:29:51.542080Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sample_submission_original = pd.read_csv(Path(BASE_PATH) / 'sample_submission.csv')\nsample_submission_original['case_id'] = sample_submission_original['case_id'].astype(str)\nprint(f\"Original sample_submission: {len(sample_submission_original)} cases\")\n\ntest_imgs_available = sorted(list(test_img_dir.glob('**/*.png')))\nprint(f\"Available test images: {len(test_imgs_available)}\")\n\nif len(test_imgs_available) > len(sample_submission_original) * 2:\n    print(f\"\\n Found {len(test_imgs_available)} test images,\")\n    print(f\"but sample_submission has only {len(sample_submission_original)} case_id\")\n    print(f\"Creating expanded submission for all found images...\")\n    \n    all_case_ids = [str(Path(p).stem) for p in test_imgs_available]\n    sample_submission = pd.DataFrame({\n        'case_id': all_case_ids\n    })\n    \n    print(f\"Created expanded submission with {len(sample_submission)} case_id\")\n    print(f\"First 10 case_id: {sample_submission['case_id'].head(10).tolist()}\")\nelse:\n    sample_submission = sample_submission_original.copy()\n    print(f\"Using original sample_submission with {len(sample_submission)} case_id\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:29:51.543466Z","iopub.execute_input":"2026-01-04T10:29:51.543687Z","iopub.status.idle":"2026-01-04T10:29:51.575750Z","shell.execute_reply.started":"2026-01-04T10:29:51.543664Z","shell.execute_reply":"2026-01-04T10:29:51.575199Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_imgs_all = sorted(list(test_img_dir.glob('**/*.png')))\ntest_imgs_dict = {}\nfor p in test_imgs_all:\n    stem = Path(p).stem\n    test_imgs_dict[str(stem)] = p\n    test_imgs_dict[stem] = p\n    try:\n        test_imgs_dict[int(stem)] = p\n    except:\n        pass\n\nmodels_list = [model_unet, model_dino]\nif model_dino is None:\n    models_list = [model_unet]\n    ensemble_weights = [1.0]\nelse:\n    ensemble_weights = ENSEMBLE_WEIGHTS\n\nfor model in models_list:\n    if model is not None:\n        model.eval()\n\npredictions_dict = {}\n\ntorch.cuda.empty_cache()\n\nprint(f\"\\n Processing {len(sample_submission)} cases with ensemble\")\nprint(f\"   Models: {len([m for m in models_list if m is not None])}\")\nprint(f\"   Weights: {ensemble_weights}\")\n\nwith torch.no_grad():\n    for idx, row in tqdm(sample_submission.iterrows(), total=len(sample_submission), desc='Predicting'):\n        case_id = row['case_id']\n        case_id_str = str(case_id)\n        \n        img_path = None\n        if case_id_str in test_imgs_dict:\n            img_path = test_imgs_dict[case_id_str]\n        elif case_id in test_imgs_dict:\n            img_path = test_imgs_dict[case_id]\n        else:\n            for test_stem, test_path in test_imgs_dict.items():\n                if str(test_stem) == case_id_str or str(test_stem) == str(case_id):\n                    img_path = test_path\n                    break\n        \n        if img_path is None:\n            predictions_dict[case_id_str] = \"authentic\"\n            continue\n        \n        try:\n            orig_img = Image.open(img_path).convert('RGB')\n            \n            label, mask, dbg = pipeline_final(models_list, ensemble_weights, orig_img, device, img_size=720)\n            \n            if label == \"authentic\":\n                predictions_dict[case_id_str] = \"authentic\"\n            else:\n                if mask is not None:\n                    mask = postprocess_mask(mask, min_area=50)\n                    if mask.sum() == 0:\n                        predictions_dict[case_id_str] = \"authentic\"\n                    else:\n                        rle = rle_encode((mask > 0).astype(np.uint8))\n                        predictions_dict[case_id_str] = rle\n                else:\n                    predictions_dict[case_id_str] = \"authentic\"\n        except Exception as e:\n            print(f\"Error processing {case_id}: {e}\")\n            predictions_dict[case_id_str] = \"authentic\"\n        \n        if (idx + 1) % 10 == 0:\n            torch.cuda.empty_cache()\n\nprint(f'\\nProcessed {len(predictions_dict)} predictions')\n\nsubmission = sample_submission.copy()\nsubmission['case_id'] = submission['case_id'].astype(str)\nsubmission['annotation'] = submission['case_id'].map(predictions_dict).fillna('authentic')\n\nprint(f\"\\nPredictions summary:\")\nprint(f\"   Total cases: {len(submission)}\")\nprint(f\"   With predictions: {len(predictions_dict)}\")\nprint(f\"   Authentic: {(submission['annotation'] == 'authentic').sum()}\")\nprint(f\"   Forged: {(submission['annotation'] != 'authentic').sum()}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:29:51.576483Z","iopub.execute_input":"2026-01-04T10:29:51.576688Z","iopub.status.idle":"2026-01-04T10:29:53.716042Z","shell.execute_reply.started":"2026-01-04T10:29:51.576662Z","shell.execute_reply":"2026-01-04T10:29:53.715327Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission_final = sample_submission[['case_id']].merge(\n    submission[['case_id', 'annotation']],\n    on='case_id',\n    how='left'\n)\nsubmission_final['annotation'] = submission_final['annotation'].fillna('authentic')\n\nforged_annotations = submission_final[submission_final['annotation'] != 'authentic']['annotation']\nif len(forged_annotations) > 0:\n    try:\n        for ann in forged_annotations.head(3):\n            json.loads(ann)\n        print(f\"RLE encoding is valid (JSON format)\")\n    except Exception as e:\n        print(f\"Problem with RLE encoding: {e}\")\n\nsubmission_final.to_csv('submission.csv', index=False)\nprint(f'\\nSubmission saved! Shape: {submission_final.shape}')\nprint(f\"\\nFinal statistics:\")\nprint(submission_final.head(10))\nprint(f\"\\n   Total case_id: {len(submission_final)}\")\nprint(f\"   Authentic: {(submission_final['annotation'] == 'authentic').sum()}\")\nprint(f\"   Forged: {(submission_final['annotation'] != 'authentic').sum()}\")\n\nmissing = set(sample_submission_original['case_id'].astype(str)) - set(submission_final['case_id'].astype(str))\nif missing:\n    print(f\"\\nWARNING: Missing case_id: {list(missing)[:10]}...\")\nelse:\n    print(f\"\\nAll case_id from sample_submission processed!\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-04T10:29:53.717421Z","iopub.execute_input":"2026-01-04T10:29:53.717655Z","iopub.status.idle":"2026-01-04T10:29:53.750999Z","shell.execute_reply.started":"2026-01-04T10:29:53.717637Z","shell.execute_reply":"2026-01-04T10:29:53.750249Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Links\n**Pankaj Gupta | https://www.kaggle.com/pankajiitr**\n**Hossam Hamouda | https://www.kaggle.com/hossam82**","metadata":{}}]}