{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":126777,"databundleVersionId":15314950,"sourceType":"competition"}],"dockerImageVersionId":31261,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:18:45.255093Z","iopub.execute_input":"2026-01-23T11:18:45.255578Z","iopub.status.idle":"2026-01-23T11:18:50.431675Z","shell.execute_reply.started":"2026-01-23T11:18:45.255551Z","shell.execute_reply":"2026-01-23T11:18:50.430438Z"},"scrolled":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n# How My CNN + Attention + Metric Learning Pipeline Worked (and Why It Reached ~0.82 Public Score)\n\n## Overview of the Approach\n\nMy solution was built as a **progressive enhancement pipeline**, where each component fixed a specific weakness of the previous one:\n\n\nPretrained CNN\n→ Attention\n→ Embedding Head\n→ ArcFace Loss\n→ Triplet Loss\n→ Cosine Similarity (Inference)\n\n\n\nThe task was **Re-Identification**, so the goal was **not classification accuracy**, but **learning a discriminative embedding space** where:\n- Images of the **same identity are close**\n- Images of **different identities are far**\n\n---\n\n## 1. Pretrained CNN Backbone (Feature Extractor)\n\n### What it did\n- Used a CNN pretrained on ImageNet (e.g., ResNet / EfficientNet)\n- Extracted mid- to high-level visual features (edges, textures, shapes)\n\n### Why it mattered\n- Transfer learning gave strong low-level and mid-level representations\n- Prevented overfitting on a small dataset\n- Provided a stable starting point instead of learning from scratch\n\n### Limitation\n- CNN features alone treat **all spatial regions and channels equally**\n- Background and irrelevant regions leaked into the representation\n\n---\n\n## 2. Attention Module (CBAM / SE-style)\n\n### What it did\n- **Channel Attention**: learned which feature channels mattered more\n- **Spatial Attention**: learned which spatial regions mattered more\n- Applied **multiplicative reweighting** to feature maps\n\nMathematically:\n\n\nF_out = F ⊙ ChannelWeights ⊙ SpatialWeights\n\n\n### Why it helped\n- Suppressed background noise\n- Emphasized identity-specific regions (patterns, textures)\n- Produced **cleaner and more stable features** before pooling\n\n### Practical effect\n- Reduced variance caused by pose, lighting, and occlusion\n- Improved generalization across different views of the same identity\n\n---\n\n## 3. Embedding Head (Projection Layer)\n\n### What it did\n- Converted CNN features into a fixed-length embedding (e.g., 512-D)\n- Applied BatchNorm and L2-normalization\n\n### Why it mattered\n- Mapped raw features into a **metric space**\n- Normalization made cosine similarity meaningful\n- Stabilized training for metric losses\n\n---\n\n## 4. ArcFace Loss (Margin-Based Classification)\n\n### What it did\n- Treated each identity as a class\n- Enforced an **angular margin** between identities in embedding space\n\nConceptually:\n- Same identity → embeddings pulled closer\n- Different identities → embeddings pushed apart with a margin\n\n### Why it helped\n- Created **compact, well-separated clusters**\n- Gave strong **global structure** to the embedding space\n- Much better than plain CrossEntropy for ReID\n\n### Contribution to score\n- Major jump in public score due to better inter-class separation\n\n---\n\n## 5. Triplet Loss (Relative Distance Refinement)\n\n### What it did\n- Operated on (anchor, positive, negative) samples\n- Enforced:\n\ndistance(anchor, positive) + margin < distance(anchor, negative)\n\n### Why it helped\n- Fixed **local ranking mistakes** that ArcFace alone could not\n- Improved ordering of nearest neighbors (critical for mAP)\n- Especially useful for visually similar identities\n\n### How it was used\n- Added after ArcFace had stabilized training\n- Used with a smaller weight to avoid instability\n\n---\n\n## 6. Why This Combination Worked Well\n\nEach component solved a **different problem**:\n\n| Component | Solved |\n|--------|------|\n| Pretrained CNN | Feature extraction & generalization |\n| Attention | Noise suppression & focus |\n| ArcFace | Global identity separation |\n| Triplet | Local ranking correctness |\n\nTogether:\n- Attention cleaned the features\n- ArcFace structured the embedding space\n- Triplet refined relative distances\n\nThis alignment matched the **evaluation metric (mAP)**, not just training loss.\n\n---\n\n## 7. Why It Reached ~0.82 on the Public Leaderboard\n\n- The model learned **identity-discriminative embeddings**, not class logits\n- Background influence was reduced via attention\n- Angular margins (ArcFace) ensured strong separation\n- Relative ordering (Triplet) improved retrieval ranking\n- Cosine similarity matched how the embeddings were trained\n\n### Key insight\n> The pipeline optimized **ranking quality**, not raw accuracy — exactly what the leaderboard measured.\n\n---\n\n## Final Takeaway\n\nThis solution worked because it respected the nature of the problem:\n\n- **ReID ≠ Classification**\n- **Embeddings > Softmax outputs**\n- **Ranking-aware losses > Accuracy-focused losses**\n\nThat alignment is what pushed the score to ~0.82 on the public board.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"BASE_DIR = \"/kaggle/input/jaguar-re-id\"\n\nTRAIN_CSV = f\"{BASE_DIR}/train.csv\"\nTEST_CSV  = f\"{BASE_DIR}/test.csv\"\n\nTRAIN_IMG_DIR = f\"{BASE_DIR}/train/train\"\nTEST_IMG_DIR  = f\"{BASE_DIR}/test/test\"\nimport pandas as pd\n\ntrain_df = pd.read_csv(TRAIN_CSV)\ntest_df  = pd.read_csv(TEST_CSV)\n\nprint(train_df.head())\nprint(test_df.head())\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:18:50.432721Z","iopub.execute_input":"2026-01-23T11:18:50.433200Z","iopub.status.idle":"2026-01-23T11:18:50.582079Z","shell.execute_reply.started":"2026-01-23T11:18:50.433156Z","shell.execute_reply":"2026-01-23T11:18:50.581053Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport random\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\n\nfrom torchvision import transforms\nimport timm\nfrom torch.utils.data import Dataset\nfrom PIL import Image\nimport torch\nimport os","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:18:50.583360Z","iopub.execute_input":"2026-01-23T11:18:50.583784Z","iopub.status.idle":"2026-01-23T11:19:08.038434Z","shell.execute_reply.started":"2026-01-23T11:18:50.583750Z","shell.execute_reply":"2026-01-23T11:19:08.037622Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.040066Z","iopub.execute_input":"2026-01-23T11:19:08.040314Z","iopub.status.idle":"2026-01-23T11:19:08.059296Z","shell.execute_reply.started":"2026-01-23T11:19:08.040291Z","shell.execute_reply":"2026-01-23T11:19:08.058420Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from torch.utils.data import Dataset\nfrom PIL import Image\nimport os\n\nclass JaguarDataset(Dataset):\n    def __init__(self, df, img_dir, label2id, transform=None):\n        self.transform = transform\n        self.samples = []\n\n        for _, row in df.iterrows():\n            img_path = os.path.join(img_dir, row[\"filename\"])\n            if os.path.exists(img_path):\n                label = label2id[row[\"ground_truth\"]]  # STRING → INT\n                self.samples.append((img_path, label))\n\n        if len(self.samples) == 0:\n            raise RuntimeError(\"No valid images found. Check paths.\")\n\n    def __len__(self):\n        return len(self.samples)\n\n    def __getitem__(self, idx):\n        img_path, label = self.samples[idx]\n        image = Image.open(img_path).convert(\"RGB\")\n\n        if self.transform:\n            image = self.transform(image)\n\n        return image, label\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.060594Z","iopub.execute_input":"2026-01-23T11:19:08.060967Z","iopub.status.idle":"2026-01-23T11:19:08.069308Z","shell.execute_reply.started":"2026-01-23T11:19:08.060934Z","shell.execute_reply":"2026-01-23T11:19:08.068760Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_transforms = transforms.Compose([\n    transforms.Resize((224, 224)),\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.ColorJitter(brightness=0.2, contrast=0.2),\n    transforms.RandomRotation(10),\n    transforms.ToTensor(),\n    \n    transforms.Normalize(\n        mean=[0.485, 0.456, 0.406],\n        std=[0.229, 0.224, 0.225]\n    )\n])\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.070162Z","iopub.execute_input":"2026-01-23T11:19:08.070473Z","iopub.status.idle":"2026-01-23T11:19:08.087084Z","shell.execute_reply.started":"2026-01-23T11:19:08.070431Z","shell.execute_reply":"2026-01-23T11:19:08.086363Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class ChannelAttention(nn.Module):\n    def __init__(self, channels, reduction=16):\n        super().__init__()\n        self.avg_pool = nn.AdaptiveAvgPool2d(1)\n        self.fc = nn.Sequential(\n            nn.Linear(channels, channels // reduction),\n            nn.ReLU(),\n            nn.Linear(channels // reduction, channels),\n            nn.Sigmoid()\n        )\n\n    def forward(self, x):\n        b, c, _, _ = x.size()\n        y = self.avg_pool(x).view(b, c)\n        y = self.fc(y).view(b, c, 1, 1)\n        return x * y\n\n\nclass SpatialAttention(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv = nn.Conv2d(2, 1, kernel_size=7, padding=3)\n        self.sigmoid = nn.Sigmoid()\n\n    def forward(self, x):\n        avg = torch.mean(x, dim=1, keepdim=True)\n        mx, _ = torch.max(x, dim=1, keepdim=True)\n        y = torch.cat([avg, mx], dim=1)\n        y = self.sigmoid(self.conv(y))\n        return x * y\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.087871Z","iopub.execute_input":"2026-01-23T11:19:08.088099Z","iopub.status.idle":"2026-01-23T11:19:08.108136Z","shell.execute_reply.started":"2026-01-23T11:19:08.088078Z","shell.execute_reply":"2026-01-23T11:19:08.107396Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class AttentionEmbeddingNet(nn.Module):\n    def __init__(self, backbone_name=\"resnet50\", embedding_dim=512):\n        super().__init__()\n\n        self.backbone = timm.create_model(\n            backbone_name, pretrained=True, num_classes=0\n        )\n        channels = self.backbone.num_features\n\n        self.ca = ChannelAttention(channels)\n        self.sa = SpatialAttention()\n\n        self.embedding = nn.Linear(channels, embedding_dim)\n\n    def forward(self, x):\n        features = self.backbone.forward_features(x)\n        features = self.ca(features)\n        features = self.sa(features)\n\n        features = F.adaptive_avg_pool2d(features, 1).squeeze(-1).squeeze(-1)\n        embeddings = F.normalize(self.embedding(features))\n        return embeddings\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.109198Z","iopub.execute_input":"2026-01-23T11:19:08.109587Z","iopub.status.idle":"2026-01-23T11:19:08.127390Z","shell.execute_reply.started":"2026-01-23T11:19:08.109553Z","shell.execute_reply":"2026-01-23T11:19:08.126841Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class ArcFace(nn.Module):\n    def __init__(self, embedding_dim, num_classes, s=30.0, m=0.5):\n        super().__init__()\n        self.s = s\n        self.m = m\n        self.weight = nn.Parameter(torch.randn(num_classes, embedding_dim))\n        nn.init.xavier_uniform_(self.weight)\n\n    def forward(self, embeddings, labels):\n        cosine = F.linear(\n            F.normalize(embeddings),\n            F.normalize(self.weight)\n        )\n        phi = cosine - self.m\n\n        one_hot = torch.zeros_like(cosine)\n        one_hot.scatter_(1, labels.view(-1, 1), 1)\n\n        logits = self.s * (one_hot * phi + (1 - one_hot) * cosine)\n        return logits\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.128536Z","iopub.execute_input":"2026-01-23T11:19:08.128757Z","iopub.status.idle":"2026-01-23T11:19:08.148817Z","shell.execute_reply.started":"2026-01-23T11:19:08.128737Z","shell.execute_reply":"2026-01-23T11:19:08.148255Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class TripletLoss(nn.Module):\n    def __init__(self, margin=0.3):\n        super().__init__()\n        self.margin = margin\n\n    def forward(self, embeddings, labels):\n        dist = torch.cdist(embeddings, embeddings)\n\n        loss = 0.0\n        for i in range(len(labels)):\n            pos = dist[i][labels == labels[i]]\n            neg = dist[i][labels != labels[i]]\n\n            hardest_pos = pos.max()\n            hardest_neg = neg.min()\n\n            loss += F.relu(hardest_pos - hardest_neg + self.margin)\n\n        return loss / len(labels)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.151629Z","iopub.execute_input":"2026-01-23T11:19:08.151830Z","iopub.status.idle":"2026-01-23T11:19:08.164003Z","shell.execute_reply.started":"2026-01-23T11:19:08.151811Z","shell.execute_reply":"2026-01-23T11:19:08.163391Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"label2id = {label: i for i, label in enumerate(train_df[\"ground_truth\"].unique())}\nid2label = {v: k for k, v in label2id.items()}\n\nprint(label2id)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.164810Z","iopub.execute_input":"2026-01-23T11:19:08.165127Z","iopub.status.idle":"2026-01-23T11:19:08.181620Z","shell.execute_reply.started":"2026-01-23T11:19:08.165106Z","shell.execute_reply":"2026-01-23T11:19:08.180893Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_dataset = JaguarDataset(\n    df=train_df,\n    img_dir=TRAIN_IMG_DIR,\n    label2id=label2id,\n    transform=train_transforms\n)\n\ntrain_loader = DataLoader(\n    train_dataset,\n    batch_size=32,\n    shuffle=True,\n    num_workers=2,\n    drop_last=True\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:08.182592Z","iopub.execute_input":"2026-01-23T11:19:08.183222Z","iopub.status.idle":"2026-01-23T11:19:11.541465Z","shell.execute_reply.started":"2026-01-23T11:19:08.183189Z","shell.execute_reply":"2026-01-23T11:19:11.540811Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\nmodel = AttentionEmbeddingNet().to(device)\narcface = ArcFace(512, num_classes=31).to(device)\ntriplet_loss = TripletLoss()\n\noptimizer = torch.optim.Adam(\n    list(model.parameters()) + list(arcface.parameters()),\n    lr=1e-4\n)\n\nce_loss = nn.CrossEntropyLoss()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:11.542366Z","iopub.execute_input":"2026-01-23T11:19:11.542623Z","iopub.status.idle":"2026-01-23T11:19:14.294765Z","shell.execute_reply.started":"2026-01-23T11:19:11.542601Z","shell.execute_reply":"2026-01-23T11:19:14.293785Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"images, labels = next(iter(train_loader))\n\nprint(type(images), type(labels))\nprint(images.shape)\nprint(labels.shape)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:14.295927Z","iopub.execute_input":"2026-01-23T11:19:14.296183Z","iopub.status.idle":"2026-01-23T11:19:36.550331Z","shell.execute_reply.started":"2026-01-23T11:19:14.296159Z","shell.execute_reply":"2026-01-23T11:19:36.549239Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# for epoch in range(10):\n#     model.train()\n#     for images, labels in train_loader:\n#         images, labels = images.to(device), labels.to(device)\n\n#         embeddings = model(images)\n#         logits = arcface(embeddings, labels)\n\n#         loss_arc = ce_loss(logits, labels)\n#         loss_trip = triplet_loss(embeddings, labels)\n\n#         loss = loss_arc + 0.5 * loss_trip\n\n#         optimizer.zero_grad()\n#         loss.backward()\n#         optimizer.step()\n\n#     print(f\"Epoch {epoch+1} | Loss: {loss.item():.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:36.552579Z","iopub.execute_input":"2026-01-23T11:19:36.553153Z","iopub.status.idle":"2026-01-23T11:19:36.558262Z","shell.execute_reply.started":"2026-01-23T11:19:36.553106Z","shell.execute_reply":"2026-01-23T11:19:36.557454Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import time\nfrom tqdm import tqdm\nnum_epochs=15\nfor epoch in range(num_epochs):\n    model.train()\n\n    pbar = tqdm(train_loader, desc=f\"Epoch {epoch+1}/{num_epochs}\")\n\n    for images, labels in pbar:\n        images, labels = images.to(device), labels.to(device)\n\n        embeddings = model(images)\n        logits = arcface(embeddings, labels)\n\n        loss_arc = ce_loss(logits, labels)\n        loss_trip = triplet_loss(embeddings, labels)\n        loss = loss_arc + 0.5 * loss_trip\n\n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n\n        pbar.set_postfix(\n            loss=f\"{loss.item():.4f}\"\n        )\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-23T11:19:36.559546Z","iopub.execute_input":"2026-01-23T11:19:36.559869Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true,"scrolled":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from torch.utils.data import Dataset\nfrom PIL import Image\nimport os\n\nclass TestImageDataset(Dataset):\n    def __init__(self, image_list, img_dir, transform=None):\n        self.image_list = image_list\n        self.img_dir = img_dir\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.image_list)\n\n    def __getitem__(self, idx):\n        img_name = self.image_list[idx]\n        img_path = os.path.join(self.img_dir, img_name)\n\n        image = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            image = self.transform(image)\n\n        return img_name, image\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pandas as pd\n\ntest_df = pd.read_csv(\"/kaggle/input/jaguar-re-id/test.csv\")\n\nall_images = pd.unique(\n    test_df[[\"query_image\", \"gallery_image\"]].values.ravel()\n)\n\nprint(\"Total unique test images:\", len(all_images))\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from torch.utils.data import DataLoader\nimport torch\n\ntest_dataset = TestImageDataset(\n    image_list=all_images,\n    img_dir=TEST_IMG_DIR,\n    transform=train_transforms  # same as training\n)\n\ntest_loader = DataLoader(\n    test_dataset,\n    batch_size=64,\n    shuffle=False,\n    num_workers=2\n)\n\nmodel.eval()\nembeddings_dict = {}\n\nwith torch.no_grad():\n    for names, images in test_loader:\n        images = images.to(device)\n        embeds = model(images)\n        embeds = torch.nn.functional.normalize(embeds, dim=1)\n\n        for name, emb in zip(names, embeds):\n            embeddings_dict[name] = emb.cpu()\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import torch.nn.functional as F\n\nsimilarities = []\n\nfor _, row in test_df.iterrows():\n    q_emb = embeddings_dict[row[\"query_image\"]]\n    g_emb = embeddings_dict[row[\"gallery_image\"]]\n\n    sim = F.cosine_similarity(\n        q_emb.unsqueeze(0),\n        g_emb.unsqueeze(0)\n    )\n    \n    sim = ((sim + 1.0) / 2.0).clamp(0.0, 1.0)\n    similarities.append(sim.item())\n\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission = pd.read_csv(\"/kaggle/input/jaguar-re-id/sample_submission.csv\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(submission.head())\nprint(submission[\"similarity\"].min(), submission[\"similarity\"].max())\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission = pd.DataFrame({\n    \"row_id\": test_df[\"row_id\"],\n    \"similarity\": similarities\n})\n\nsubmission.to_csv(\"submission.csv\", index=False)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}