{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","pygments_lexer":"ipython3"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"competition","sourceId":14774,"databundleVersionId":875431,"isSourceIdPinned":false}],"dockerImageVersionId":31401,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"b89acefe-6cd5-4d74-977a-12e49173d3e5","cell_type":"markdown","source":"# APTOS v8 Training + Offline Weight Export\n\nNotebook ini adalah tahap training/export. Internet boleh ON di tahap ini jika ingin memakai pretrained timm/torchvision. Output pentingnya adalah folder dan zip `aptos_v8_offline_weights`, yang kemudian diupload sebagai Kaggle Dataset/Input untuk notebook submission offline.","metadata":{}},{"id":"3ddc5dcc","cell_type":"markdown","source":"# APTOS 2019 DR Classification v8: K-Fold Multi-Model OOF Ensemble\n\nThis notebook continues from the v7 C-only K-fold/OOF baseline. The goal is a limited multi-model K-fold experiment: train C and B across the same stratified folds, build model-level OOF predictions, ensemble them only from OOF probabilities, and select submission candidates by OOF metrics rather than Kaggle test labels.\n","metadata":{}},{"id":"d02001ed","cell_type":"markdown","source":"## Why Multi-Model K-Fold Now\n\nv7 showed that the C-only OOF baseline is valid but still below the 90% accuracy target, with Severe recall remaining weak. The next evidence-based step is limited model diversity inside K-fold, not pseudo-labeling or external data. v8 therefore runs C + B by default and keeps D, focal C, 5-fold, 512px, pseudo-label, and external data disabled unless explicitly enabled later.\n","metadata":{}},{"id":"335edb1f","cell_type":"markdown","source":"## Kaggle Public vs Private Leaderboard Context\n\nThe APTOS public leaderboard uses approximately 15% of test data, while the private leaderboard uses approximately 85% and reflects final standings. OOF validation is a stronger local estimate than one split, but it is still not public or private leaderboard performance. Kaggle `test.csv` has no labels and must only be used for submission inference.\n","metadata":{}},{"id":"7e089fb3","cell_type":"markdown","source":"## Environment, Imports, and Output Directories\n\nThe notebook writes v8 artifacts to separate result/cache directories. Existing v6/v7 notebooks and outputs are not overwritten.\n","metadata":{}},{"id":"db9365bc","cell_type":"code","source":"import os\nimport sys\nimport gc\nimport json\nimport math\nimport time\nimport random\nimport warnings\nimport subprocess\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom tqdm.auto import tqdm\nfrom PIL import Image\n\ntry:\n    import cv2\n    CV2_AVAILABLE = True\nexcept Exception:\n    cv2 = None\n    CV2_AVAILABLE = False\n\ntry:\n    import albumentations as A\n    from albumentations.pytorch import ToTensorV2\n    ALBUMENTATIONS_AVAILABLE = True\nexcept Exception:\n    A = None\n    ToTensorV2 = None\n    ALBUMENTATIONS_AVAILABLE = False\n\ntry:\n    import timm\n    TIMM_AVAILABLE = True\nexcept Exception:\n    timm = None\n    TIMM_AVAILABLE = False\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn.parameter import Parameter\nfrom torch.utils.data import Dataset, DataLoader, WeightedRandomSampler\nfrom torch.cuda.amp import autocast, GradScaler\nfrom torchvision import transforms, models\n\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\nfrom sklearn.metrics import (\n    accuracy_score,\n    precision_score,\n    recall_score,\n    f1_score,\n    confusion_matrix,\n    classification_report,\n    roc_auc_score,\n    cohen_kappa_score,\n)\nfrom sklearn.preprocessing import label_binarize\n\nwarnings.filterwarnings(\"ignore\")\nsns.set_theme(style=\"whitegrid\")\n\nSEED = 42\nrandom.seed(SEED)\nnp.random.seed(SEED)\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\ntorch.backends.cudnn.benchmark = True\n\nDEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nUSE_AMP = torch.cuda.is_available()\nPIN_MEMORY = torch.cuda.is_available()\nNUM_WORKERS = 0\n\nWORK_DIR = Path(\"/kaggle/working\") if Path(\"/kaggle/working\").exists() else Path.cwd()\nRESULTS_DIR = WORK_DIR / \"aptos_v8_kfold_multimodel_results\"\nCACHE_ROOT = WORK_DIR / \"aptos_cache_v8_kfold_multimodel\"\nRESULTS_DIR.mkdir(parents=True, exist_ok=True)\nCACHE_ROOT.mkdir(parents=True, exist_ok=True)\n\nNUM_CLASSES = 5\nCLASS_NAMES = {0: \"No_DR\", 1: \"Mild\", 2: \"Moderate\", 3: \"Severe\", 4: \"Proliferative_DR\"}\nCLASS_LABELS = [CLASS_NAMES[i] for i in range(NUM_CLASSES)]\nTARGET_INTERNAL_TEST_ACCURACY = 0.90\n\nFAILED_BASELINE_RESULTS = pd.DataFrame([\n    {\"experiment_id\": \"A0_v1_failed\", \"model\": \"InceptionV3 224 CE\", \"split\": \"internal_test\", \"accuracy\": 0.709091, \"macro_f1\": 0.508014, \"weighted_f1\": 0.707867, \"roc_auc_ovr_macro\": 0.878700, \"quadratic_weighted_kappa\": 0.768561, \"note\": \"Run awal; 224 input dan CE baseline lemah.\"},\n    {\"experiment_id\": \"A0_v1_failed\", \"model\": \"Hybrid EfficientNet-B3 + Swin-T CE\", \"split\": \"internal_test\", \"accuracy\": 0.787273, \"macro_f1\": 0.667663, \"weighted_f1\": 0.795294, \"roc_auc_ovr_macro\": 0.920236, \"quadratic_weighted_kappa\": 0.847185, \"note\": \"Run awal; hybrid lebih baik dari Inception tetapi masih jauh dari target.\"},\n    {\"experiment_id\": \"A0_v2_failed\", \"model\": \"InceptionV3 224 tuned CE\", \"split\": \"internal_test\", \"accuracy\": 0.783636, \"macro_f1\": 0.610322, \"weighted_f1\": 0.777911, \"roc_auc_ovr_macro\": 0.896969, \"quadratic_weighted_kappa\": 0.843329, \"note\": \"Run V2; mulai overfit dan masih 224.\"},\n    {\"experiment_id\": \"A0_v2_failed\", \"model\": \"Hybrid EfficientNet-B3 + SwinV2-T CE\", \"split\": \"internal_test\", \"accuracy\": 0.741818, \"macro_f1\": 0.604498, \"weighted_f1\": 0.751088, \"roc_auc_ovr_macro\": 0.896593, \"quadratic_weighted_kappa\": 0.839087, \"note\": \"Run V2; hybrid tidak stabil setelah unfreeze.\"},\n    {\"experiment_id\": \"A0_forensic_failed\", \"model\": \"InceptionV3 299 CE\", \"split\": \"internal_test\", \"accuracy\": 0.803636, \"macro_f1\": 0.613256, \"weighted_f1\": 0.791966, \"roc_auc_ovr_macro\": 0.911851, \"quadratic_weighted_kappa\": 0.847555, \"note\": \"Forensic run; internal test, bukan Kaggle test.\"},\n    {\"experiment_id\": \"A0_forensic_failed\", \"model\": \"Hybrid B3 + Swin-T gated CE\", \"split\": \"internal_test\", \"accuracy\": 0.780000, \"macro_f1\": 0.591502, \"weighted_f1\": 0.777479, \"roc_auc_ovr_macro\": 0.916836, \"quadratic_weighted_kappa\": 0.850749, \"note\": \"Forensic run; hybrid tidak mengalahkan Inception.\"},\n    {\"experiment_id\": \"A0_ordinal_failed\", \"model\": \"InceptionV3 299 GeM Ordinal\", \"split\": \"internal_test\", \"accuracy\": 0.732727, \"macro_f1\": 0.520194, \"weighted_f1\": 0.735711, \"roc_auc_ovr_macro\": np.nan, \"quadratic_weighted_kappa\": 0.875372, \"note\": \"Ordinal SmoothL1 menaikkan QWK tetapi menurunkan accuracy/macro F1.\"},\n    {\"experiment_id\": \"A0_ordinal_failed\", \"model\": \"Hybrid B3 + Swin-T 224 GeM Ordinal\", \"split\": \"internal_test\", \"accuracy\": 0.720000, \"macro_f1\": 0.539422, \"weighted_f1\": 0.733377, \"roc_auc_ovr_macro\": np.nan, \"quadratic_weighted_kappa\": 0.879503, \"note\": \"Hybrid ordinal masih lemah dan input 224 terlalu kecil.\"},\n    {\"experiment_id\": \"A0_ordinal_failed\", \"model\": \"Ensemble Inception + Hybrid Ordinal\", \"split\": \"internal_test\", \"accuracy\": 0.754545, \"macro_f1\": 0.555186, \"weighted_f1\": 0.757145, \"roc_auc_ovr_macro\": np.nan, \"quadratic_weighted_kappa\": 0.889620, \"note\": \"QWK tertinggi lama tetapi exact accuracy masih rendah.\"},\n])\nFAILED_BASELINE_RESULTS.to_csv(RESULTS_DIR / \"previous_failed_runs_summary.csv\", index=False)\n\ndisplay(FAILED_BASELINE_RESULTS)\nprint(f\"Device: {DEVICE}\")\nprint(f\"AMP: {USE_AMP}, OpenCV: {CV2_AVAILABLE}, Albumentations: {ALBUMENTATIONS_AVAILABLE}, timm: {TIMM_AVAILABLE}\")\nprint(f\"Results dir: {RESULTS_DIR}\")\nprint(f\"Cache root: {CACHE_ROOT}\")\n","metadata":{},"outputs":[],"execution_count":null},{"id":"218a4e2f","cell_type":"markdown","source":"## V8 Configuration\n\nDefault v8 runs 3-fold C+B only. Model D, C focal, pseudo-label, external data, 512px, and full 5-fold are present as flags but disabled by default to keep Kaggle runtime controlled.\n","metadata":{}},{"id":"f271a44c","cell_type":"code","source":"REBUILD_CACHE = False\n\nRUN_KFOLD_MULTIMODEL = True\nN_FOLDS = 3\nFOLD_SEED = 42\nRUN_FULL_5FOLD = False\nFORCE_RETRAIN_KFOLD = False\n\nRUN_KFOLD_MODEL_C = True\nRUN_KFOLD_MODEL_B = True\nRUN_KFOLD_MODEL_D = True\nRUN_KFOLD_FOCAL_C = True\n\nRUN_KFOLD_ENSEMBLE = True\nRUN_OOF_THRESHOLD_TUNING = True\nRUN_OOF_CALIBRATION = True\nRUN_TEMPERATURE_SCALING = False\nRUN_KFOLD_TEST_SUBMISSION = True\n\nRUN_PSEUDO_LABEL_STAGE = False\nRUN_EXTERNAL_DATA = False\nRUN_512_EXPERIMENT = False\nRUN_FULL_5FOLD = False\n\nKFOLD_SELECTION_METRIC = \"oof_composite\"\nENSEMBLE_SOFTMAX_TEMPERATURE = 0.05\nKFOLD_EARLY_STOP_PATIENCE = 8\n\nV7_REFERENCE = {\n    \"C_only_argmax\": {\n        \"pipeline\": \"v7_c_only_oof\",\n        \"models\": \"C\",\n        \"prediction_method\": \"argmax\",\n        \"oof_accuracy\": 0.847897,\n        \"oof_macro_f1\": 0.703738,\n        \"oof_weighted_f1\": 0.843121,\n        \"oof_qwk\": 0.911411,\n        \"oof_roc_auc\": 0.950453,\n        \"oof_composite\": 0.823704,\n        \"severe_recall\": 0.331606,\n        \"mild_recall\": 0.656757,\n        \"proliferative_recall\": 0.627119,\n        \"notes\": \"v7 3-fold C-only argmax OOF reference\",\n    },\n    \"C_only_threshold_accuracy\": {\n        \"pipeline\": \"v7_c_only_oof\",\n        \"models\": \"C\",\n        \"prediction_method\": \"threshold_accuracy\",\n        \"oof_accuracy\": 0.852540,\n        \"oof_macro_f1\": 0.690487,\n        \"oof_weighted_f1\": 0.844295,\n        \"oof_qwk\": 0.917764,\n        \"oof_roc_auc\": 0.950453,\n        \"oof_composite\": 0.823491,\n        \"severe_recall\": np.nan,\n        \"mild_recall\": np.nan,\n        \"proliferative_recall\": np.nan,\n        \"notes\": \"v7 threshold optimized for OOF accuracy; class recalls not printed in v7 summary\",\n    },\n    \"C_only_threshold_composite\": {\n        \"pipeline\": \"v7_c_only_oof\",\n        \"models\": \"C\",\n        \"prediction_method\": \"threshold_composite\",\n        \"oof_accuracy\": 0.847897,\n        \"oof_macro_f1\": 0.700316,\n        \"oof_weighted_f1\": 0.845022,\n        \"oof_qwk\": 0.921112,\n        \"oof_roc_auc\": 0.950453,\n        \"oof_composite\": 0.825587,\n        \"severe_recall\": np.nan,\n        \"mild_recall\": np.nan,\n        \"proliferative_recall\": np.nan,\n        \"notes\": \"v7 selected OOF composite reference\",\n    },\n}\n\nKFOLD_MODEL_SPECS = [\n    {\n        \"model_key\": \"C\",\n        \"enabled\": RUN_KFOLD_MODEL_C,\n        \"experiment_id\": \"C_seresnext50_384_ce_kfold\",\n        \"model_name\": \"seresnext50_32x4d\",\n        \"model_candidates\": [\"seresnext50_32x4d\", \"seresnext50_32x4d.gluon_in1k\", \"legacy_seresnext50_32x4d\", \"seresnextaa101d_32x8d\"],\n        \"input_size\": 384,\n        \"loss_type\": \"cross_entropy\",\n        \"pooling_type\": \"gem\",\n        \"batch_size\": 4,\n        \"accum_steps\": 8,\n        \"lr_main\": 2e-4,\n        \"lr_finetune\": 2e-5,\n        \"warmup_head_epochs\": 2,\n        \"epochs_main\": 30,\n        \"epochs_finetune\": 5,\n        \"augmentation_strength\": \"strong_controlled\",\n        \"class_balance\": \"none\",\n        \"runtime_notes\": \"SE-ResNeXt C model from v6/v7, main K-fold baseline.\",\n    },\n    {\n        \"model_key\": \"B\",\n        \"enabled\": RUN_KFOLD_MODEL_B,\n        \"experiment_id\": \"B_inception_v4_384_ce_kfold\",\n        \"model_name\": \"inception_v4\",\n        \"model_candidates\": [\"inception_v4\", \"inception_v4.tf_in1k\", \"legacy_inception_v4\", \"inception_resnet_v2\"],\n        \"input_size\": 384,\n        \"loss_type\": \"cross_entropy\",\n        \"pooling_type\": \"gem\",\n        \"batch_size\": 4,\n        \"accum_steps\": 8,\n        \"lr_main\": 2e-4,\n        \"lr_finetune\": 2e-5,\n        \"warmup_head_epochs\": 2,\n        \"epochs_main\": 30,\n        \"epochs_finetune\": 5,\n        \"augmentation_strength\": \"strong_controlled\",\n        \"class_balance\": \"none\",\n        \"runtime_notes\": \"InceptionV4-style diversity model for limited K-fold ensemble.\",\n    },\n    {\n        \"model_key\": \"D\",\n        \"enabled\": RUN_KFOLD_MODEL_D,\n        \"experiment_id\": \"D_efficientnet_b4_384_ce_kfold\",\n        \"model_name\": \"tf_efficientnet_b4\",\n        \"model_candidates\": [\"tf_efficientnet_b4\", \"tf_efficientnet_b4_ns\", \"efficientnet_b4\", \"convnext_tiny\", \"convnext_small\"],\n        \"input_size\": 384,\n        \"loss_type\": \"cross_entropy\",\n        \"pooling_type\": \"gem\",\n        \"batch_size\": 4,\n        \"accum_steps\": 8,\n        \"lr_main\": 2e-4,\n        \"lr_finetune\": 2e-5,\n        \"warmup_head_epochs\": 2,\n        \"epochs_main\": 30,\n        \"epochs_finetune\": 5,\n        \"augmentation_strength\": \"strong_controlled\",\n        \"class_balance\": \"none\",\n        \"runtime_notes\": \"Optional EfficientNet/ConvNeXt fallback diversity model; disabled by default.\",\n    },\n    {\n        \"model_key\": \"C_focal\",\n        \"enabled\": RUN_KFOLD_FOCAL_C,\n        \"experiment_id\": \"C_seresnext50_384_focal_kfold\",\n        \"model_name\": \"seresnext50_32x4d\",\n        \"model_candidates\": [\"seresnext50_32x4d\", \"seresnext50_32x4d.gluon_in1k\", \"legacy_seresnext50_32x4d\"],\n        \"input_size\": 384,\n        \"loss_type\": \"cross_entropy\",\n        \"pooling_type\": \"gem\",\n        \"batch_size\": 4,\n        \"accum_steps\": 8,\n        \"lr_main\": 2e-4,\n        \"lr_finetune\": 2e-5,\n        \"warmup_head_epochs\": 2,\n        \"epochs_main\": 30,\n        \"epochs_finetune\": 5,\n        \"augmentation_strength\": \"strong_controlled\",\n        \"class_balance\": \"focal_loss\",\n        \"focal_gamma\": 2.0,\n        \"runtime_notes\": \"Optional C focal-loss model for minority-class recall; disabled by default.\",\n    },\n]\n\nif RUN_FULL_5FOLD:\n    N_FOLDS = 5\nACTIVE_KFOLD_SPECS = [spec for spec in KFOLD_MODEL_SPECS if spec.get(\"enabled\", False)]\nKFOLD_RESULTS_DIR = RESULTS_DIR / \"kfold_multimodel\"\nKFOLD_RESULTS_DIR.mkdir(parents=True, exist_ok=True)\n\nprint(\"K-fold multimodel v8 config:\")\nprint(json.dumps({\n    \"RUN_KFOLD_MULTIMODEL\": RUN_KFOLD_MULTIMODEL,\n    \"N_FOLDS\": N_FOLDS,\n    \"RUN_KFOLD_MODEL_C\": RUN_KFOLD_MODEL_C,\n    \"RUN_KFOLD_MODEL_B\": RUN_KFOLD_MODEL_B,\n    \"RUN_KFOLD_MODEL_D\": RUN_KFOLD_MODEL_D,\n    \"RUN_KFOLD_FOCAL_C\": RUN_KFOLD_FOCAL_C,\n    \"RUN_KFOLD_ENSEMBLE\": RUN_KFOLD_ENSEMBLE,\n    \"KFOLD_SELECTION_METRIC\": KFOLD_SELECTION_METRIC,\n    \"RUN_PSEUDO_LABEL_STAGE\": RUN_PSEUDO_LABEL_STAGE,\n    \"RUN_EXTERNAL_DATA\": RUN_EXTERNAL_DATA,\n    \"RUN_512_EXPERIMENT\": RUN_512_EXPERIMENT,\n    \"active_models\": [s[\"model_key\"] for s in ACTIVE_KFOLD_SPECS],\n}, indent=2))\n","metadata":{},"outputs":[],"execution_count":null},{"id":"afe9f814","cell_type":"markdown","source":"## Dataset Loading\n\nThe loader reads labeled `train.csv`, unlabeled `test.csv`, and `sample_submission.csv`. The historical single split is retained only as reference metadata; K-fold training uses stratified folds from the full labeled train set.\n","metadata":{}},{"id":"2c0f7ac4","cell_type":"code","source":"MANUAL_COMPETITION_ROOT = None\n\n\ndef candidate_roots():\n    roots = []\n    if MANUAL_COMPETITION_ROOT is not None:\n        roots.append(Path(MANUAL_COMPETITION_ROOT))\n    input_root = Path(\"/kaggle/input\")\n    roots.extend([\n        Path(\"/kaggle/input/aptos2019-blindness-detection\"),\n        Path(\"/kaggle/input/aptos-2019-blindness-detection\"),\n        Path(\"/kaggle/input/competitions/aptos2019-blindness-detection\"),\n    ])\n    if input_root.exists():\n        roots.extend([p for p in input_root.rglob(\"*\") if p.is_dir()])\n    unique, seen = [], set()\n    for root in roots:\n        try:\n            key = root.resolve()\n        except Exception:\n            key = root\n        if key not in seen:\n            seen.add(key)\n            unique.append(root)\n    return unique\n\n\ndef image_dirs_for(root, kind):\n    names = {\n        \"train\": [\"train_images\", \"train_images/train_images\", \"images\", \"train\"],\n        \"test\": [\"test_images\", \"test_images/test_images\", \"test\"],\n    }[kind]\n    return [root / name for name in names if (root / name).exists() and (root / name).is_dir()]\n\n\ndef find_image_path(image_id, image_dirs):\n    for image_dir in image_dirs:\n        for ext in [\".png\", \".jpg\", \".jpeg\"]:\n            path = Path(image_dir) / f\"{image_id}{ext}\"\n            if path.exists():\n                return str(path)\n    return None\n\n\ndef resolve_aptos_files():\n    for root in candidate_roots():\n        train_csv = root / \"train.csv\"\n        test_csv = root / \"test.csv\"\n        sample_submission = root / \"sample_submission.csv\"\n        train_dirs = image_dirs_for(root, \"train\")\n        test_dirs = image_dirs_for(root, \"test\")\n        if train_csv.exists() and train_dirs:\n            return {\n                \"root\": root,\n                \"train_csv\": train_csv,\n                \"train_image_dirs\": train_dirs,\n                \"test_csv\": test_csv if test_csv.exists() else None,\n                \"test_image_dirs\": test_dirs,\n                \"sample_submission\": sample_submission if sample_submission.exists() else None,\n            }\n    raise FileNotFoundError(\"APTOS dataset tidak ditemukan. Tambahkan dataset competition ke Kaggle Notebook.\")\n\n\npaths = resolve_aptos_files()\nlabeled_df = pd.read_csv(paths[\"train_csv\"])\nlabeled_df[\"image_id\"] = labeled_df[\"id_code\"].astype(str)\nlabeled_df[\"diagnosis\"] = labeled_df[\"diagnosis\"].astype(int)\nlabeled_df[\"original_path\"] = labeled_df[\"image_id\"].apply(lambda x: find_image_path(x, paths[\"train_image_dirs\"]))\nif labeled_df[\"original_path\"].isna().any():\n    raise FileNotFoundError(\"Sebagian train images tidak ditemukan.\")\nlabeled_df = labeled_df[[\"image_id\", \"diagnosis\", \"original_path\"]].copy()\nlabeled_df[\"class_name\"] = labeled_df[\"diagnosis\"].map(CLASS_NAMES)\n\nsample_submission_df = pd.read_csv(paths[\"sample_submission\"]) if paths[\"sample_submission\"] else pd.DataFrame(columns=[\"id_code\", \"diagnosis\"])\nif paths[\"test_csv\"] is not None:\n    kaggle_test_set = pd.read_csv(paths[\"test_csv\"])\nelse:\n    kaggle_test_set = sample_submission_df[[\"id_code\"]].copy()\nkaggle_test_set[\"image_id\"] = kaggle_test_set[\"id_code\"].astype(str)\nkaggle_test_set[\"original_path\"] = kaggle_test_set[\"image_id\"].apply(lambda x: find_image_path(x, paths[\"test_image_dirs\"]))\nif len(kaggle_test_set) and kaggle_test_set[\"original_path\"].isna().any():\n    raise FileNotFoundError(\"Sebagian Kaggle test images tidak ditemukan.\")\n\ntrain_val_df, internal_test_split = train_test_split(\n    labeled_df,\n    test_size=0.15,\n    stratify=labeled_df[\"diagnosis\"],\n    random_state=SEED,\n)\ntrain_split, val_split = train_test_split(\n    train_val_df,\n    test_size=0.15 / 0.85,\n    stratify=train_val_df[\"diagnosis\"],\n    random_state=SEED,\n)\ntrain_split = train_split.copy()\nval_split = val_split.copy()\ninternal_test_split = internal_test_split.copy()\ntrain_split[\"split\"] = \"train_split\"\nval_split[\"split\"] = \"val_split\"\ninternal_test_split[\"split\"] = \"internal_test_split\"\nlabeled_split_df = pd.concat([train_split, val_split, internal_test_split], ignore_index=True)\n\ndef split_ids(name):\n    return set(labeled_split_df.loc[labeled_split_df[\"split\"] == name, \"image_id\"])\n\noverlap_check = {\n    \"train_val\": len(split_ids(\"train_split\") & split_ids(\"val_split\")),\n    \"train_internal_test\": len(split_ids(\"train_split\") & split_ids(\"internal_test_split\")),\n    \"val_internal_test\": len(split_ids(\"val_split\") & split_ids(\"internal_test_split\")),\n}\nassert all(v == 0 for v in overlap_check.values()), overlap_check\n\nlabeled_split_df.to_csv(RESULTS_DIR / \"labeled_split_metadata.csv\", index=False)\nkaggle_test_set.to_csv(RESULTS_DIR / \"kaggle_test_metadata_unlabeled.csv\", index=False)\n\nprint(f\"Competition root: {paths['root']}\")\nprint(f\"Train rows: {len(labeled_df)}\")\nprint(f\"Kaggle test rows without labels: {len(kaggle_test_set)}\")\nprint(\"Overlap check:\", overlap_check)\ndisplay(pd.crosstab(labeled_split_df[\"split\"], labeled_split_df[\"class_name\"], margins=True))\ndisplay(labeled_df[\"diagnosis\"].value_counts().sort_index().rename(index=CLASS_NAMES).to_frame(\"count\"))\ndisplay(kaggle_test_set.head())\n","metadata":{},"outputs":[],"execution_count":null},{"id":"c15b2651","cell_type":"markdown","source":"## Plain Resize Cache Utilities\n\nPlain resize remains the primary preprocessing. Caches are versioned by input size and dataset kind. Kaggle test cache is intentionally delayed until the submission cell.\n","metadata":{}},{"id":"b80cb181","cell_type":"code","source":"PREPROCESS_CONFIGS = {\n    \"plain_resize_384\": {\"size\": 384, \"mode\": \"plain\"},\n    \"plain_resize_512\": {\"size\": 512, \"mode\": \"plain\"},\n    \"crop_resize_384\": {\"size\": 384, \"mode\": \"crop\"},\n    \"crop_gaussian_resize_384\": {\"size\": 384, \"mode\": \"crop_gaussian\"},\n}\nBLACK_THRESHOLD = 7\n\n\ndef read_rgb_image(path):\n    if CV2_AVAILABLE:\n        img_bgr = cv2.imread(str(path), cv2.IMREAD_COLOR)\n        if img_bgr is None:\n            raise FileNotFoundError(path)\n        return cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)\n    return np.array(Image.open(path).convert(\"RGB\"))\n\n\ndef resize_rgb(img, size):\n    if CV2_AVAILABLE:\n        return cv2.resize(img, (size, size), interpolation=cv2.INTER_AREA)\n    return np.array(Image.fromarray(img).resize((size, size), Image.BILINEAR))\n\n\ndef crop_black_border(img, threshold=BLACK_THRESHOLD):\n    gray = img.mean(axis=2)\n    mask = gray > threshold\n    if not mask.any():\n        return img\n    y, x = np.where(mask)\n    return img[y.min():y.max() + 1, x.min():x.max() + 1]\n\n\ndef pad_to_square(img, fill=0):\n    h, w = img.shape[:2]\n    side = max(h, w)\n    top = (side - h) // 2\n    bottom = side - h - top\n    left = (side - w) // 2\n    right = side - w - left\n    return np.pad(img, ((top, bottom), (left, right), (0, 0)), mode=\"constant\", constant_values=fill)\n\n\ndef gaussian_enhance(img, sigma=10):\n    if not CV2_AVAILABLE:\n        return img\n    blur = cv2.GaussianBlur(img, (0, 0), sigmaX=sigma)\n    out = cv2.addWeighted(img, 1.6, blur, -0.6, 16)\n    return np.clip(out, 0, 255).astype(np.uint8)\n\n\ndef preprocess_image(path, config_name):\n    cfg = PREPROCESS_CONFIGS[config_name]\n    img = read_rgb_image(path)\n    if cfg[\"mode\"] in [\"crop\", \"crop_gaussian\"]:\n        img = pad_to_square(crop_black_border(img), fill=0)\n    if cfg[\"mode\"] == \"crop_gaussian\":\n        img = gaussian_enhance(img)\n    return resize_rgb(img, cfg[\"size\"]).astype(np.uint8)\n\n\ndef safe_stem(text):\n    return \"\".join(ch if ch.isalnum() or ch in [\"-\", \"_\"] else \"_\" for ch in str(text))\n\n\ndef build_cache(metadata, config_name, dataset_kind):\n    cache_dir = CACHE_ROOT / config_name / dataset_kind\n    cache_dir.mkdir(parents=True, exist_ok=True)\n    rows = []\n    for i, (_, row) in enumerate(tqdm(metadata.reset_index(drop=True).iterrows(), total=len(metadata), desc=f\"cache {dataset_kind} {config_name}\")):\n        split_name = row.get(\"split\", dataset_kind)\n        image_id = row[\"image_id\"]\n        cache_path = cache_dir / f\"{safe_stem(split_name)}_{i:05d}_{safe_stem(image_id)}.npy\"\n        if REBUILD_CACHE or not cache_path.exists():\n            arr = preprocess_image(row[\"original_path\"], config_name)\n            np.save(cache_path, arr.astype(np.uint8), allow_pickle=False)\n        rec = {\n            \"image_id\": image_id,\n            \"id_code\": row.get(\"id_code\", image_id),\n            \"split\": split_name,\n            \"original_path\": row[\"original_path\"],\n            \"cache_path\": str(cache_path),\n            \"preprocess_config\": config_name,\n            \"dataset_kind\": dataset_kind,\n        }\n        if \"diagnosis\" in row and not pd.isna(row[\"diagnosis\"]):\n            rec[\"diagnosis\"] = int(row[\"diagnosis\"])\n            rec[\"class_name\"] = CLASS_NAMES[int(row[\"diagnosis\"])]\n        rows.append(rec)\n    cache_df = pd.DataFrame(rows)\n    cache_df.to_csv(cache_dir / \"metadata_cache.csv\", index=False)\n    return cache_df\n\n\ndef show_preprocessing_examples(metadata, config_names=(\"plain_resize_384\", \"crop_resize_384\", \"crop_gaussian_resize_384\"), n=2):\n    sample = metadata.sample(n=min(n, len(metadata)), random_state=SEED)\n    for _, row in sample.iterrows():\n        fig, axes = plt.subplots(1, len(config_names) + 1, figsize=(4.2 * (len(config_names) + 1), 4))\n        axes[0].imshow(read_rgb_image(row[\"original_path\"]))\n        axes[0].set_title(f\"Original\\n{row.get('class_name', '')}\")\n        axes[0].axis(\"off\")\n        for ax, cfg_name in zip(axes[1:], config_names):\n            arr = preprocess_image(row[\"original_path\"], cfg_name)\n            ax.imshow(arr)\n            ax.set_title(f\"{cfg_name}\\n{arr.shape}\")\n            ax.axis(\"off\")\n        plt.tight_layout()\n        plt.show()\n","metadata":{},"outputs":[],"execution_count":null},{"id":"ec84f2a1","cell_type":"markdown","source":"## Stratified Fold Assignment and Labeled Caches\n\nThis cell assigns balanced folds, checks leakage, then builds/reuses labeled caches for the active model input sizes.\n","metadata":{}},{"id":"0020228b","cell_type":"code","source":"def assign_stratified_folds(metadata, n_folds=N_FOLDS, seed=FOLD_SEED):\n    fold_meta = metadata.copy().reset_index(drop=True)\n    fold_meta[\"fold\"] = -1\n    splitter = StratifiedKFold(n_splits=n_folds, shuffle=True, random_state=seed)\n    for fold, (_, valid_idx) in enumerate(splitter.split(fold_meta, fold_meta[\"diagnosis\"])):\n        fold_meta.loc[valid_idx, \"fold\"] = fold\n    if (fold_meta[\"fold\"] < 0).any():\n        raise RuntimeError(\"Some rows did not receive a fold assignment.\")\n    return fold_meta\n\nkfold_metadata = assign_stratified_folds(labeled_df, n_folds=N_FOLDS, seed=FOLD_SEED)\nkfold_metadata[\"split\"] = \"kfold_labeled\"\nkfold_metadata.to_csv(KFOLD_RESULTS_DIR / \"kfold_metadata.csv\", index=False)\n\nfold_distribution = pd.crosstab(kfold_metadata[\"fold\"], kfold_metadata[\"class_name\"], margins=True)\nfold_distribution.to_csv(KFOLD_RESULTS_DIR / \"fold_class_distribution.csv\")\nprint(\"Fold class distribution:\")\ndisplay(fold_distribution)\n\nleakage_check = {}\nfor fold in range(N_FOLDS):\n    val_ids = set(kfold_metadata.loc[kfold_metadata[\"fold\"] == fold, \"image_id\"])\n    train_ids = set(kfold_metadata.loc[kfold_metadata[\"fold\"] != fold, \"image_id\"])\n    leakage_check[f\"fold_{fold}\"] = len(val_ids & train_ids)\nassert all(v == 0 for v in leakage_check.values()), leakage_check\nprint(\"K-fold leakage check:\", leakage_check)\n\nneeded_preprocess = sorted({f\"plain_resize_{spec['input_size']}\" for spec in ACTIVE_KFOLD_SPECS})\nkfold_cache_by_preprocess = {}\nfor cfg_name in needed_preprocess:\n    cache_df = build_cache(kfold_metadata, cfg_name, \"labeled_kfold\")\n    cache_df = cache_df.merge(kfold_metadata[[\"image_id\", \"fold\"]], on=\"image_id\", how=\"left\")\n    cache_df.to_csv(KFOLD_RESULTS_DIR / f\"kfold_cache_metadata_{cfg_name}.csv\", index=False)\n    kfold_cache_by_preprocess[cfg_name] = cache_df\nprint(\"Built/reused labeled K-fold caches:\", list(kfold_cache_by_preprocess.keys()))\nprint(\"Kaggle test cache is still delayed until submission.\")\n","metadata":{},"outputs":[],"execution_count":null},{"id":"bc89a8fd","cell_type":"markdown","source":"## Augmentation, Dataset, and DataLoader Utilities\n\nTraining folds use strong controlled augmentation; validation and test inference are deterministic. Validation rows never receive random augmentation.\n","metadata":{}},{"id":"c873f524","cell_type":"code","source":"IMAGENET_MEAN = [0.485, 0.456, 0.406]\nIMAGENET_STD = [0.229, 0.224, 0.225]\n\n\nclass AlbumentationsTransform:\n    def __init__(self, transform):\n        self.transform = transform\n\n    def __call__(self, img):\n        return self.transform(image=np.asarray(img))[\"image\"]\n\n\ndef albumentations_eval_transform(img_size, hflip=False):\n    ops = [A.Resize(img_size, img_size)]\n    if hflip:\n        ops.append(A.HorizontalFlip(p=1.0))\n    ops.extend([A.Normalize(mean=IMAGENET_MEAN, std=IMAGENET_STD), ToTensorV2()])\n    return AlbumentationsTransform(A.Compose(ops))\n\n\ndef torchvision_eval_transform(img_size, hflip=False):\n    ops = [transforms.ToPILImage(), transforms.Resize((img_size, img_size))]\n    if hflip:\n        ops.append(transforms.RandomHorizontalFlip(p=1.0))\n    ops.extend([transforms.ToTensor(), transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD)])\n    return transforms.Compose(ops)\n\n\ndef get_eval_transform(img_size, hflip=False):\n    if ALBUMENTATIONS_AVAILABLE and CV2_AVAILABLE:\n        return albumentations_eval_transform(img_size, hflip=hflip)\n    return torchvision_eval_transform(img_size, hflip=hflip)\n\n\ndef get_transforms(img_size, strength=\"strong_controlled\"):\n    eval_tf = get_eval_transform(img_size, hflip=False)\n    if strength == \"none\":\n        return eval_tf, eval_tf\n    if ALBUMENTATIONS_AVAILABLE and CV2_AVAILABLE:\n        train_tf = AlbumentationsTransform(A.Compose([\n            A.Resize(img_size, img_size),\n            A.Affine(\n                scale=(0.85, 1.15),\n                translate_percent=(-0.12, 0.12),\n                rotate=(-180, 180),\n                shear=(-10, 10),\n                mode=cv2.BORDER_CONSTANT,\n                cval=0,\n                p=0.90,\n            ),\n            A.HorizontalFlip(p=0.5),\n            A.VerticalFlip(p=0.5),\n            A.RandomBrightnessContrast(brightness_limit=0.20, contrast_limit=0.20, p=0.75),\n            A.HueSaturationValue(hue_shift_limit=10, sat_shift_limit=20, val_shift_limit=20, p=0.50),\n            A.OneOf([\n                A.GaussianBlur(blur_limit=(3, 5), p=1.0),\n                A.Sharpen(alpha=(0.15, 0.35), lightness=(0.8, 1.2), p=1.0),\n            ], p=0.25),\n            A.Normalize(mean=IMAGENET_MEAN, std=IMAGENET_STD),\n            ToTensorV2(),\n        ]))\n        return train_tf, eval_tf\n\n    train_tf = transforms.Compose([\n        transforms.ToPILImage(),\n        transforms.RandomAffine(degrees=180, translate=(0.12, 0.12), scale=(0.85, 1.15), shear=10, fill=0),\n        transforms.RandomHorizontalFlip(p=0.5),\n        transforms.RandomVerticalFlip(p=0.5),\n        transforms.ColorJitter(brightness=0.20, contrast=0.20, saturation=0.20, hue=10 / 180),\n        transforms.RandomApply([transforms.GaussianBlur(kernel_size=3, sigma=(0.1, 1.5))], p=0.20),\n        transforms.RandomAdjustSharpness(sharpness_factor=2.0, p=0.20),\n        transforms.ToTensor(),\n        transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),\n    ])\n    return train_tf, eval_tf\n\n\ndef get_tta_transforms(img_size, level=\"light\"):\n    transforms_named = [(\"original\", get_eval_transform(img_size, hflip=False))]\n    if level in [\"light\", \"light_plus\"]:\n        transforms_named.append((\"hflip\", get_eval_transform(img_size, hflip=True)))\n    return transforms_named\n\n\nclass AptosLabeledDataset(Dataset):\n    def __init__(self, metadata, transform=None, loss_type=\"cross_entropy\"):\n        self.metadata = metadata.reset_index(drop=True).copy()\n        self.transform = transform\n        self.loss_type = loss_type\n\n    def __len__(self):\n        return len(self.metadata)\n\n    def __getitem__(self, idx):\n        row = self.metadata.iloc[idx]\n        img = np.load(row[\"cache_path\"]).astype(np.uint8)\n        if self.transform is not None:\n            img = self.transform(img)\n        label = int(row[\"diagnosis\"])\n        if self.loss_type == \"smooth_l1\":\n            target = torch.tensor([float(label)], dtype=torch.float32)\n        else:\n            target = torch.tensor(label, dtype=torch.long)\n        return img, target, row[\"image_id\"]\n\n\nclass AptosUnlabeledDataset(Dataset):\n    def __init__(self, metadata, transform=None):\n        self.metadata = metadata.reset_index(drop=True).copy()\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.metadata)\n\n    def __getitem__(self, idx):\n        row = self.metadata.iloc[idx]\n        img = np.load(row[\"cache_path\"]).astype(np.uint8)\n        if self.transform is not None:\n            img = self.transform(img)\n        return img, row.get(\"id_code\", row[\"image_id\"])\n\n\ndef make_labeled_loaders(cache_df, cfg):\n    train_tf, eval_tf = get_transforms(cfg[\"input_size\"], cfg.get(\"augmentation_strength\", \"strong_controlled\"))\n    train_meta = cache_df[cache_df[\"split\"] == \"train_split\"].reset_index(drop=True)\n    val_meta = cache_df[cache_df[\"split\"] == \"val_split\"].reset_index(drop=True)\n    internal_test_meta = cache_df[cache_df[\"split\"] == \"internal_test_split\"].reset_index(drop=True)\n    train_ds = AptosLabeledDataset(train_meta, transform=train_tf, loss_type=cfg[\"loss_type\"])\n    val_ds = AptosLabeledDataset(val_meta, transform=eval_tf, loss_type=cfg[\"loss_type\"])\n    internal_test_ds = AptosLabeledDataset(internal_test_meta, transform=eval_tf, loss_type=cfg[\"loss_type\"])\n\n    sampler = None\n    shuffle_train = True\n    if cfg.get(\"class_balance\") == \"weighted_sampler\":\n        labels = train_meta[\"diagnosis\"].astype(int).values\n        counts = np.bincount(labels, minlength=NUM_CLASSES).astype(float)\n        class_weights = counts.sum() / np.maximum(counts, 1.0)\n        sample_weights = class_weights[labels]\n        sampler = WeightedRandomSampler(\n            weights=torch.as_tensor(sample_weights, dtype=torch.double),\n            num_samples=len(sample_weights),\n            replacement=True,\n        )\n        shuffle_train = False\n        print(f\"Using WeightedRandomSampler for {cfg['experiment_id']} with class counts {counts.astype(int).tolist()}\")\n\n    train_loader = DataLoader(train_ds, batch_size=cfg[\"batch_size\"], shuffle=shuffle_train, sampler=sampler, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY)\n    val_loader = DataLoader(val_ds, batch_size=cfg[\"batch_size\"], shuffle=False, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY)\n    internal_test_loader = DataLoader(internal_test_ds, batch_size=cfg[\"batch_size\"], shuffle=False, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY)\n    return train_loader, val_loader, internal_test_loader, train_meta, val_meta, internal_test_meta\n\n\ndef make_labeled_eval_loader(cache_df, cfg, split_kind, transform_override=None):\n    eval_tf = transform_override or get_eval_transform(cfg[\"input_size\"], hflip=False)\n    meta = cache_df[cache_df[\"split\"] == split_kind].reset_index(drop=True)\n    ds = AptosLabeledDataset(meta, transform=eval_tf, loss_type=cfg[\"loss_type\"])\n    return DataLoader(ds, batch_size=cfg[\"batch_size\"], shuffle=False, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY)\n\n\ndef make_unlabeled_loader(cache_df, cfg, transform_override=None):\n    eval_tf = transform_override or get_eval_transform(cfg[\"input_size\"], hflip=False)\n    ds = AptosUnlabeledDataset(cache_df, transform=eval_tf)\n    return DataLoader(ds, batch_size=cfg[\"batch_size\"], shuffle=False, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY)\n","metadata":{},"outputs":[],"execution_count":null},{"id":"9e103ca4","cell_type":"markdown","source":"## Metrics and Threshold Helpers\n\nMetrics are kept separate: accuracy, macro F1, weighted F1, ROC AUC, and QWK are not interchangeable. Composite is used only as a configured selection metric.\n","metadata":{}},{"id":"50c9bc40","cell_type":"code","source":"def qwk_score(y_true, y_pred):\n    return cohen_kappa_score(np.asarray(y_true).astype(int), np.asarray(y_pred).astype(int), weights=\"quadratic\")\n\n\ndef regression_to_class(pred_cont, thresholds):\n    return np.digitize(np.asarray(pred_cont).reshape(-1), np.asarray(thresholds, dtype=float)).clip(0, 4).astype(int)\n\n\ndef optimize_qwk_thresholds(y_true, pred_cont, init=(0.5, 1.5, 2.5, 3.5), rounds=3, steps=41):\n    y_true = np.asarray(y_true).astype(int)\n    pred_cont = np.asarray(pred_cont).reshape(-1)\n    thresholds = np.array(init, dtype=float)\n    best = qwk_score(y_true, regression_to_class(pred_cont, thresholds))\n    for _ in range(rounds):\n        for i in range(4):\n            low = 0.0 if i == 0 else thresholds[i - 1] + 0.05\n            high = 4.0 if i == 3 else thresholds[i + 1] - 0.05\n            if low >= high:\n                continue\n            for cand in np.linspace(low, high, steps):\n                trial = thresholds.copy()\n                trial[i] = cand\n                if np.any(np.diff(trial) <= 0):\n                    continue\n                score = qwk_score(y_true, regression_to_class(pred_cont, trial))\n                if score > best:\n                    best = score\n                    thresholds = trial\n    return thresholds.tolist(), float(best)\n\n\ndef compute_specificity(cm):\n    out = {}\n    total = cm.sum()\n    for cls in range(NUM_CLASSES):\n        tp = cm[cls, cls]\n        fp = cm[:, cls].sum() - tp\n        fn = cm[cls, :].sum() - tp\n        tn = total - tp - fp - fn\n        out[CLASS_NAMES[cls]] = tn / (tn + fp) if (tn + fp) else 0.0\n    return out\n\n\ndef class_prob_to_expected_score(probs):\n    return np.asarray(probs) @ np.arange(NUM_CLASSES)\n\n\ndef metrics_from_predictions(y_true, y_pred, y_prob=None):\n    cm = confusion_matrix(y_true, y_pred, labels=list(range(NUM_CLASSES)))\n    spec = compute_specificity(cm)\n    auc_macro, auc_weighted = np.nan, np.nan\n    if y_prob is not None:\n        try:\n            y_bin = label_binarize(y_true, classes=list(range(NUM_CLASSES)))\n            auc_macro = roc_auc_score(y_bin, y_prob, average=\"macro\", multi_class=\"ovr\")\n            auc_weighted = roc_auc_score(y_bin, y_prob, average=\"weighted\", multi_class=\"ovr\")\n        except Exception:\n            pass\n    metrics = {\n        \"accuracy\": accuracy_score(y_true, y_pred),\n        \"macro_precision\": precision_score(y_true, y_pred, average=\"macro\", zero_division=0),\n        \"weighted_precision\": precision_score(y_true, y_pred, average=\"weighted\", zero_division=0),\n        \"macro_recall_sensitivity\": recall_score(y_true, y_pred, average=\"macro\", zero_division=0),\n        \"weighted_recall\": recall_score(y_true, y_pred, average=\"weighted\", zero_division=0),\n        \"macro_specificity\": float(np.mean(list(spec.values()))),\n        \"macro_f1\": f1_score(y_true, y_pred, average=\"macro\", zero_division=0),\n        \"weighted_f1\": f1_score(y_true, y_pred, average=\"weighted\", zero_division=0),\n        \"roc_auc_ovr_macro\": auc_macro,\n        \"roc_auc_ovr_weighted\": auc_weighted,\n        \"quadratic_weighted_kappa\": qwk_score(y_true, y_pred),\n    }\n    metrics[\"validation_composite\"] = 0.4 * metrics[\"accuracy\"] + 0.3 * metrics[\"macro_f1\"] + 0.3 * metrics[\"quadratic_weighted_kappa\"]\n    report = pd.DataFrame(classification_report(\n        y_true, y_pred, labels=list(range(NUM_CLASSES)), target_names=CLASS_LABELS,\n        zero_division=0, output_dict=True\n    )).T\n    return metrics, report, cm\n\n\ndef append_summary(row):\n    path = RESULTS_DIR / \"experiment_summary.csv\"\n    if path.exists():\n        df = pd.read_csv(path)\n        df = pd.concat([df, pd.DataFrame([row])], ignore_index=True)\n        df = df.drop_duplicates(subset=[\"experiment_id\", \"model\", \"split\", \"loss_type\"], keep=\"last\")\n    else:\n        df = pd.DataFrame([row])\n    df.to_csv(path, index=False)\n\n\ndef log_eval(experiment_id, model_label, split, metrics, report, cm, cfg, extra=None):\n    row = {\n        \"experiment_id\": experiment_id,\n        \"model\": model_label,\n        \"split\": split,\n        **metrics,\n        \"model_name\": cfg[\"model_name\"],\n        \"input_size\": cfg[\"input_size\"],\n        \"loss_type\": cfg[\"loss_type\"],\n        \"pooling_type\": cfg[\"pooling_type\"],\n        \"batch_size\": cfg[\"batch_size\"],\n        \"accum_steps\": cfg[\"accum_steps\"],\n        \"lr_main\": cfg[\"lr_main\"],\n        \"lr_finetune\": cfg[\"lr_finetune\"],\n        \"epochs_main\": cfg[\"epochs_main\"],\n        \"epochs_finetune\": cfg[\"epochs_finetune\"],\n        \"preprocessing_version\": cfg[\"preprocessing_version\"],\n        \"augmentation_strength\": cfg[\"augmentation_strength\"],\n        \"class_balance\": cfg.get(\"class_balance\", \"none\"),\n        \"focal_gamma\": cfg.get(\"focal_gamma\", np.nan),\n        \"runtime_notes\": cfg.get(\"runtime_notes\", \"\"),\n    }\n    if extra:\n        row.update(extra)\n    append_summary(row)\n    report.to_csv(RESULTS_DIR / f\"{experiment_id}_{split}_classification_report.csv\")\n    pd.DataFrame(cm, index=CLASS_LABELS, columns=CLASS_LABELS).to_csv(RESULTS_DIR / f\"{experiment_id}_{split}_confusion_matrix.csv\")\n    print(f\"\\n{experiment_id} | {model_label} | {split}\")\n    for key in [\"accuracy\", \"macro_f1\", \"weighted_f1\", \"quadratic_weighted_kappa\", \"validation_composite\"]:\n        print(f\"{key:30s}: {metrics[key]:.4f}\")\n    display(report)\n    plt.figure(figsize=(7, 6))\n    sns.heatmap(cm, annot=True, fmt=\"d\", cmap=\"Blues\", xticklabels=CLASS_LABELS, yticklabels=CLASS_LABELS)\n    plt.title(f\"{experiment_id} {split} confusion matrix\")\n    plt.xlabel(\"Predicted\")\n    plt.ylabel(\"True\")\n    plt.xticks(rotation=25)\n    plt.tight_layout()\n    plt.show()\n","metadata":{},"outputs":[],"execution_count":null},{"id":"58fbd04f","cell_type":"markdown","source":"## Model Builder\n\nModels are resolved through `timm`. If a requested model name is unavailable, the resolver tries configured candidates and records the change in runtime notes instead of stopping the whole notebook.\n","metadata":{}},{"id":"c4f0624a","cell_type":"code","source":"def gem(x, p=3, eps=1e-6):\n    return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1.0 / p)\n\n\nclass GeM(nn.Module):\n    def __init__(self, p=3, eps=1e-6):\n        super().__init__()\n        self.p = Parameter(torch.ones(1) * p)\n        self.eps = eps\n\n    def forward(self, x):\n        return gem(x, self.p, self.eps)\n\n\nclass TimmFeatureModel(nn.Module):\n    def __init__(self, model_name, input_size, out_dim, pooling_type=\"gem\", pretrained=True):\n        super().__init__()\n        if not TIMM_AVAILABLE:\n            raise ImportError(\"timm is not available in this Kaggle environment.\")\n        available = set(timm.list_models(pretrained=False))\n        if model_name not in available:\n            raise ValueError(f\"{model_name} is not available in timm.\")\n        self.model_name = model_name\n        self.backbone = timm.create_model(model_name, pretrained=pretrained, num_classes=0, global_pool=\"\")\n        self.pooling_type = pooling_type\n        self.pool = GeM() if pooling_type == \"gem\" else nn.AdaptiveAvgPool2d(1)\n        self.feature_dim = self._infer_feature_dim(input_size)\n        self.head = nn.Linear(self.feature_dim, out_dim)\n\n    def _forward_features(self, x):\n        if hasattr(self.backbone, \"forward_features\"):\n            feats = self.backbone.forward_features(x)\n        else:\n            feats = self.backbone(x)\n        if isinstance(feats, (tuple, list)):\n            feats = feats[-1]\n        return feats\n\n    def _pool_features(self, feats):\n        if feats.ndim == 4:\n            return self.pool(feats).flatten(1)\n        if feats.ndim == 3:\n            return feats.mean(dim=1)\n        if feats.ndim == 2:\n            return feats\n        raise RuntimeError(f\"Unexpected feature shape: {feats.shape}\")\n\n    def _infer_feature_dim(self, input_size):\n        self.backbone.eval()\n        with torch.no_grad():\n            dummy = torch.zeros(1, 3, input_size, input_size)\n            feats = self._pool_features(self._forward_features(dummy))\n        return feats.shape[1]\n\n    def forward(self, x):\n        feats = self._pool_features(self._forward_features(x))\n        return self.head(feats)\n\n\nclass TorchvisionFallbackModel(nn.Module):\n    def __init__(self, model_name, out_dim, pooling_type=\"gem\", pretrained=True):\n        super().__init__()\n        self.model_name = model_name\n        self.pool = GeM() if pooling_type == \"gem\" else nn.AdaptiveAvgPool2d(1)\n        if model_name == \"convnext_tiny\":\n            weights = models.ConvNeXt_Tiny_Weights.IMAGENET1K_V1 if pretrained else None\n            base = models.convnext_tiny(weights=weights)\n            self.features = base.features\n            feature_dim = base.classifier[2].in_features\n        elif model_name == \"efficientnet_b4\":\n            weights = models.EfficientNet_B4_Weights.IMAGENET1K_V1 if pretrained else None\n            base = models.efficientnet_b4(weights=weights)\n            self.features = base.features\n            feature_dim = base.classifier[1].in_features\n        else:\n            weights = models.EfficientNet_B3_Weights.IMAGENET1K_V1 if pretrained else None\n            base = models.efficientnet_b3(weights=weights)\n            self.features = base.features\n            feature_dim = base.classifier[1].in_features\n        self.head = nn.Linear(feature_dim, out_dim)\n\n    def forward(self, x):\n        feats = self.features(x)\n        feats = self.pool(feats).flatten(1)\n        return self.head(feats)\n\n\ndef list_timm_candidates(pattern):\n    if not TIMM_AVAILABLE:\n        return []\n    try:\n        return sorted(timm.list_models(f\"*{pattern}*\", pretrained=False))[:25]\n    except Exception:\n        available = sorted(timm.list_models(pretrained=False))\n        return [name for name in available if pattern in name][:25]\n\n\ndef resolve_timm_model_name(cfg):\n    if not TIMM_AVAILABLE:\n        return None\n    available = set(timm.list_models(pretrained=False))\n    candidates = cfg.get(\"model_candidates\") or [cfg[\"model_name\"]]\n    for cand in candidates:\n        if cand in available:\n            if cand != cfg[\"model_name\"]:\n                msg = f\"Resolved requested model {cfg['model_name']} to available timm candidate {cand}.\"\n                print(msg)\n                cfg[\"runtime_notes\"] = (cfg.get(\"runtime_notes\", \"\") + \" \" + msg).strip()\n            return cand\n    family_hint = cfg.get(\"model_name\", \"\").split(\"_\")[0]\n    near = list_timm_candidates(family_hint)\n    msg = (\n        f\"No requested candidate available for {cfg['experiment_id']}: {candidates}. \"\n        f\"Nearby timm models containing '{family_hint}': {near[:12]}\"\n    )\n    print(msg)\n    cfg[\"runtime_notes\"] = (cfg.get(\"runtime_notes\", \"\") + \" \" + msg).strip()\n    return None\n\n\ndef build_model_from_config(cfg):\n    out_dim = 1 if cfg[\"loss_type\"] == \"smooth_l1\" else NUM_CLASSES\n    resolved_name = resolve_timm_model_name(cfg)\n    if resolved_name is not None:\n        model = TimmFeatureModel(resolved_name, cfg[\"input_size\"], out_dim, pooling_type=cfg[\"pooling_type\"], pretrained=True)\n        model_label = f\"timm_{resolved_name}_{cfg['input_size']}_{cfg['loss_type']}_{cfg['pooling_type']}\"\n        return model, model_label, resolved_name\n\n    fallback_candidates = cfg.get(\"model_candidates\", [])\n    if any(\"convnext\" in name for name in fallback_candidates + [cfg.get(\"model_name\", \"\")]):\n        tv_name = \"convnext_tiny\"\n    elif any(\"efficientnet_b4\" in name or \"b4\" in name for name in fallback_candidates + [cfg.get(\"model_name\", \"\")]):\n        tv_name = \"efficientnet_b4\"\n    else:\n        raise ValueError(f\"No compatible timm model found for {cfg['experiment_id']} and no safe torchvision fallback is defined.\")\n    print(f\"Using torchvision fallback {tv_name} for {cfg['experiment_id']}.\")\n    cfg[\"runtime_notes\"] = (cfg.get(\"runtime_notes\", \"\") + f\" Used torchvision fallback {tv_name}.\").strip()\n    model = TorchvisionFallbackModel(tv_name, out_dim, pooling_type=cfg[\"pooling_type\"], pretrained=True)\n    model_label = f\"torchvision_{tv_name}_{cfg['input_size']}_{cfg['loss_type']}_{cfg['pooling_type']}\"\n    return model, model_label, tv_name\n\n\ndef set_requires_grad(module, value):\n    for p in module.parameters():\n        p.requires_grad = value\n\n\ndef freeze_backbone_train_head(model):\n    set_requires_grad(model, False)\n    if hasattr(model, \"head\"):\n        set_requires_grad(model.head, True)\n\n\ndef unfreeze_all(model):\n    set_requires_grad(model, True)\n","metadata":{},"outputs":[],"execution_count":null},{"id":"6e18e152","cell_type":"markdown","source":"## Training, Prediction, and Checkpoint Utilities\n\nThis reuses the stable v7 training and checkpoint format. Checkpoints are saved as `.pth` state dict plus JSON metadata.\n","metadata":{}},{"id":"e8cdfba4","cell_type":"code","source":"def clear_memory():\n    gc.collect()\n    if torch.cuda.is_available():\n        torch.cuda.empty_cache()\n\n\nclass FocalLoss(nn.Module):\n    def __init__(self, gamma=2.0, weight=None):\n        super().__init__()\n        self.gamma = gamma\n        self.weight = weight\n\n    def forward(self, logits, targets):\n        ce = F.cross_entropy(logits, targets, weight=self.weight, reduction=\"none\")\n        pt = torch.exp(-ce)\n        loss = ((1 - pt) ** self.gamma) * ce\n        return loss.mean()\n\n\ndef class_weight_tensor():\n    train_labels = labeled_split_df.loc[labeled_split_df[\"split\"] == \"train_split\", \"diagnosis\"].astype(int).values\n    counts = np.bincount(train_labels, minlength=NUM_CLASSES).astype(float)\n    weights = counts.sum() / np.maximum(counts, 1.0)\n    weights = weights / weights.mean()\n    return torch.tensor(weights, dtype=torch.float32, device=DEVICE)\n\n\ndef criterion_for(cfg):\n    if cfg[\"loss_type\"] == \"smooth_l1\":\n        return nn.SmoothL1Loss()\n    balance = cfg.get(\"class_balance\", \"none\")\n    if balance == \"class_weight\":\n        return nn.CrossEntropyLoss(weight=class_weight_tensor())\n    if balance == \"focal_loss\":\n        return FocalLoss(gamma=float(cfg.get(\"focal_gamma\", 2.0)), weight=class_weight_tensor())\n    return nn.CrossEntropyLoss()\n\n\ndef train_one_epoch(model, loader, criterion, optimizer, cfg, scaler):\n    model.train(True)\n    total_loss = 0.0\n    y_true, pred_collect = [], []\n    optimizer.zero_grad(set_to_none=True)\n    for step, (images, targets, _) in enumerate(tqdm(loader, desc=\"train\", leave=False), start=1):\n        images = images.to(DEVICE, non_blocking=True)\n        targets = targets.to(DEVICE, non_blocking=True)\n        with autocast(enabled=USE_AMP):\n            outputs = model(images)\n            if cfg[\"loss_type\"] == \"smooth_l1\":\n                outputs = outputs.float().clamp(-0.5, 4.5)\n                loss = criterion(outputs, targets) / cfg[\"accum_steps\"]\n            else:\n                loss = criterion(outputs, targets) / cfg[\"accum_steps\"]\n        scaler.scale(loss).backward()\n        if step % cfg[\"accum_steps\"] == 0 or step == len(loader):\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=5.0)\n            scaler.step(optimizer)\n            scaler.update()\n            optimizer.zero_grad(set_to_none=True)\n        total_loss += float(loss.item()) * cfg[\"accum_steps\"] * images.size(0)\n        if cfg[\"loss_type\"] == \"smooth_l1\":\n            y_true.extend(targets.detach().cpu().view(-1).numpy().astype(int).tolist())\n            pred_collect.extend(outputs.detach().cpu().view(-1).numpy().tolist())\n        else:\n            y_true.extend(targets.detach().cpu().numpy().astype(int).tolist())\n            pred_collect.extend(torch.softmax(outputs.detach().cpu(), dim=1).numpy().tolist())\n    avg_loss = total_loss / len(loader.dataset)\n    if cfg[\"loss_type\"] == \"smooth_l1\":\n        y_pred = regression_to_class(pred_collect, [0.5, 1.5, 2.5, 3.5])\n    else:\n        y_pred = np.asarray(pred_collect).argmax(axis=1)\n    acc = accuracy_score(y_true, y_pred)\n    return avg_loss, acc\n\n\n@torch.no_grad()\ndef predict_labeled(model, loader, cfg):\n    model.eval()\n    y_true, ids = [], []\n    probs, cont = [], []\n    total_loss = 0.0\n    criterion = criterion_for(cfg)\n    for images, targets, image_ids in tqdm(loader, desc=\"predict\", leave=False):\n        images = images.to(DEVICE, non_blocking=True)\n        targets_dev = targets.to(DEVICE, non_blocking=True)\n        with autocast(enabled=USE_AMP):\n            outputs = model(images)\n            if cfg[\"loss_type\"] == \"smooth_l1\":\n                outputs = outputs.float().clamp(-0.5, 4.5)\n                loss = criterion(outputs, targets_dev)\n                cont.extend(outputs.detach().cpu().view(-1).numpy().tolist())\n                y_true.extend(targets.numpy().reshape(-1).astype(int).tolist())\n            else:\n                loss = criterion(outputs, targets_dev)\n                prob = torch.softmax(outputs.float(), dim=1)\n                probs.extend(prob.detach().cpu().numpy().tolist())\n                y_true.extend(targets.numpy().astype(int).tolist())\n        total_loss += float(loss.item()) * images.size(0)\n        ids.extend(list(image_ids))\n    if cfg[\"loss_type\"] == \"smooth_l1\":\n        return {\"loss\": total_loss / len(loader.dataset), \"y_true\": np.array(y_true), \"pred_cont\": np.array(cont), \"ids\": np.array(ids)}\n    return {\"loss\": total_loss / len(loader.dataset), \"y_true\": np.array(y_true), \"probs\": np.array(probs), \"ids\": np.array(ids)}\n\n\n@torch.no_grad()\ndef predict_unlabeled(model, loader, cfg):\n    model.eval()\n    ids, probs, cont = [], [], []\n    for images, id_codes in tqdm(loader, desc=\"predict\", leave=False):\n        images = images.to(DEVICE, non_blocking=True)\n        with autocast(enabled=USE_AMP):\n            outputs = model(images)\n            if cfg[\"loss_type\"] == \"smooth_l1\":\n                cont.extend(outputs.float().clamp(-0.5, 4.5).detach().cpu().view(-1).numpy().tolist())\n            else:\n                probs.extend(torch.softmax(outputs.float(), dim=1).detach().cpu().numpy().tolist())\n        ids.extend(list(id_codes))\n    if cfg[\"loss_type\"] == \"smooth_l1\":\n        return pd.DataFrame({\"id_code\": ids, \"pred_cont\": cont})\n    return pd.DataFrame({\"id_code\": ids, \"probs\": list(probs)})\n\n\ndef evaluate_prediction_dict(pred, cfg, thresholds=None, tune_threshold=False):\n    y_true = pred[\"y_true\"]\n    if cfg[\"loss_type\"] == \"smooth_l1\":\n        if tune_threshold:\n            thresholds, _ = optimize_qwk_thresholds(y_true, pred[\"pred_cont\"])\n        thresholds = thresholds or [0.5, 1.5, 2.5, 3.5]\n        y_pred = regression_to_class(pred[\"pred_cont\"], thresholds)\n        y_prob = None\n    else:\n        y_prob = pred[\"probs\"]\n        y_pred = y_prob.argmax(axis=1)\n        thresholds = None\n    metrics, report, cm = metrics_from_predictions(y_true, y_pred, y_prob=y_prob)\n    metrics[\"eval_loss\"] = pred[\"loss\"]\n    return metrics, report, cm, thresholds\n\n\ndef json_sanitize(obj):\n    if isinstance(obj, dict):\n        return {str(k): json_sanitize(v) for k, v in obj.items()}\n    if isinstance(obj, (list, tuple)):\n        return [json_sanitize(v) for v in obj]\n    if isinstance(obj, (np.integer,)):\n        return int(obj)\n    if isinstance(obj, (np.floating,)):\n        return float(obj)\n    if isinstance(obj, np.ndarray):\n        return obj.tolist()\n    if isinstance(obj, Path):\n        return str(obj)\n    if isinstance(obj, (float, np.floating)) and pd.isna(obj):\n        return None\n    return obj\n\n\ndef checkpoint_paths(experiment_id):\n    return {\n        \"state\": RESULTS_DIR / f\"{experiment_id}_best_state.pth\",\n        \"meta\": RESULTS_DIR / f\"{experiment_id}_best_meta.json\",\n        \"legacy\": RESULTS_DIR / f\"{experiment_id}_best.pt\",\n    }\n\n\ndef checkpoint_available(cfg):\n    paths = checkpoint_paths(cfg[\"experiment_id\"])\n    return paths[\"state\"].exists() or paths[\"legacy\"].exists()\n\n\ndef save_safe_checkpoint(model, cfg, best, model_label, resolved_name):\n    paths = checkpoint_paths(cfg[\"experiment_id\"])\n    torch.save(model.state_dict(), paths[\"state\"])\n    metadata = {\n        \"experiment_id\": cfg[\"experiment_id\"],\n        \"cfg\": json_sanitize(cfg),\n        \"best\": json_sanitize(best),\n        \"model_label\": model_label,\n        \"resolved_model_name\": resolved_name,\n        \"state_path\": str(paths[\"state\"]),\n    }\n    paths[\"meta\"].write_text(json.dumps(metadata, indent=2))\n    return paths\n\n\ndef torch_load_own_legacy_checkpoint(path):\n    # Safe only for checkpoints produced by this notebook/session. Do not use this for unknown files.\n    try:\n        return torch.load(path, map_location=DEVICE, weights_only=False)\n    except TypeError:\n        return torch.load(path, map_location=DEVICE)\n\n\ndef load_checkpoint_into_model(model, cfg):\n    paths = checkpoint_paths(cfg[\"experiment_id\"])\n    if paths[\"state\"].exists():\n        state_dict = torch.load(paths[\"state\"], map_location=DEVICE)\n        model.load_state_dict(state_dict)\n        metadata = json.loads(paths[\"meta\"].read_text()) if paths[\"meta\"].exists() else {}\n        best = metadata.get(\"best\", {})\n        source = \"state_dict_json_metadata\"\n        return best, str(paths[\"state\"]), source\n    if paths[\"legacy\"].exists():\n        legacy = torch_load_own_legacy_checkpoint(paths[\"legacy\"])\n        model.load_state_dict(legacy[\"model_state\"])\n        best = json_sanitize(legacy.get(\"best\", {}))\n        # Convert the old mixed checkpoint to the new safe format for future runs.\n        model_label = legacy.get(\"model_label\", \"recovered_model\")\n        resolved_name = best.get(\"resolved_model_name\", cfg[\"model_name\"])\n        save_safe_checkpoint(model, cfg, best, model_label, resolved_name)\n        source = \"legacy_mixed_checkpoint_weights_only_false\"\n        return best, str(paths[\"legacy\"]), source\n    raise FileNotFoundError(f\"No checkpoint found for {cfg['experiment_id']}\")\n\n\ndef evaluate_model_from_loaded_checkpoint(model, cfg, model_label, resolved_name, best, checkpoint_path, recovery_source, runtime_seconds=0.0):\n    cache_df = labeled_cache_by_config.get(cfg[\"preprocessing_version\"])\n    if cache_df is None:\n        cache_df = build_cache(labeled_split_df, cfg[\"preprocessing_version\"], \"labeled_splits\")\n        labeled_cache_by_config[cfg[\"preprocessing_version\"]] = cache_df\n    _, val_loader, internal_test_loader, _, _, _ = make_labeled_loaders(cache_df, cfg)\n\n    val_pred = predict_labeled(model, val_loader, cfg)\n    val_metrics, val_report, val_cm, thresholds = evaluate_prediction_dict(\n        val_pred,\n        cfg,\n        thresholds=best.get(\"thresholds\"),\n        tune_threshold=(cfg[\"loss_type\"] == \"smooth_l1\" and best.get(\"thresholds\") is None),\n    )\n    internal_pred = predict_labeled(model, internal_test_loader, cfg)\n    internal_metrics, internal_report, internal_cm, _ = evaluate_prediction_dict(\n        internal_pred,\n        cfg,\n        thresholds=thresholds,\n        tune_threshold=False,\n    )\n    extra = {\n        \"best_epoch\": best.get(\"epoch\"),\n        \"best_thresholds\": json.dumps(thresholds) if thresholds is not None else \"\",\n        \"resolved_model_name\": resolved_name,\n        \"runtime_seconds\": runtime_seconds,\n        \"oom_notes\": \"\",\n        \"checkpoint_path\": checkpoint_path,\n        \"checkpoint_recovery_source\": recovery_source,\n    }\n    log_eval(cfg[\"experiment_id\"], model_label, \"val\", val_metrics, val_report, val_cm, cfg, extra)\n    log_eval(cfg[\"experiment_id\"], model_label, \"internal_test\", internal_metrics, internal_report, internal_cm, cfg, extra)\n    return {\n        \"cfg\": cfg,\n        \"model\": model,\n        \"model_label\": model_label,\n        \"resolved_model_name\": resolved_name,\n        \"thresholds\": thresholds,\n        \"val_metrics\": val_metrics,\n        \"internal_metrics\": internal_metrics,\n        \"checkpoint\": checkpoint_path,\n        \"recovery_source\": recovery_source,\n    }\n\n\ndef recover_experiment_from_checkpoint(cfg):\n    if not checkpoint_available(cfg):\n        return None\n    print(f\"Recovering checkpoint for {cfg['experiment_id']} without retraining.\")\n    model, model_label, resolved_name = build_model_from_config(cfg)\n    model = model.to(DEVICE)\n    best, checkpoint_path, recovery_source = load_checkpoint_into_model(model, cfg)\n    result = evaluate_model_from_loaded_checkpoint(\n        model,\n        cfg,\n        model_label,\n        resolved_name,\n        best,\n        checkpoint_path,\n        recovery_source,\n        runtime_seconds=0.0,\n    )\n    return result\n\n\ndef train_model_for_config(cfg):\n    start = time.time()\n    cache_df = labeled_cache_by_config.get(cfg[\"preprocessing_version\"])\n    if cache_df is None:\n        cache_df = build_cache(labeled_split_df, cfg[\"preprocessing_version\"], \"labeled_splits\")\n        labeled_cache_by_config[cfg[\"preprocessing_version\"]] = cache_df\n    train_loader, val_loader, internal_test_loader, _, _, _ = make_labeled_loaders(cache_df, cfg)\n    model, model_label, resolved_name = build_model_from_config(cfg)\n    model = model.to(DEVICE)\n    criterion = criterion_for(cfg)\n    scaler = GradScaler(enabled=USE_AMP)\n    best = {\"validation_composite\": -np.inf, \"epoch\": 0, \"thresholds\": None}\n    ckpt_paths = checkpoint_paths(cfg[\"experiment_id\"])\n    history = []\n    global_epoch = 0\n\n    phases = []\n    if cfg.get(\"warmup_head_epochs\", 0) > 0:\n        phases.append((\"head_warmup\", cfg[\"warmup_head_epochs\"], cfg[\"lr_main\"], freeze_backbone_train_head))\n    phases.append((\"main_training\", cfg[\"epochs_main\"], cfg[\"lr_main\"], unfreeze_all))\n    phases.append((\"low_lr_finetune\", cfg[\"epochs_finetune\"], cfg[\"lr_finetune\"], unfreeze_all))\n\n    for phase_name, epochs, lr, trainable_fn in phases:\n        trainable_fn(model)\n        optimizer = torch.optim.Adam([p for p in model.parameters() if p.requires_grad], lr=lr, weight_decay=1e-5)\n        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode=\"max\", factor=0.5, patience=5, min_lr=1e-7)\n        bad = 0\n        print(f\"\\n{cfg['experiment_id']} | {model_label} | phase={phase_name} | lr={lr}\")\n        for _ in range(epochs):\n            global_epoch += 1\n            train_loss, train_acc = train_one_epoch(model, train_loader, criterion, optimizer, cfg, scaler)\n            val_pred = predict_labeled(model, val_loader, cfg)\n            val_metrics, _, _, thresholds = evaluate_prediction_dict(val_pred, cfg, tune_threshold=(cfg[\"loss_type\"] == \"smooth_l1\"))\n            scheduler.step(val_metrics[\"validation_composite\"])\n            improved = val_metrics[\"validation_composite\"] > best[\"validation_composite\"] + 1e-5\n            if improved:\n                best = {\n                    **val_metrics,\n                    \"epoch\": global_epoch,\n                    \"thresholds\": thresholds,\n                    \"resolved_model_name\": resolved_name,\n                    \"model_label\": model_label,\n                }\n                save_safe_checkpoint(model, cfg, best, model_label, resolved_name)\n                bad = 0\n                print(f\"Saved best epoch={global_epoch}, composite={best['validation_composite']:.4f}, acc={best['accuracy']:.4f}, macro_f1={best['macro_f1']:.4f}, qwk={best['quadratic_weighted_kappa']:.4f}\")\n            else:\n                bad += 1\n            history.append({\n                \"global_epoch\": global_epoch,\n                \"phase\": phase_name,\n                \"train_loss\": train_loss,\n                \"train_acc\": train_acc,\n                \"val_loss\": val_metrics[\"eval_loss\"],\n                \"val_acc\": val_metrics[\"accuracy\"],\n                \"val_macro_f1\": val_metrics[\"macro_f1\"],\n                \"val_qwk\": val_metrics[\"quadratic_weighted_kappa\"],\n                \"val_composite\": val_metrics[\"validation_composite\"],\n                \"lr\": optimizer.param_groups[0][\"lr\"],\n                \"thresholds\": json.dumps(thresholds) if thresholds is not None else \"\",\n            })\n            print(f\"Epoch {global_epoch:03d} | train_acc={train_acc:.4f} val_acc={val_metrics['accuracy']:.4f} val_macro_f1={val_metrics['macro_f1']:.4f} val_qwk={val_metrics['quadratic_weighted_kappa']:.4f} composite={val_metrics['validation_composite']:.4f}\")\n            if bad >= 12:\n                print(f\"Early stopping phase {phase_name}.\")\n                break\n\n    if ckpt_paths[\"state\"].exists() or ckpt_paths[\"legacy\"].exists():\n        best, checkpoint_path, recovery_source = load_checkpoint_into_model(model, cfg)\n    else:\n        checkpoint_path = \"\"\n        recovery_source = \"no_checkpoint\"\n\n    history_df = pd.DataFrame(history)\n    history_df.to_csv(RESULTS_DIR / f\"{cfg['experiment_id']}_history.csv\", index=False)\n\n    return evaluate_model_from_loaded_checkpoint(\n        model,\n        cfg,\n        model_label,\n        resolved_name,\n        best,\n        checkpoint_path,\n        recovery_source,\n        runtime_seconds=round(time.time() - start, 2),\n    )\n","metadata":{},"outputs":[],"execution_count":null},{"id":"52ef3116","cell_type":"markdown","source":"## Fold-Local Criterion Patch\n\nOptional focal/class-weight experiments need weights computed from each training fold, not from the old single split. Default C+B uses ordinary cross entropy, but this patch keeps optional focal C methodologically correct.\n","metadata":{}},{"id":"90a29cf8","cell_type":"code","source":"def criterion_for(cfg):\n    if cfg[\"loss_type\"] == \"smooth_l1\":\n        return nn.SmoothL1Loss()\n    balance = cfg.get(\"class_balance\", \"none\")\n    weight = None\n    if cfg.get(\"class_weight_values\") is not None:\n        weight = torch.tensor(cfg[\"class_weight_values\"], dtype=torch.float32, device=DEVICE)\n    elif balance in [\"class_weight\", \"focal_loss\"]:\n        weight = class_weight_tensor()\n    if balance == \"class_weight\":\n        return nn.CrossEntropyLoss(weight=weight)\n    if balance == \"focal_loss\":\n        return FocalLoss(gamma=float(cfg.get(\"focal_gamma\", 2.0)), weight=weight)\n    return nn.CrossEntropyLoss()\n\nprint(\"criterion_for patched for fold-local class weights when focal/class_weight is enabled.\")\n","metadata":{},"outputs":[],"execution_count":null},{"id":"c8d83449","cell_type":"markdown","source":"## K-Fold Multi-Model Runner\n\nFor every active model and fold, this cell trains or recovers the fold checkpoint, saves validation-fold probabilities, writes history/report/confusion matrix files, and continues if one model/fold fails.\n","metadata":{}},{"id":"dd0bf289","cell_type":"code","source":"def spec_by_key(model_key):\n    for spec in KFOLD_MODEL_SPECS:\n        if spec[\"model_key\"] == model_key:\n            return spec\n    raise KeyError(model_key)\n\n\ndef make_kfold_cfg(spec, fold):\n    cfg = dict(spec)\n    cfg[\"experiment_id\"] = f\"{spec['experiment_id']}_fold{fold}\"\n    cfg[\"preprocessing_version\"] = f\"plain_resize_{spec['input_size']}\"\n    cfg[\"fold\"] = int(fold)\n    return cfg\n\n\ndef fold_cache_for_training(cache_df, fold):\n    fold_df = cache_df.copy()\n    fold_df[\"split\"] = np.where(fold_df[\"fold\"].astype(int) == int(fold), \"val_split\", \"train_split\")\n    return fold_df\n\n\ndef add_fold_class_weights(cfg, train_meta):\n    if cfg.get(\"class_balance\", \"none\") in [\"class_weight\", \"focal_loss\"]:\n        counts = np.bincount(train_meta[\"diagnosis\"].astype(int).values, minlength=NUM_CLASSES).astype(float)\n        weights = counts.sum() / np.maximum(counts, 1.0)\n        weights = weights / weights.mean()\n        cfg[\"class_weight_values\"] = weights.tolist()\n    return cfg\n\n\ndef save_model_fold_artifacts(model_key, fold, history, report, cm, pred):\n    prefix = f\"{model_key}_fold_{fold}\"\n    history_df = pd.DataFrame(history)\n    history_path = KFOLD_RESULTS_DIR / f\"{prefix}_history.csv\"\n    history_df.to_csv(history_path, index=False)\n    pd.DataFrame(cm, index=CLASS_LABELS, columns=CLASS_LABELS).to_csv(KFOLD_RESULTS_DIR / f\"{prefix}_confusion_matrix.csv\")\n    report.to_csv(KFOLD_RESULTS_DIR / f\"{prefix}_classification_report.csv\")\n    prob_cols = [f\"prob_{i}\" for i in range(NUM_CLASSES)]\n    pred_df = pd.DataFrame(pred[\"probs\"], columns=prob_cols)\n    pred_df.insert(0, \"image_id\", pred[\"ids\"])\n    pred_df.insert(1, \"model_key\", model_key)\n    pred_df.insert(2, \"fold\", fold)\n    pred_df.insert(3, \"diagnosis\", pred[\"y_true\"].astype(int))\n    pred_df[\"pred_argmax\"] = pred_df[prob_cols].values.argmax(axis=1)\n    pred_df.to_csv(KFOLD_RESULTS_DIR / f\"{prefix}_val_predictions.csv\", index=False)\n    return history_path, pred_df\n\n\ndef summarize_fold_generalization(model_key, fold, history_df, best_epoch):\n    if history_df.empty:\n        return {\n            \"model_key\": model_key,\n            \"fold\": fold,\n            \"best_epoch\": best_epoch,\n            \"final_epoch\": np.nan,\n            \"final_train_acc\": np.nan,\n            \"final_val_acc\": np.nan,\n            \"train_val_gap\": np.nan,\n            \"overfit_warning\": True,\n            \"stagnation_epoch\": np.nan,\n            \"generalization_note\": \"missing history\",\n        }\n    final = history_df.iloc[-1]\n    gap = float(final.get(\"train_acc\", np.nan) - final.get(\"val_acc\", np.nan))\n    best_val = float(history_df[\"val_composite\"].max()) if \"val_composite\" in history_df else np.nan\n    stagnation_epoch = history_df[history_df[\"val_composite\"] >= best_val - 1e-8][\"global_epoch\"].min() if \"val_composite\" in history_df else np.nan\n    overfit_loss = False\n    if len(history_df) >= 3:\n        tail = history_df.tail(3)\n        overfit_loss = bool(tail[\"train_loss\"].iloc[-1] < tail[\"train_loss\"].iloc[0] and tail[\"val_loss\"].iloc[-1] > tail[\"val_loss\"].iloc[0])\n    overfit_warning = bool(gap > 0.10 or overfit_loss)\n    if overfit_warning and gap > 0.10:\n        note = \"clear overfit: train accuracy keeps exceeding validation\"\n    elif overfit_warning:\n        note = \"possible overfit: train loss improves while validation loss worsens\"\n    elif best_epoch < int(final[\"global_epoch\"]) - 5:\n        note = \"early peak: best validation happened well before final epoch\"\n    else:\n        note = \"acceptable fold generalization\"\n    return {\n        \"model_key\": model_key,\n        \"fold\": fold,\n        \"best_epoch\": best_epoch,\n        \"final_epoch\": int(final[\"global_epoch\"]),\n        \"final_train_acc\": float(final[\"train_acc\"]),\n        \"final_val_acc\": float(final[\"val_acc\"]),\n        \"train_val_gap\": gap,\n        \"overfit_warning\": overfit_warning,\n        \"stagnation_epoch\": stagnation_epoch,\n        \"generalization_note\": note,\n    }\n\n\ndef train_or_recover_model_fold(spec, fold):\n    model_key = spec[\"model_key\"]\n    cfg = make_kfold_cfg(spec, fold)\n    cache_df = kfold_cache_by_preprocess[cfg[\"preprocessing_version\"]]\n    fold_df = fold_cache_for_training(cache_df, fold)\n    train_loader, val_loader, _, train_meta, val_meta, _ = make_labeled_loaders(fold_df, cfg)\n    cfg = add_fold_class_weights(cfg, train_meta)\n    model, model_label, resolved_name = build_model_from_config(cfg)\n    model = model.to(DEVICE)\n    ckpt_exists = checkpoint_available(cfg)\n    status = \"recovered\" if ckpt_exists and not FORCE_RETRAIN_KFOLD else \"trained\"\n    history = []\n    best = {\"validation_composite\": -np.inf, \"epoch\": 0, \"thresholds\": None}\n    start = time.time()\n\n    if ckpt_exists and not FORCE_RETRAIN_KFOLD:\n        print(f\"[{model_key} fold {fold}] Recovering existing checkpoint for {cfg['experiment_id']}.\")\n        best, checkpoint_path, recovery_source = load_checkpoint_into_model(model, cfg)\n    else:\n        print(f\"[{model_key} fold {fold}] Training {cfg['experiment_id']} from pretrained weights.\")\n        criterion = criterion_for(cfg)\n        scaler = GradScaler(enabled=USE_AMP)\n        global_epoch = 0\n        phases = []\n        if cfg.get(\"warmup_head_epochs\", 0) > 0:\n            phases.append((\"head_warmup\", cfg[\"warmup_head_epochs\"], cfg[\"lr_main\"], freeze_backbone_train_head))\n        phases.append((\"main_training\", cfg[\"epochs_main\"], cfg[\"lr_main\"], unfreeze_all))\n        phases.append((\"low_lr_finetune\", cfg[\"epochs_finetune\"], cfg[\"lr_finetune\"], unfreeze_all))\n\n        for phase_name, epochs, lr, trainable_fn in phases:\n            trainable_fn(model)\n            optimizer = torch.optim.Adam([p for p in model.parameters() if p.requires_grad], lr=lr, weight_decay=1e-5)\n            scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode=\"max\", factor=0.5, patience=5, min_lr=1e-7)\n            bad = 0\n            print(f\"\\n{model_key} fold {fold} | {model_label} | phase={phase_name} | lr={lr}\")\n            for _ in range(int(epochs)):\n                global_epoch += 1\n                train_loss, train_acc = train_one_epoch(model, train_loader, criterion, optimizer, cfg, scaler)\n                val_pred = predict_labeled(model, val_loader, cfg)\n                val_metrics, _, _, thresholds = evaluate_prediction_dict(val_pred, cfg, tune_threshold=False)\n                scheduler.step(val_metrics[\"validation_composite\"])\n                improved = val_metrics[\"validation_composite\"] > best[\"validation_composite\"] + 1e-5\n                if improved:\n                    best = {\n                        **val_metrics,\n                        \"epoch\": global_epoch,\n                        \"thresholds\": thresholds,\n                        \"resolved_model_name\": resolved_name,\n                        \"model_label\": model_label,\n                    }\n                    save_safe_checkpoint(model, cfg, best, model_label, resolved_name)\n                    bad = 0\n                    print(f\"[{model_key} fold {fold}] Saved best epoch={global_epoch}, composite={best['validation_composite']:.4f}, acc={best['accuracy']:.4f}, macro_f1={best['macro_f1']:.4f}, qwk={best['quadratic_weighted_kappa']:.4f}\")\n                else:\n                    bad += 1\n                history.append({\n                    \"model_key\": model_key,\n                    \"fold\": fold,\n                    \"global_epoch\": global_epoch,\n                    \"phase\": phase_name,\n                    \"train_loss\": train_loss,\n                    \"train_acc\": train_acc,\n                    \"val_loss\": val_metrics[\"eval_loss\"],\n                    \"val_acc\": val_metrics[\"accuracy\"],\n                    \"val_macro_f1\": val_metrics[\"macro_f1\"],\n                    \"val_qwk\": val_metrics[\"quadratic_weighted_kappa\"],\n                    \"val_composite\": val_metrics[\"validation_composite\"],\n                    \"lr\": optimizer.param_groups[0][\"lr\"],\n                })\n                print(f\"{model_key} fold {fold} Epoch {global_epoch:03d} | train_acc={train_acc:.4f} val_acc={val_metrics['accuracy']:.4f} val_macro_f1={val_metrics['macro_f1']:.4f} val_qwk={val_metrics['quadratic_weighted_kappa']:.4f} composite={val_metrics['validation_composite']:.4f}\")\n                if bad >= KFOLD_EARLY_STOP_PATIENCE:\n                    print(f\"[{model_key} fold {fold}] Early stopping phase {phase_name}.\")\n                    break\n        best, checkpoint_path, recovery_source = load_checkpoint_into_model(model, cfg)\n\n    val_pred = predict_labeled(model, val_loader, cfg)\n    val_metrics, val_report, val_cm, _ = evaluate_prediction_dict(val_pred, cfg, tune_threshold=False)\n    history_path, pred_df = save_model_fold_artifacts(model_key, fold, history, val_report, val_cm, val_pred)\n    hist_df = pd.DataFrame(history)\n    gen = summarize_fold_generalization(model_key, fold, hist_df, int(best.get(\"epoch\", 0)))\n    row = {\n        \"model_key\": model_key,\n        \"fold\": fold,\n        \"experiment_id\": cfg[\"experiment_id\"],\n        \"model_name\": cfg[\"model_name\"],\n        \"resolved_model_name\": resolved_name,\n        \"status\": status,\n        \"checkpoint_path\": checkpoint_path,\n        \"recovery_source\": recovery_source,\n        \"runtime_seconds\": round(time.time() - start, 2),\n        \"train_rows\": len(train_meta),\n        \"val_rows\": len(val_meta),\n        \"best_epoch\": int(best.get(\"epoch\", 0)),\n        \"val_accuracy\": val_metrics[\"accuracy\"],\n        \"val_macro_f1\": val_metrics[\"macro_f1\"],\n        \"val_weighted_f1\": val_metrics[\"weighted_f1\"],\n        \"val_qwk\": val_metrics[\"quadratic_weighted_kappa\"],\n        \"val_roc_auc\": val_metrics[\"roc_auc_ovr_macro\"],\n        \"val_composite\": val_metrics[\"validation_composite\"],\n        \"val_loss\": val_metrics[\"eval_loss\"],\n        \"history_path\": str(history_path),\n        \"runtime_notes\": cfg.get(\"runtime_notes\", \"\"),\n        **gen,\n    }\n    del model\n    clear_memory()\n    return row, pred_df\n\n\nfold_metrics_rows = []\noof_pred_frames_by_model = {spec[\"model_key\"]: [] for spec in ACTIVE_KFOLD_SPECS}\nfold_errors = []\n\nif RUN_KFOLD_MULTIMODEL:\n    for spec in ACTIVE_KFOLD_SPECS:\n        model_key = spec[\"model_key\"]\n        print(f\"\\n=== Processing K-fold model {model_key}: {spec['experiment_id']} ===\")\n        for fold in range(N_FOLDS):\n            try:\n                row, pred_df = train_or_recover_model_fold(spec, fold)\n                fold_metrics_rows.append(row)\n                oof_pred_frames_by_model[model_key].append(pred_df)\n                print(f\"[{model_key} fold {fold}] success: val_acc={row['val_accuracy']:.4f}, val_macro_f1={row['val_macro_f1']:.4f}, val_qwk={row['val_qwk']:.4f}\")\n            except RuntimeError as e:\n                msg = str(e)\n                status = \"OOM\" if \"out of memory\" in msg.lower() else \"runtime_error\"\n                if status == \"OOM\":\n                    clear_memory()\n                fold_errors.append({\"model_key\": model_key, \"fold\": fold, \"status\": status, \"error\": msg[:1000]})\n                print(f\"[{model_key} fold {fold}] {status}: {msg[:300]}\")\n            except Exception as e:\n                fold_errors.append({\"model_key\": model_key, \"fold\": fold, \"status\": \"error\", \"error\": str(e)[:1000]})\n                print(f\"[{model_key} fold {fold}] error: {str(e)[:300]}\")\n\nfold_metrics_by_model = pd.DataFrame(fold_metrics_rows)\nfold_metrics_by_model.to_csv(KFOLD_RESULTS_DIR / \"fold_metrics_by_model.csv\", index=False)\npd.DataFrame(fold_errors).to_csv(KFOLD_RESULTS_DIR / \"kfold_model_errors_and_oom_notes.csv\", index=False)\n\nhistory_frames = []\nfor history_file in sorted(KFOLD_RESULTS_DIR.glob(\"*_fold_*_history.csv\")):\n    try:\n        hist = pd.read_csv(history_file)\n        if len(hist):\n            history_frames.append(hist)\n    except Exception:\n        pass\nif history_frames:\n    pd.concat(history_frames, ignore_index=True).to_csv(KFOLD_RESULTS_DIR / \"per_fold_history_by_model.csv\", index=False)\n\noof_predictions_by_model = {}\nprob_cols = [f\"prob_{i}\" for i in range(NUM_CLASSES)]\nfor model_key, frames in oof_pred_frames_by_model.items():\n    if frames:\n        oof_df = pd.concat(frames, ignore_index=True).sort_values(\"image_id\").reset_index(drop=True)\n        oof_predictions_by_model[model_key] = oof_df\n        oof_df.to_csv(KFOLD_RESULTS_DIR / f\"oof_predictions_{model_key}.csv\", index=False)\n        np.save(KFOLD_RESULTS_DIR / f\"oof_probabilities_{model_key}.npy\", oof_df[prob_cols].values.astype(np.float32), allow_pickle=False)\n\nprint(\"K-fold multimodel run complete.\")\ndisplay(fold_metrics_by_model)\nif fold_errors:\n    display(pd.DataFrame(fold_errors))\nprint(\"OOF models available:\", list(oof_predictions_by_model.keys()))\n","metadata":{},"outputs":[],"execution_count":null},{"id":"29d07a09","cell_type":"markdown","source":"## OOF Model and Ensemble Metrics\n\nThis cell evaluates C-only, B-only, and OOF ensembles. Ensemble weights are derived only from OOF metrics, not internal test or Kaggle test. Thresholds for accuracy, QWK, and composite are optimized only on OOF predictions.\n","metadata":{}},{"id":"006baf48","cell_type":"code","source":"def expected_score_from_probs(probs):\n    return np.asarray(probs) @ np.arange(NUM_CLASSES)\n\n\ndef threshold_score(y_true, scores, thresholds, metric_name):\n    pred = regression_to_class(scores, thresholds)\n    metrics, _, _ = metrics_from_predictions(y_true, pred, y_prob=None)\n    if metric_name == \"accuracy\":\n        return metrics[\"accuracy\"]\n    if metric_name == \"qwk\":\n        return metrics[\"quadratic_weighted_kappa\"]\n    if metric_name == \"composite\":\n        return metrics[\"validation_composite\"]\n    if metric_name == \"macro_f1\":\n        return metrics[\"macro_f1\"]\n    raise ValueError(metric_name)\n\n\ndef optimize_thresholds_for_metric(y_true, scores, metric_name, init=(0.5, 1.5, 2.5, 3.5), rounds=3, steps=31):\n    thresholds = np.array(init, dtype=float)\n    best_score = threshold_score(y_true, scores, thresholds, metric_name)\n    for _ in range(rounds):\n        for i in range(4):\n            low = 0.0 if i == 0 else thresholds[i - 1] + 0.05\n            high = 4.0 if i == 3 else thresholds[i + 1] - 0.05\n            if low >= high:\n                continue\n            for cand in np.linspace(low, high, steps):\n                trial = thresholds.copy()\n                trial[i] = cand\n                if np.any(np.diff(trial) <= 0):\n                    continue\n                score = threshold_score(y_true, scores, trial, metric_name)\n                if score > best_score:\n                    best_score = score\n                    thresholds = trial\n    return thresholds.tolist(), float(best_score)\n\n\ndef class_recalls_from_report(report):\n    return {\n        \"mild_recall\": float(report.loc[\"Mild\", \"recall\"]) if \"Mild\" in report.index else np.nan,\n        \"severe_recall\": float(report.loc[\"Severe\", \"recall\"]) if \"Severe\" in report.index else np.nan,\n        \"proliferative_recall\": float(report.loc[\"Proliferative_DR\", \"recall\"]) if \"Proliferative_DR\" in report.index else np.nan,\n    }\n\n\ndef evaluate_prob_candidate(candidate_name, candidate_type, probs, y_true, members, weights=None):\n    rows = []\n    report_outputs = {}\n    cm_outputs = {}\n    pred_argmax = np.asarray(probs).argmax(axis=1)\n    methods = [(\"argmax\", pred_argmax, None, np.nan, False)]\n    scores = expected_score_from_probs(probs)\n    if RUN_OOF_THRESHOLD_TUNING:\n        for metric_name in [\"accuracy\", \"qwk\", \"composite\"]:\n            thresholds, opt_score = optimize_thresholds_for_metric(y_true, scores, metric_name)\n            pred_thr = regression_to_class(scores, thresholds)\n            methods.append((f\"threshold_{metric_name}\", pred_thr, thresholds, opt_score, True))\n    for method_name, pred, thresholds, opt_score, threshold_tuned in methods:\n        metrics, report, cm = metrics_from_predictions(y_true, pred, y_prob=probs)\n        recalls = class_recalls_from_report(report)\n        full_name = candidate_name if method_name == \"argmax\" else f\"{candidate_name}_{method_name}\"\n        rows.append({\n            \"candidate_name\": full_name,\n            \"base_candidate\": candidate_name,\n            \"candidate_type\": candidate_type,\n            \"prediction_method\": method_name,\n            \"members\": \"+\".join(members),\n            \"weights\": json.dumps(list(map(float, weights))) if weights is not None else \"\",\n            \"threshold_tuned\": bool(threshold_tuned),\n            \"thresholds\": json.dumps(thresholds) if thresholds is not None else \"\",\n            \"optimized_score\": opt_score,\n            \"oof_accuracy\": metrics[\"accuracy\"],\n            \"oof_macro_f1\": metrics[\"macro_f1\"],\n            \"oof_weighted_f1\": metrics[\"weighted_f1\"],\n            \"oof_qwk\": metrics[\"quadratic_weighted_kappa\"],\n            \"oof_roc_auc\": metrics[\"roc_auc_ovr_macro\"],\n            \"oof_composite\": metrics[\"validation_composite\"],\n            **recalls,\n        })\n        report_outputs[full_name] = report\n        cm_outputs[full_name] = cm\n    return rows, report_outputs, cm_outputs\n\n\ndef align_oof_models(oof_dict, model_keys):\n    aligned = None\n    for key in model_keys:\n        df = oof_dict[key].copy()\n        cols = [\"image_id\", \"diagnosis\"] + [f\"prob_{i}\" for i in range(NUM_CLASSES)]\n        df = df[cols].rename(columns={f\"prob_{i}\": f\"{key}_prob_{i}\" for i in range(NUM_CLASSES)})\n        aligned = df if aligned is None else aligned.merge(df, on=[\"image_id\", \"diagnosis\"], how=\"inner\")\n    return aligned.sort_values(\"image_id\").reset_index(drop=True)\n\n\ndef softmax_weights(scores, temperature=ENSEMBLE_SOFTMAX_TEMPERATURE):\n    arr = np.asarray(scores, dtype=float)\n    arr = arr - arr.max()\n    ex = np.exp(arr / max(float(temperature), 1e-6))\n    return ex / ex.sum()\n\n\ndef multiclass_ece(y_true, probs, n_bins=15):\n    probs = np.asarray(probs)\n    y_true = np.asarray(y_true).astype(int)\n    conf = probs.max(axis=1)\n    pred = probs.argmax(axis=1)\n    correct = (pred == y_true).astype(float)\n    bins = np.linspace(0.0, 1.0, n_bins + 1)\n    ece = 0.0\n    rows = []\n    for lo, hi in zip(bins[:-1], bins[1:]):\n        mask = (conf > lo) & (conf <= hi)\n        if not mask.any():\n            rows.append({\"bin_low\": lo, \"bin_high\": hi, \"count\": 0, \"accuracy\": np.nan, \"confidence\": np.nan})\n            continue\n        bin_acc = correct[mask].mean()\n        bin_conf = conf[mask].mean()\n        ece += mask.mean() * abs(bin_acc - bin_conf)\n        rows.append({\"bin_low\": lo, \"bin_high\": hi, \"count\": int(mask.sum()), \"accuracy\": bin_acc, \"confidence\": bin_conf})\n    return float(ece), pd.DataFrame(rows)\n\n\nprob_cols = [f\"prob_{i}\" for i in range(NUM_CLASSES)]\noof_candidate_registry = {}\noof_metric_rows = []\nreport_registry = {}\ncm_registry = {}\nmodel_argmax_scores = {}\n\nfor model_key, oof_df in oof_predictions_by_model.items():\n    if len(oof_df) != len(labeled_df):\n        print(f\"Warning: model {model_key} OOF rows={len(oof_df)}, expected={len(labeled_df)}. Metrics still computed for available OOF rows.\")\n    probs = oof_df[prob_cols].values.astype(np.float32)\n    y_true = oof_df[\"diagnosis\"].astype(int).values\n    rows, reports, cms = evaluate_prob_candidate(f\"M_{model_key}_oof\", \"single_model_oof\", probs, y_true, [model_key])\n    oof_metric_rows.extend(rows)\n    report_registry.update(reports)\n    cm_registry.update(cms)\n    oof_candidate_registry[f\"M_{model_key}_oof\"] = {\"members\": [model_key], \"weights\": [1.0], \"probs\": probs, \"y_true\": y_true}\n    model_argmax_scores[model_key] = rows[0][\"oof_composite\"]\n\ncomplete_model_keys = [k for k, df in oof_predictions_by_model.items() if len(df) == len(labeled_df)]\nif RUN_KFOLD_ENSEMBLE and len(complete_model_keys) >= 2:\n    aligned = align_oof_models(oof_predictions_by_model, complete_model_keys)\n    y_true = aligned[\"diagnosis\"].astype(int).values\n    member_probs = {key: aligned[[f\"{key}_prob_{i}\" for i in range(NUM_CLASSES)]].values.astype(np.float32) for key in complete_model_keys}\n    composite_scores = np.array([model_argmax_scores.get(key, 1.0) for key in complete_model_keys], dtype=float)\n    ensemble_defs = []\n    equal_w = np.ones(len(complete_model_keys), dtype=float) / len(complete_model_keys)\n    stable_w = composite_scores / composite_scores.sum()\n    soft_w = softmax_weights(composite_scores)\n    ensemble_defs.extend([\n        (\"E_all_active_simple_avg\", equal_w),\n        (\"E_all_active_weighted_avg\", stable_w),\n        (\"E_all_active_soft_weighted\", soft_w),\n    ])\n    if set([\"C\", \"B\"]).issubset(set(complete_model_keys)):\n        cb_keys = [\"C\", \"B\"]\n        cb_scores = np.array([model_argmax_scores.get(key, 1.0) for key in cb_keys], dtype=float)\n        ensemble_defs.extend([\n            (\"E_CB_simple_avg\", np.array([0.5, 0.5]), cb_keys),\n            (\"E_CB_weighted_avg\", cb_scores / cb_scores.sum(), cb_keys),\n            (\"E_CB_soft_weighted\", softmax_weights(cb_scores), cb_keys),\n        ])\n    if len(complete_model_keys) >= 3:\n        for drop_key in complete_model_keys:\n            keep = [k for k in complete_model_keys if k != drop_key]\n            ensemble_defs.append((f\"E_leave_one_out_without_{drop_key}\", np.ones(len(keep)) / len(keep), keep))\n\n    for item in ensemble_defs:\n        if len(item) == 2:\n            name, weights = item\n            keys = complete_model_keys\n        else:\n            name, weights, keys = item\n        probs = np.zeros_like(member_probs[keys[0]])\n        for w, key in zip(weights, keys):\n            probs += float(w) * member_probs[key]\n        rows, reports, cms = evaluate_prob_candidate(name, \"ensemble_oof\", probs, y_true, keys, weights)\n        oof_metric_rows.extend(rows)\n        report_registry.update(reports)\n        cm_registry.update(cms)\n        oof_candidate_registry[name] = {\"members\": keys, \"weights\": list(map(float, weights)), \"probs\": probs, \"y_true\": y_true}\n\nall_oof_metrics = pd.DataFrame(oof_metric_rows)\nif len(all_oof_metrics):\n    all_oof_metrics = all_oof_metrics.sort_values(\"oof_composite\", ascending=False).reset_index(drop=True)\n    selection_col = KFOLD_SELECTION_METRIC\n    if selection_col not in all_oof_metrics.columns:\n        selection_col = \"oof_composite\"\n    best_idx = all_oof_metrics[selection_col].astype(float).idxmax()\n    all_oof_metrics[\"selection_status\"] = \"\"\n    all_oof_metrics.loc[best_idx, \"selection_status\"] = f\"selected_by_{selection_col}\"\n    all_oof_metrics.to_csv(KFOLD_RESULTS_DIR / \"kfold_multimodel_summary.csv\", index=False)\n    model_rows = all_oof_metrics[all_oof_metrics[\"candidate_type\"] == \"single_model_oof\"]\n    ensemble_rows = all_oof_metrics[all_oof_metrics[\"candidate_type\"] == \"ensemble_oof\"]\n    model_rows.to_csv(KFOLD_RESULTS_DIR / \"oof_model_metrics.csv\", index=False)\n    ensemble_rows.to_csv(KFOLD_RESULTS_DIR / \"oof_ensemble_metrics.csv\", index=False)\n    for name, report in report_registry.items():\n        report.to_csv(KFOLD_RESULTS_DIR / f\"{safe_stem(name)}_classification_report.csv\")\n    for name, cm in cm_registry.items():\n        pd.DataFrame(cm, index=CLASS_LABELS, columns=CLASS_LABELS).to_csv(KFOLD_RESULTS_DIR / f\"{safe_stem(name)}_confusion_matrix.csv\")\n    display(all_oof_metrics.head(20))\nelse:\n    print(\"No OOF metrics available. Check kfold_model_errors_and_oom_notes.csv.\")\n\nif RUN_OOF_CALIBRATION and len(oof_candidate_registry):\n    cal_rows = []\n    for name, item in oof_candidate_registry.items():\n        if name.startswith(\"M_\") or name in [\"E_CB_simple_avg\", \"E_all_active_simple_avg\"]:\n            y_true = item[\"y_true\"]\n            probs = item[\"probs\"]\n            ece, reliability = multiclass_ece(y_true, probs)\n            nll = -np.log(np.clip(probs[np.arange(len(y_true)), y_true], 1e-7, 1.0)).mean()\n            cal_rows.append({\n                \"candidate_name\": name,\n                \"members\": \"+\".join(item[\"members\"]),\n                \"average_max_probability\": float(probs.max(axis=1).mean()),\n                \"ece_15_bins\": ece,\n                \"nll\": float(nll),\n            })\n            reliability.to_csv(KFOLD_RESULTS_DIR / f\"{safe_stem(name)}_reliability_bins.csv\", index=False)\n    calibration_df = pd.DataFrame(cal_rows).sort_values(\"ece_15_bins\")\n    calibration_df.to_csv(KFOLD_RESULTS_DIR / \"oof_calibration_diagnostics.csv\", index=False)\n    display(calibration_df)\n","metadata":{},"outputs":[],"execution_count":null},{"id":"364b1a9a","cell_type":"markdown","source":"## K-Fold Multi-Model Test Prediction and Submission\n\nAt submission time only, each fold checkpoint predicts Kaggle `test.csv`. Probabilities are averaged across folds and then across models according to the selected OOF candidate. No local metric is computed from Kaggle test.\n","metadata":{}},{"id":"a33bcad7","cell_type":"code","source":"def build_kaggle_test_cache_for_spec(spec):\n    if len(kaggle_test_set) == 0:\n        return pd.DataFrame()\n    test_meta = kaggle_test_set.copy()\n    test_meta[\"split\"] = \"kaggle_test\"\n    return build_cache(test_meta, f\"plain_resize_{spec['input_size']}\", \"kaggle_test\")\n\n\ndef load_model_fold_for_inference(spec, fold):\n    cfg = make_kfold_cfg(spec, fold)\n    model, model_label, resolved_name = build_model_from_config(cfg)\n    model = model.to(DEVICE)\n    best, checkpoint_path, source = load_checkpoint_into_model(model, cfg)\n    return cfg, model, {\"model_key\": spec[\"model_key\"], \"fold\": fold, \"model_label\": model_label, \"resolved_model_name\": resolved_name, \"checkpoint_path\": checkpoint_path, \"source\": source}\n\n\ndef dataframe_probs_to_array(pred_df):\n    return np.vstack(pred_df[\"probs\"].values).astype(np.float32)\n\n\ndef average_test_probabilities_for_model(spec):\n    model_key = spec[\"model_key\"]\n    test_cache_df = build_kaggle_test_cache_for_spec(spec)\n    fold_probs = []\n    info_rows = []\n    successful_folds = sorted(fold_metrics_by_model.loc[fold_metrics_by_model[\"model_key\"] == model_key, \"fold\"].astype(int).unique().tolist()) if len(fold_metrics_by_model) else []\n    for fold in successful_folds:\n        cfg, model, info = load_model_fold_for_inference(spec, fold)\n        loader = make_unlabeled_loader(test_cache_df, cfg)\n        pred_df = predict_unlabeled(model, loader, cfg)\n        probs = dataframe_probs_to_array(pred_df)\n        fold_probs.append(probs)\n        info_rows.append({**info, \"num_test_rows\": len(pred_df)})\n        del model\n        clear_memory()\n    if not fold_probs:\n        raise RuntimeError(f\"No checkpoints are available for model {model_key} test prediction.\")\n    avg_probs = np.mean(np.stack(fold_probs, axis=0), axis=0)\n    return avg_probs, pd.DataFrame(info_rows)\n\n\ndef save_submission_from_predictions(name, pred_labels):\n    if len(sample_submission_df):\n        sub = sample_submission_df[[\"id_code\"]].copy()\n    else:\n        sub = kaggle_test_set[[\"id_code\"]].copy()\n    if len(sub) != len(pred_labels):\n        raise RuntimeError(f\"Submission row count mismatch: template={len(sub)}, predictions={len(pred_labels)}\")\n    sub[\"diagnosis\"] = np.asarray(pred_labels).astype(int).clip(0, 4)\n    path = WORK_DIR / name\n    sub.to_csv(path, index=False)\n    return path\n\n\ndef labels_from_candidate_probs(probs, prediction_method, thresholds_text=\"\"):\n    if prediction_method == \"argmax\":\n        return probs.argmax(axis=1)\n    if prediction_method.startswith(\"threshold_\"):\n        thresholds = json.loads(thresholds_text) if thresholds_text else [0.5, 1.5, 2.5, 3.5]\n        return regression_to_class(expected_score_from_probs(probs), thresholds)\n    return probs.argmax(axis=1)\n\n\nif RUN_KFOLD_TEST_SUBMISSION and len(kaggle_test_set) and len(all_oof_metrics):\n    print(\"Building Kaggle test cache and fold predictions only now, at submission stage.\")\n    test_probs_by_model = {}\n    inference_tables = []\n    for spec in ACTIVE_KFOLD_SPECS:\n        model_key = spec[\"model_key\"]\n        if model_key not in oof_predictions_by_model:\n            continue\n        probs, info = average_test_probabilities_for_model(spec)\n        test_probs_by_model[model_key] = probs\n        inference_tables.append(info)\n        np.save(KFOLD_RESULTS_DIR / f\"kfold_test_avg_probabilities_{model_key}.npy\", probs.astype(np.float32), allow_pickle=False)\n        save_submission_from_predictions(f\"submission_kfold_{model_key}_argmax.csv\", probs.argmax(axis=1))\n    if inference_tables:\n        pd.concat(inference_tables, ignore_index=True).to_csv(KFOLD_RESULTS_DIR / \"kfold_test_inference_models.csv\", index=False)\n\n    submission_candidates = []\n    for name, item in oof_candidate_registry.items():\n        members = item[\"members\"]\n        weights = item[\"weights\"]\n        if not all(member in test_probs_by_model for member in members):\n            continue\n        probs = np.zeros_like(test_probs_by_model[members[0]])\n        for w, member in zip(weights, members):\n            probs += float(w) * test_probs_by_model[member]\n        rows = all_oof_metrics[all_oof_metrics[\"base_candidate\"].eq(name) | all_oof_metrics[\"candidate_name\"].eq(name)]\n        for _, row in rows.iterrows():\n            labels = labels_from_candidate_probs(probs, row[\"prediction_method\"], row.get(\"thresholds\", \"\"))\n            file_name = f\"submission_kfold_{safe_stem(row['candidate_name'])}.csv\"\n            path = save_submission_from_predictions(file_name, labels)\n            submission_candidates.append({\"candidate_name\": row[\"candidate_name\"], \"submission_path\": str(path)})\n\n    pd.DataFrame(submission_candidates).to_csv(KFOLD_RESULTS_DIR / \"kfold_submission_candidates.csv\", index=False)\n    selection_col = KFOLD_SELECTION_METRIC if KFOLD_SELECTION_METRIC in all_oof_metrics.columns else \"oof_composite\"\n    ranked = all_oof_metrics.sort_values(selection_col, ascending=False).reset_index(drop=True)\n    selected = ranked.iloc[0]\n    lookup = {row[\"candidate_name\"]: row[\"submission_path\"] for row in submission_candidates}\n    selected_path = lookup.get(selected[\"candidate_name\"])\n    if selected_path is None:\n        fallback = submission_candidates[0] if submission_candidates else None\n        if fallback is None:\n            raise RuntimeError(\"No submission candidate could be produced.\")\n        selected_path = fallback[\"submission_path\"]\n        print(f\"Selected candidate {selected['candidate_name']} was not submission-ready; falling back to {fallback['candidate_name']}.\")\n    final_path = WORK_DIR / \"final_submission.csv\"\n    pd.read_csv(selected_path).to_csv(final_path, index=False)\n    final_selection = selected.to_frame().T.copy()\n    final_selection[\"selection_metric\"] = KFOLD_SELECTION_METRIC\n    final_selection[\"submission_path\"] = str(final_path)\n    final_selection.to_csv(KFOLD_RESULTS_DIR / \"final_kfold_multimodel_selection_summary.csv\", index=False)\n    print(f\"Selected v8 K-fold candidate: {selected['candidate_name']}; metric={KFOLD_SELECTION_METRIC}; submission={final_path}\")\n    final_sub = pd.read_csv(final_path)\n    print(f\"Rows: {len(final_sub)}; predicted classes: {sorted(final_sub['diagnosis'].unique().tolist())}\")\nelse:\n    print(\"K-fold multimodel submission skipped. Either submission flag is off, no OOF metrics exist, or Kaggle test metadata is unavailable.\")\n","metadata":{},"outputs":[],"execution_count":null},{"id":"e65ec57d","cell_type":"markdown","source":"## V7 C-Only OOF vs V8 Multi-Model OOF Comparison\n\nThis section compares v7 C-only reference metrics with v8 model/ensemble OOF candidates and summarizes generalization per model and fold.\n","metadata":{}},{"id":"b5ca9903","cell_type":"code","source":"comparison_rows = list(V7_REFERENCE.values())\nif len(all_oof_metrics):\n    selected_col = KFOLD_SELECTION_METRIC if KFOLD_SELECTION_METRIC in all_oof_metrics.columns else \"oof_composite\"\n    best_by_metric = all_oof_metrics.sort_values(selected_col, ascending=False).iloc[0]\n    best_by_acc = all_oof_metrics.sort_values(\"oof_accuracy\", ascending=False).iloc[0]\n    best_by_macro = all_oof_metrics.sort_values(\"oof_macro_f1\", ascending=False).iloc[0]\n    for label, row in [\n        (\"v8_selected_by_metric\", best_by_metric),\n        (\"v8_best_accuracy\", best_by_acc),\n        (\"v8_best_macro_f1\", best_by_macro),\n    ]:\n        comparison_rows.append({\n            \"pipeline\": label,\n            \"models\": row[\"members\"],\n            \"prediction_method\": row[\"prediction_method\"],\n            \"oof_accuracy\": row[\"oof_accuracy\"],\n            \"oof_macro_f1\": row[\"oof_macro_f1\"],\n            \"oof_weighted_f1\": row[\"oof_weighted_f1\"],\n            \"oof_qwk\": row[\"oof_qwk\"],\n            \"oof_roc_auc\": row[\"oof_roc_auc\"],\n            \"oof_composite\": row[\"oof_composite\"],\n            \"severe_recall\": row[\"severe_recall\"],\n            \"mild_recall\": row[\"mild_recall\"],\n            \"proliferative_recall\": row[\"proliferative_recall\"],\n            \"notes\": row[\"candidate_name\"],\n        })\ncomparison_df = pd.DataFrame(comparison_rows)\ncomparison_df.to_csv(KFOLD_RESULTS_DIR / \"v7_c_only_vs_v8_multimodel_oof_comparison.csv\", index=False)\ndisplay(comparison_df)\n\nif len(fold_metrics_by_model):\n    diag_cols = [\n        \"model_key\", \"fold\", \"val_accuracy\", \"val_macro_f1\", \"val_qwk\", \"val_composite\",\n        \"final_train_acc\", \"final_val_acc\", \"train_val_gap\", \"best_epoch\", \"final_epoch\",\n        \"overfit_warning\", \"generalization_note\"\n    ]\n    print(\"K-Fold Multi-Model Generalization Diagnostics:\")\n    display(fold_metrics_by_model[diag_cols].sort_values([\"model_key\", \"fold\"]))\n    diag_summary = fold_metrics_by_model.groupby(\"model_key\").agg(\n        mean_val_accuracy=(\"val_accuracy\", \"mean\"),\n        std_val_accuracy=(\"val_accuracy\", \"std\"),\n        mean_val_macro_f1=(\"val_macro_f1\", \"mean\"),\n        mean_val_qwk=(\"val_qwk\", \"mean\"),\n        mean_train_val_gap=(\"train_val_gap\", \"mean\"),\n        overfit_folds=(\"overfit_warning\", \"sum\"),\n    ).reset_index()\n    diag_summary.to_csv(KFOLD_RESULTS_DIR / \"kfold_multimodel_generalization_summary.csv\", index=False)\n    display(diag_summary)\n","metadata":{},"outputs":[],"execution_count":null},{"id":"b6ecc21c","cell_type":"code","source":"from pathlib import Path\nimport pandas as pd\n\nfinal_path = Path(\"/kaggle/working/final_submission.csv\")\nsubmission_path = Path(\"/kaggle/working/submission.csv\")\n\nif not final_path.exists():\n    raise FileNotFoundError(\"final_submission.csv tidak ditemukan. Pastikan submission cell sebelumnya berhasil.\")\n\nsub = pd.read_csv(final_path)\n\nassert list(sub.columns) == [\"id_code\", \"diagnosis\"], sub.columns.tolist()\nassert sub[\"diagnosis\"].between(0, 4).all()\nsub[\"diagnosis\"] = sub[\"diagnosis\"].astype(int)\n\nsub.to_csv(submission_path, index=False)\n\nprint(\"Saved Kaggle-required submission file:\", submission_path)\nprint(\"Rows:\", len(sub))\nprint(\"Columns:\", sub.columns.tolist())\nprint(\"Predicted classes:\", sorted(sub[\"diagnosis\"].unique().tolist()))\ndisplay(sub.head())","metadata":{},"outputs":[],"execution_count":null},{"id":"e5660c77","cell_type":"markdown","source":"## Stage-2 Pseudo-Label Plan (Default Off)\n\n`RUN_PSEUDO_LABEL_STAGE = False` by default. Pseudo-labeling should only be considered after OOF ensemble performance is stable. If enabled later, use soft probabilities, start with a high confidence threshold such as 0.90–0.95, audit pseudo-label class distribution, and keep validation strictly from labeled `train.csv`.\n","metadata":{}},{"id":"e296afdd","cell_type":"markdown","source":"## External Data Future Work (Not Enabled)\n\nExternal datasets such as APTOS 2015, IDRiD, and Messidor may help private-leaderboard-oriented performance, but they require duplicate checks, label alignment, domain-shift analysis, and leakage controls. v8 does not load or train on external data.\n","metadata":{}},{"id":"cfd5b65a","cell_type":"markdown","source":"## Run Checklist\n\nExpected outputs include `fold_metrics_by_model.csv`, `per_fold_history_by_model.csv`, per-model OOF predictions/probabilities, `oof_model_metrics.csv`, `oof_ensemble_metrics.csv`, `kfold_multimodel_summary.csv`, calibration diagnostics, per-candidate reports/confusion matrices, and `final_submission.csv`. The target remains validation/OOF generalization; high train accuracy alone is not success.\n","metadata":{}},{"id":"5d42b62c-2729-48b6-bf8e-fc77b17f5b86","cell_type":"markdown","source":"## Export Offline Weights\n\nJalankan cell ini setelah training dan submission generation selesai.","metadata":{}},{"id":"9fd04c43-34f4-4592-a31d-907b3a5bae5d","cell_type":"code","source":"# Export trained checkpoints and metadata for offline Kaggle submission.\n# Run this after the K-Fold training, OOF selection, and final submission cells have succeeded.\n\nimport shutil\nimport zipfile\nfrom datetime import datetime\n\nEXPORT_DIR = WORK_DIR / \"aptos_v8_offline_weights\"\nCHECKPOINT_EXPORT_DIR = EXPORT_DIR / \"checkpoints\"\nMETADATA_EXPORT_DIR = EXPORT_DIR / \"metadata\"\nSUBMISSION_EXPORT_DIR = EXPORT_DIR / \"submissions\"\n\nif EXPORT_DIR.exists():\n    shutil.rmtree(EXPORT_DIR)\nCHECKPOINT_EXPORT_DIR.mkdir(parents=True, exist_ok=True)\nMETADATA_EXPORT_DIR.mkdir(parents=True, exist_ok=True)\nSUBMISSION_EXPORT_DIR.mkdir(parents=True, exist_ok=True)\n\nrequired_experiment_ids = []\nfor spec in ACTIVE_KFOLD_SPECS:\n    for fold in range(N_FOLDS):\n        required_experiment_ids.append(f\"{spec['experiment_id']}_fold{fold}\")\n\nmissing = []\nexported_checkpoints = []\nfor experiment_id in required_experiment_ids:\n    state_path = RESULTS_DIR / f\"{experiment_id}_best_state.pth\"\n    meta_path = RESULTS_DIR / f\"{experiment_id}_best_meta.json\"\n    if not state_path.exists():\n        missing.append(str(state_path))\n        continue\n    if not meta_path.exists():\n        missing.append(str(meta_path))\n        continue\n    shutil.copy2(state_path, CHECKPOINT_EXPORT_DIR / state_path.name)\n    shutil.copy2(meta_path, CHECKPOINT_EXPORT_DIR / meta_path.name)\n    exported = {\n        \"experiment_id\": experiment_id,\n        \"state_file\": f\"checkpoints/{state_path.name}\",\n        \"meta_file\": f\"checkpoints/{meta_path.name}\",\n    }\n    exported_checkpoints.append(exported)\n\nif missing:\n    raise FileNotFoundError(\"Missing checkpoint(s) required for offline inference:\\n\" + \"\\n\".join(missing))\n\nmetadata_sources = [\n    KFOLD_RESULTS_DIR / \"fold_metrics_by_model.csv\",\n    KFOLD_RESULTS_DIR / \"kfold_multimodel_summary.csv\",\n    KFOLD_RESULTS_DIR / \"oof_model_metrics.csv\",\n    KFOLD_RESULTS_DIR / \"oof_ensemble_metrics.csv\",\n    KFOLD_RESULTS_DIR / \"final_kfold_multimodel_selection_summary.csv\",\n    KFOLD_RESULTS_DIR / \"kfold_submission_candidates.csv\",\n    KFOLD_RESULTS_DIR / \"kfold_test_inference_models.csv\",\n    KFOLD_RESULTS_DIR / \"kfold_model_errors_and_oom_notes.csv\",\n    KFOLD_RESULTS_DIR / \"kfold_metadata.csv\",\n    RESULTS_DIR / \"labeled_split_metadata.csv\",\n    RESULTS_DIR / \"kaggle_test_metadata_unlabeled.csv\",\n]\nexported_metadata = []\nfor src in metadata_sources:\n    if src.exists():\n        dst = METADATA_EXPORT_DIR / src.name\n        shutil.copy2(src, dst)\n        exported_metadata.append(f\"metadata/{dst.name}\")\n\nfor src in [WORK_DIR / \"final_submission.csv\", WORK_DIR / \"submission.csv\"]:\n    if src.exists():\n        dst = SUBMISSION_EXPORT_DIR / src.name\n        shutil.copy2(src, dst)\n\nselected_candidate = \"\"\nselection_summary_path = KFOLD_RESULTS_DIR / \"final_kfold_multimodel_selection_summary.csv\"\nif selection_summary_path.exists():\n    try:\n        selection_df = pd.read_csv(selection_summary_path)\n        if len(selection_df) and \"candidate_name\" in selection_df.columns:\n            selected_candidate = str(selection_df.iloc[0][\"candidate_name\"])\n    except Exception:\n        selected_candidate = \"\"\n\nmanifest = {\n    \"created_at_utc\": datetime.utcnow().isoformat() + \"Z\",\n    \"purpose\": \"APTOS v8 offline inference weights for Kaggle Internet OFF submission\",\n    \"n_folds\": int(N_FOLDS),\n    \"selection_metric\": KFOLD_SELECTION_METRIC,\n    \"selected_candidate\": selected_candidate,\n    \"active_model_keys\": [spec[\"model_key\"] for spec in ACTIVE_KFOLD_SPECS],\n    \"active_specs\": json_sanitize(ACTIVE_KFOLD_SPECS),\n    \"checkpoints\": exported_checkpoints,\n    \"metadata_files\": exported_metadata,\n    \"submission_files\": [f\"submissions/{p.name}\" for p in SUBMISSION_EXPORT_DIR.glob(\"*.csv\")],\n    \"offline_inference_rule\": \"Create model with pretrained=False / weights=None, then load these local .pth files.\",\n}\n(EXPORT_DIR / \"offline_weight_manifest.json\").write_text(json.dumps(manifest, indent=2))\n\nzip_base = WORK_DIR / \"aptos_v8_offline_weights\"\nzip_path = Path(shutil.make_archive(str(zip_base), \"zip\", root_dir=EXPORT_DIR))\n\nprint(f\"Exported {len(exported_checkpoints)} checkpoint state_dict files to: {EXPORT_DIR}\")\nprint(f\"Exported metadata files: {len(exported_metadata)}\")\nprint(f\"Zip ready for Kaggle Dataset upload: {zip_path}\")\ndisplay(pd.DataFrame(exported_checkpoints))\n","metadata":{},"outputs":[],"execution_count":null}]}