{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":13836,"databundleVersionId":1718836,"sourceType":"competition"}],"dockerImageVersionId":31236,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"students : Ahmed Ahmed Ibrahim Abusetta, Mohmed Maher","metadata":{}},{"cell_type":"markdown","source":"### **Project Goal:**\nDeveloped a comparative study of deep learning approaches for cassava leaf disease classification using **5 disease types**.\n\n### **Objectives Achieved:**\n\n1. **Built and trained a custom CNN from scratch** \n - Designed a 4-layer CNN with batch normalization and dropout \n - Trained on 21,397 cassava leaf images \n - Achieved competitive accuracy demonstrating dataset is large enough for training from scratch\n\n2. **Implemented transfer learning with EfficientNet-B0** \n - Replaced MobileNetV2 with more efficient EfficientNet-B0 \n - Fine-tuned pretrained ImageNet weights \n - Compared parameter efficiency vs. custom CNN\n\n3. **Comprehensive comparison framework** \n - Accuracy metrics: Training, Validation, and Test accuracy \n - Efficiency metrics: Parameters, training time, model size \n - Performance analysis: Confusion matrices, precision/recall\n\n4. **Applied industry best practices** \n - Stratified train/val/test splitting (70/20/10) \n - Data augmentation (flips, rotation, color jitter) \n - Learning rate scheduling with ReduceLROnPlateau \n - Mixed precision training (FP16) \n - Model checkpointing and early stopping \n - Interactive progress bars with tqdm","metadata":{}},{"cell_type":"markdown","source":"# š Cassava Leaf Disease Classification - Final Results\n\n## š Dataset Overview\n- **Total Images**: 21,397\n- **Classes**: 5\n- **Test Set**: 2,140 images\n- **Environment**: Kaggle\n\n### š Class Distribution (Severe Imbalance)\n| Class | Disease | Samples | Percentage | Key Challenge |\n|-------|---------|---------|------------|---------------|\n| 0 | Cassava Bacterial Blight (CBB) | 1,087 | 5.1% | Rare disease |\n| 1 | Cassava Brown Streak Disease (CBSD) | 2,189 | 10.2% | Moderate |\n| 2 | Cassava Green Mottle (CGM) | 2,386 | 11.2% | Moderate |\n| 3 | Cassava Mosaic Disease (CMD) | 13,158 | 61.5% | **Dominant class** |\n| 4 | Healthy | 2,577 | 12.0% | Baseline |\n\n**ā ļø Imbalance Ratio**: 12.1Ć (CMD dominates dataset)\n\n## šļø Experimental Setup\n### Data Pipeline\n- **Split**: 70%/20%/10% (Train/Validation/Test) - Stratified\n- **Batches**: 469 train | 134 val | 67 test\n- **Augmentation**: Horizontal flip, rotation (Ā±15Ā°), color jitter\n- **Normalization**: ImageNet stats (mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225])\n\n### Model Architectures\n| Aspect | Simple CNN (From Scratch) | EfficientNet-B0 (Transfer) |\n|--------|---------------------------|----------------------------|\n| **Design** | Custom 4-layer CNN | Pretrained + Custom Head |\n| **Params** | 6.8M trainable | 6.4K trainable / 4.0M total |\n| **Pretrained** | ā No | ā
ImageNet |\n| **Training** | Random initialization | Fine-tuning frozen backbone |\n| **Efficiency** | 100% trainable | **0.16% trainable** |\n\n## š Performance Comparison\n### š Summary Metrics\n| Metric | Simple CNN | EfficientNet-B0 | Winner |\n|--------|------------|-----------------|--------|\n| **Test Accuracy** | **71.54%** | 69.53% | š„ **Simple CNN** |\n| **Val Accuracy** | ~72% | ~70% | š„ **Simple CNN** |\n| **Model Size** | 6.8M params | 4.0M params | š„ EfficientNet |\n| **Trainable Params** | 6,814,981 | **6,405** | š„ **EfficientNet** |\n| **Parameter Efficiency** | 100% trainable | **0.16% trainable** | š„ **EfficientNet** |\n\n\n## š Performance Analysis\n\n### š Class-by-Class Performance\n| Class | Disease | Simple CNN (F1) | EfficientNet (F1) | Best Model | Key Insight |\n|-------|---------|-----------------|-------------------|------------|-------------|\n| 0 | CBB | 0.33 | 0.32 | **Tie** | Both struggle (rare class) |\n| 1 | CBSD | **0.46** | 0.40 | š„ **Simple CNN** | Simple CNN +47% better |\n| 2 | CGM | 0.28 | 0.31 | š„ **EfficientNet** | EfficientNet +11% better |\n| 3 | CMD | **0.88** | 0.86 | š„ **Simple CNN** | Both good, CNN better |\n| 4 | Healthy | **0.40** | 0.45 | š„ **EfficientNet** | EfficientNet +13% better |\n\n### š” Model Strengths & Weaknesses\n| | Simple CNN | EfficientNet-B0 |\n|--|------------|-----------------|\n| **ā
Strengths** | Higher overall accuracy
Better for CBSD & CMD
No transfer bias | Parameter efficient
Better for CGM & Healthy
Faster convergence |\n| **ā Weaknesses** | Poor on minority classes
All params require training
Larger size | Lower overall accuracy
Transfer learning mismatch
Struggles with cassava features |\n\n## šÆ Critical Insights\n\n### 1. **Dataset Size Matters**\n- 21,397 images sufficient for training from scratch\n- Simple CNN learns cassava-specific features better than ImageNet transfer\n\n### 2. **Class Imbalance Drives Results**\n- CMD (61.5% of data) dominates predictions\n- Both models achieve 86-88% F1 on CMD but struggle with rare classes\n- **Macro F1**: Only 0.47 (vs weighted 0.67-0.68)\n\n### 3. **Transfer Learning Limitations**\n- EfficientNet trained on general ImageNet (1,000 classes)\n- Cassava disease features may not transfer well\n- Domain gap reduces transfer learning benefits\n\n### 4. **Parameter Efficiency Tradeoff**\n- EfficientNet: 6.4K trainable params (99.84% frozen)\n- Simple CNN: 6.8M trainable params\n- **Insight**: More parameters ā better accuracy for domain-specific tasks\n\n\n## š Conclusion\n\n**Simple CNN (71.54%) outperforms EfficientNet-B0 (69.53%)** for cassava leaf disease classification, demonstrating that:\n\n1. **Domain-specific training** from scratch can beat transfer learning\n2. **Dataset size** (21K images) enables effective custom model training \n3. **Class imbalance** is the primary challenge (12.1Ć ratio)\n4. **Parameter efficiency** doesn't always translate to better accuracy\n\n**Final Recommendation**: Deploy Simple CNN with class weighting for production, while continuing to collect more balanced data for minority disease classes.\n\n---\n\n*Results generated from 2,140 test samples | Training: 10 epochs | Batch size: 32 | Optimizer: Adam (1e-4)*","metadata":{}},{"cell_type":"code","source":"# =============================================================================\n# IMPORT LIBRARIES\n# =============================================================================\n\nimport os\nimport json\nimport time\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Scikit-learn utilities\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix, classification_report\n\n# PyTorch libraries\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import transforms, models\nfrom torch.cuda.amp import autocast, GradScaler\n\n# Set visualization style\nplt.style.use('seaborn-v0_8-whitegrid')\nsns.set_palette(\"husl\")\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2025-12-20T00:29:01.730043Z","iopub.execute_input":"2025-12-20T00:29:01.73063Z","iopub.status.idle":"2025-12-20T00:29:01.73691Z","shell.execute_reply.started":"2025-12-20T00:29:01.730599Z","shell.execute_reply":"2025-12-20T00:29:01.736178Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\n# =============================================================================\n# SETUP AND CONFIGURATION\n# =============================================================================\n\ndef set_seed(seed=42):\n \"\"\"Ensure reproducible results\"\"\"\n np.random.seed(seed)\n torch.manual_seed(seed)\n if torch.cuda.is_available():\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = False\n\nset_seed(42)\n\n# Check for GPU\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nprint(f\"Device: {device}\")\nif device.type == 'cuda':\n print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n\n# =============================================================================\n# PROJECT CONFIGURATION\n# =============================================================================\n\n# Model and training parameters\nCONFIG = {\n 'image_size': 224, # Standard size for pretrained models\n 'batch_size': 32, # Balance memory and training stability\n 'epochs': 10, # Number of training cycles\n 'learning_rate': 1e-4, # Learning rate for optimization\n 'num_classes': 5, # 5 disease types\n 'num_workers': 4, # Data loading workers\n 'seed': 42, # Random seed\n 'patience': 2, # Learning rate scheduler patience\n 'lr_factor': 0.5 # LR reduction factor\n}\n\n# Disease class names\nCLASS_NAMES = [\n 'Cassava Bacterial Blight (CBB)',\n 'Cassava Brown Streak Disease (CBSD)', \n 'Cassava Green Mottle (CGM)',\n 'Cassava Mosaic Disease (CMD)',\n 'Healthy'\n]\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:01.747078Z","iopub.execute_input":"2025-12-20T00:29:01.747666Z","iopub.status.idle":"2025-12-20T00:29:01.761705Z","shell.execute_reply.started":"2025-12-20T00:29:01.747638Z","shell.execute_reply":"2025-12-20T00:29:01.761049Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# DATA LOADING AND PREPROCESSING\n# =============================================================================\n\n# Load the dataset\nif os.path.exists('/kaggle/input/cassava-leaf-disease-classification'):\n DATA_DIR = '/kaggle/input/cassava-leaf-disease-classification'\n print(\"Running on Kaggle\")\nelse:\n DATA_DIR = './cassava-leaf-disease-classification'\n print(\"Running locally\")\n\n# File paths\nTRAIN_DIR = os.path.join(DATA_DIR, 'train_images')\nLABEL_FILE = os.path.join(DATA_DIR, 'train.csv')\n\n# Load labels\nlabels_df = pd.read_csv(LABEL_FILE)\nlabels_df['image_path'] = labels_df['image_id'].apply(\n lambda x: os.path.join(TRAIN_DIR, x)\n)\n\nprint(f\"\\nDataset Statistics:\")\nprint(f\"Total images: {len(labels_df):,}\")\nprint(f\"Classes: {len(CLASS_NAMES)}\")\n\n# Show class distribution\nprint(\"\\nClass Distribution:\")\nclass_counts = labels_df['label'].value_counts().sort_index()\nfor label, count in class_counts.items():\n percentage = (count / len(labels_df)) * 100\n print(f\"Class {label}: {count:>5,} images ({percentage:5.1f}%)\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:01.762925Z","iopub.execute_input":"2025-12-20T00:29:01.763148Z","iopub.status.idle":"2025-12-20T00:29:01.826853Z","shell.execute_reply.started":"2025-12-20T00:29:01.763126Z","shell.execute_reply":"2025-12-20T00:29:01.826009Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# DATA SPLITTING\n# =============================================================================\n\n# Split data: 70% train, 20% validation, 10% test\ntrain_df, temp_df = train_test_split(\n labels_df,\n test_size=0.3,\n stratify=labels_df['label'],\n random_state=CONFIG['seed']\n)\n\nval_df, test_df = train_test_split(\n temp_df,\n test_size=1/3, # 10% of total data\n stratify=temp_df['label'],\n random_state=CONFIG['seed']\n)\n\nprint(f\"\\nData Split:\")\nprint(f\"Training set: {len(train_df):>6,} images (70%)\")\nprint(f\"Validation set: {len(val_df):>6,} images (20%)\")\nprint(f\"Test set: {len(test_df):>6,} images (10%)\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:01.828333Z","iopub.execute_input":"2025-12-20T00:29:01.828594Z","iopub.status.idle":"2025-12-20T00:29:01.854218Z","shell.execute_reply.started":"2025-12-20T00:29:01.828571Z","shell.execute_reply":"2025-12-20T00:29:01.853545Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# DATA VISUALIZATION\n# =============================================================================\n\ndef plot_class_distribution(df, class_names):\n \"\"\"Visualize the distribution of classes\"\"\"\n fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n \n # Count classes\n class_counts = df['label'].value_counts().sort_index()\n colors = plt.cm.Set2(np.linspace(0, 1, len(class_names)))\n \n # Bar plot\n axes[0].bar(range(len(class_names)), class_counts.values, \n color=colors, edgecolor='black')\n axes[0].set_xlabel('Class Label')\n axes[0].set_ylabel('Number of Samples')\n axes[0].set_title('Class Distribution')\n axes[0].set_xticks(range(len(class_names)))\n axes[0].set_xticklabels([f'Class {i}' for i in range(len(class_names))])\n \n # Add count labels\n for i, count in enumerate(class_counts.values):\n axes[0].text(i, count + max(class_counts.values)*0.02, \n f'{count:,}', ha='center', va='bottom')\n \n # Pie chart\n axes[1].pie(class_counts.values, labels=[f'Class {i}' for i in range(len(class_names))],\n autopct='%1.1f%%', colors=colors, startangle=90)\n axes[1].set_title('Percentage Distribution')\n \n plt.tight_layout()\n plt.show()\n \n # Check for imbalance\n imbalance = class_counts.max() / class_counts.min()\n print(f\"\\nClass imbalance ratio: {imbalance:.1f}x\")\n if imbalance > 3:\n print(\"Warning: Significant class imbalance detected\")\n\nplot_class_distribution(labels_df, CLASS_NAMES)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:01.855155Z","iopub.execute_input":"2025-12-20T00:29:01.855904Z","iopub.status.idle":"2025-12-20T00:29:02.116689Z","shell.execute_reply.started":"2025-12-20T00:29:01.855862Z","shell.execute_reply":"2025-12-20T00:29:02.115817Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# DATASET CLASS\n# =============================================================================\n\nclass CassavaDataset(Dataset):\n \"\"\"Custom Dataset for loading cassava leaf images\"\"\"\n \n def __init__(self, dataframe, transform=None):\n self.dataframe = dataframe\n self.transform = transform\n \n def __len__(self):\n return len(self.dataframe)\n \n def __getitem__(self, idx):\n # Get image path and label\n img_path = self.dataframe.iloc[idx]['image_path']\n label = self.dataframe.iloc[idx]['label']\n \n # Load image\n image = plt.imread(img_path)\n \n # Apply transformations \n if self.transform:\n image = self.transform(image)\n \n return image, label\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.118579Z","iopub.execute_input":"2025-12-20T00:29:02.118918Z","iopub.status.idle":"2025-12-20T00:29:02.12509Z","shell.execute_reply.started":"2025-12-20T00:29:02.118892Z","shell.execute_reply":"2025-12-20T00:29:02.124401Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# DATA TRANSFORMATIONS\n# =============================================================================\n\n# ImageNet normalization values\nIMAGENET_MEAN = [0.485, 0.456, 0.406]\nIMAGENET_STD = [0.229, 0.224, 0.225]\n\n# Training transformations (with augmentation)\ntrain_transform = transforms.Compose([\n transforms.ToPILImage(),\n transforms.Resize((CONFIG['image_size'], CONFIG['image_size'])),\n transforms.RandomHorizontalFlip(p=0.5),\n transforms.RandomRotation(15),\n transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2),\n transforms.ToTensor(),\n transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD)\n])\n\n# Validation/Test transformations (no augmentation)\nval_transform = transforms.Compose([\n transforms.ToPILImage(),\n transforms.Resize((CONFIG['image_size'], CONFIG['image_size'])),\n transforms.ToTensor(),\n transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD)\n])\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.126098Z","iopub.execute_input":"2025-12-20T00:29:02.126557Z","iopub.status.idle":"2025-12-20T00:29:02.147179Z","shell.execute_reply.started":"2025-12-20T00:29:02.126523Z","shell.execute_reply":"2025-12-20T00:29:02.146526Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# CREATE DATALOADERS\n# =============================================================================\n\n# Create datasets\ntrain_dataset = CassavaDataset(train_df, transform=train_transform)\nval_dataset = CassavaDataset(val_df, transform=val_transform)\ntest_dataset = CassavaDataset(test_df, transform=val_transform)\n\n# Create dataloaders\ntrain_loader = DataLoader(\n train_dataset,\n batch_size=CONFIG['batch_size'],\n shuffle=True,\n num_workers=CONFIG['num_workers'],\n pin_memory=True\n)\n\nval_loader = DataLoader(\n val_dataset,\n batch_size=CONFIG['batch_size'],\n shuffle=False,\n num_workers=CONFIG['num_workers'],\n pin_memory=True\n)\n\ntest_loader = DataLoader(\n test_dataset,\n batch_size=CONFIG['batch_size'],\n shuffle=False,\n num_workers=CONFIG['num_workers'],\n pin_memory=True\n)\n\nprint(f\"\\nDataLoader Statistics:\")\nprint(f\"Training batches: {len(train_loader)}\")\nprint(f\"Validation batches: {len(val_loader)}\")\nprint(f\"Test batches: {len(test_loader)}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.148107Z","iopub.execute_input":"2025-12-20T00:29:02.148408Z","iopub.status.idle":"2025-12-20T00:29:02.164214Z","shell.execute_reply.started":"2025-12-20T00:29:02.148378Z","shell.execute_reply":"2025-12-20T00:29:02.163667Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# MODEL ARCHITECTURES\n# =============================================================================\n\nclass SimpleCNN(nn.Module):\n \"\"\"Simple CNN model trained from scratch\"\"\"\n \n def __init__(self, num_classes=5):\n super(SimpleCNN, self).__init__()\n \n # Feature extractor\n self.features = nn.Sequential(\n # Block 1\n nn.Conv2d(3, 32, kernel_size=3, padding=1),\n nn.BatchNorm2d(32),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(2),\n \n # Block 2\n nn.Conv2d(32, 64, kernel_size=3, padding=1),\n nn.BatchNorm2d(64),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(2),\n \n # Block 3\n nn.Conv2d(64, 128, kernel_size=3, padding=1),\n nn.BatchNorm2d(128),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(2),\n \n # Block 4\n nn.Conv2d(128, 256, kernel_size=3, padding=1),\n nn.BatchNorm2d(256),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(2)\n )\n \n # Classifier\n self.classifier = nn.Sequential(\n nn.AdaptiveAvgPool2d((7, 7)),\n nn.Flatten(),\n nn.Linear(256 * 7 * 7, 512),\n nn.ReLU(inplace=True),\n nn.Dropout(0.5),\n nn.Linear(512, num_classes)\n )\n \n def forward(self, x):\n x = self.features(x)\n x = self.classifier(x)\n return x\n \n def count_parameters(self):\n return sum(p.numel() for p in self.parameters() if p.requires_grad)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.165083Z","iopub.execute_input":"2025-12-20T00:29:02.165331Z","iopub.status.idle":"2025-12-20T00:29:02.186597Z","shell.execute_reply.started":"2025-12-20T00:29:02.165311Z","shell.execute_reply":"2025-12-20T00:29:02.185846Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# EfficientNet-B0 MODEL \n# =============================================================================\n\nclass EfficientNetTransfer(nn.Module):\n \"\"\"\n EfficientNet-B0 model with transfer learning.\n \n Efficiency Advantages:\n - 5.3M parameters \n - Compound scaling method optimizes depth, width, resolution\n \"\"\"\n \n def __init__(self, num_classes=5, freeze_backbone=True):\n super(EfficientNetTransfer, self).__init__()\n \n # Load pretrained EfficientNet-B0\n self.base_model = models.efficientnet_b0(pretrained=True)\n \n # Freeze backbone layers for transfer learning\n if freeze_backbone:\n for param in self.base_model.parameters():\n param.requires_grad = False\n \n # Get number of features in the classifier\n num_features = self.base_model.classifier[1].in_features\n \n # Replace classifier head\n self.base_model.classifier = nn.Sequential(\n nn.Dropout(p=0.2, inplace=True),\n nn.Linear(num_features, 256),\n nn.ReLU(inplace=True),\n nn.BatchNorm1d(512),\n nn.Dropout(p=0.3),\n nn.Linear(256, num_classes)\n )\n \n def forward(self, x):\n return self.base_model(x)\n \n def count_parameters(self):\n total = sum(p.numel() for p in self.parameters())\n trainable = sum(p.numel() for p in self.parameters() if p.requires_grad)\n return total, trainable\n \n def unfreeze_layers(self, num_layers=20):\n \"\"\"Unfreeze last few layers for fine-tuning\"\"\"\n # Count total layers\n all_params = list(self.base_model.parameters())\n \n # Unfreeze last 'num_layers' parameters\n for i, param in enumerate(all_params[-num_layers:]):\n param.requires_grad = True\n \n print(f\"Unfroze last {num_layers} layers for fine-tuning\")\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.187673Z","iopub.execute_input":"2025-12-20T00:29:02.187953Z","iopub.status.idle":"2025-12-20T00:29:02.204006Z","shell.execute_reply.started":"2025-12-20T00:29:02.187932Z","shell.execute_reply":"2025-12-20T00:29:02.203209Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# TRAINING UTILITIES\n# =============================================================================\n\nfrom tqdm import tqdm\n\ndef train_epoch(model, dataloader, criterion, optimizer, device, scaler=None):\n \"\"\"Train for one epoch with progress bar\"\"\"\n model.train()\n running_loss = 0.0\n correct = 0\n total = 0\n \n # Create progress bar for batches\n pbar = tqdm(enumerate(dataloader), total=len(dataloader), \n desc=\"Training\", leave=False)\n \n for batch_idx, (images, labels) in pbar:\n images, labels = images.to(device), labels.to(device)\n \n # Zero gradients\n optimizer.zero_grad()\n \n # Forward pass (with mixed precision if using GPU)\n if scaler and device.type == 'cuda':\n with torch.amp.autocast(device_type='cuda'):\n outputs = model(images)\n loss = criterion(outputs, labels)\n \n # Backward pass with scaling\n scaler.scale(loss).backward()\n scaler.step(optimizer)\n scaler.update()\n else:\n outputs = model(images)\n loss = criterion(outputs, labels)\n loss.backward()\n optimizer.step()\n \n # Statistics\n running_loss += loss.item()\n _, predicted = outputs.max(1)\n total += labels.size(0)\n correct += predicted.eq(labels).sum().item()\n \n # Update progress bar description\n current_loss = running_loss / (batch_idx + 1)\n current_acc = 100.0 * correct / total\n pbar.set_postfix({\n 'Loss': f'{current_loss:.4f}',\n 'Acc': f'{current_acc:.2f}%',\n 'LR': f'{optimizer.param_groups[0][\"lr\"]:.6f}' # FIXED: Added f-string\n })\n \n epoch_loss = running_loss / len(dataloader)\n epoch_acc = 100.0 * correct / total\n \n return epoch_loss, epoch_acc\ndef validate(model, dataloader, criterion, device):\n \"\"\"Validate the model with progress bar\"\"\"\n model.eval()\n running_loss = 0.0\n correct = 0\n total = 0\n \n # Create progress bar for validation\n val_pbar = tqdm(enumerate(dataloader), total=len(dataloader), \n desc=\"Validating\", leave=False)\n \n with torch.no_grad():\n for batch_idx, (images, labels) in val_pbar:\n images, labels = images.to(device), labels.to(device)\n \n outputs = model(images)\n loss = criterion(outputs, labels)\n \n running_loss += loss.item()\n _, predicted = outputs.max(1)\n total += labels.size(0)\n correct += predicted.eq(labels).sum().item()\n \n # Update validation progress bar\n current_loss = running_loss / (batch_idx + 1)\n current_acc = 100.0 * correct / total\n val_pbar.set_postfix({\n 'Loss': f'{current_loss:.4f}',\n 'Acc': f'{current_acc:.2f}%'\n })\n \n val_loss = running_loss / len(dataloader)\n val_acc = 100.0 * correct / total\n \n return val_loss, val_acc\ndef test_model(model, test_loader, device):\n \"\"\"Test model and generate predictions\"\"\"\n model.eval()\n all_predictions = []\n all_labels = []\n \n with torch.no_grad():\n for images, labels in test_loader:\n images = images.to(device)\n outputs = model(images)\n _, predicted = outputs.max(1)\n \n all_predictions.extend(predicted.cpu().numpy())\n all_labels.extend(labels.numpy())\n \n return np.array(all_predictions), np.array(all_labels)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.206368Z","iopub.execute_input":"2025-12-20T00:29:02.206986Z","iopub.status.idle":"2025-12-20T00:29:02.226869Z","shell.execute_reply.started":"2025-12-20T00:29:02.206962Z","shell.execute_reply":"2025-12-20T00:29:02.226028Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def train_model(model_type='simple_cnn', num_epochs=10):\n \"\"\"Train either SimpleCNN or EfficientNet with progress bars\"\"\"\n \n print(f\"\\n{'='*60}\")\n print(f\"Training {model_type.upper()}\")\n print('='*60)\n \n # Initialize model\n if model_type == 'simple_cnn':\n model = SimpleCNN(num_classes=CONFIG['num_classes']).to(device)\n model_name = \"SimpleCNN (From Scratch)\"\n else: # efficientnet\n model = EfficientNetTransfer(\n num_classes=CONFIG['num_classes'],\n freeze_backbone=True\n ).to(device)\n model_name = \"EfficientNet-B0 (Transfer Learning)\"\n \n # Count parameters\n if model_type == 'simple_cnn':\n total_params = model.count_parameters()\n trainable_params = total_params\n else:\n total_params, trainable_params = model.count_parameters()\n \n print(f\"Model: {model_name}\")\n print(f\"Parameters: {trainable_params:,} trainable / {total_params:,} total\")\n \n # Loss function and optimizer\n criterion = nn.CrossEntropyLoss()\n optimizer = optim.Adam(model.parameters(), lr=CONFIG['learning_rate'])\n \n # Learning rate scheduler\n scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n optimizer, mode='min', factor=CONFIG['lr_factor'],\n patience=CONFIG['patience']\n )\n \n # Mixed precision training\n scaler = torch.amp.GradScaler('cuda') if device.type == 'cuda' else None\n \n # Training history\n history = {\n 'train_loss': [], 'train_acc': [],\n 'val_loss': [], 'val_acc': [],\n 'learning_rate': []\n }\n \n # Create progress bar for epochs\n epoch_pbar = tqdm(range(num_epochs), desc=\"Epochs\", position=0, leave=True)\n \n start_time = time.time()\n best_val_acc = 0.0\n \n for epoch in epoch_pbar:\n # Update epoch progress bar description\n epoch_pbar.set_description(f\"Epoch {epoch+1}/{num_epochs}\")\n \n # Train\n train_loss, train_acc = train_epoch(\n model, train_loader, criterion, optimizer, device, scaler\n )\n \n # Validate\n val_loss, val_acc = validate(model, val_loader, criterion, device)\n \n # Update learning rate\n scheduler.step(val_loss)\n \n # Record history\n history['train_loss'].append(train_loss)\n history['train_acc'].append(train_acc)\n history['val_loss'].append(val_loss)\n history['val_acc'].append(val_acc)\n history['learning_rate'].append(optimizer.param_groups[0]['lr'])\n \n # Update epoch progress bar postfix\n epoch_pbar.set_postfix({\n 'Train Acc': f'{train_acc:.2f}%',\n 'Val Acc': f'{val_acc:.2f}%',\n 'Best Val': f'{best_val_acc:.2f}%',\n 'LR': f'{optimizer.param_groups[0][\"lr\"]:.2e}'\n })\n \n # Save best model\n if val_acc > best_val_acc:\n best_val_acc = val_acc\n torch.save({\n 'epoch': epoch,\n 'model_state_dict': model.state_dict(),\n 'optimizer_state_dict': optimizer.state_dict(),\n 'val_acc': val_acc,\n 'val_loss': val_loss,\n }, f'best_{model_type}.pth')\n epoch_pbar.write(f\" ā Saved best model (Val Acc: {val_acc:.2f}%)\")\n \n epoch_pbar.close()\n training_time = time.time() - start_time\n \n print(f\"\\n{'='*40}\")\n print(f\"Training completed in {training_time/60:.1f} minutes\")\n print(f\"Best validation accuracy: {best_val_acc:.2f}%\")\n print(f\"{'='*40}\")\n \n return model, history","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.22773Z","iopub.execute_input":"2025-12-20T00:29:02.22795Z","iopub.status.idle":"2025-12-20T00:29:02.248681Z","shell.execute_reply.started":"2025-12-20T00:29:02.22793Z","shell.execute_reply":"2025-12-20T00:29:02.248056Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# VISUALIZATION FUNCTIONS\n# =============================================================================\n\ndef plot_training_history(history, model_name):\n \"\"\"Plot training and validation metrics\"\"\"\n fig, axes = plt.subplots(1, 3, figsize=(15, 4))\n \n epochs = range(1, len(history['train_loss']) + 1)\n \n # Loss plot\n axes[0].plot(epochs, history['train_loss'], 'b-', label='Training Loss', linewidth=2)\n axes[0].plot(epochs, history['val_loss'], 'r-', label='Validation Loss', linewidth=2)\n axes[0].set_xlabel('Epochs')\n axes[0].set_ylabel('Loss')\n axes[0].set_title(f'{model_name} - Loss')\n axes[0].legend()\n axes[0].grid(True, alpha=0.3)\n \n # Accuracy plot\n axes[1].plot(epochs, history['train_acc'], 'b-', label='Training Accuracy', linewidth=2)\n axes[1].plot(epochs, history['val_acc'], 'r-', label='Validation Accuracy', linewidth=2)\n axes[1].set_xlabel('Epochs')\n axes[1].set_ylabel('Accuracy (%)')\n axes[1].set_title(f'{model_name} - Accuracy')\n axes[1].legend()\n axes[1].grid(True, alpha=0.3)\n \n # Learning rate plot\n axes[2].plot(epochs, history['learning_rate'], 'g-', marker='o', linewidth=2)\n axes[2].set_xlabel('Epochs')\n axes[2].set_ylabel('Learning Rate')\n axes[2].set_title(f'{model_name} - Learning Rate')\n axes[2].set_yscale('log')\n axes[2].grid(True, alpha=0.3)\n \n plt.tight_layout()\n plt.show()\n\ndef plot_confusion_matrix(y_true, y_pred, class_names, model_name):\n \"\"\"Plot confusion matrix for model evaluation\"\"\"\n cm = confusion_matrix(y_true, y_pred)\n \n plt.figure(figsize=(10, 8))\n sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',\n xticklabels=[c[:15] for c in class_names],\n yticklabels=[c[:15] for c in class_names])\n plt.title(f'Confusion Matrix - {model_name}')\n plt.ylabel('True Label')\n plt.xlabel('Predicted Label')\n plt.tight_layout()\n plt.show()\n \n # Print classification report\n print(f\"\\nClassification Report - {model_name}:\")\n print(\"-\" * 60)\n print(classification_report(y_true, y_pred, target_names=[c[:15] for c in class_names]))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.24945Z","iopub.execute_input":"2025-12-20T00:29:02.249929Z","iopub.status.idle":"2025-12-20T00:29:02.270663Z","shell.execute_reply.started":"2025-12-20T00:29:02.249897Z","shell.execute_reply":"2025-12-20T00:29:02.269918Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# COMPARISON AND RESULTS\n# =============================================================================\n\ndef compare_models(results):\n \"\"\"Compare performance of different models\"\"\"\n print(\"\\n\" + \"=\"*70)\n print(\"MODEL COMPARISON SUMMARY\")\n print(\"=\"*70)\n \n print(f\"{'Model':<30} {'Test Acc (%)':<12} {'Params':<12} {'Trainable Params':<15}\")\n print(\"-\"*70)\n \n for model_name, metrics in results.items():\n print(f\"{model_name:<30} {metrics['test_acc']:<12.2f} \"\n f\"{metrics['total_params']:<12,} {metrics['trainable_params']:<15,}\")\n \n print(\"=\"*70)\n \n # Plot comparison\n fig, axes = plt.subplots(1, 2, figsize=(14, 6))\n \n # Accuracy comparison\n model_names = list(results.keys())\n test_accs = [results[m]['test_acc'] for m in model_names]\n \n bars = axes[0].bar(model_names, test_accs, color=['skyblue', 'lightgreen'])\n axes[0].set_ylabel('Test Accuracy (%)')\n axes[0].set_title('Model Accuracy Comparison')\n axes[0].set_ylim([0, 100])\n \n # Add value labels on bars\n for bar, acc in zip(bars, test_accs):\n axes[0].text(bar.get_x() + bar.get_width()/2, bar.get_height() + 1,\n f'{acc:.1f}%', ha='center', va='bottom')\n \n # Parameter comparison\n total_params = [results[m]['total_params'] for m in model_names]\n trainable_params = [results[m]['trainable_params'] for m in model_names]\n \n x = np.arange(len(model_names))\n width = 0.35\n \n axes[1].bar(x - width/2, total_params, width, label='Total Params', color='skyblue')\n axes[1].bar(x + width/2, trainable_params, width, label='Trainable Params', color='lightcoral')\n axes[1].set_ylabel('Number of Parameters')\n axes[1].set_title('Model Size Comparison')\n axes[1].set_xticks(x)\n axes[1].set_xticklabels(model_names, rotation=15)\n axes[1].legend()\n axes[1].set_yscale('log')\n \n plt.tight_layout()\n plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.271533Z","iopub.execute_input":"2025-12-20T00:29:02.271765Z","iopub.status.idle":"2025-12-20T00:29:02.294014Z","shell.execute_reply.started":"2025-12-20T00:29:02.271744Z","shell.execute_reply":"2025-12-20T00:29:02.2933Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# CELL 1: TRAIN SIMPLE CNN\n# =============================================================================\n\nprint(\"\\n\" + \"=\"*60)\nprint(\"TRAINING SIMPLE CNN (FROM SCRATCH)\")\nprint(\"=\"*60)\n\n# Check if model already exists\nimport os\nif os.path.exists('best_simple_cnn.pth'):\n print(\"Loading existing Simple CNN model...\")\n checkpoint = torch.load('best_simple_cnn.pth')\n simple_cnn_model = SimpleCNN(num_classes=CONFIG['num_classes']).to(device)\n simple_cnn_model.load_state_dict(checkpoint['model_state_dict'])\n # Load history if saved separately\n if os.path.exists('cnn_history.json'):\n with open('cnn_history.json', 'r') as f:\n cnn_history = json.load(f)\n else:\n print(\"No history found, plotting may not work correctly\")\n cnn_history = None\nelse:\n# # Train from scratch\n simple_cnn_model, cnn_history = train_model('simple_cnn', CONFIG['epochs'])\n # Save history\n with open('cnn_history.json', 'w') as f:\n json.dump(cnn_history, f)\n\n# Plot training history\nif cnn_history:\n plot_training_history(cnn_history, \"Simple CNN\")\n\n# Test Simple CNN\ntest_loss, test_acc = validate(simple_cnn_model, test_loader, nn.CrossEntropyLoss(), device)\npredictions, true_labels = test_model(simple_cnn_model, test_loader, device)\n\n# Store results\nresults = {}\nresults['Simple CNN'] = {\n 'test_acc': test_acc,\n 'test_loss': test_loss,\n 'total_params': simple_cnn_model.count_parameters(),\n 'trainable_params': simple_cnn_model.count_parameters()\n}\n\nprint(f\"\\nSimple CNN Test Accuracy: {test_acc:.2f}%\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:29:02.294896Z","iopub.execute_input":"2025-12-20T00:29:02.295142Z","iopub.status.idle":"2025-12-20T00:29:20.546933Z","shell.execute_reply.started":"2025-12-20T00:29:02.295114Z","shell.execute_reply":"2025-12-20T00:29:20.545901Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# CELL 2: TRAIN EFFICIENTNET (FIXED VERSION)\n# =============================================================================\n\nprint(\"\\n\" + \"=\"*60)\nprint(\"TRAINING EFFICIENTNET-B0 (TRANSFER LEARNING)\")\nprint(\"=\"*60)\n\n# IMPORTANT: Define the SIMPLE architecture that matches your checkpoint\nclass EfficientNetTransferSimple(nn.Module):\n \"\"\"\n SIMPLE version that matches your saved checkpoint\n (1280 features ā 5 classes directly)\n \"\"\"\n def __init__(self, num_classes=5, freeze_backbone=True):\n super().__init__()\n # Use updated weights syntax to avoid warnings\n self.base_model = models.efficientnet_b0(weights=models.EfficientNet_B0_Weights.DEFAULT)\n \n if freeze_backbone:\n for param in self.base_model.parameters():\n param.requires_grad = False\n \n # SIMPLE classifier (matches checkpoint)\n num_features = self.base_model.classifier[1].in_features\n self.base_model.classifier[1] = nn.Linear(num_features, num_classes)\n \n def forward(self, x):\n return self.base_model(x)\n \n def count_parameters(self):\n total = sum(p.numel() for p in self.parameters())\n trainable = sum(p.numel() for p in self.parameters() if p.requires_grad)\n return total, trainable\n\n# Check if model already exists\nif os.path.exists('best_efficientnet.pth'):\n print(\"Loading existing EfficientNet model...\")\n \n # Load checkpoint\n checkpoint = torch.load('best_efficientnet.pth')\n \n # Create model with SIMPLE architecture\n efficientnet_model = EfficientNetTransferSimple(\n num_classes=CONFIG['num_classes'],\n freeze_backbone=True\n ).to(device)\n \n # Load state dict\n efficientnet_model.load_state_dict(checkpoint['model_state_dict'])\n print(f\"ā Model loaded successfully!\")\n print(f\" Previous validation accuracy: {checkpoint.get('val_acc', 'N/A'):.2f}%\")\n \n # Load history if saved separately\n if os.path.exists('efficientnet_history.json'):\n with open('efficientnet_history.json', 'r') as f:\n efficientnet_history = json.load(f)\n print(\"ā Training history loaded\")\n else:\n print(\"ā ļø No history found, plotting may not work correctly\")\n efficientnet_history = None\nelse:\n print(\"No existing model found. Training from scratch...\")\n \n # Train from scratch (using simple architecture)\n efficientnet_model, efficientnet_history = train_model('efficientnet', CONFIG['epochs'])\n \n # Save history\n with open('efficientnet_history.json', 'w') as f:\n json.dump(efficientnet_history, f, indent=2)\n print(\"ā Model and history saved\")\n\n# Plot training history\nif efficientnet_history:\n plot_training_history(efficientnet_history, \"EfficientNet-B0\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:30:28.265163Z","iopub.execute_input":"2025-12-20T00:30:28.26588Z","iopub.status.idle":"2025-12-20T00:30:29.066105Z","shell.execute_reply.started":"2025-12-20T00:30:28.26584Z","shell.execute_reply":"2025-12-20T00:30:29.065428Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# CELL 3: TEST EFFICIENTNET\n# =============================================================================\n\nprint(\"\\n\" + \"=\"*60)\nprint(\"TESTING EFFICIENTNET-B0\")\nprint(\"=\"*60)\n\n# Load best model if not already loaded\nif 'efficientnet_model' not in locals():\n checkpoint = torch.load('best_efficientnet.pth')\n efficientnet_model = EfficientNetTransfer(\n num_classes=CONFIG['num_classes'],\n freeze_backbone=True\n ).to(device)\n efficientnet_model.load_state_dict(checkpoint['model_state_dict'])\n\n# Test EfficientNet\ntest_loss_eff, test_acc_eff = validate(efficientnet_model, test_loader, nn.CrossEntropyLoss(), device)\npredictions_eff, true_labels_eff = test_model(efficientnet_model, test_loader, device)\n\n# Get parameters\ntotal_params, trainable_params = efficientnet_model.count_parameters()\n\n# Store results\nresults['EfficientNet-B0'] = {\n 'test_acc': test_acc_eff,\n 'test_loss': test_loss_eff,\n 'total_params': total_params,\n 'trainable_params': trainable_params\n}\n\nprint(f\"EfficientNet-B0 Test Accuracy: {test_acc_eff:.2f}%\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:30:48.826259Z","iopub.execute_input":"2025-12-20T00:30:48.826597Z","iopub.status.idle":"2025-12-20T00:31:06.604595Z","shell.execute_reply.started":"2025-12-20T00:30:48.82657Z","shell.execute_reply":"2025-12-20T00:31:06.603792Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# CELL 4: COMPARE MODELS\n# =============================================================================\n\nprint(\"\\n\" + \"=\"*60)\nprint(\"MODEL COMPARISON\")\nprint(\"=\"*60)\n\n# Compare models\ncompare_models(results)\n\n# Generate confusion matrices\nplot_confusion_matrix(true_labels, predictions, CLASS_NAMES, \"Simple CNN\")\nplot_confusion_matrix(true_labels_eff, predictions_eff, CLASS_NAMES, \"EfficientNet-B0\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:31:10.900929Z","iopub.execute_input":"2025-12-20T00:31:10.901267Z","iopub.status.idle":"2025-12-20T00:31:11.897326Z","shell.execute_reply.started":"2025-12-20T00:31:10.901241Z","shell.execute_reply":"2025-12-20T00:31:11.896562Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =============================================================================\n# CELL 5: SAVE RESULTS AND CONCLUSION\n# =============================================================================\n\nprint(\"\\n\" + \"=\"*70)\nprint(\"PROJECT CONCLUSION\")\nprint(\"=\"*70)\n\nprint(\"\\nKey Findings:\")\nprint(\"1. Transfer learning (EfficientNet) significantly outperforms training from scratch\")\nprint(\"2. EfficientNet achieves higher accuracy with fewer trainable parameters\")\nprint(\"3. Compound scaling in EfficientNet provides better efficiency-accuracy tradeoff\")\nprint(f\"4. Best model: EfficientNet-B0 with {test_acc_eff:.2f}% test accuracy\")\n\nprint(\"\\nWhy EfficientNet is more efficient than MobileNet:\")\nprint(\"ā Uses compound scaling (balances depth, width, resolution)\")\nprint(\"ā Better FLOPs-to-accuracy ratio\")\nprint(\"ā More parameter efficient\")\nprint(\"ā State-of-the-art on ImageNet with fewer parameters\")\n\n# Save comprehensive results\nresults_summary = {\n 'config': CONFIG,\n 'results': results,\n 'class_names': CLASS_NAMES,\n 'dataset_stats': {\n 'total_samples': len(labels_df),\n 'train_samples': len(train_df),\n 'val_samples': len(val_df),\n 'test_samples': len(test_df),\n 'class_distribution': class_counts.to_dict()\n },\n 'training_info': {\n 'simple_cnn_epochs': CONFIG['epochs'],\n 'efficientnet_epochs': CONFIG['epochs'],\n 'device': str(device),\n 'timestamp': time.strftime(\"%Y-%m-%d %H:%M:%S\")\n }\n}\n\nwith open('training_results.json', 'w') as f:\n json.dump(results_summary, f, indent=2)\n\nprint(f\"\\nā
Results saved to 'training_results.json'\")\nprint(\"ā
Models saved as 'best_simple_cnn.pth' and 'best_efficientnet.pth'\")\nprint(\"ā
Training histories saved as JSON files\")\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-20T00:32:50.228154Z","iopub.execute_input":"2025-12-20T00:32:50.22887Z","iopub.status.idle":"2025-12-20T00:32:50.239249Z","shell.execute_reply.started":"2025-12-20T00:32:50.228838Z","shell.execute_reply":"2025-12-20T00:32:50.238363Z"}},"outputs":[],"execution_count":null}]}