{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.13"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceType":"competition","sourceId":13836,"databundleVersionId":1718836,"isSourceIdPinned":false}],"dockerImageVersionId":31193,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"a005860b","cell_type":"markdown","source":"# Cassava Leaf Disease Classification\n\n# Executive Summary\n\n### Key Steps:\n1. **Data Loading & Preparation**: Custom dataset loader built for non-standard folder structure, splitting data into 70% training, 20% validation, and 10% test sets.\n2. **Preprocessing**: Images resized to 298x224 with augmentation (horizontal flip, rotation, color jitter) and normalization.\n3. **Model Development**: Started with custom CNN, then progressed to pre-trained models (ResNet50, DenseNet121, EfficientNet, MobileNet).\n4. **Training**: Used CrossEntropyLoss, Adam optimizer (lr=1e-4), batch size 64, with validation monitoring and model checkpointing.\n\n### Model Performance:\n\n| Model | Validation Accuracy | Test Accuracy |\n|-------|-------------------|---------------|\n| Custom CNN | 68.54% | 68.80% |\n| EfficientNet_B3 (random weights) | 63.78% | 64.55% |\n| EfficientNet_B3 (torchvision) | 84.09% | 82.91% |\n| EfficientNet_B3 (timm, JFT-300M) | 83.45%| 83.33% |\n| **MobileNet V2** | **83.06%** | **84.40%** |\n\n**Best Model**: MobileNet V2 achieved the highest test accuracy of 84.40%.\n---\n\n## Pipeline (In detail)\n\n### Loading the Dataset\n\n#### Exploration\n\n- Explored the data directory structure: All training images are placed in a single folder.\n- `train.csv` had the `image_id` *(Image file name)* : `label` mapping.\n- `label_num_to_disease_map.json` had `label_id`:`label_text` mapping.  \n  Which helped us map each id to the actual text that represents it *(Used in Visualization)*\n\n#### Loading Data\n\n- As the folder structure isn't in the standard format, we needed to use the `Dataset` class from `torch.utils.data` and build the batch loading pipeline manually.\n  - `__init__` *(constructor)*:\n    - Used `os.walk` to get all the directories, and files *(Actually only image files existed)*.\n    - Stored image file names in `image_files` list.\n    - Stored the label corresponding to each image in `labels` list.\n    - Added `transform` property for later usage.\n  - `__len__` returns the length of `image_files` list as the length of the dataset.\n  - `__getitem__` returns single sample and its label *(Loading the batch)*\n    - Built image loading pipeline\n    - Transforming the image *(Resizing and Augmentation)* using transform.\n    - Returning the image object with its label in a one-hot encoded format.\n\n- Built the `transforms` object for each set:\n  - **Training**:\n    - Resized the images from 800x600 to 298x224: *(Roughly the same ratio to prevent distortion)*\n      - Which is not actually the most efficient size as some other notebooks got accuracy score above 90% using dimensions of 512x384 but this required a gradient accumulation and some additional steps to prevent the memory from exploding in our faces every time :), so we decided to stay happy by using this size for now.\n    - Added some augmentation steps:\n      - **RandomHorizontalFlip:**\n      - **RandomAffine:**\n        - Rotation degrees of 15\n        - Slight random translation\n        - Slight random scaling\n      - **ColorJitter:**  which randomly changed brightness, contrast, saturation, and hue.\n    - Finally, converted the image to a tensor and normalized it using ImageNet mean and std values *(Obtained from the notebooks shared for this dataset)*.\n- Using `random_split`, we splitted the dataset into:\n  - **70%** as Training Set\n  - **20%** as Validation Set\n  - **10%** as Test Set\n\n- Set the suitable transform for each split.\n- Built the data loaders using `DataLoader` Class, each loader with its set and batch_size.\n\n### Visualization\n\n- Used the `visualize_images` function *copied from the notebook provided in the material with some modifications to fit the current dataset structure*.\n- Plotted a sample of each class.\n\n### Building the Training Pipeline:\n- We used the exact same pipeline in the notebook provided in the material with some modifications:\n  - Used `tqdm` to visualize the training progress with training loss monitored in the header.\n  - Added validation phase after each epoch to monitor the model performance on unseen data.\n  - Added model checkpointing step for preserving the best model based on validation loss.\n\n### Building Models\n\n#### Custom CNN Model\n\n- Used `torchinfo.summary` to help determine the **CNN** part output shape *(By creating an instance of he model and providing the input_size)*, to determine the **Classifier** *(fully-connected)* input dimenstions.\n  *Noticed later that some pre-trained models use `AdaptiveAveragePooling` to set a fixed output shape.*\n\n  - **68.54%** accuracy on Validation set\n  - **68.80%** on Test set.\n\n#### Pre-trained Models\n\n- Used `torchvision.models` to load pre-trained models.\n- Removed the last layer in the (Classifier) and replaced it with a new one that fits our number of classes *(5 classes)*.\n- We didn't have to use `torchinfo.summary` here as the pre-trained models already had fixed output shapes before the last layer thanks to the `AdabtiveAveragePooling` Layer just before the classifier part.\n- Training Config:\n  - Used **CrossEntropyLoss** as the loss function.\n  - Used **Adam** optimizer with a learning rate of **1e-4**.\n  - Batch size of **64**.\n- Used the following pre-trained models:\n  - **ResNet50**: Large so the memroy crashed\n  - **DenseNet121**: Large so the memroy crashed\n  - **EfficientNet_B0**: Worked fine and got **82%** accuracy on validation set.\n- We decided to continue with **MobileNet V2** and **EfficientNet_B3** as they gave better results.\n  - Tried to use different versions:\n    - **MobileNet_V2**:\n      - **83.06%** accuracy on validation set\n      - **84.40%** on test set.\n    - **tf_efficientnet_b3.ns_jft_in1k** on Hugging face: An improved version pre-trained on a larger dataset *(JFT-300M)*.\n    Loaded the model using `timm` library (Py**T**orch **Im**age **M**odels).\n      - **83.45%** accuracy on validation set\n      - **83.33%** on Test set.\n    - Pre-Trained vanilla **efficientnet_b3** from `torchvision.models`:\n      - **84.09%** accuracy on validation set\n      - **82.91%** on Test set.\n    - vanilla **efficientnet_b3** with randomly initialized weights:\n      - **63.78%** accuracy on validation set\n      - **64.55%** on test set.\n\n#### Finally\n\nWe tended to submit the model on the dataset's competition page using the best model we had, but as we think the submission system was down for a long time, so we couldn't get the final test accuracy on the competition page.","metadata":{}},{"id":"5f684fbf","cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport torchvision.transforms as transforms\nfrom torchvision import datasets, models\nfrom torchvision.io.image import decode_image, ImageReadMode\nfrom torch.utils.data import DataLoader, Dataset, random_split\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport copy\nimport torchvision\nfrom collections import defaultdict\nimport random\nimport PIL\n\nfrom tqdm import tqdm\nfrom copy import deepcopy\nimport gc","metadata":{},"outputs":[],"execution_count":null},{"id":"888ed0a9","cell_type":"code","source":"# Device configuration\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nprint(f\"Using device: {device}\")","metadata":{},"outputs":[],"execution_count":null},{"id":"8eec1bea","cell_type":"markdown","source":"# Load data","metadata":{}},{"id":"afcf0cca","cell_type":"code","source":"import os","metadata":{},"outputs":[],"execution_count":null},{"id":"edea9a81","cell_type":"code","source":"DATA_DIR = '/kaggle/input/cassava-leaf-disease-classification'\nTRAIN_DIR = os.path.join(DATA_DIR, 'train_images')","metadata":{},"outputs":[],"execution_count":null},{"id":"676a496b","cell_type":"code","source":"import json\nmapping_path = os.path.join(DATA_DIR, 'label_num_to_disease_map.json')\nwith open(mapping_path, 'r') as f:\n    idx_to_class = json.load(f)\n\nidx_to_class","metadata":{},"outputs":[],"execution_count":null},{"id":"45952a7d","cell_type":"code","source":"import pandas as pd\nmapping_csv = pd.read_csv(os.path.join(DATA_DIR, 'train.csv'))\nmapping_csv.head()","metadata":{},"outputs":[],"execution_count":null},{"id":"61057fa7","cell_type":"code","source":"test_img = decode_image(os.path.join(TRAIN_DIR, '1000015157.jpg'), mode=ImageReadMode.RGB)\nimg_shape = test_img.shape\nprint(\"Single image shape:\", img_shape)","metadata":{},"outputs":[],"execution_count":null},{"id":"34341246","cell_type":"code","source":"class CustomDataset(Dataset):\n    def __init__(self, root_dir, img_label_df, transform=None):\n        self.root_dir = root_dir\n        self.transform = transform\n        self.image_files = []\n        self.labels = []\n        for root, dirs, files in os.walk(root_dir):\n            for file in files:\n                self.image_files.append(os.path.join(root, file))\n                label = img_label_df[img_label_df['image_id'] == file]['label'].iloc[0]\n                self.labels.append(label)\n \n    def __len__(self):\n        return len(self.image_files)\n \n    def __getitem__(self, idx):\n        image_path = self.image_files[idx]\n        image = PIL.Image.open(image_path)\n        if self.transform:\n            image = self.transform(image)\n        else:\n            image = torch.Tensor(image)\n\n        label = self.labels[idx]\n        ohe_label = torch.eye(5)[label]\n        return image, ohe_label","metadata":{},"outputs":[],"execution_count":null},{"id":"83a2eeed","cell_type":"code","source":"mean = [0.485, 0.456, 0.406]\nstd = [0.229, 0.224, 0.225]\n\ntrain_transform = transforms.Compose([\n    transforms.Resize(224),\n    transforms.RandomHorizontalFlip(),\n    transforms.RandomAffine(degrees=15, translate=(0.0625, 0.0625), scale=(0.9, 1.1)),\n    transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.2, hue=0.2),\n    transforms.ToTensor(),\n    transforms.ConvertImageDtype(torch.float32),\n    transforms.Normalize(mean=mean, std=std),\n])\n\nval_transform = transforms.Compose([\n    transforms.Resize(224),\n    transforms.ToTensor(),\n    transforms.ConvertImageDtype(torch.float32),\n    transforms.Normalize(mean=mean, std=std),\n])","metadata":{},"outputs":[],"execution_count":null},{"id":"5dc93419","cell_type":"code","source":"dataset = CustomDataset(TRAIN_DIR, mapping_csv)\n\ntrain_size = int(0.7 * len(dataset))\nval_size = int(0.2 * len(dataset))\ntest_size = len(dataset) - train_size - val_size\ntrain_data, val_data, test_data = random_split(dataset, (train_size, val_size, test_size))","metadata":{},"outputs":[],"execution_count":null},{"id":"78176303","cell_type":"code","source":"train_data.dataset.transform = train_transform\nval_data.dataset.transform = val_transform\ntest_data.dataset.transform = val_transform","metadata":{},"outputs":[],"execution_count":null},{"id":"678ef9f2","cell_type":"code","source":"train_loader = DataLoader(train_data, batch_size=64, num_workers=4)\nval_loader = DataLoader(val_data, batch_size=64, num_workers=4)\ntest_loader = DataLoader(test_data, batch_size=64, num_workers=4)","metadata":{},"outputs":[],"execution_count":null},{"id":"161d62e1","cell_type":"markdown","source":"# Visualization","metadata":{}},{"id":"f1611f0d","cell_type":"code","source":"def visualise_images(dataset, grid, mapping):\n    \"\"\"\n    Displays a grid of images from a dataset, with one random image per class.\n\n    Args:\n        dataset: The dataset object containing the images and labels.\n        grid (tuple): A tuple specifying the number of rows and columns for the image grid.\n    \"\"\"\n\n    # Create a shallow copy of the dataset to avoid modifying the original\n    dataset_copy = copy.copy(dataset)\n    # Set the transform on the copied dataset to convert images to tensors\n    dataset_copy.transform = torchvision.transforms.ToTensor()\n\n    # Create a DataLoader to handle batching and shuffling of the data\n    loader = DataLoader(dataset_copy, batch_size=64, shuffle=True)\n\n    # Unpack the grid dimensions from the input tuple\n    rows, cols = grid\n    # Calculate the total number of images to display in the grid\n    num_images_to_show = rows * cols\n\n    # Get the dataset object from the DataLoader\n    dataset_to_show = loader.dataset\n\n    # Create a dictionary to store lists of indices for each class\n    class_indices = defaultdict(list)\n    # Iterate through the dataset to populate the class_indices dictionary\n    for idx, target in enumerate(dataset_to_show.labels):\n        class_indices[target].append(idx)\n        \n    # Get the list of class names from the dataset\n    class_names = list(mapping.values())\n    # Create a figure and a set of subplots for the grid layout\n    fig, axes = plt.subplots(rows, cols, figsize=(cols * 4, rows * 4))\n\n    # Iterate over each subplot in the grid\n    for i, ax in enumerate(axes.flat):\n        # If the current index is out of bounds, turn off the subplot axis\n        if i >= num_images_to_show or i >= len(class_names):\n            ax.axis('off')\n            continue\n            \n        # Set the class label based on the current iteration index\n        class_label = i\n        # Get the list of image indices for the current class\n        indices_for_class = class_indices[class_label]\n        # If there are no images for this class, turn off the subplot axis\n        if not indices_for_class:\n            ax.axis('off')\n            continue\n\n        # Choose a random image index from the list for the current class\n        random_image_index = random.choice(indices_for_class)\n        \n        # Retrieve the image tensor and its corresponding label from the dataset\n        image_tensor, _ = dataset_to_show[random_image_index]\n        \n        # Convert the tensor to a NumPy array and transpose dimensions for display\n        img_to_display = image_tensor.numpy().transpose((1, 2, 0))\n        \n        # Get the name of the class corresponding to the class label\n        class_name = class_names[class_label]\n        \n        # Display the image on the current subplot\n        ax.imshow(img_to_display)\n        \n        # Set the title of the subplot to the capitalized class name\n        ax.set_title(class_name.capitalize(), fontsize=16)\n        # Turn off the axis for a cleaner look\n        ax.axis('off')\n\n    # Adjust subplot parameters for a tight layout\n    plt.tight_layout(rect=[0, 0.03, 1, 0.95])\n    # Display the plot\n    plt.show()\n\n    # Clean up the copied dataset to free up memory\n    del dataset_copy","metadata":{},"outputs":[],"execution_count":null},{"id":"ec906192","cell_type":"code","source":"visualise_images(train_data.dataset, grid=(1, 5), mapping=idx_to_class)","metadata":{},"outputs":[],"execution_count":null},{"id":"ce9c3f50","cell_type":"markdown","source":"# Building the model","metadata":{}},{"id":"b5e4a340","cell_type":"code","source":"from torchinfo import summary","metadata":{},"outputs":[],"execution_count":null},{"id":"0d2a0418","cell_type":"markdown","source":"# Custom CNN Model","metadata":{}},{"id":"ef26d8c9","cell_type":"code","source":"next(iter(train_loader))[0].shape","metadata":{},"outputs":[],"execution_count":null},{"id":"62206569","cell_type":"code","source":"class CustomCNN(nn.Module):\n    def __init__(self, num_classes):\n        super().__init__()\n        self.cnn = nn.Sequential(\n            nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1),\n            nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, padding=1),\n            nn.ReLU(),\n            nn.BatchNorm2d(64),\n            nn.MaxPool2d(2),\n\n            nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, padding=1),\n            nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, padding=1),\n            nn.ReLU(),\n            nn.BatchNorm2d(128),\n            nn.MaxPool2d(2),\n\n            nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, padding=1),\n            nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1),\n            nn.ReLU(),\n            nn.BatchNorm2d(256),\n            nn.MaxPool2d(2),  \n        )\n        \n        self.flat = nn.Flatten()\n        \n        self.classifier = nn.Sequential(\n            nn.Linear(265216, 128),\n            nn.ReLU(),\n\n            nn.Linear(128, 64),\n            nn.ReLU(),\n            \n            nn.Linear(64, num_classes),\n        )\n        \n    def forward(self, x):\n        x = self.cnn(x)        \n        x = self.flat(x)\n        x = self.classifier(x)\n        \n        return x\n\nsummary(CustomCNN(5), input_size= [2, 3, 224, 298])","metadata":{},"outputs":[],"execution_count":null},{"id":"b17aa460","cell_type":"code","source":"custom_model = CustomCNN(5)\ncustom_model.to(device)\nprint()","metadata":{},"outputs":[],"execution_count":null},{"id":"00d60ac4","cell_type":"markdown","source":"## Using Pre-Defined Model Architectures","metadata":{}},{"id":"19ba2385","cell_type":"markdown","source":"#### MobileNet_V2","metadata":{}},{"id":"c13175fa","cell_type":"code","source":"mn_model = models.mobilenet_v2(weights='DEFAULT')\nmn_model.classifier[1] = nn.Linear(in_features= 1280, out_features=5)\nmn_model.to(device)\nprint()","metadata":{},"outputs":[],"execution_count":null},{"id":"d33083ab","cell_type":"markdown","source":"#### Semi-Supervised EfficientNet","metadata":{}},{"id":"fec77e05","cell_type":"code","source":"import timm\n\nhf_model = timm.create_model('tf_efficientnet_b3.ns_jft_in1k', pretrained=True)\nn_features = hf_model.classifier.in_features\nhf_model.classifier = nn.Linear(n_features, 5)\nhf_model.to(device)\nprint()","metadata":{},"outputs":[],"execution_count":null},{"id":"973bc6b4","cell_type":"markdown","source":"#### Pre-trained","metadata":{}},{"id":"dc1e23f5","cell_type":"code","source":"pt_model = models.efficientnet_b3(weights='DEFAULT')\npt_model.classifier[1] = nn.Linear(1536, 5)\npt_model.to(device)\nprint()","metadata":{},"outputs":[],"execution_count":null},{"id":"c68f3231","cell_type":"markdown","source":"#### Pre-Built (Only Architecture)","metadata":{}},{"id":"dab69e80","cell_type":"code","source":"arch_model = models.efficientnet_b3()\narch_model.classifier[1] = nn.Linear(1536, 5)\narch_model.to(device)\nprint()","metadata":{},"outputs":[],"execution_count":null},{"id":"8f64a640","cell_type":"markdown","source":"# Training loop functions","metadata":{}},{"id":"a32f19fd","cell_type":"code","source":"def train_one_epoch(model, train_loader, optimizer, loss_func, device):\n    model.train()\n    running_loss = 0.0\n    \n    pbar = tqdm(train_loader)\n    for i, (images, labels) in enumerate(pbar, 1):\n        images, labels = images.to(device), labels.to(device)\n        \n        optimizer.zero_grad()\n        outputs = model(images)\n        \n        loss = loss_func(outputs, labels)\n        loss.backward()\n        optimizer.step()\n        \n        running_loss += loss.item() * images.size(0)\n        pbar.set_description(f\"Train Loss: {running_loss / (i * train_loader.batch_size):.4f}\")\n    \n    epoch_loss = running_loss / len(train_loader.dataset)\n    return epoch_loss\n\n\ndef validate_one_epoch(model, val_loader, loss_func, device):\n    model.eval()\n    running_val_loss = 0.0\n    correct = 0\n    total = 0\n    \n    with torch.no_grad():\n        for images, labels in val_loader:\n            images, labels = images.to(device), labels.to(device)\n            \n            outputs = model(images)\n            \n            val_loss = loss_func(outputs, labels)\n            running_val_loss += val_loss.item() * images.size(0)\n            \n            _, predicted = torch.max(outputs, 1)\n            total += labels.size(0)\n            correct += (predicted == torch.max(labels, dim=1)[1]).sum().item()\n    \n    epoch_val_loss = running_val_loss / len(val_loader.dataset)\n    epoch_accuracy = 100.0 * correct / total\n    \n    return epoch_val_loss, epoch_accuracy\n\n\ndef update_best_model(current_model, best_model, current_val_loss, least_val_loss):\n    if best_model is None or current_val_loss < least_val_loss:\n        if best_model is not None:\n            del best_model\n            gc.collect()\n            torch.cuda.empty_cache()\n        \n        best_model = deepcopy(current_model)\n        least_val_loss = current_val_loss\n    \n    return best_model, least_val_loss\n\n\ndef train_model(model, optimizer, loss_func, num_epochs, train_loader, val_loader, verbose=True):\n    best_model = None\n    least_val_loss = float('inf')\n\n    train_losses = []\n    val_losses = []\n    val_accuracies = []\n\n    for epoch in range(num_epochs):\n        # Training phase\n        epoch_loss = train_one_epoch(model, train_loader, optimizer, loss_func, device)\n        train_losses.append(epoch_loss)\n        \n        # Validation phase\n        epoch_val_loss, epoch_accuracy = validate_one_epoch(model, val_loader, loss_func, device)\n        val_losses.append(epoch_val_loss)\n        val_accuracies.append(epoch_accuracy)\n        \n        # Update best model\n        best_model, least_val_loss = update_best_model(model, best_model, epoch_val_loss, least_val_loss)\n        \n        if verbose:\n            print(f\"Epoch [{epoch+1}/{num_epochs}], Train Loss: {epoch_loss:.4f}, Val Loss: {epoch_val_loss:.4f}, Val Accuracy: {epoch_accuracy:.2f}%\")\n    \n    metrics = [train_losses, val_losses, val_accuracies]\n    return model, best_model, metrics","metadata":{},"outputs":[],"execution_count":null},{"id":"bd721bad","cell_type":"markdown","source":"# Training the models","metadata":{}},{"id":"0766f3ae","cell_type":"markdown","source":"### 1. Custom CNN Model","metadata":{}},{"id":"b442cc0c","cell_type":"code","source":"loss_func = nn.CrossEntropyLoss().to(device)\noptimizer = optim.Adam(custom_model.parameters(), lr= 1e-4)\ncustom_model, best_custom_model, custom_model_metrics = train_model(custom_model, optimizer, loss_func, 10, train_loader, val_loader)\n\nval_loss, val_acc = validate_one_epoch(best_custom_model, val_loader, loss_func, device)\nprint(f\"Validation Accuracy: {val_acc:.2f}%, Validation Loss: {val_loss:.4f}\")\n\nloss, acc = validate_one_epoch(best_custom_model, test_loader, loss_func, device)\nprint(f\"Test Accuracy: {acc:.2f}%, Test Loss: {loss:.4f}\")\n\ntorch.save(best_custom_model, 'custom_cnn.torch')","metadata":{},"outputs":[],"execution_count":null},{"id":"91f6137d","cell_type":"markdown","source":"### 2. MobileNet_V2","metadata":{}},{"id":"d6d1e951","cell_type":"code","source":"loss_func = nn.CrossEntropyLoss().to(device)\noptimizer = optim.Adam(mn_model.parameters(), lr= 1e-4)\nmn_model, best_mn_model, mn_model_metrics = train_model(mn_model, optimizer, loss_func, 10, train_loader, val_loader)\n\nval_loss, val_acc = validate_one_epoch(best_mn_model, val_loader, loss_func, device)\nprint(f\"Validation Accuracy: {val_acc:.2f}%, Validation Loss: {val_loss:.4f}\")\n\nloss, acc = validate_one_epoch(best_mn_model, test_loader, loss_func, device)\nprint(f\"Test Accuracy: {acc:.2f}%, Test Loss: {loss:.4f}\")\n\ntorch.save(best_mn_model, 'mobilenetv2.torch')","metadata":{},"outputs":[],"execution_count":null},{"id":"14400581","cell_type":"markdown","source":"### 3. Huggingface Semi-Supervised EfficientNet","metadata":{}},{"id":"b0123ced","cell_type":"code","source":"loss_func = nn.CrossEntropyLoss().to(device)\noptimizer = optim.Adam(hf_model.parameters(), lr=1e-4)\nhf_model, best_hf_model, hf_model_metrics = train_model(hf_model, optimizer, loss_func, 10, train_loader, val_loader)\n\nval_loss, val_acc = validate_one_epoch(best_hf_model, val_loader, loss_func, device)\nprint(f\"Validation Accuracy: {val_acc:.2f}%, Validation Loss: {val_loss:.4f}\")\n\nloss, acc = validate_one_epoch(best_hf_model, test_loader, loss_func, device)\nprint(f\"Test Accuracy: {acc:.2f}%, Test Loss: {loss:.4f}\")\n\ntorch.save(best_hf_model, 'hf_EfficientNet.torch')","metadata":{},"outputs":[],"execution_count":null},{"id":"1258ab33","cell_type":"markdown","source":"### 4. Pre-trained Model","metadata":{}},{"id":"d2763fbf","cell_type":"code","source":"loss_func = nn.CrossEntropyLoss().to(device)\noptimizer = optim.Adam(pt_model.parameters(), lr=1e-4)\npt_model, best_pt_model, pt_model_metrics = train_model(pt_model, optimizer, loss_func, 10, train_loader, val_loader)\nloss, acc = validate_one_epoch(best_pt_model, test_loader, loss_func, device)\n\nval_loss, val_acc = validate_one_epoch(best_pt_model, val_loader, loss_func, device)\nprint(f\"Validation Accuracy: {val_acc:.2f}%, Validation Loss: {val_loss:.4f}\")\n\nloss, acc = validate_one_epoch(best_pt_model, test_loader, loss_func, device)\nprint(f\"Test Accuracy: {acc:.2f}%, Test Loss: {loss:.4f}\")\n\ntorch.save(best_pt_model, 'pt_efficientnet.torch')","metadata":{},"outputs":[],"execution_count":null},{"id":"4aa3e96a","cell_type":"markdown","source":"### 5. Only Architecture (Randomly Initialized) Model","metadata":{}},{"id":"f45a891a","cell_type":"code","source":"loss_func = nn.CrossEntropyLoss().to(device)\noptimizer = optim.Adam(arch_model.parameters(), lr=1e-4)\narch_model, best_arch_model, arch_model_metrics = train_model(arch_model, optimizer, loss_func, 10, train_loader, test_loader)\nloss, acc = validate_one_epoch(best_arch_model, test_loader, loss_func, device)\n\nval_loss, val_acc = validate_one_epoch(best_arch_model, val_loader, loss_func, device)\nprint(f\"Validation Accuracy: {val_acc:.2f}%, Validation Loss: {val_loss:.4f}\")\n\nloss, acc = validate_one_epoch(best_arch_model, test_loader, loss_func, device)\nprint(f\"Test Accuracy: {acc:.2f}%, Test Loss: {loss:.4f}\")\n\ntorch.save(best_arch_model, 'arch_efficientnet.torch')","metadata":{},"outputs":[],"execution_count":null},{"id":"dfc27b4a","cell_type":"markdown","source":"# Submission","metadata":{}},{"id":"e004a406","cell_type":"code","source":"TEST_DIR= os.path.join(DATA_DIR, 'test_images')\nsubmission_df = pd.DataFrame(columns=['image_id', 'label'])\n\nimage_files = []\nfor root, dirs, files in os.walk(TEST_DIR):\n    for file in files:\n        image =  PIL.Image.open(os.path.join(TEST_DIR, file))\n        image = val_transform(image)\n        image = image.unsqueeze(0).to(device)\n        with torch.no_grad():\n            proba = best_pt_model(image)\n            pred = proba.max(1)[1]\n            submission_df.loc[len(submission_df)] = [file, pred.item()]\n\nsubmission_df.to_csv('submission.csv')","metadata":{},"outputs":[],"execution_count":null}]}