\n \n## Transfer Learning\n\n#### Why transfer learning and how to use it?\n \nIf dataset is very large, it can easily take more than a few weeks to train. It's good to use transfer learning in this case. A few ways to use it:\n- **As fixed feature extractor**: froze the weights in the convoluntion layers. Can add/train fully connected layers.\n- **Fine-tuning the CNN**: you data is big, similar to the transfer learning dataset but not exactly the same. You can 'de-forze' and train some CNN layers weights too.\n- **Use only part of CNN layer**: add a few of your own CNN layers also.\n - A typical example for medical images.\n - **Medical images** are harder to collect and even harder to label since they need to labeled by doctors.\n - Trained by big ImageNet data, the first few layers can find some primitive features that can be used for medical images. We can use the earlier layers and add some later layers to train specifically for medical images.\n \n \n \n#### When to use a pre-trained network?\n- **โNew dataset is small (ex: 20,000 with 100 categories) and similar to the original dataset (ex: 2,000,000 with 1000 categories)**\n - Since the network is huge, you have froze the weight, which means you aren't training the model. Even not consider computation resource that you can actually re-train the model, the model is going to overfit since it's too big and more suitable for much bigger data.\n \n- **โ
New dataset is large and similar to the original dataset**\n - can gain training time by using either fixed or fresh weights\n- **โNew dataset is small but very different from the original dataset**\n - Overfit as in case in. Additionally, if very different, the transfer learning efficiency would be low.\n- **โ
New dataset is large and very different from the original dataset**\n - Can still use part of feature extractor of earlier layers. Still can save lots of training time.\n \nVarious [models](https://keras.io/applications/) pre-trained on ImageNet.","metadata":{}},{"cell_type":"markdown","source":"
\n \n## InceptionV3 \n#### Insights from InceptionV3 Architecture:\n- Lower Layers:\n - Basic convolutional Operations: such as Conv2d_1a_3x3, Conv2d_2a_3x3, and Conv2d_2b_3x3 \n - Fundamental local features extractraction: such as edges, textures, colors, and gradients\n - Simple Shapes and Contrast: In addition to low-level features, identify simple shapes and capturing contrast variations in the images.\n- Intermediate Layers:\n - Mixed Operations: blocks such as Mixed_3a, Mixed_4a, and Mixed_5a, combining different filter sizes and operations in parallel.\n - Repetitive Structures capturing: such as repetitive structures within the images, recognizing patterns that repeat across the dataset.\n - Color Variations and Relationships: further understand color variations and establish hierarchical relationships between different parts of objects in the images.\n - Object Boundaries: delineating object boundaries, enhancing the model's ability to recognize distinct objects and their spatial relationships.\n- Deeper Inception Blocks:\n - Mixed Operations: Blocks such as Mixed_5b, Mixed_6a, Mixed_7a\n - Complex Shapes and Structures: improved understanding of complex shapes and intricate structures present in the images.\n - Discrimination of Patterns: enhanced discrimination of complex patterns, distinguish fine-grained details and textures.\n\n#### My Adaptation Plan:\n- Feature Extraction:\n - Utilize earlier blocks for low-level features \n- Intermediate Layers Modification:\n - Introduce extra convolution layers for medical image nuances.\n - Post 'mixed2' layer in InceptionV3, the feature map is roughly (9 * 9). Add padding to prevent rapid size reduction (retain spatial information)\n- Enhancements with 1x1 Filters:\n - Incorporate 1x1 filters for improved learning.\n- Handle Overfitting:\n - Batch normalization\n - Dropout layers.\n- Image Augmentations for better generalization.","metadata":{}},{"cell_type":"markdown","source":"## Model 1 - transfer learning with Inception V3","metadata":{}},{"cell_type":"code","source":"# Build model 1\n\n# Load the InceptionV3 model\nincept_model = InceptionV3(weights='imagenet', include_top=False, input_shape=(96, 96, 3))\n\n# Freeze the weights of the layers.\nfor layer in incept_model.layers:\n layer.trainable = False\n\n# get the last_layer of inception we want to use\nlast_layer = incept_model.get_layer('mixed2')\nprint('last layer output shape: ', last_layer.output_shape)\n\n# Create a new model using the functional API\nbase_model = Model(inputs=incept_model.input, outputs=last_layer.output)\n\n# Add layers to the model\nmymodel_3 = Sequential([\n base_model,\n \n Conv2D(64, (3, 3), padding='same', activation='relu'), # keep the width and height\n BatchNormalization(),\n Conv2D(64, (1, 1),activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n \n Conv2D(128, (3, 3), activation='relu'),\n BatchNormalization(), \n Conv2D(64, (1, 1), activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n\n Flatten(),\n Dense(128, activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n \n Dense(128, activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n \n Dense(1, activation='sigmoid')\n])\n\n# Print the model summary\nmymodel_3.build(input_shape=(None, 96, 96, 3))\nmymodel_3.summary()","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:00:48.184581Z","iopub.execute_input":"2025-05-27T08:00:48.18518Z","iopub.status.idle":"2025-05-27T08:00:55.693657Z","shell.execute_reply.started":"2025-05-27T08:00:48.185152Z","shell.execute_reply":"2025-05-27T08:00:55.69285Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from tensorflow.keras.optimizers.schedules import ExponentialDecay\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.metrics import AUC\n\n# Define the learning rate schedule (exponential decay)\ninitial_learning_rate = 0.001\ndecay_steps = 1000\ndecay_rate = 0.9 \nlr_schedule = ExponentialDecay(initial_learning_rate, decay_steps, decay_rate, staircase=True)\n\n# Compile the model\nmymodel_3.compile(\n optimizer=Adam(learning_rate=lr_schedule),\n loss='binary_crossentropy',\n metrics=['accuracy', AUC()] # Added 'accuracy' here\n)\n","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:02:09.989484Z","iopub.execute_input":"2025-05-27T08:02:09.98981Z","iopub.status.idle":"2025-05-27T08:02:10.94928Z","shell.execute_reply.started":"2025-05-27T08:02:09.989777Z","shell.execute_reply":"2025-05-27T08:02:10.948593Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Fit model 3\nBATCH_SIZE = 128\nEPOCHS = 5\n\n# data generator\ndatagen = ImageDataGenerator(\n rescale=1./255,\n rotation_range=20,\n width_shift_range=0.2,\n height_shift_range=0.2,\n vertical_flip=True,\n horizontal_flip=True,\n)\n\ndatagen_val = ImageDataGenerator(\n rescale=1./255,\n)\n\ntrain_generator = datagen.flow_from_directory(directory=train_dir,\n batch_size=BATCH_SIZE,\n class_mode='binary',\n target_size=(96, 96))\n\nval_generator = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=BATCH_SIZE, \n class_mode='binary',\n target_size=(96,96))\n\nsteps_per_epoch = len(train_generator) \nvalidation_steps = len(val_generator)\n\n# Fit model 3\nhistory_3 = mymodel_3.fit(\n train_generator,\n validation_data=val_generator,\n epochs=EPOCHS,\n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n verbose=1\n)","metadata":{"execution":{"iopub.status.busy":"2025-05-27T09:59:00.390741Z","iopub.execute_input":"2025-05-27T09:59:00.391061Z","iopub.status.idle":"2025-05-27T09:59:03.406283Z","shell.execute_reply.started":"2025-05-27T09:59:00.391036Z","shell.execute_reply":"2025-05-27T09:59:03.40543Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(history_3.history.keys())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T08:43:45.48086Z","iopub.execute_input":"2025-05-27T08:43:45.481501Z","iopub.status.idle":"2025-05-27T08:43:45.485774Z","shell.execute_reply.started":"2025-05-27T08:43:45.481474Z","shell.execute_reply":"2025-05-27T08:43:45.484816Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Extract final values from training history\nfinal_loss = history_3.history['loss'][-1]\nfinal_accuracy = history_3.history['accuracy'][-1]\nfinal_auc = history_3.history['auc'][-1]\n\nfinal_val_loss = history_3.history['val_loss'][-1]\nfinal_val_accuracy = history_3.history['val_accuracy'][-1]\nfinal_val_auc = history_3.history['val_auc'][-1]\n\n# Print final metrics\nprint(\"\\n=== Final Training & Validation Metrics ===\")\nprint(f\"Train Loss: {final_loss:.4f}\")\nprint(f\"Train Accuracy: {final_accuracy:.4f}\")\nprint(f\"Train AUC: {final_auc:.4f}\")\nprint(f\"Validation Loss: {final_val_loss:.4f}\")\nprint(f\"Validation Accuracy: {final_val_accuracy:.4f}\")\nprint(f\"Validation AUC: {final_val_auc:.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T08:45:06.739289Z","iopub.execute_input":"2025-05-27T08:45:06.739984Z","iopub.status.idle":"2025-05-27T08:45:06.745714Z","shell.execute_reply.started":"2025-05-27T08:45:06.739956Z","shell.execute_reply":"2025-05-27T08:45:06.744841Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot metrics\nepochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(10, 4))\n\n# Accuracy plot\nplt.subplot(1, 2, 1)\nplt.plot(epochs, history_3.history['accuracy'], label='Train Accuracy')\nplt.plot(epochs, history_3.history['val_accuracy'], label='Val Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Accuracy Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss plot\nplt.subplot(1, 2, 2)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:50:13.890734Z","iopub.execute_input":"2025-05-27T08:50:13.891474Z","iopub.status.idle":"2025-05-27T08:50:14.28542Z","shell.execute_reply.started":"2025-05-27T08:50:13.891446Z","shell.execute_reply":"2025-05-27T08:50:14.284587Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Model 3 Performance:\n- Achieved satisfactory training AUC of around 0.96 and a commendable validation AUC of 0.97 in the last epoch.\n- validation AUC plateaued after 8 epochs.\n\n### Next Steps:\n- Model evaluation, focusing on metrics such as precision, recall, F1 score, and confusion matrix to gain a holistic understanding of the model's performance. This approach ensures a balanced assessment of the model's predictive capabilities beyond AUC, providing insights into its precision, recall, and overall classification effectiveness.","metadata":{}},{"cell_type":"code","source":"# Predict validation set \nsamples_val = len(os.listdir(os.path.join(val_dir, '1'))) + len(os.listdir(os.path.join(val_dir, '0')))\nval_generator_for_pred = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=1, \n class_mode='binary',\n target_size=(96,96),\n shuffle=False)\n\n\nval_predictions = mymodel_3.predict_generator(val_generator_for_pred,steps=samples_val, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:46:44.387677Z","iopub.execute_input":"2025-05-27T08:46:44.388594Z","iopub.status.idle":"2025-05-27T08:48:22.650591Z","shell.execute_reply.started":"2025-05-27T08:46:44.388564Z","shell.execute_reply":"2025-05-27T08:48:22.649815Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Actual vs. predicted classes for validation set\n\n# actual\nval_labels = val_generator_for_pred.classes\n\n# convert probability to classes using 0.5 shreshold\nbinary_val_predictions = (val_predictions > 0.5).astype(int)\n","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:49:15.603978Z","iopub.execute_input":"2025-05-27T08:49:15.604292Z","iopub.status.idle":"2025-05-27T08:49:15.609303Z","shell.execute_reply.started":"2025-05-27T08:49:15.60427Z","shell.execute_reply":"2025-05-27T08:49:15.608367Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot ROC curve\n\n# Calculate AUC\nfpr, tpr, thresholds = roc_curve(val_labels, val_predictions)\nroc_auc = auc(fpr, tpr)\n\n# Plot \nplt.figure(figsize=(4, 4))\nplt.plot(fpr, tpr, color='orange', lw=2, label='ROC curve (AUC = {:.2f})'.format(roc_auc))\nplt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label='Random Guessing')\nplt.xlabel('FPR')\nplt.ylabel('TPR')\nplt.title('ROC Curve')\nplt.legend(loc='lower right')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:49:16.783868Z","iopub.execute_input":"2025-05-27T08:49:16.784207Z","iopub.status.idle":"2025-05-27T08:49:16.945791Z","shell.execute_reply.started":"2025-05-27T08:49:16.784182Z","shell.execute_reply":"2025-05-27T08:49:16.944956Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Get recall, predicision, F1, accuracy scores for validation set\nrecall = recall_score(val_labels, binary_val_predictions)\nprecision = precision_score(val_labels, binary_val_predictions)\nF1 = f1_score(val_labels, binary_val_predictions)\nacc= accuracy_score(val_labels, binary_val_predictions)\n\nprint(\"Recall on validation set:\", recall)\nprint(\"Precision on validation set:\", precision)\nprint(\"F1 score n on validation set:\", F1)\nprint(\"Accuracy score n on validation set:\", acc)","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:49:20.413251Z","iopub.execute_input":"2025-05-27T08:49:20.413569Z","iopub.status.idle":"2025-05-27T08:49:20.438359Z","shell.execute_reply.started":"2025-05-27T08:49:20.413545Z","shell.execute_reply":"2025-05-27T08:49:20.437616Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot confusion matrix\ncm = confusion_matrix(val_labels, binary_val_predictions)\ndisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Non-cancer', 'Cancer'])\ndisp.plot(cmap='Blues', values_format='d')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2025-05-27T08:50:45.313554Z","iopub.execute_input":"2025-05-27T08:50:45.314425Z","iopub.status.idle":"2025-05-27T08:50:45.468689Z","shell.execute_reply.started":"2025-05-27T08:50:45.314394Z","shell.execute_reply":"2025-05-27T08:50:45.467877Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Xception Model - Model 2","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.applications import Xception\nfrom tensorflow.keras.models import Model, Sequential\nfrom tensorflow.keras.layers import Conv2D, BatchNormalization, Dropout, Dense, GlobalAveragePooling2D\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping\nfrom tensorflow.keras.metrics import AUC\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\n\n# Load Xception\nxception_model = Xception(weights='imagenet', include_top=False, input_shape=(96, 96, 3))\n\n# Unfreeze top 20 layers for fine-tuning\nfor layer in xception_model.layers[:-20]:\n layer.trainable = False\nfor layer in xception_model.layers[-20:]:\n layer.trainable = True\n\n# Define base model\nlast_layer = xception_model.get_layer('block14_sepconv2_act')\nbase_model = Model(inputs=xception_model.input, outputs=last_layer.output)\n\n# Define the complete model\nmymodel_3 = Sequential([\n base_model,\n \n GlobalAveragePooling2D(), # Replaces Flatten + reduces params\n\n Dense(256, activation='relu', kernel_regularizer='l2'),\n BatchNormalization(),\n Dropout(0.4),\n \n Dense(128, activation='relu', kernel_regularizer='l2'),\n BatchNormalization(),\n Dropout(0.3),\n \n Dense(1, activation='sigmoid')\n])\n\n# Compile with dynamic LR adjustment\nmymodel_3.compile(\n optimizer=Adam(learning_rate=1e-3),\n loss='binary_crossentropy',\n metrics=['accuracy',AUC()]\n)\n\n# Callbacks\ncallbacks = [\n ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=1, verbose=1),\n EarlyStopping(monitor='val_loss', patience=2, restore_best_weights=True, verbose=1)\n]\n\n# Fit model\nhistory_3 = mymodel_3.fit(\n train_generator,\n validation_data=val_generator,\n epochs=5,\n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n callbacks=callbacks,\n verbose=1\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:01:31.171134Z","iopub.execute_input":"2025-05-27T10:01:31.171933Z","iopub.status.idle":"2025-05-27T10:44:06.806997Z","shell.execute_reply.started":"2025-05-27T10:01:31.171906Z","shell.execute_reply":"2025-05-27T10:44:06.806266Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(history_3.history.keys())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:44:52.590438Z","iopub.execute_input":"2025-05-27T10:44:52.590769Z","iopub.status.idle":"2025-05-27T10:44:52.595536Z","shell.execute_reply.started":"2025-05-27T10:44:52.590744Z","shell.execute_reply":"2025-05-27T10:44:52.594632Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Extract final values from training history\nfinal_loss = history_3.history['loss'][-1]\nfinal_accuracy = history_3.history['accuracy'][-1]\nfinal_auc = history_3.history['auc_2'][-1]\n\nfinal_val_loss = history_3.history['val_loss'][-1]\nfinal_val_accuracy = history_3.history['val_accuracy'][-1]\nfinal_val_auc = history_3.history['val_auc_2'][-1]\nfinal_lr = history_3.history['lr'][-1]\n\n# Print final metrics\nprint(\"\\n=== Final Training & Validation Metrics ===\")\nprint(f\"Train Loss: {final_loss:.4f}\")\nprint(f\"Train Accuracy: {final_accuracy:.4f}\")\nprint(f\"Train AUC: {final_auc:.4f}\")\nprint(f\"Validation Loss: {final_val_loss:.4f}\")\nprint(f\"Validation Accuracy: {final_val_accuracy:.4f}\")\nprint(f\"Validation AUC: {final_val_auc:.4f}\")\nprint(f\"Final Learning Rate: {final_lr:.6f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:45:51.918274Z","iopub.execute_input":"2025-05-27T10:45:51.919037Z","iopub.status.idle":"2025-05-27T10:45:51.925067Z","shell.execute_reply.started":"2025-05-27T10:45:51.919011Z","shell.execute_reply":"2025-05-27T10:45:51.924243Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"epochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(15, 4))\n\n# Accuracy Plot\nplt.subplot(1, 3, 1)\nplt.plot(epochs, history_3.history['accuracy'], label='Train Accuracy')\nplt.plot(epochs, history_3.history['val_accuracy'], label='Val Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Accuracy Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# AUC Plot\nplt.subplot(1, 3, 2)\nplt.plot(epochs, history_3.history['auc_2'], label='Train AUC')\nplt.plot(epochs, history_3.history['val_auc_2'], label='Val AUC')\nplt.xlabel('Epochs')\nplt.ylabel('AUC')\nplt.title('AUC Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss Plot\nplt.subplot(1, 3, 3)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:46:26.091893Z","iopub.execute_input":"2025-05-27T10:46:26.092497Z","iopub.status.idle":"2025-05-27T10:46:26.685974Z","shell.execute_reply.started":"2025-05-27T10:46:26.092467Z","shell.execute_reply":"2025-05-27T10:46:26.684934Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predict validation set \nsamples_val = len(os.listdir(os.path.join(val_dir, '1'))) + len(os.listdir(os.path.join(val_dir, '0')))\nval_generator_for_pred = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=1, \n class_mode='binary',\n target_size=(96,96),\n shuffle=False)\n\n\nval_predictions = mymodel_3.predict_generator(val_generator_for_pred,steps=samples_val, verbose=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:47:04.643507Z","iopub.execute_input":"2025-05-27T10:47:04.644126Z","iopub.status.idle":"2025-05-27T10:49:35.396494Z","shell.execute_reply.started":"2025-05-27T10:47:04.644097Z","shell.execute_reply":"2025-05-27T10:49:35.395738Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Actual vs. predicted classes for validation set\n\n# actual\nval_labels = val_generator_for_pred.classes\n\n# convert probability to classes using 0.5 shreshold\nbinary_val_predictions = (val_predictions > 0.5).astype(int)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:49:39.765779Z","iopub.execute_input":"2025-05-27T10:49:39.766337Z","iopub.status.idle":"2025-05-27T10:49:39.770401Z","shell.execute_reply.started":"2025-05-27T10:49:39.76631Z","shell.execute_reply":"2025-05-27T10:49:39.769589Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot ROC curve\n\n# Calculate AUC\nfpr, tpr, thresholds = roc_curve(val_labels, val_predictions)\nroc_auc = auc(fpr, tpr)\n\n# Plot \nplt.figure(figsize=(4, 4))\nplt.plot(fpr, tpr, color='orange', lw=2, label='ROC curve (AUC = {:.2f})'.format(roc_auc))\nplt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label='Random Guessing')\nplt.xlabel('FPR')\nplt.ylabel('TPR')\nplt.title('ROC Curve')\nplt.legend(loc='lower right')\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:49:41.408224Z","iopub.execute_input":"2025-05-27T10:49:41.409027Z","iopub.status.idle":"2025-05-27T10:49:41.562675Z","shell.execute_reply.started":"2025-05-27T10:49:41.408999Z","shell.execute_reply":"2025-05-27T10:49:41.561814Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Get recall, predicision, F1, accuracy scores for validation set\nrecall = recall_score(val_labels, binary_val_predictions)\nprecision = precision_score(val_labels, binary_val_predictions)\nF1 = f1_score(val_labels, binary_val_predictions)\nacc= accuracy_score(val_labels, binary_val_predictions)\n\nprint(\"Recall on validation set:\", recall)\nprint(\"Precision on validation set:\", precision)\nprint(\"F1 score n on validation set:\", F1)\nprint(\"Accuracy score n on validation set:\", acc)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:49:44.106696Z","iopub.execute_input":"2025-05-27T10:49:44.107023Z","iopub.status.idle":"2025-05-27T10:49:44.130021Z","shell.execute_reply.started":"2025-05-27T10:49:44.106995Z","shell.execute_reply":"2025-05-27T10:49:44.129208Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot confusion matrix\ncm = confusion_matrix(val_labels, binary_val_predictions)\ndisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Non-cancer', 'Cancer'])\ndisp.plot(cmap='Blues', values_format='d')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:49:48.723001Z","iopub.execute_input":"2025-05-27T10:49:48.723654Z","iopub.status.idle":"2025-05-27T10:49:48.864638Z","shell.execute_reply.started":"2025-05-27T10:49:48.723627Z","shell.execute_reply":"2025-05-27T10:49:48.863754Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# DenseNet121 - Model 3","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.applications import DenseNet121\nfrom tensorflow.keras.models import Model, Sequential\nfrom tensorflow.keras.layers import Conv2D, BatchNormalization, Dropout, Dense, GlobalAveragePooling2D\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping\nfrom tensorflow.keras.metrics import AUC\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\n\n# Load DenseNet121 with ImageNet weights\ndensenet_model = DenseNet121(weights='imagenet', include_top=False, input_shape=(96, 96, 3))\n\n# Unfreeze the last 30 layers for fine-tuning\nfor layer in densenet_model.layers[:-30]:\n layer.trainable = False\nfor layer in densenet_model.layers[-30:]:\n layer.trainable = True\n\n# Define base model\nlast_layer = densenet_model.get_layer('conv5_block16_2_conv') # Last dense block before global pooling\nbase_model = Model(inputs=densenet_model.input, outputs=last_layer.output)\n\n# Define the complete model\nmymodel_3 = Sequential([\n base_model,\n \n GlobalAveragePooling2D(), # Reduces parameters, acts like flattening\n\n Dense(512, activation='relu', kernel_regularizer='l2'),\n BatchNormalization(),\n Dropout(0.5),\n \n Dense(256, activation='relu', kernel_regularizer='l2'),\n BatchNormalization(),\n Dropout(0.4),\n \n Dense(128, activation='relu', kernel_regularizer='l2'),\n BatchNormalization(),\n Dropout(0.3),\n \n Dense(1, activation='sigmoid') # For binary classification\n])\n\n# Compile with dynamic LR adjustment\nmymodel_3.compile(\n optimizer=Adam(learning_rate=1e-3),\n loss='binary_crossentropy',\n metrics=['accuracy',AUC()]\n)\n\n# Callbacks for learning rate adjustment and early stopping\ncallbacks = [\n ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=2, verbose=1),\n EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True, verbose=1)\n]\n\n# Define aggressive data augmentation strategies\ndatagen = ImageDataGenerator(\n rescale=1./255,\n rotation_range=30,\n width_shift_range=0.2,\n height_shift_range=0.2,\n shear_range=0.2,\n zoom_range=0.2,\n horizontal_flip=True,\n fill_mode='nearest'\n)\n\ndatagen_val = ImageDataGenerator(rescale=1./255)\n\n# Assuming `train_dir` and `val_dir` are your directories\ntrain_generator = datagen.flow_from_directory(\n directory=train_dir,\n batch_size=128,\n class_mode='binary',\n target_size=(96, 96)\n)\n\nval_generator = datagen_val.flow_from_directory(\n directory=val_dir,\n batch_size=128,\n class_mode='binary',\n target_size=(96, 96)\n)\n\nsteps_per_epoch = len(train_generator)\nvalidation_steps = len(val_generator)\n\n# Fit the model\nhistory_3 = mymodel_3.fit(\n train_generator,\n validation_data=val_generator,\n epochs=5,\n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n callbacks=callbacks,\n verbose=1\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T10:49:58.583786Z","iopub.execute_input":"2025-05-27T10:49:58.584182Z","iopub.status.idle":"2025-05-27T11:33:26.78659Z","shell.execute_reply.started":"2025-05-27T10:49:58.584151Z","shell.execute_reply":"2025-05-27T11:33:26.785867Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(history_3.history.keys())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:34:32.639454Z","iopub.execute_input":"2025-05-27T11:34:32.640054Z","iopub.status.idle":"2025-05-27T11:34:32.644474Z","shell.execute_reply.started":"2025-05-27T11:34:32.640023Z","shell.execute_reply":"2025-05-27T11:34:32.643639Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Extract final values using correct keys\nfinal_loss = history_3.history['loss'][-1]\nfinal_accuracy = history_3.history['accuracy'][-1]\nfinal_auc = history_3.history['auc_3'][-1]\n\nfinal_val_loss = history_3.history['val_loss'][-1]\nfinal_val_accuracy = history_3.history['val_accuracy'][-1]\nfinal_val_auc = history_3.history['val_auc_3'][-1]\n\n# Print final metrics\nprint(\"\\n=== Final Training & Validation Metrics ===\")\nprint(f\"Train Loss: {final_loss:.4f}\")\nprint(f\"Train Accuracy: {final_accuracy:.4f}\")\nprint(f\"Train AUC: {final_auc:.4f}\")\nprint(f\"Validation Loss: {final_val_loss:.4f}\")\nprint(f\"Validation Accuracy: {final_val_accuracy:.4f}\")\nprint(f\"Validation AUC: {final_val_auc:.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:35:13.749068Z","iopub.execute_input":"2025-05-27T11:35:13.74987Z","iopub.status.idle":"2025-05-27T11:35:13.756049Z","shell.execute_reply.started":"2025-05-27T11:35:13.749842Z","shell.execute_reply":"2025-05-27T11:35:13.755235Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\n\nepochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(15, 4))\n\n# Accuracy Plot\nplt.subplot(1, 3, 1)\nplt.plot(epochs, history_3.history['accuracy'], label='Train Accuracy')\nplt.plot(epochs, history_3.history['val_accuracy'], label='Val Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Accuracy Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# AUC Plot\nplt.subplot(1, 3, 2)\nplt.plot(epochs, history_3.history['auc_3'], label='Train AUC')\nplt.plot(epochs, history_3.history['val_auc_3'], label='Val AUC')\nplt.xlabel('Epochs')\nplt.ylabel('AUC')\nplt.title('AUC Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss Plot\nplt.subplot(1, 3, 3)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:35:31.671059Z","iopub.execute_input":"2025-05-27T11:35:31.671444Z","iopub.status.idle":"2025-05-27T11:35:32.275512Z","shell.execute_reply.started":"2025-05-27T11:35:31.671415Z","shell.execute_reply":"2025-05-27T11:35:32.274642Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predict validation set \nsamples_val = len(os.listdir(os.path.join(val_dir, '1'))) + len(os.listdir(os.path.join(val_dir, '0')))\nval_generator_for_pred = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=1, \n class_mode='binary',\n target_size=(96,96),\n shuffle=False)\n\n\nval_predictions = mymodel_3.predict_generator(val_generator_for_pred,steps=samples_val, verbose=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:35:40.128484Z","iopub.execute_input":"2025-05-27T11:35:40.12907Z","iopub.status.idle":"2025-05-27T11:39:49.728416Z","shell.execute_reply.started":"2025-05-27T11:35:40.129038Z","shell.execute_reply":"2025-05-27T11:39:49.7276Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Actual vs. predicted classes for validation set\n\n# actual\nval_labels = val_generator_for_pred.classes\n\n# convert probability to classes using 0.5 shreshold\nbinary_val_predictions = (val_predictions > 0.5).astype(int)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:39:49.730145Z","iopub.execute_input":"2025-05-27T11:39:49.730449Z","iopub.status.idle":"2025-05-27T11:39:49.73487Z","shell.execute_reply.started":"2025-05-27T11:39:49.730412Z","shell.execute_reply":"2025-05-27T11:39:49.733941Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot ROC curve\n\n# Calculate AUC\nfpr, tpr, thresholds = roc_curve(val_labels, val_predictions)\nroc_auc = auc(fpr, tpr)\n\n# Plot \nplt.figure(figsize=(4, 4))\nplt.plot(fpr, tpr, color='orange', lw=2, label='ROC curve (AUC = {:.2f})'.format(roc_auc))\nplt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label='Random Guessing')\nplt.xlabel('FPR')\nplt.ylabel('TPR')\nplt.title('ROC Curve')\nplt.legend(loc='lower right')\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:39:51.298651Z","iopub.execute_input":"2025-05-27T11:39:51.299315Z","iopub.status.idle":"2025-05-27T11:39:51.458166Z","shell.execute_reply.started":"2025-05-27T11:39:51.29929Z","shell.execute_reply":"2025-05-27T11:39:51.4573Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Get recall, predicision, F1, accuracy scores for validation set\nrecall = recall_score(val_labels, binary_val_predictions)\nprecision = precision_score(val_labels, binary_val_predictions)\nF1 = f1_score(val_labels, binary_val_predictions)\nacc= accuracy_score(val_labels, binary_val_predictions)\n\nprint(\"Recall on validation set:\", recall)\nprint(\"Precision on validation set:\", precision)\nprint(\"F1 score n on validation set:\", F1)\nprint(\"Accuracy score n on validation set:\", acc)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:39:54.028745Z","iopub.execute_input":"2025-05-27T11:39:54.029405Z","iopub.status.idle":"2025-05-27T11:39:54.05217Z","shell.execute_reply.started":"2025-05-27T11:39:54.029364Z","shell.execute_reply":"2025-05-27T11:39:54.051301Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot confusion matrix\ncm = confusion_matrix(val_labels, binary_val_predictions)\ndisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Non-cancer', 'Cancer'])\ndisp.plot(cmap='Blues', values_format='d')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:39:55.085273Z","iopub.execute_input":"2025-05-27T11:39:55.085615Z","iopub.status.idle":"2025-05-27T11:39:55.234488Z","shell.execute_reply.started":"2025-05-27T11:39:55.085589Z","shell.execute_reply":"2025-05-27T11:39:55.233586Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# MobileNet","metadata":{}},{"cell_type":"code","source":"# Fit model \nBATCH_SIZE = 128\nEPOCHS = 5\n\n# data generator\ndatagen = ImageDataGenerator(\n rescale=1./255,\n rotation_range=20,\n width_shift_range=0.2,\n height_shift_range=0.2,\n vertical_flip=True,\n horizontal_flip=True,\n)\n\ndatagen_val = ImageDataGenerator(\n rescale=1./255,\n)\n\ntrain_generator = datagen.flow_from_directory(directory=train_dir,\n batch_size=BATCH_SIZE,\n class_mode='binary',\n target_size=(96, 96))\n\nval_generator = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=BATCH_SIZE, \n class_mode='binary',\n target_size=(96,96))\n\nsteps_per_epoch = len(train_generator) \nvalidation_steps = len(val_generator)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:40:01.395129Z","iopub.execute_input":"2025-05-27T11:40:01.396031Z","iopub.status.idle":"2025-05-27T11:40:04.33943Z","shell.execute_reply.started":"2025-05-27T11:40:01.396003Z","shell.execute_reply":"2025-05-27T11:40:04.338736Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from tensorflow.keras.applications import MobileNetV2\nfrom tensorflow.keras.models import Model, Sequential\nfrom tensorflow.keras.layers import GlobalAveragePooling2D, Dense, BatchNormalization, Dropout\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping\nfrom tensorflow.keras.metrics import AUC\n\n# Load MobileNetV2 with ImageNet weights\nmobilenet_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(96, 96, 3))\n\n# Freeze most layers and unfreeze the top few for fine-tuning\nfor layer in mobilenet_model.layers[:-20]:\n layer.trainable = False\nfor layer in mobilenet_model.layers[-20:]:\n layer.trainable = True\n\n# Define the base output\nlast_layer = mobilenet_model.output\n\n# Build the full model\nx = GlobalAveragePooling2D()(last_layer)\nx = Dense(256, activation='relu', kernel_regularizer='l2')(x)\nx = BatchNormalization()(x)\nx = Dropout(0.4)(x)\n\nx = Dense(128, activation='relu', kernel_regularizer='l2')(x)\nx = BatchNormalization()(x)\nx = Dropout(0.3)(x)\n\npredictions = Dense(1, activation='sigmoid')(x)\n\nmymodel = Model(inputs=mobilenet_model.input, outputs=predictions)\n\n# Compile the model\nmymodel_3.compile(\n optimizer=Adam(learning_rate=1e-3),\n loss='binary_crossentropy',\n metrics=['accuracy',AUC()]\n)\n\n# Define callbacks\ncallbacks = [\n ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=1, verbose=1),\n EarlyStopping(monitor='val_loss', patience=2, restore_best_weights=True, verbose=1)\n]\n\n# Fit the model\nhistory_3 = mymodel_3.fit(\n train_generator,\n validation_data=val_generator,\n epochs=5,\n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n callbacks=callbacks,\n verbose=1\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T11:40:39.863187Z","iopub.execute_input":"2025-05-27T11:40:39.86398Z","iopub.status.idle":"2025-05-27T12:23:56.521975Z","shell.execute_reply.started":"2025-05-27T11:40:39.863952Z","shell.execute_reply":"2025-05-27T12:23:56.521369Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(history_3.history.keys())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:24:02.330915Z","iopub.execute_input":"2025-05-27T12:24:02.331245Z","iopub.status.idle":"2025-05-27T12:24:02.335755Z","shell.execute_reply.started":"2025-05-27T12:24:02.331217Z","shell.execute_reply":"2025-05-27T12:24:02.334924Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"final_loss = history_3.history['loss'][-1]\nfinal_accuracy = history_3.history['accuracy'][-1]\nfinal_auc = history_3.history['auc_4'][-1] # changed key\n\nfinal_val_loss = history_3.history['val_loss'][-1]\nfinal_val_accuracy = history_3.history['val_accuracy'][-1]\nfinal_val_auc = history_3.history['val_auc_4'][-1] # changed key\n\n# Print final metrics\nprint(\"\\n=== Final Training & Validation Metrics ===\")\nprint(f\"Train Loss: {final_loss:.4f}\")\nprint(f\"Train Accuracy: {final_accuracy:.4f}\")\nprint(f\"Train AUC: {final_auc:.4f}\")\nprint(f\"Validation Loss: {final_val_loss:.4f}\")\nprint(f\"Validation Accuracy: {final_val_accuracy:.4f}\")\nprint(f\"Validation AUC: {final_val_auc:.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:24:55.868156Z","iopub.execute_input":"2025-05-27T12:24:55.86849Z","iopub.status.idle":"2025-05-27T12:24:55.874746Z","shell.execute_reply.started":"2025-05-27T12:24:55.868464Z","shell.execute_reply":"2025-05-27T12:24:55.873935Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"epochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(15, 4))\n\n# Accuracy Plot\nplt.subplot(1, 3, 1)\nplt.plot(epochs, history_3.history['accuracy'], label='Train Accuracy')\nplt.plot(epochs, history_3.history['val_accuracy'], label='Val Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Accuracy Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# AUC Plot\nplt.subplot(1, 3, 2)\nplt.plot(epochs, history_3.history['auc_4'], label='Train AUC') # changed key\nplt.plot(epochs, history_3.history['val_auc_4'], label='Val AUC') # changed key\nplt.xlabel('Epochs')\nplt.ylabel('AUC')\nplt.title('AUC Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss Plot\nplt.subplot(1, 3, 3)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:25:08.791306Z","iopub.execute_input":"2025-05-27T12:25:08.791669Z","iopub.status.idle":"2025-05-27T12:25:09.401565Z","shell.execute_reply.started":"2025-05-27T12:25:08.791641Z","shell.execute_reply":"2025-05-27T12:25:09.400666Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predict validation set \nsamples_val = len(os.listdir(os.path.join(val_dir, '1'))) + len(os.listdir(os.path.join(val_dir, '0')))\nval_generator_for_pred = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=1, \n class_mode='binary',\n target_size=(96,96),\n shuffle=False)\n\n\nval_predictions = mymodel_3.predict_generator(val_generator_for_pred,steps=samples_val, verbose=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:25:16.02434Z","iopub.execute_input":"2025-05-27T12:25:16.02471Z","iopub.status.idle":"2025-05-27T12:29:20.618561Z","shell.execute_reply.started":"2025-05-27T12:25:16.024683Z","shell.execute_reply":"2025-05-27T12:29:20.617726Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Actual vs. predicted classes for validation set\n\n# actual\nval_labels = val_generator_for_pred.classes\n\n# convert probability to classes using 0.5 shreshold\nbinary_val_predictions = (val_predictions > 0.5).astype(int)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:29:24.158508Z","iopub.execute_input":"2025-05-27T12:29:24.158824Z","iopub.status.idle":"2025-05-27T12:29:24.163601Z","shell.execute_reply.started":"2025-05-27T12:29:24.1588Z","shell.execute_reply":"2025-05-27T12:29:24.162662Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot ROC curve\n\n# Calculate AUC\nfpr, tpr, thresholds = roc_curve(val_labels, binary_val_predictions)\nroc_auc = auc(fpr, tpr)\n\n# Plot \nplt.figure(figsize=(4, 4))\nplt.plot(fpr, tpr, color='orange', lw=2, label='ROC curve (AUC = {:.2f})'.format(roc_auc))\nplt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label='Random Guessing')\nplt.xlabel('FPR')\nplt.ylabel('TPR')\nplt.title('ROC Curve')\nplt.legend(loc='lower right')\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:29:30.635364Z","iopub.execute_input":"2025-05-27T12:29:30.636007Z","iopub.status.idle":"2025-05-27T12:29:30.789725Z","shell.execute_reply.started":"2025-05-27T12:29:30.635982Z","shell.execute_reply":"2025-05-27T12:29:30.788855Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Get recall, predicision, F1, accuracy scores for validation set\nrecall = recall_score(val_labels, binary_val_predictions)\nprecision = precision_score(val_labels, binary_val_predictions)\nF1 = f1_score(val_labels, binary_val_predictions)\nacc= accuracy_score(val_labels, binary_val_predictions)\n\nprint(\"Recall on validation set:\", recall)\nprint(\"Precision on validation set:\", precision)\nprint(\"F1 score n on validation set:\", F1)\nprint(\"Accuracy score n on validation set:\", acc)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:29:34.14133Z","iopub.execute_input":"2025-05-27T12:29:34.142104Z","iopub.status.idle":"2025-05-27T12:29:34.167161Z","shell.execute_reply.started":"2025-05-27T12:29:34.142071Z","shell.execute_reply":"2025-05-27T12:29:34.166276Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot confusion matrix\ncm = confusion_matrix(val_labels, binary_val_predictions)\ndisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Non-cancer', 'Cancer'])\ndisp.plot(cmap='Blues', values_format='d')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:29:36.211037Z","iopub.execute_input":"2025-05-27T12:29:36.211328Z","iopub.status.idle":"2025-05-27T12:29:36.357635Z","shell.execute_reply.started":"2025-05-27T12:29:36.211306Z","shell.execute_reply":"2025-05-27T12:29:36.35676Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Inception without Data Augmentation","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.applications import InceptionV3\nfrom tensorflow.keras.models import Model, Sequential\nfrom tensorflow.keras.layers import Conv2D, Flatten, Dense, BatchNormalization, Dropout\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping\nfrom tensorflow.keras.metrics import AUC\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.optimizers.schedules import ExponentialDecay\n\n# Load the InceptionV3 model\nincept_model = InceptionV3(weights='imagenet', include_top=False, input_shape=(96, 96, 3))\n\n# Freeze the weights of the layers.\nfor layer in incept_model.layers:\n layer.trainable = False\n\n# get the last_layer of inception we want to use\nlast_layer = incept_model.get_layer('mixed2')\nprint('last layer output shape: ', last_layer.output_shape)\n\n# Create a new model using the functional API\nbase_model = Model(inputs=incept_model.input, outputs=last_layer.output)\n\n# Add layers to the model\nmymodel_3 = Sequential([\n base_model,\n \n Conv2D(64, (3, 3), padding='same', activation='relu'),\n BatchNormalization(),\n Conv2D(64, (1, 1), activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n \n Conv2D(128, (3, 3), activation='relu'),\n BatchNormalization(), \n Conv2D(64, (1, 1), activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n\n Flatten(),\n Dense(128, activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n \n Dense(128, activation='relu'),\n BatchNormalization(),\n Dropout(0.2),\n \n Dense(1, activation='sigmoid')\n])\n\n# Print the model summary\nmymodel_3.build(input_shape=(None, 96, 96, 3))\nmymodel_3.summary()\n\n# Define the learning rate schedule\ninitial_learning_rate = 0.001\ndecay_steps = 1000\ndecay_rate = 0.9 \nlr_schedule = ExponentialDecay(initial_learning_rate, decay_steps, decay_rate, staircase=True)\n\n# Compile the model\nmymodel_3.compile(\n optimizer=Adam(learning_rate=lr_schedule),\n loss='binary_crossentropy',\n metrics=[AUC()]\n)\n\n# Training config\nBATCH_SIZE = 128\nEPOCHS = 5\n\n# ImageDataGenerators without augmentation\ndatagen = ImageDataGenerator(rescale=1./255)\ndatagen_val = ImageDataGenerator(rescale=1./255)\n\ntrain_generator = datagen.flow_from_directory(directory=train_dir,\n batch_size=BATCH_SIZE,\n class_mode='binary',\n target_size=(96, 96))\n\nval_generator = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=BATCH_SIZE, \n class_mode='binary',\n target_size=(96, 96))\n\nsteps_per_epoch = len(train_generator)\nvalidation_steps = len(val_generator)\n\n# Fit model 3\nhistory_3 = mymodel_3.fit(\n train_generator,\n validation_data=val_generator,\n epochs=EPOCHS,\n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n verbose=1\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:33:05.329727Z","iopub.execute_input":"2025-05-04T15:33:05.330388Z","iopub.status.idle":"2025-05-04T15:45:25.305645Z","shell.execute_reply.started":"2025-05-04T15:33:05.330359Z","shell.execute_reply":"2025-05-04T15:45:25.304848Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(history_3.history.keys())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:45:31.543103Z","iopub.execute_input":"2025-05-04T15:45:31.543875Z","iopub.status.idle":"2025-05-04T15:45:31.548127Z","shell.execute_reply.started":"2025-05-04T15:45:31.543845Z","shell.execute_reply":"2025-05-04T15:45:31.547289Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot metrics\nepochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(10, 4))\n\n# AUC plot\nplt.subplot(1, 2, 1)\nplt.plot(epochs, history_3.history['auc_1'], label='Train AUC')\nplt.plot(epochs, history_3.history['val_auc_1'], label='Val AUC')\nplt.xlabel('Epochs')\nplt.ylabel('AUC')\nplt.title('AUC Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss plot\nplt.subplot(1, 2, 2)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:45:55.501277Z","iopub.execute_input":"2025-05-04T15:45:55.501608Z","iopub.status.idle":"2025-05-04T15:45:55.897749Z","shell.execute_reply.started":"2025-05-04T15:45:55.501586Z","shell.execute_reply":"2025-05-04T15:45:55.896905Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predict validation set \nsamples_val = len(os.listdir(os.path.join(val_dir, '1'))) + len(os.listdir(os.path.join(val_dir, '0')))\nval_generator_for_pred = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=1, \n class_mode='binary',\n target_size=(96,96),\n shuffle=False)\n\n\nval_predictions = mymodel_3.predict_generator(val_generator_for_pred,steps=samples_val, verbose=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:45:58.682531Z","iopub.execute_input":"2025-05-04T15:45:58.68322Z","iopub.status.idle":"2025-05-04T15:47:34.019467Z","shell.execute_reply.started":"2025-05-04T15:45:58.683192Z","shell.execute_reply":"2025-05-04T15:47:34.018657Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Actual vs. predicted classes for validation set\n\n# actual\nval_labels = val_generator_for_pred.classes\n\n# convert probability to classes using 0.5 shreshold\nbinary_val_predictions = (val_predictions > 0.5).astype(int)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:47:39.756255Z","iopub.execute_input":"2025-05-04T15:47:39.756671Z","iopub.status.idle":"2025-05-04T15:47:39.761522Z","shell.execute_reply.started":"2025-05-04T15:47:39.756649Z","shell.execute_reply":"2025-05-04T15:47:39.760455Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot ROC curve\n\n# Calculate AUC\nfpr, tpr, thresholds = roc_curve(val_labels, binary_val_predictions)\nroc_auc = auc(fpr, tpr)\n\n# Plot \nplt.figure(figsize=(4, 4))\nplt.plot(fpr, tpr, color='orange', lw=2, label='ROC curve (AUC = {:.2f})'.format(roc_auc))\nplt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label='Random Guessing')\nplt.xlabel('FPR')\nplt.ylabel('TPR')\nplt.title('ROC Curve')\nplt.legend(loc='lower right')\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:47:42.19174Z","iopub.execute_input":"2025-05-04T15:47:42.192444Z","iopub.status.idle":"2025-05-04T15:47:42.350364Z","shell.execute_reply.started":"2025-05-04T15:47:42.192416Z","shell.execute_reply":"2025-05-04T15:47:42.349359Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Get recall, predicision, F1, accuracy scores for validation set\nrecall = recall_score(val_labels, binary_val_predictions)\nprecision = precision_score(val_labels, binary_val_predictions)\nF1 = f1_score(val_labels, binary_val_predictions)\nacc= accuracy_score(val_labels, binary_val_predictions)\n\nprint(\"Recall on validation set:\", recall)\nprint(\"Precision on validation set:\", precision)\nprint(\"F1 score n on validation set:\", F1)\nprint(\"Accuracy score n on validation set:\", acc)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:47:46.240677Z","iopub.execute_input":"2025-05-04T15:47:46.241501Z","iopub.status.idle":"2025-05-04T15:47:46.263646Z","shell.execute_reply.started":"2025-05-04T15:47:46.241471Z","shell.execute_reply":"2025-05-04T15:47:46.262801Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot confusion matrix\ncm = confusion_matrix(val_labels, binary_val_predictions)\ndisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Non-cancer', 'Cancer'])\ndisp.plot(cmap='Blues', values_format='d')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-04T15:48:31.171771Z","iopub.execute_input":"2025-05-04T15:48:31.172444Z","iopub.status.idle":"2025-05-04T15:48:31.319459Z","shell.execute_reply.started":"2025-05-04T15:48:31.172418Z","shell.execute_reply":"2025-05-04T15:48:31.318553Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# VGG16","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.applications import VGG16\nfrom tensorflow.keras.models import Model\nfrom tensorflow.keras.layers import GlobalAveragePooling2D, Dense, BatchNormalization, Dropout\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping\nfrom tensorflow.keras.metrics import AUC\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\n\n# Load VGG16 with ImageNet weights\nvgg_model = VGG16(weights='imagenet', include_top=False, input_shape=(96, 96, 3))\n\n# Freeze most layers and fine-tune last few\nfor layer in vgg_model.layers[:-10]:\n layer.trainable = False\nfor layer in vgg_model.layers[-10:]:\n layer.trainable = True\n\n# Define the output layer from base model\nx = vgg_model.output\nx = GlobalAveragePooling2D()(x)\n\n# Add custom layers\nx = Dense(256, activation='relu', kernel_regularizer='l2')(x)\nx = BatchNormalization()(x)\nx = Dropout(0.4)(x)\n\nx = Dense(128, activation='relu', kernel_regularizer='l2')(x)\nx = BatchNormalization()(x)\nx = Dropout(0.3)(x)\n\npredictions = Dense(1, activation='sigmoid')(x)\n\n# Build final model\nmymodel_3 = Model(inputs=vgg_model.input, outputs=predictions)\n\n# Compile\nmymodel_3.compile(\n optimizer=Adam(learning_rate=1e-3),\n loss='binary_crossentropy',\n metrics=['accuracy',AUC()]\n)\n\n# Callbacks\ncallbacks = [\n ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=1, verbose=1),\n EarlyStopping(monitor='val_loss', patience=2, restore_best_weights=True, verbose=1)\n]\n\n# Train the model\nhistory_3 = mymodel_3.fit(\n train_generator,\n validation_data=val_generator,\n epochs=5,\n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n callbacks=callbacks,\n verbose=1\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T12:30:08.681802Z","iopub.execute_input":"2025-05-27T12:30:08.682721Z","iopub.status.idle":"2025-05-27T13:12:03.148678Z","shell.execute_reply.started":"2025-05-27T12:30:08.68269Z","shell.execute_reply":"2025-05-27T13:12:03.147799Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(history_3.history.keys())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:12:06.745769Z","iopub.execute_input":"2025-05-27T13:12:06.746417Z","iopub.status.idle":"2025-05-27T13:12:06.750672Z","shell.execute_reply.started":"2025-05-27T13:12:06.746387Z","shell.execute_reply":"2025-05-27T13:12:06.749886Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Extract final values from training history\nfinal_loss = history_3.history['loss'][-1]\nfinal_accuracy = history_3.history['accuracy'][-1]\nfinal_auc = history_3.history['auc_6'][-1]\n\nfinal_val_loss = history_3.history['val_loss'][-1]\nfinal_val_accuracy = history_3.history['val_accuracy'][-1]\nfinal_val_auc = history_3.history['val_auc_6'][-1]\n\n# Print final metrics\nprint(\"\\n=== Final Training & Validation Metrics ===\")\nprint(f\"Train Loss: {final_loss:.4f}\")\nprint(f\"Train Accuracy: {final_accuracy:.4f}\")\nprint(f\"Train AUC: {final_auc:.4f}\")\nprint(f\"Validation Loss: {final_val_loss:.4f}\")\nprint(f\"Validation Accuracy: {final_val_accuracy:.4f}\")\nprint(f\"Validation AUC: {final_val_auc:.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:13:11.905039Z","iopub.execute_input":"2025-05-27T13:13:11.905407Z","iopub.status.idle":"2025-05-27T13:13:11.911783Z","shell.execute_reply.started":"2025-05-27T13:13:11.905361Z","shell.execute_reply":"2025-05-27T13:13:11.910923Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"epochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(15, 4))\n\n# Accuracy Plotepochs = np.arange(1, len(history_3.history['loss']) + 1)\nplt.figure(figsize=(15, 4))\n\n# Accuracy Plot\nplt.subplot(1, 3, 1)\nplt.plot(epochs, history_3.history['accuracy'], label='Train Accuracy')\nplt.plot(epochs, history_3.history['val_accuracy'], label='Val Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Accuracy Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# AUC Plot\nplt.subplot(1, 3, 2)\nplt.plot(epochs, history_3.history['auc_6'], label='Train AUC')\nplt.plot(epochs, history_3.history['val_auc_6'], label='Val AUC')\nplt.xlabel('Epochs')\nplt.ylabel('AUC')\nplt.title('AUC Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss Plot\nplt.subplot(1, 3, 3)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()\n\nplt.subplot(1, 3, 1)\nplt.plot(epochs, history_3.history['accuracy'], label='Train Accuracy')\nplt.plot(epochs, history_3.history['val_accuracy'], label='Val Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Accuracy Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# AUC Plot\nplt.subplot(1, 3, 2)\nplt.plot(epochs, history_3.history['auc_6'], label='Train AUC')\nplt.plot(epochs, history_3.history['val_auc_6'], label='Val AUC')\nplt.xlabel('Epochs')\nplt.ylabel('AUC')\nplt.title('AUC Over Epochs')\nplt.legend()\nplt.grid(True)\n\n# Loss Plot\nplt.subplot(1, 3, 3)\nplt.plot(epochs, history_3.history['loss'], label='Train Loss')\nplt.plot(epochs, history_3.history['val_loss'], label='Val Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.title('Loss Over Epochs')\nplt.legend()\nplt.grid(True)\n\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:14:14.001201Z","iopub.execute_input":"2025-05-27T13:14:14.001558Z","iopub.status.idle":"2025-05-27T13:14:15.043599Z","shell.execute_reply.started":"2025-05-27T13:14:14.001531Z","shell.execute_reply":"2025-05-27T13:14:15.042801Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predict validation set \nsamples_val = len(os.listdir(os.path.join(val_dir, '1'))) + len(os.listdir(os.path.join(val_dir, '0')))\nval_generator_for_pred = datagen_val.flow_from_directory(directory=val_dir, \n batch_size=1, \n class_mode='binary',\n target_size=(96,96),\n shuffle=False)\n\n\nval_predictions = mymodel_3.predict_generator(val_generator_for_pred,steps=samples_val, verbose=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:14:18.172771Z","iopub.execute_input":"2025-05-27T13:14:18.173098Z","iopub.status.idle":"2025-05-27T13:15:38.255255Z","shell.execute_reply.started":"2025-05-27T13:14:18.173066Z","shell.execute_reply":"2025-05-27T13:15:38.254515Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Actual vs. predicted classes for validation set\n\n# actual\nval_labels = val_generator_for_pred.classes\n\n# convert probability to classes using 0.5 shreshold\nbinary_val_predictions = (val_predictions > 0.5).astype(int)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:15:47.873433Z","iopub.execute_input":"2025-05-27T13:15:47.873758Z","iopub.status.idle":"2025-05-27T13:15:47.878284Z","shell.execute_reply.started":"2025-05-27T13:15:47.873733Z","shell.execute_reply":"2025-05-27T13:15:47.877423Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot ROC curve\n\n# Calculate AUC\nfpr, tpr, thresholds = roc_curve(val_labels, binary_val_predictions)\nroc_auc = auc(fpr, tpr)\n\n# Plot \nplt.figure(figsize=(4, 4))\nplt.plot(fpr, tpr, color='orange', lw=2, label='ROC curve (AUC = {:.2f})'.format(roc_auc))\nplt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label='Random Guessing')\nplt.xlabel('FPR')\nplt.ylabel('TPR')\nplt.title('ROC Curve')\nplt.legend(loc='lower right')\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:15:59.095839Z","iopub.execute_input":"2025-05-27T13:15:59.09652Z","iopub.status.idle":"2025-05-27T13:15:59.245694Z","shell.execute_reply.started":"2025-05-27T13:15:59.096491Z","shell.execute_reply":"2025-05-27T13:15:59.244759Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Get recall, predicision, F1, accuracy scores for validation set\nrecall = recall_score(val_labels, binary_val_predictions)\nprecision = precision_score(val_labels, binary_val_predictions)\nF1 = f1_score(val_labels, binary_val_predictions)\nacc= accuracy_score(val_labels, binary_val_predictions)\n\nprint(\"Recall on validation set:\", recall)\nprint(\"Precision on validation set:\", precision)\nprint(\"F1 score n on validation set:\", F1)\nprint(\"Accuracy score n on validation set:\", acc)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:16:02.064646Z","iopub.execute_input":"2025-05-27T13:16:02.06545Z","iopub.status.idle":"2025-05-27T13:16:02.087077Z","shell.execute_reply.started":"2025-05-27T13:16:02.065411Z","shell.execute_reply":"2025-05-27T13:16:02.086345Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Plot confusion matrix\ncm = confusion_matrix(val_labels, binary_val_predictions)\ndisp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Non-cancer', 'Cancer'])\ndisp.plot(cmap='Blues', values_format='d')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:16:13.181957Z","iopub.execute_input":"2025-05-27T13:16:13.182587Z","iopub.status.idle":"2025-05-27T13:16:13.321278Z","shell.execute_reply.started":"2025-05-27T13:16:13.182557Z","shell.execute_reply":"2025-05-27T13:16:13.320409Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Updated model names\nmodels = ['InceptionV3', 'Xception', 'DenseNet121', 'MobileNet', 'VGG16']\n\n# Corresponding accuracy scores from validation\naccuracies = [\n 0.9215, # InceptionV3\n 0.9031, # Xception\n 0.8795, # DenseNet121\n 0.9253, # MobileNet\n 0.9077 # VGG16\n]\n\n# Plotting accuracy\nplt.figure(figsize=(10, 6))\nbars = plt.bar(models, accuracies, color='skyblue')\n\n# Add labels\nfor bar, acc in zip(bars, accuracies):\n plt.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.005, f'{acc:.4f}', \n ha='center', va='bottom', fontsize=10)\n\nplt.ylim(0.85, 0.95)\nplt.title('Model Validation Accuracy Comparison', fontsize=16)\nplt.ylabel('Accuracy', fontsize=12)\nplt.xlabel('Model', fontsize=12)\nplt.grid(axis='y', linestyle='--', alpha=0.7)\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:20:00.729214Z","iopub.execute_input":"2025-05-27T13:20:00.729561Z","iopub.status.idle":"2025-05-27T13:20:00.90502Z","shell.execute_reply.started":"2025-05-27T13:20:00.729537Z","shell.execute_reply":"2025-05-27T13:20:00.904152Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Models\nmodels = ['InceptionV3', 'Xception', 'DenseNet121', 'MobileNet', 'VGG16']\n\n# Recall, Precision, and F1-score values from provided metrics\nrecall_scores = [0.8705, 0.9299, 0.9438, 0.9222, 0.9892]\nprecision_scores = [0.9315, 0.8465, 0.7967, 0.8968, 0.4316]\nf1_scores = [0.8999, 0.8862, 0.8640, 0.9093, 0.6010]\n\n# Plot Recall\nplt.figure(figsize=(8, 5))\nplt.bar(models, recall_scores, color='skyblue')\nplt.title('Recall on Validation Set')\nplt.ylabel('Recall')\nplt.ylim(0.8, 1.0)\nplt.xticks(rotation=45)\nplt.grid(axis='y')\nplt.tight_layout()\nplt.show()\n\n# Plot Precision\nplt.figure(figsize=(8, 5))\nplt.bar(models, precision_scores, color='salmon')\nplt.title('Precision on Validation Set')\nplt.ylabel('Precision')\nplt.ylim(0.4, 1.0)\nplt.xticks(rotation=45)\nplt.grid(axis='y')\nplt.tight_layout()\nplt.show()\n\n# Plot F1-Score\nplt.figure(figsize=(8, 5))\nplt.bar(models, f1_scores, color='mediumseagreen')\nplt.title('F1 Score on Validation Set')\nplt.ylabel('F1 Score')\nplt.ylim(0.5, 1.0)\nplt.xticks(rotation=45)\nplt.grid(axis='y')\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-27T13:20:14.297283Z","iopub.execute_input":"2025-05-27T13:20:14.29762Z","iopub.status.idle":"2025-05-27T13:20:14.794978Z","shell.execute_reply.started":"2025-05-27T13:20:14.297594Z","shell.execute_reply":"2025-05-27T13:20:14.794116Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Methodology used","metadata":{}},{"cell_type":"markdown","source":"\"\"\" Model 1 โ InceptionV3 + Custom Conv Head\nMethodology:\nBase Model: Pretrained InceptionV3 (ImageNet weights) with include_top=False.\n\nLayer Freezing: All layers are frozen (feature extractor only).\n\nCut-off Layer: Model output taken from mixed2 layer (intermediate layer).\n\nArchitecture:\n\nSeveral custom Conv2D โ BatchNorm โ Dropout layers.\n\nFollowed by Flatten, multiple Dense layers with BatchNorm, and dropout.\n\nEnds with a Dense(1, activation='sigmoid') for binary classification.\n\nNo fine-tuning of base model.\n\nPurpose: Leverage early features from InceptionV3 and add a deep CNN head for learning.\n\nโ
Model 2 โ Xception + GlobalAveragePooling Head\nMethodology:\nBase Model: Xception pretrained on ImageNet, include_top=False.\n\nFine-Tuning: Top 20 layers of the model are unfrozen and trained.\n\nCut-off Layer: block14_sepconv2_act (final convolutional activation layer).\n\nArchitecture:\n\nGlobalAveragePooling2D instead of flattening.\n\nTwo Dense layers with L2 regularization, BatchNorm, and Dropout.\n\nFinal sigmoid output for binary classification.\n\nPurpose: Lighter model with fewer parameters using GAP, leveraging deeper pretrained features.\n\nโ
Model 3 โ DenseNet121 + Deep Dense Head\nMethodology:\nBase Model: DenseNet121 pretrained with include_top=False.\n\nFine-Tuning: Last 30 layers unfrozen for fine-tuning.\n\nCut-off Layer: conv5_block16_2_conv (last convolution block).\n\nArchitecture:\n\nGlobalAveragePooling2D after base.\n\n3 Dense layers (512, 256, 128) with L2, BatchNorm, and Dropout.\n\nFinal sigmoid classification layer.\n\nData Augmentation: Extensive real-time augmentation used.\n\nPurpose: Use DenseNetโs deep features with an extensive classifier on top.\n\nโ
Model 4 โ MobileNetV2 + Lightweight Dense Head\nMethodology:\nBase Model: MobileNetV2 with include_top=False, pretrained on ImageNet.\n\nFine-Tuning: Last 20 layers unfrozen.\n\nCut-off Layer: Output layer of base model.\n\nArchitecture:\n\nGlobalAveragePooling2D\n\nTwo Dense layers with L2, BatchNorm, Dropout.\n\nFinal sigmoid output layer.\n\nData Augmentation: Moderate augmentation with both horizontal and vertical flips.\n\nPurpose: Lightweight model for faster training and inference with efficient mobile architecture.\n\nโ
Model 5 โ VGG16 + GAP & Dense Classifier\nMethodology:\nBase Model: VGG16, pretrained, include_top=False.\n\nFine-Tuning: Last 10 layers unfrozen for training.\n\nCut-off Layer: Last layer of base model (block5).\n\nArchitecture:\n\nGlobalAveragePooling2D\n\nTwo Dense layers with L2, BatchNorm, and Dropout.\n\nFinal sigmoid output layer.\n\nPurpose: Simple architecture with strong early-layer feature maps and classic classifier head.\"\"\"\n\n","metadata":{}},{"cell_type":"markdown","source":"# Differences in Methodology Across the Models\nBase Architecture Used:\n\nEach model uses a different pretrained CNN:\nInceptionV3, Xception, DenseNet121, MobileNetV2, and VGG16, all pretrained on ImageNet.\n\nLayer Freezing & Fine-Tuning Strategy:\n\nModel 1 (InceptionV3): All layers frozen (no fine-tuning).\n\nModels 2โ5: Partial fine-tuning by unfreezing the top N layers:\n\nXception: last 20 layers\n\nDenseNet121: last 30 layers\n\nMobileNetV2: last 20 layers\n\nVGG16: last 10 layers\n\nOutput Layer from Base Model:\n\nModel 1 taps into an intermediate layer (mixed2) for feature extraction.\n\nModels 2โ5 use the final output of the base CNN or near-final layers for better abstracted features.\n\nType of Classifier Head:\n\nModel 1: Uses multiple Conv2D + BatchNorm + Dropout layers before flattening and dense layers (custom CNN head).\n\nModels 2โ5: Use GlobalAveragePooling2D followed by stacked Dense layers (simpler, parameter-efficient).\n\nClassifier Depth and Regularization:\n\nModel 1: Deeper custom head, no L2 regularization.\n\nModels 2โ5: Shallower or moderately deep heads with L2 regularization on dense layers to combat overfitting.\n\nData Augmentation Usage:\n\nModel 3 (DenseNet121) and Model 4 (MobileNetV2): Use aggressive or moderate real-time image augmentation.\n\nOthers: Not explicitly shown or lighter augmentation assumed.\n\nModel Compilation & Callbacks:\n\nAll models use Adam optimizer, binary_crossentropy loss, and AUC as a metric.\n\nAll incorporate ReduceLROnPlateau and EarlyStopping, but patience levels vary slightly.\n\nThese differences reflect trade-offs in model complexity, training time, generalization ability, and resource efficiency, with deeper heads like in Model 1 favoring learning capacity and lighter ones (Model 4) favoring speed and deployment feasibility.\n\n\n\n\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"# ๐ Thesis Title\n\"Evaluating the Performance of Deep Learning Models on Imbalanced Datasets for Novel Disease Detection\"\n\n๐ก Importance of the Thesis\nDetecting new or rare diseases presents a major challenge in medical AI due to the imbalance in available data. Typically, there are far fewer positive cases (patients with the disease) than negative ones (healthy or other conditions). This class imbalance causes most traditional models to favor the majority class, leading to false security and missed diagnoses.\n\nThis thesis aims to analyze the effectiveness of five state-of-the-art deep learning models in handling such imbalanced medical data, to ensure early and accurate detection of new diseases.\n\n๐ Why Accuracy Isnโt Enough\nIn imbalanced datasets, accuracy can be misleading. A model predicting only the majority class can still score high on accuracy. For this reason, other metrics are far more important:\n\nMetric\tImportance in Imbalanced Datasets\nPrecision\tProportion of predicted positive cases that are actually correct. High precision reduces false alarms.\nRecall\tProportion of actual positive cases that are correctly identified. Crucial for catching real cases.\nF1 Score\tHarmonic mean of precision and recall. A balanced metric for imbalanced datasets.\nAUC\tMeasures overall ability to distinguish between classes at all thresholds. Higher is bett","metadata":{}},{"cell_type":"markdown","source":"| Model | Accuracy (%) | Precision | Recall | F1 Score | AUC | Validation Loss |\n| --------------- | ------------ | --------- | --------- | --------- | ---------- | --------------- |\n| **InceptionV3** | 92.15 | 0.931 | 0.870 | 0.899 | 0.9729 | 0.2006 |\n| **Xception** | 90.31 | 0.847 | 0.930 | 0.886 | 0.9711 | 0.2406 |\n| **DenseNet121** | 87.95 | 0.797 | 0.944 | 0.864 | 0.9664 | 0.3218 |\n| **MobileNetV2** | **92.53** | 0.897 | 0.922 | **0.909** | **0.9759** | 0.2089 |\n| **VGG16** | 46.69 | 0.432 | **0.989** | 0.601 | 0.9626 | 0.2477 |\n","metadata":{}},{"cell_type":"markdown","source":"# Best Performing Model: MobileNetV2\nโ
Reasons for Superior Performance:\nHighest F1 Score (0.909): Balanced performance for both precision and recall.\n\nHigh Recall (0.922): Captures a large number of actual disease cases.\n\nGood Precision (0.897): Ensures most flagged cases are truly positive.\n\nHighest AUC (0.9759): Best overall discrimination ability.\n\nLow Validation Loss (0.2089): Indicates better generalization and less overfitting.\n\n๐งฑ Architectural Advantage:\nDepthwise Separable Convolutions: Efficiently reduce parameters and overfitting.\n\nLightweight and Fast: MobileNet is designed to perform well with fewer computations.\n\nTransfer Learning: Fine-tuning of the top 20 layers allows adaptation to disease data without losing general features.\n\nโ Model Limitations โ Why Others Didnโt Win\nInceptionV3: Excellent precision, but lower recall than MobileNet โ may miss some actual disease cases.\n\nXception: High recall but compromises on precision โ more false positives.\n\nDenseNet121: Slight overfitting visible in higher validation loss.\n\nVGG16: Extremely high recall but terrible precision โ predicts nearly everything as disease, causing too many false alarms.\n\n๐ฌ Final Takeaway\nThis thesis highlights that MobileNetV2 is best suited for detecting new diseases from imbalanced datasets due to its architecture and balanced metric performance. The findings stress that in sensitive applications like healthcare, metrics like F1 Score, Recall, and AUC are more important than mere accuracy.\n\n","metadata":{}},{"cell_type":"markdown","source":"# Results","metadata":{}},{"cell_type":"markdown","source":"#Inception V3\n=== Final Training & Validation Metrics ===\nTrain Loss: 0.2252\nTrain Accuracy: 0.9101\nTrain AUC: 0.9658\nValidation Loss: 0.2006\nValidation Accuracy: 0.9215\nValidation AUC: 0.9729\n\nRecall on validation set: 0.8704785441473826\nPrecision on validation set: 0.9314821492967905\nF1 score n on validation set: 0.8999477382265839\nAccuracy score n on validation set: 0.9214640594375313\n\nXception Model\nTrain Loss: 0.2260\nTrain Accuracy: 0.9119\nTrain AUC: 0.9678\nValidation Loss: 0.2406\nValidation Accuracy: 0.9031\nValidation AUC: 0.9711\nFinal Learning Rate: 0.000500\n\nRecall on validation set: 0.9299033924960683\nPrecision on validation set: 0.8465078228857756\nF1 score n on validation set: 0.8862480595257214\nAccuracy score n on validation set: 0.9031405260039199(Prediction accuracy)\n\nDenseNet121\n=== Final Training & Validation Metrics ===\nTrain Loss: 0.2705\nTrain Accuracy: 0.8990\nTrain AUC: 0.9582\nValidation Loss: 0.3218\nValidation Accuracy: 0.8795\nValidation AUC: 0.9664\n\nRecall on validation set: 0.9438328465513368\nPrecision on validation set: 0.7967001706808269\nF1 score n on validation set: 0.8640477169888934\nAccuracy score n on validation set: 0.8794840238844067\n\nMobileNet\n=== Final Training & Validation Metrics ===\nTrain Loss: 0.2256\nTrain Accuracy: 0.9161\nTrain AUC: 0.9692\nValidation Loss: 0.2089\nValidation Accuracy: 0.9253\nValidation AUC: 0.9759\n\nRecall on validation set: 0.9221523253201528\nPrecision on validation set: 0.8967664409001529\nF1 score n on validation set: 0.9092822330527249\nAccuracy score n on validation set: 0.9253384383973745\n\nVGG16 Model\n=== Final Training & Validation Metrics ===\nTrain Loss: 0.2289\nTrain Accuracy: 0.9142\nTrain AUC: 0.9667\nValidation Loss: 0.2477\nValidation Accuracy: 0.9077\nValidation AUC: 0.9626\n\nRecall on validation set: 0.9892159065378566\nPrecision on validation set: 0.4315820427367183\nF1 score n on validation set: 0.6009690848290453\nAccuracy score n on validation set: 0.4669766169834541\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"#
\n\n## References (really helpful notebooks๐๐ฝ)\nhttps://www.kaggle.com/code/jieshends2020/baseline-keras-cnn-roc-fast-10min-0-925-lb/edit
\nhttps://www.kaggle.com/code/akarshu121/cancer-detection-with-cnn-for-beginners
\nhttps://www.kaggle.com/code/qitvision/a-complete-ml-pipeline-fast-ai
\nhttps://www.kaggle.com/code/CVxTz/cnn-starter-nasnet-mobile-0-9709-lb
\nhttps://www.kaggle.com/code/suicaokhoailang/wip-densenet121-baseline-with-fastai
\nhttps://www.kaggle.com/code/vbookshelf/cnn-how-to-use-160-000-images-without-crashing
\n","metadata":{}}]}