{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Diabetic Retinopathy Classification with Deep Learning\n\n## Introduction\n\n**Problem Statement:** Diabetic retinopathy is a diabetes-related eye disease that affects the blood vessels in the retina. It is a leading cause of blindness in diabetic individuals. Early detection and timely intervention are crucial in preventing vision loss. Deep learning models have shown promise in automating the classification of diabetic retinopathy into different severity categories from retinal images.\n\nIn this project, I aim to develop a deep learning model for the automated classification of diabetic retinopathy into five categories.\n\n## Dataset\n\n**Dataset Source:** Kaggle\n\n**Dataset Description:** I will be using a Kaggle dataset of retinal images containing cases from all five diabetic retinopathy categories. The dataset is labeled with information about the severity of diabetic retinopathy. It is essential to have a reliable and well-annotated dataset for training and evaluating my model.\n\n## Environment\n\n**Notebook Environment:** Kaggle Jupyter Notebook\n\n**Accelerated Computing:** I will utilize a TPU (Tensor Processing Unit) VM with specifications for efficient deep learning computations, specifically the TPU VM v3.8.\n\n## Model Architecture\n\n**Convolutional Neural Networks (CNNs):** My model will be based on CNNs, which are highly effective for image classification tasks. I will experiment with different architectures and hyperparameters to achieve the best possible performance in classifying diabetic retinopathy.\n\n## Evaluation\n\n**Performance Metrics:** I will evaluate the model's performance using standard evaluation metrics such as accuracy, precision, recall, F1-score, and confusion matrices for each severity category.\n\n**Clinical Interpretation:** Additionally, I will consider clinical validation and interpretation of the results to assess the model's practical utility in assisting healthcare professionals.\n\n## Code Overview\n\nThe code below implements the diabetic retinopathy classification model using TensorFlow and Keras. I will preprocess the dataset, build the model, train it, and evaluate its performance across all five severity categories.\n\nLet's get started!\n","metadata":{}},{"cell_type":"markdown","source":"# Connecting and Instantiating TPU\n\nThe following code cell establishes the connection to a TPU (Tensor Processing Unit) and instantiates it for use in training the deep learning model. TPUs are specialized hardware accelerators that can significantly speed up the training process for deep neural networks.\n","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\n\nprint(\"TensorFlow version:\", tf.__version__)\n\ntry: # detect TPUs\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver.connect() # TPU detection\n    strategy = tf.distribute.TPUStrategy(tpu)\nexcept ValueError: # detect GPUs\n    strategy = tf.distribute.MirroredStrategy() # for GPU or multi-GPU machines\n    #strategy = tf.distribute.get_strategy() # default strategy that works on CPU and single GPU\n    #strategy = tf.distribute.experimental.MultiWorkerMirroredStrategy() # for clusters of multi-GPU machines\n\nprint(\"Number of accelerators: \", strategy.num_replicas_in_sync)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T08:44:41.902747Z","iopub.execute_input":"2023-09-21T08:44:41.903136Z","iopub.status.idle":"2023-09-21T08:45:31.246736Z","shell.execute_reply.started":"2023-09-21T08:44:41.903105Z","shell.execute_reply":"2023-09-21T08:45:31.245719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preprocessing: Image Resizing\n\nIn preparation for training our diabetic retinopathy detection model, we perform data preprocessing by resizing the images.\n\n- **Image Directory:** We start with the Kaggle Diabetic Retinopathy Dataset stored in '/kaggle/input/aptos2019-blindness-detection/train_images'.\n\n- **Target Size:** All images are resized to a common target size of (331, 331) pixels, which is suitable for our model.\n\n- **Resized Image Storage:** Resized images are saved in the 'resized_images_final' directory.\n\n- **Resizing Method:** We use the LANCZOS filter for high-quality image resizing.\n\n- **Supportive Preprocessing:** This image resizing step is part of the overall preprocessing pipeline to prepare the data for our diabetic retinopathy detection model.\n\nThis ensures that all images are standardized in size before feeding them into the model for training.\n","metadata":{}},{"cell_type":"code","source":"from PIL import Image, ImageOps\nimport os\nimport shutil\n\n# Define the image directory\nimage_dir = '/kaggle/input/aptos2019-blindness-detection/train_images'\n\n# Define the target size for resizing\ntarget_size = (331, 331)\n\n# Create a directory to store resized images (if it doesn't exist)\nresized_dir = 'resized_images_final'\n\nif(shutil.os.path.exists(resized_dir)):\n    shutil.rmtree(resized_dir)\n    print(f\"Directory '{resized_dir}' has been deleted.\")\n\n\nos.makedirs(resized_dir, exist_ok=True)\n\n# Get a list of image filenames in the directory\nimage_filenames = os.listdir(image_dir)\n\n# Loop through each image and resize it\nfor image_filename in image_filenames:\n    # Open the image\n    image_path = os.path.join(image_dir, image_filename)\n    image = Image.open(image_path)\n    \n    # Resize the image to the target size with LANCZOS filter\n    resized_image = ImageOps.fit(image, target_size, Image.LANCZOS)\n    \n    # Save the resized image to the resized directory\n    resized_image.save(os.path.join(resized_dir, image_filename))\n    \nprint(f\"Resized {len(image_filenames)} images to {target_size}.\")\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T08:45:41.346146Z","iopub.execute_input":"2023-09-21T08:45:41.346516Z","iopub.status.idle":"2023-09-21T08:59:37.526643Z","shell.execute_reply.started":"2023-09-21T08:45:41.346489Z","shell.execute_reply":"2023-09-21T08:59:37.525581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preprocessing: TFRecord Creation\n\nTo efficiently handle and load the dataset during training, we convert the resized images into the TFRecord format. This format is optimized for working with large datasets and can significantly accelerate data loading.\n\n- **Image Directory:** We start with the resized images stored in the 'resized_images_final/' directory.\n\n- **Output Directory:** Split TFRecord files are generated and stored in '/kaggle/working/split_tfrecords/'.\n\n- **Number of Split Files:** We create 18 split TFRecord files for effective data management.\n\n- **CSV File for Labels:** Image labels are obtained from the 'train.csv' file, which contains information about each image, including its diagnosis label.\n\n- **Mapping Labels:** We create a dictionary to map image filenames to their corresponding diagnosis labels.\n\n- **TFRecord Encoding:** Images and labels are encoded into TFRecord examples. Images are serialized as bytes, and labels as integers.\n\n- **Splitting Records:** Images and labels are split into multiple TFRecord files to facilitate parallel processing during training.\n\nThis TFRecord creation process enhances data efficiency and is an essential step in our diabetic retinopathy detection project.\n","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport os\nimport pandas as pd\n\n# Define the directory where your images are stored\nimage_dir = 'resized_images_final/'\n\n# Define the input TFRecord file\n# input_tfrecord = '/kaggle/input/aptos2019-blindness-detection/datasetFinal.tfrecord'\n\n# Define the output directory for split TFRecord files\noutput_dir = '/kaggle/working/split_tfrecords/'\n\n# Ensure the output directory exists\nos.makedirs(output_dir, exist_ok=True)\n\n# Define the number of split TFRecord files you want to create\nnum_split_files = 18\n\n# Read the CSV file to get image labels\ncsv_file = '/kaggle/input/aptos2019-blindness-detection/train.csv'\ndf = pd.read_csv(csv_file)\nprint(df.head)\n# Create a dictionary to map image filenames to labels\nlabel_dict = dict(zip(df['id_code'], df['diagnosis']))\n\n# Function to encode image and label into a TFRecord example\ndef create_tfrecord_example(image_path, label):\n    with tf.io.gfile.GFile(image_path, 'rb') as f:\n        image = f.read()\n\n    feature = {\n        'image': tf.train.Feature(bytes_list=tf.train.BytesList(value=[image])),\n        'label': tf.train.Feature(int64_list=tf.train.Int64List(value=[label])),\n    }\n\n    example = tf.train.Example(features=tf.train.Features(feature=feature))\n    return example\n\n# Create a TFRecord writer for each split\nfor i in range(num_split_files):\n    # Create a TFRecord writer for the split\n    split_filename = os.path.join(output_dir, f'split_{i}.tfrecord')\n    with tf.io.TFRecordWriter(split_filename) as writer:\n        # Calculate the range of records to write to this split\n        start = i * (len(df) // num_split_files)\n        end = (i + 1) * (len(df) // num_split_files)\n\n        for index in range(start, end):\n            image_id = df['id_code'][index]\n            label = df['diagnosis'][index]\n            image_path = os.path.join(image_dir, f\"{image_id}.png\")\n            tf_example = create_tfrecord_example(image_path, label)\n            writer.write(tf_example.SerializeToString())\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T09:00:04.159603Z","iopub.execute_input":"2023-09-21T09:00:04.160022Z","iopub.status.idle":"2023-09-21T09:00:05.677647Z","shell.execute_reply.started":"2023-09-21T09:00:04.159988Z","shell.execute_reply":"2023-09-21T09:00:05.676378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preprocessing: TFRecord Data Loading and Preprocessing\n\nWe proceed with loading and preprocessing the dataset stored in TFRecord format for efficient training. This step ensures that our diabetic retinopathy detection model is trained on well-preprocessed data.\n\n- **TFRecord Directory:** We start with TFRecord files stored in the 'split_tfrecords/' directory.\n\n- **Image Size and Parameters:** Images are loaded and preprocessed with specified parameters:\n    - **Image Size:** Images are resized to [331, 331] pixels for consistency.\n    - **Number of Classes:** Our model is designed to classify into 5 classes.\n    - **Batch Size:** We use a batch size of 16 (scaled by the number of TPU replicas) for training.\n    \n- **Validation Split:** A portion of the dataset (20%) is set aside for validation.\n\n- **TFRecord Parsing:** Images and labels are parsed from TFRecord examples using TensorFlow operations.\n\n- **Image Preprocessing:** Images undergo preprocessing to enhance features, including:\n    - Conversion to grayscale.\n    - Histogram equalization.\n    - Normalization to [0, 1].\n\n- **Data Augmentation:** Data augmentation is applied to training data, including random flip and saturation adjustment.\n\n- **Dataset Loading:** Training and validation datasets are created by loading and preprocessing TFRecord data.\n\nThis comprehensive preprocessing pipeline prepares our data for training and validation, ensuring that it is well-suited for the diabetic retinopathy detection task.\n","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport os\nimport shutil\n# import tensorflow.experimental.numpy as tnp  # Import TensorFlow's NumPy API\nimport cv2\nimport numpy as np\n\nAUTO = tf.data.experimental.AUTOTUNE\n\n# Define the image size\nIMAGE_SIZE = [331, 331]\n\n# Define other parameters\nnum_classes = 5\nbatch_size = 16 * strategy.num_replicas_in_sync\n\n# Define the directory path for TFRecord files\ntfrecords_dir = 'split_tfrecords/'\n\n\n# Set the validation split ratio\nvalidation_split = 0.20\n\n# Get the list of TFRecord filenames\nfilenames = tf.io.gfile.glob(os.path.join(tfrecords_dir, 'split_*.tfrecord'))\n\n# Calculate the split point\nsplit = len(filenames) - int(len(filenames) * validation_split)\n\n# Split filenames into training and validation\ntrain_filenames = filenames[:split]\nvalidation_filenames = filenames[split:]\n\n# Updated TFRecord feature description\nfeature_description = {\n    'image': tf.io.FixedLenFeature([], tf.string),\n    'label': tf.io.FixedLenFeature([], tf.int64),\n}\n\n\ndef parse_tfrecord(example):\n    example = tf.io.parse_single_example(example, feature_description)\n    image = tf.io.decode_png(example['image'], channels=3)\n    \n    # Convert the image array to 8-bit unsigned integer data type\n    image = tf.image.convert_image_dtype(image, tf.uint8)\n\n    # Set the static shape of the image tensor\n#     image.set_shape([None, None, 3])  # Assuming 3 channels for RGB images\n    \n    # Apply image enhancement preprocessing using OpenCV\n    def preprocess_image(image):\n        img_array = image.numpy()\n        img_array_gray = cv2.cvtColor(img_array, cv2.COLOR_RGB2GRAY)  # Convert to grayscale\n        img_array_eq = cv2.equalizeHist(img_array_gray)  # Apply histogram equalization\n        img_array_eq_rgb = cv2.cvtColor(img_array_eq, cv2.COLOR_GRAY2RGB)  # Convert back to RGB\n        # Ensure pixel values are within the expected range (0-255)\n        img_array_eq_rgb = np.clip(img_array_eq_rgb, 0, 255)\n        # Normalize the image to the range [0, 1] and convert to TensorFlow tensor\n        normalized = tf.convert_to_tensor(img_array_eq_rgb / 255.0, dtype=tf.float32)\n        return normalized\n\n    image = tf.py_function(preprocess_image, [image], tf.float32)\n    \n    # Convert to float32 and normalize to [0, 1]\n    image = tf.image.convert_image_dtype(image, tf.float32)\n    \n    # Set the static shape of the image tensor\n    image.set_shape([None, None, 3])  # Assuming 3 channels for RGB images\n    \n    # Resize the image to the defined size\n    image = tf.image.resize(image, IMAGE_SIZE)\n    \n    label = example['label']\n    \n    # Convert label to one-hot encoded format\n    one_hot_label = tf.one_hot(label, num_classes)\n    \n    return image, one_hot_label\n\n\ndef load_dataset(filenames):\n    records = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO)\n    return records.map(parse_tfrecord, num_parallel_calls=AUTO)\n\ndef data_augment(image, label):\n    modified = tf.image.random_flip_left_right(image)\n    modified = tf.image.random_saturation(modified, 0, 2)\n    return modified, label\n\ndef get_training_dataset():\n    dataset = load_dataset(train_filenames)\n    augmented = dataset.map(data_augment, num_parallel_calls=AUTO)\n    return augmented.repeat().shuffle(2048).batch(batch_size).prefetch(AUTO)\n\ndef get_validation_dataset():\n    return load_dataset(validation_filenames).batch(batch_size).prefetch(AUTO)\n\n# Create training and validation datasets\ntraining_dataset = get_training_dataset()\nvalidation_dataset = get_validation_dataset()\n\n# Print dataset information\nprint(training_dataset)\nprint(validation_dataset)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T10:32:31.404570Z","iopub.execute_input":"2023-09-21T10:32:31.405621Z","iopub.status.idle":"2023-09-21T10:32:31.581067Z","shell.execute_reply.started":"2023-09-21T10:32:31.405580Z","shell.execute_reply":"2023-09-21T10:32:31.579512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Sample Image Visualization\n\nTo better understand the data and model performance, we visualize a sample of images from the dataset along with their corresponding labels. This visualization provides insights into the quality and diversity of the dataset.\n\n- **Image Display:** We display a set of nine images in a 3x3 grid, highlighting their characteristics.\n\n- **Labels:** The labels corresponding to each image are displayed as titles below the images. These labels are categorized into five classes:\n    - 0: 'No DR'\n    - 1: 'Mild'\n    - 2: 'Moderate'\n    - 3: 'Severe'\n    - 4: 'Proliferative DR'\n\n- **Label Mapping:** We map numeric label values to their respective class names using a predefined dictionary.\n\n- **Visualization Function:** We define functions (`display_one_flower` and `display_nine_flowers`) to streamline the image display process.\n\n- **Iterator:** Images are extracted from the training dataset and displayed using the defined functions.\n\n- **Note:** You can re-run this cell to visualize a new batch of images with their labels. This enables you to gain a better understanding of the dataset's content.\n\nThis visualization step aids in assessing the dataset's quality and helps visualize how the model performs on different diabetic retinopathy severity levels.\n","metadata":{}},{"cell_type":"code","source":"\nimport matplotlib.pyplot as plt\n\ndef display_one_image(image, title, subplot, color):\n  plt.subplot(subplot)\n  plt.axis('off')\n  plt.imshow(image)\n  plt.title(title, fontsize=16, color=color)\n  \n# If model is provided, use it to generate predictions.\ndef display_nine_images(images, titles, title_colors=None):\n  subplot = 331\n  plt.figure(figsize=(13,13))\n  for i in range(9):\n    color = 'black' if title_colors is None else title_colors[i]\n    display_one_image(images[i], titles[i], 331+i, color)\n  plt.tight_layout()\n  plt.subplots_adjust(wspace=0.1, hspace=0.1)\n  plt.show()\n\ndef get_dataset_iterator(dataset, n_examples):\n  return dataset.unbatch().batch(n_examples).as_numpy_iterator()\n\ntraining_viz_iterator = get_dataset_iterator(training_dataset, 9)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:48:38.343887Z","iopub.execute_input":"2023-09-21T12:48:38.344282Z","iopub.status.idle":"2023-09-21T12:48:38.480799Z","shell.execute_reply.started":"2023-09-21T12:48:38.344252Z","shell.execute_reply":"2023-09-21T12:48:38.479467Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# Define a dictionary to map label numbers to label names\nlabel_names = {\n    0: 'No DR',\n    1: 'Mild',\n    2: 'Moderate',\n    3: 'Severe',\n    4: 'Proliferative DR'\n}\n\n# Re-run this cell to show a new batch of images\nimages, classes = next(training_viz_iterator)\nclass_idxs = tf.argmax(classes, axis=-1).numpy() # transform from one-hot array to class number\nlabels = [label_names[idx] for idx in class_idxs]\ndisplay_nine_images(images, labels)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:48:43.455217Z","iopub.execute_input":"2023-09-21T12:48:43.455578Z","iopub.status.idle":"2023-09-21T12:48:50.470584Z","shell.execute_reply.started":"2023-09-21T12:48:43.455550Z","shell.execute_reply":"2023-09-21T12:48:50.469240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_viz_iterator = get_dataset_iterator(validation_dataset, 9)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Class Weights Calculation\n\nClass weights play a crucial role in training a machine learning model, especially when dealing with imbalanced datasets. In this context, we calculate class weights for the training dataset, considering the imbalance in the distribution of diabetic retinopathy severity classes.\n\n- **One-Hot Encoding:** Initially, we extract the one-hot encoded labels for the training dataset. One-hot encoding represents each class with a binary vector, where only one element is \"1,\" corresponding to the class label.\n\n- **Conversion to Normal Labels:** To calculate class weights, we convert these one-hot encoded labels back to the normal label format. The label with the highest value in the one-hot encoding is considered the class label for each instance.\n\n- **Class Weight Computation:** We compute the class weights using the `compute_class_weight` function from the scikit-learn library. These weights are calculated in a way that compensates for class imbalance by assigning higher weights to underrepresented classes and lower weights to overrepresented classes. The parameter `class_weight='balanced'` indicates that we want balanced class weights.\n\n- **Dictionary Format:** Finally, we convert the calculated class weights into a dictionary format expected by TensorFlow. This dictionary maps class indices to their respective class weights. The model can use these weights during training to give more importance to the underrepresented classes.\n\nClass weights are essential for training a model that can effectively handle imbalanced datasets. They help the model focus on learning from the minority classes, which is particularly important for the detection of diabetic retinopathy severity.\n","metadata":{}},{"cell_type":"code","source":"# Extract one-hot encoded labels for the training dataset\ntrain_labels_one_hot = []\nfor record in load_dataset(train_filenames):\n    _, one_hot_label = record\n    train_labels_one_hot.append(one_hot_label.numpy())\n\ntrain_labels_one_hot = np.array(train_labels_one_hot)\n\n# Convert one-hot encoded labels back to normal label format\ntrain_labels = np.argmax(train_labels_one_hot, axis=1)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T10:33:42.431445Z","iopub.execute_input":"2023-09-21T10:33:42.432594Z","iopub.status.idle":"2023-09-21T10:33:49.773552Z","shell.execute_reply.started":"2023-09-21T10:33:42.432557Z","shell.execute_reply":"2023-09-21T10:33:49.771945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_labels.shape)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T10:33:54.912164Z","iopub.execute_input":"2023-09-21T10:33:54.912669Z","iopub.status.idle":"2023-09-21T10:33:54.918627Z","shell.execute_reply.started":"2023-09-21T10:33:54.912637Z","shell.execute_reply":"2023-09-21T10:33:54.917347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.utils.class_weight import compute_class_weight\n\n# Calculate class weights\nclass_weights = compute_class_weight(class_weight = 'balanced', classes = np.unique(train_labels), y = train_labels)\n\n# Convert class_weights to a dictionary format expected by TensorFlow\nclass_weight_dict = dict(enumerate(class_weights))\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T10:33:59.240942Z","iopub.execute_input":"2023-09-21T10:33:59.241416Z","iopub.status.idle":"2023-09-21T10:33:59.250100Z","shell.execute_reply.started":"2023-09-21T10:33:59.241381Z","shell.execute_reply":"2023-09-21T10:33:59.248718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(class_weight_dict)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T10:34:05.011522Z","iopub.execute_input":"2023-09-21T10:34:05.012005Z","iopub.status.idle":"2023-09-21T10:34:05.018205Z","shell.execute_reply.started":"2023-09-21T10:34:05.011968Z","shell.execute_reply":"2023-09-21T10:34:05.016851Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Architecture\n\nThe core of our diabetic retinopathy detection solution lies in the architecture of the deep learning model. This section provides an overview of the model structure and hyperparameters used for training.\n\n- **Model Architecture:** We use a Convolutional Neural Network (CNN) as the foundation for our model. The CNN consists of multiple layers that automatically learn hierarchical features from retinal images. The architecture includes convolutional layers for feature extraction, max-pooling layers for down-sampling, and fully connected layers for classification.\n\n- **Convolutional Layers:** The model begins with three convolutional layers, each followed by max-pooling and batch normalization. These layers are responsible for capturing image features at various levels of abstraction.\n\n- **Fully Connected Layers:** After feature extraction, the model includes a stack of fully connected layers. The first fully connected layer has 512 neurons and uses the ReLU activation function. A dropout layer with a rate of 0.4 is applied to mitigate overfitting. The final layer consists of neurons equal to the number of classes (5 in our case) and uses the softmax activation function to output class probabilities.\n\n- **Regularization:** To prevent overfitting, we apply L2 regularization to the convolutional layers. This helps in controlling the complexity of the model.\n\n- **Loss Function:** We use categorical cross-entropy as the loss function, which is suitable for multi-class classification problems like diabetic retinopathy severity detection.\n\n- **Optimizer:** We employ the Adam optimizer for efficient gradient descent during training.\n\n- **Image Preprocessing:** The input images are resized to a common dimension of 331x331 pixels to ensure uniformity.\n\n- **Hyperparameters:** We set the dropout rate to 0.3 and the L2 regularization parameter to 0.0001. These values were chosen through experimentation to balance model complexity and generalization.\n\nThis architecture, coupled with appropriate hyperparameters and data augmentation techniques, enables our model to detect diabetic retinopathy severity accurately. The training process leverages these components to learn relevant features from the retinal images and make predictions.\n","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport cv2\nimport tensorflow as tf\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.utils import to_categorical\nfrom tensorflow.keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping\nfrom tensorflow.keras.regularizers import l2  # Importing l2 regularization\nfrom sklearn.metrics import f1_score\nfrom tensorflow.keras.metrics import Metric\n\n# Create the model\ndef create_model(input_shape, num_classes, dropout_rate, l2_lambda):\n    model = Sequential()\n\n    model.add(Conv2D(64, (3, 3), activation='relu', input_shape=input_shape, kernel_regularizer=l2(l2_lambda)))\n    model.add(MaxPooling2D((2, 2)))\n    model.add(BatchNormalization())\n\n    model.add(Conv2D(128, (3, 3), activation='relu', kernel_regularizer=l2(l2_lambda)))\n    model.add(MaxPooling2D((2, 2)))\n    model.add(BatchNormalization())\n\n    model.add(Conv2D(256, (3, 3), activation='relu', kernel_regularizer=l2(l2_lambda)))\n    model.add(MaxPooling2D((2, 2)))\n    model.add(BatchNormalization())\n    \n    model.add(Conv2D(512, (3, 3), activation='relu', kernel_regularizer=l2(l2_lambda)))\n    model.add(MaxPooling2D((2, 2)))\n    model.add(BatchNormalization())\n\n\n    model.add(Flatten())\n\n    model.add(Dense(512, activation='relu'))\n    model.add(Dropout(dropout_rate))\n    model.add(Dense(num_classes, activation='softmax'))\n    \n    model.compile(\n    optimizer='adam',\n    loss='categorical_crossentropy',  # Use the appropriate loss function for your problem\n    metrics=['accuracy']\n    )\n\n    return model\n\n# Image dimensions for preprocessing\nimg_width, img_height = 331, 331\n\n# Define the number of classes\nnum_classes = 5\n\n# Hyperparameters for regularization and dropout\ndropout_rate = 0.4\nl2_lambda = 0.0001\n\n# Create the model with TPU strategy\nwith strategy.scope():\n    model = create_model(input_shape=(img_width, img_height, 3), num_classes=num_classes, dropout_rate=dropout_rate, l2_lambda=l2_lambda)\nmodel.summary()\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:35:18.424725Z","iopub.execute_input":"2023-09-21T12:35:18.425212Z","iopub.status.idle":"2023-09-21T12:35:20.892099Z","shell.execute_reply.started":"2023-09-21T12:35:18.425177Z","shell.execute_reply":"2023-09-21T12:35:20.890515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Learning Rate Scheduler\n\nTo facilitate effective training, we employ a learning rate scheduler. Learning rate scheduling adjusts the learning rate during training to optimize convergence. In our case, we use a custom learning rate scheduler tailored to the specific needs of our diabetic retinopathy detection model.\n\n- **Learning Rate Schedule:** The learning rate starts at 1e-4 and is reduced by 10% after each epoch. This gradual reduction allows the model to initially explore a larger learning rate for faster convergence and then fine-tune with smaller learning rates as it approaches a solution.\n\n- **Number of Epochs:** We schedule this learning rate reduction over a total of 30 epochs. During this period, the learning rate decreases in a controlled manner to enhance the model's ability to converge to an optimal solution.\n\n- **Visualization:** The accompanying plot visualizes the learning rate per epoch. You can observe the gradual decrease in the learning rate as training progresses. This dynamic adjustment is a critical component of the training process and contributes to the model's effectiveness.\n\nThe learning rate scheduler helps the model adapt to the training data and find a good balance between rapid convergence and fine-tuning. This dynamic adjustment is especially important when dealing with complex datasets like diabetic retinopathy detection, where finding the right learning rate can significantly impact training results.\n","metadata":{}},{"cell_type":"code","source":"EPOCHS = 35\n# Define learning rate scheduler\ndef lr_scheduler(EPOCHS):\n    lr = 1e-4 * (0.9 ** EPOCHS)  # Starting from 1e-4, decrease by 10% every epoch\n    return lr\n\nrang = np.arange(EPOCHS)\ny = [lr_scheduler(x) for x in rang]\nplt.plot(rang, y)\nprint('Learning rate per epoch:')","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:35:36.159568Z","iopub.execute_input":"2023-09-21T12:35:36.160395Z","iopub.status.idle":"2023-09-21T12:35:36.417405Z","shell.execute_reply.started":"2023-09-21T12:35:36.160356Z","shell.execute_reply":"2023-09-21T12:35:36.415868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Callbacks for model training\nlr_decay = LearningRateScheduler(lambda epoch: lr_scheduler(epoch), verbose=True)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:35:45.874940Z","iopub.execute_input":"2023-09-21T12:35:45.876128Z","iopub.status.idle":"2023-09-21T12:35:45.881239Z","shell.execute_reply.started":"2023-09-21T12:35:45.876089Z","shell.execute_reply":"2023-09-21T12:35:45.879764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Data Preparation\n\nBefore training our diabetic retinopathy detection model, it's essential to understand the amount of training data available and the number of steps per epoch. These details are crucial for configuring the training process effectively.\n\n- **Number of Training Images:** The variable `num_training_images` represents the total number of training images available. We calculate this by iterating through the TFRecord files containing the training data. For each file (`f`) in `train_filenames`, we count the number of records in that file. This provides an accurate count of the total training images, considering the entire dataset.\n\n- **Steps per Epoch:** `steps_per_epoch` is a critical parameter for training neural networks using mini-batch stochastic gradient descent (SGD). It determines how many batches of data are processed in each training epoch. We calculate `steps_per_epoch` by dividing the `num_training_images` by the batch size. This calculation ensures that the model trains on the entire dataset in a controlled and efficient manner, as it iterates through smaller batches of data in each epoch.\n\nThese calculations help us configure our training process to make the most efficient use of the available training data and compute resources. Understanding the number of training images and steps per epoch is vital for setting the appropriate training hyperparameters and ensuring the model's convergence.\n","metadata":{}},{"cell_type":"code","source":"# Calculate the number of training images\nnum_training_images = sum([sum(1 for _ in tf.data.TFRecordDataset(f, num_parallel_reads=AUTO)) for f in train_filenames])\nprint(num_training_images)\n# Calculate steps_per_epoch\nsteps_per_epoch = num_training_images // batch_size\nprint(steps_per_epoch)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:35:57.558126Z","iopub.execute_input":"2023-09-21T12:35:57.559341Z","iopub.status.idle":"2023-09-21T12:35:58.466297Z","shell.execute_reply.started":"2023-09-21T12:35:57.559296Z","shell.execute_reply":"2023-09-21T12:35:58.464704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training the Diabetic Retinopathy Detection Model\n\nWe've configured and prepared our model and data for training. Now, let's dive into the actual training process and how we evaluate the model's performance.\n\n- **Training Process:** The `model.fit` method is used to train our diabetic retinopathy detection model. Here's a breakdown of its key parameters:\n\n    - `training_dataset`: This is our training data, prepared as a TensorFlow dataset. It contains images and their corresponding labels.\n\n    - `steps_per_epoch`: This parameter determines how many batches of data are processed in each training epoch. It's set to ensure that the entire training dataset is processed over multiple epochs.\n\n    - `epochs`: We specify the number of training epochs, which represents how many times the model will iterate through the entire training dataset. In this case, we're training for 50 epochs.\n\n    - `validation_data`: We also provide a validation dataset (`validation_dataset`) to assess the model's performance on data it hasn't seen during training. This helps us monitor for overfitting and generalization.\n\n    - `callbacks`: A list of callbacks to be applied during training. In this case, we're using a callback called `lr_decay` for learning rate scheduling.\n\n    - `class_weight`: We pass the class weights as `class_weight_dict` to account for class imbalance in our dataset. This helps the model give more importance to underrepresented classes during training.\n\n- **Final Accuracy Mean-5:** After training for 50 epochs, we calculate the model's final accuracy. To reduce the impact of any outliers, we compute the mean accuracy of the last five validation epochs. This provides a more stable measure of the model's performance.\n\n    - `history.history[\"val_accuracy\"][-5:]` extracts the validation accuracy of the last five epochs.\n\n    - `np.mean(final_accuracy)` calculates the mean accuracy from these five values.\n\nMonitoring the final accuracy is crucial, as it indicates how well the model has learned to classify diabetic retinopathy from retinal images. A higher accuracy generally implies better performance, but it should be considered along with other evaluation metrics for a comprehensive assessment of the model's quality.\n","metadata":{}},{"cell_type":"code","source":"history = model.fit(\n    training_dataset,\n    steps_per_epoch=steps_per_epoch,\n    epochs=EPOCHS,\n    validation_data=validation_dataset,\n    callbacks=[lr_decay],\n    class_weight=class_weight_dict  # Pass the class weights here\n)\n\nfinal_accuracy = history.history[\"val_accuracy\"][-5:]\nprint(\"FINAL ACCURACY MEAN-5: \", np.mean(final_accuracy))","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:36:08.379249Z","iopub.execute_input":"2023-09-21T12:36:08.379693Z","iopub.status.idle":"2023-09-21T12:41:48.965176Z","shell.execute_reply.started":"2023-09-21T12:36:08.379659Z","shell.execute_reply":"2023-09-21T12:41:48.963641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Curve Visualization\n\nVisualizing training curves is essential for understanding how our diabetic retinopathy detection model is learning during training. We track two key metrics: accuracy and loss, over multiple epochs. Let's break down the provided code:\n\n- **Accuracy Curve:** The top subplot displays the accuracy curve. It shows how the accuracy of our model changes with each training epoch. The x-axis represents the number of epochs, while the y-axis represents accuracy. The blue line represents training accuracy, and the orange line represents validation accuracy.\n\n- **Loss Curve:** The bottom subplot displays the loss curve. Similar to the accuracy curve, it shows how the loss (error) of our model changes during training. Lower values indicate better performance. The blue line represents training loss, and the orange line represents validation loss.\n\nBy visualizing these curves, we can observe important trends:\n\n- **Training Accuracy:** The blue line indicates how well the model is fitting the training data. We want to see it steadily increase, indicating that the model is learning.\n\n- **Validation Accuracy:** The orange line shows how well the model generalizes to unseen data (validation set). Ideally, it should also increase or remain stable. If it significantly diverges from the training accuracy, it could indicate overfitting (fitting the training data too closely).\n\n- **Training Loss:** The blue line represents the training loss, which should steadily decrease as the model learns. Increasing loss would suggest the model is struggling to fit the training data.\n\n- **Validation Loss:** The orange line represents the validation loss. Similar to accuracy, we want it to decrease or remain stable. A significant increase in validation loss relative to training loss can be a sign of overfitting.\n\nThis visualization helps us make informed decisions during model training and fine-tuning. If the validation accuracy plateaus or decreases while training accuracy continues to increase, it may be necessary to adjust model complexity, regularization, or other hyperparameters to improve generalization.\n","metadata":{}},{"cell_type":"code","source":"def display_training_curves(training, validation, title, subplot):\n  ax = plt.subplot(subplot)\n  ax.plot(training)\n  ax.plot(validation)\n  ax.set_title('model '+ title)\n  ax.set_ylabel(title)\n  ax.set_xlabel('epoch')\n  ax.legend(['training', 'validation'])\n\nplt.subplots(figsize=(10,10))\nplt.tight_layout()\ndisplay_training_curves(history.history['accuracy'], history.history['val_accuracy'], 'accuracy', 211)\ndisplay_training_curves(history.history['loss'], history.history['val_loss'], 'loss', 212)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:43:55.976013Z","iopub.execute_input":"2023-09-21T12:43:55.976457Z","iopub.status.idle":"2023-09-21T12:43:56.735589Z","shell.execute_reply.started":"2023-09-21T12:43:55.976422Z","shell.execute_reply":"2023-09-21T12:43:56.734339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image Visualization with Titles\n\nIn this visualization, we display a batch of images along with their titles. Each image is accompanied by a title that provides information about the model's prediction for the disease's class.\n\n- **Image Grid:** The grid displays multiple images.\n\n- **Image Titles:** The titles above each image describe the classification result. There are two types of titles:\n    - **Correct Predictions (Black):** If the model's prediction matches the actual class label, the title appears in black. It indicates that the model correctly identified the disease's class.\n    - **Incorrect Predictions (Red):** If the model's prediction differs from the actual class label, the title appears in red. It also mentions what the correct class should have been.\n\nThis visualization allows us to assess how well the model is performing on the validation dataset. Correct predictions are an indicator of the model's accuracy, while incorrect predictions highlight areas where the model may need improvement.\n\nBy examining the images and titles, we can gain insights into the model's strengths and weaknesses. It's particularly helpful for identifying cases where the model is struggling to classify certain disease types.\n\nFeel free to re-run this cell to view different batches of images and their corresponding titles, allowing you to assess the model's performance visually.\n","metadata":{}},{"cell_type":"code","source":"def disease_title(label, prediction):\n  # Both prediction (probabilities) and label (one-hot) are arrays with one item per class.\n  class_idx = np.argmax(label, axis=-1)\n  prediction_idx = np.argmax(prediction, axis=-1)\n  if class_idx == prediction_idx:\n    return f'{label_names[prediction_idx]} [correct]', 'black'\n  else:\n    return f'{label_names[prediction_idx]} [incorrect, should be {label_names[class_idx]}]', 'red'\n\ndef get_titles(images, labels, model):\n  predictions = model.predict(images)\n  titles, colors = [], []\n  for label, prediction in zip(labels, predictions):\n    title, color = disease_title(label, prediction)\n    titles.append(title)\n    colors.append(color)\n  return titles, colors\n\nvalidation_viz_iterator = get_dataset_iterator(validation_dataset, 9)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:54:28.036232Z","iopub.execute_input":"2023-09-21T12:54:28.036629Z","iopub.status.idle":"2023-09-21T12:54:28.209625Z","shell.execute_reply.started":"2023-09-21T12:54:28.036600Z","shell.execute_reply":"2023-09-21T12:54:28.208234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Re-run this cell to show a new batch of images\nimages, classes = next(validation_viz_iterator)\ntitles, colors = get_titles(images, classes, model)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-21T12:54:37.039893Z","iopub.execute_input":"2023-09-21T12:54:37.041044Z","iopub.status.idle":"2023-09-21T12:54:44.260697Z","shell.execute_reply.started":"2023-09-21T12:54:37.041006Z","shell.execute_reply":"2023-09-21T12:54:44.259344Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_nine_images(images, titles, colors)","metadata":{"execution":{"iopub.status.busy":"2023-09-21T13:08:20.951948Z","iopub.execute_input":"2023-09-21T13:08:20.952324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Save Trained Model\n\nThe trained model has been saved as 'best_model2.h5'. This saved model file contains the learned weights, architecture, and configuration of the model. Saving the model allows us to reuse it for various purposes, such as making predictions on new data or fine-tuning it for further training.\n\nTo load this saved model in the future, you can use the following code:\n\n```python\nfrom tensorflow.keras.models import load_model\n\n# Load the saved model\nloaded_model = load_model('best_model2.h5')\n","metadata":{}},{"cell_type":"code","source":"model.save('best_model.h5')","metadata":{"execution":{"iopub.status.busy":"2023-09-21T09:28:14.676408Z","iopub.execute_input":"2023-09-21T09:28:14.676947Z","iopub.status.idle":"2023-09-21T09:28:16.863504Z","shell.execute_reply.started":"2023-09-21T09:28:14.676907Z","shell.execute_reply":"2023-09-21T09:28:16.862106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Purpose of Creating a ZIP File\n\nWe are creating a ZIP file to simplify the process of downloading the trained model. This compressed archive, named 'out1.zip', contains the model file ('best_model2.h5').\n\nBy using a ZIP file, you can easily download the model to your local machine. Once downloaded, you can unzip the file to access the saved model. This streamlined approach ensures a convenient and efficient way to obtain and work with the trained model.\n","metadata":{}},{"cell_type":"code","source":"import zipfile\n\nwith zipfile.ZipFile('out1.zip', 'w') as zipf:\n    zipf.write('/kaggle/working/best_model2.h5', 'best_model.h5')","metadata":{"execution":{"iopub.status.busy":"2023-09-17T12:00:22.363148Z","iopub.execute_input":"2023-09-17T12:00:22.363591Z","iopub.status.idle":"2023-09-17T12:00:24.639346Z","shell.execute_reply.started":"2023-09-17T12:00:22.363557Z","shell.execute_reply":"2023-09-17T12:00:24.637957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion\n\nIn this project, we've developed a deep learning model to detect and classify diabetic retinopathy (DR) into five severity categories, ranging from 'No DR' to 'Proliferative DR.' Leveraging a Kaggle dataset and the power of a TPU VM (v3.8), we've undertaken various steps:\n\n- Preprocessed and resized retinal images.\n- Created TFRecord files for efficient data input.\n- Designed a convolutional neural network (CNN) model.\n- Implemented learning rate scheduling for training optimization.\n- Managed class imbalances with class weights.\n- Trained and evaluated the model with TensorFlow and Keras.\n\nThe resulting model can assist in the early detection and severity classification of diabetic retinopathy, potentially aiding in preventing vision loss for diabetic individuals.\n\nThank you for joining this journey towards leveraging AI for healthcare. Feel free to download the model and explore its applications further.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}