{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Welcome to RetinoGuard AI! \n\nIn this project, we will be working with a dataset of retinal images from the APTOS 2019 Blindness Detection Challenge. Our goal is to train a model to detect signs of diabetic retinopathy in these images.\n\nTo accomplish this, we will be using the DenseNet201 architecture, a deep convolutional neural network that has been shown to perform well on image classification tasks. We will also be using techniques such as data augmentation and transfer learning to improve the accuracy of our model.\n\nSome of the methods we used in this notebook were inspired by the work of others in the Kaggle community. We would like to give credit to xhlulu's DenseNet Keras Starter notebook (https://www.kaggle.com/code/xhlulu/aptos-2019-densenet-keras-starter) for providing a helpful starting point, as well as to Salihacur's Step3-DenseNet-201-Model notebook (https://www.kaggle.com/code/salihacur/step3-densenet-201-model/notebook) for additional insights into model architecture.\n\nWe hope that this notebook will be a useful resource for others interested in working with image classification and deep learning techniques. Let's get started!","metadata":{"papermill":{"duration":0.005707,"end_time":"2023-04-27T08:48:09.177416","exception":false,"start_time":"2023-04-27T08:48:09.171709","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Import necessary libraries\nThis section imports necessary libraries such as TensorFlow, Keras, and Pandas.","metadata":{"papermill":{"duration":0.004468,"end_time":"2023-04-27T08:48:09.186840","exception":false,"start_time":"2023-04-27T08:48:09.182372","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport keras\nfrom keras import layers\nfrom keras.applications import DenseNet121, DenseNet201\nfrom keras.callbacks import Callback, EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, GlobalAveragePooling2D\nfrom keras.models import Sequential, Model\nfrom keras.optimizers import Adam\nfrom keras import backend as K\nfrom keras import regularizers\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import cohen_kappa_score, accuracy_score, confusion_matrix\nimport tensorflow as tf\nimport cv2\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\nfrom keras.utils import to_categorical\nfrom keras.losses import CategoricalCrossentropy","metadata":{"papermill":{"duration":8.930889,"end_time":"2023-04-27T08:48:18.123950","exception":false,"start_time":"2023-04-27T08:48:09.193061","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:06:20.518078Z","iopub.execute_input":"2023-04-29T20:06:20.518587Z","iopub.status.idle":"2023-04-29T20:06:28.007876Z","shell.execute_reply.started":"2023-04-29T20:06:20.518542Z","shell.execute_reply":"2023-04-29T20:06:28.006731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load data\nThe dataset is loaded into the notebook using the Pandas library.","metadata":{"papermill":{"duration":0.004701,"end_time":"2023-04-27T08:48:18.134076","exception":false,"start_time":"2023-04-27T08:48:18.129375","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_df = pd.read_csv(\"../input/aptos2019-blindness-detection/train.csv\")\ntest_df = pd.read_csv(\"../input/aptos2019-blindness-detection/test.csv\")\n","metadata":{"papermill":{"duration":0.036333,"end_time":"2023-04-27T08:48:18.174973","exception":false,"start_time":"2023-04-27T08:48:18.138640","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:06:28.010328Z","iopub.execute_input":"2023-04-29T20:06:28.011103Z","iopub.status.idle":"2023-04-29T20:06:28.039166Z","shell.execute_reply.started":"2023-04-29T20:06:28.011063Z","shell.execute_reply":"2023-04-29T20:06:28.038286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preprocessing Function for Images\nThe preprocess_image() function is used to preprocess an input image before it can be fed into the deep learning model for prediction. Specifically, the function performs the following steps:\n\n* Reads the image from the specified file path using OpenCV's imread() function.\n* Converts the image from the BGR color space (default in OpenCV) to the RGB color space, which is the default color space used by most deep learning frameworks.\n* Resizes the image to the desired size (224 x 224 pixels by default), which is the input size required by the model used in this notebook.\n* Enhances the contrast of the image by adding a weighted sum of the original image and a Gaussian blurred version of the image. This step helps to increase the visibility of important features in the image.\n\nThe preprocessed image is then returned by the function to be used as input for the deep learning model.","metadata":{"papermill":{"duration":0.00444,"end_time":"2023-04-27T08:48:18.184430","exception":false,"start_time":"2023-04-27T08:48:18.179990","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def preprocess_image(image_path, desired_size=299):  # Increased desired size\n    im = cv2.imread(image_path)\n    im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n    im = cv2.resize(im, (desired_size,) * 2)\n    im = cv2.addWeighted(im, 4, cv2.GaussianBlur(im, (0, 0), 30), -4, 128)\n\n    return im","metadata":{"papermill":{"duration":0.180146,"end_time":"2023-04-27T08:48:18.369834","exception":false,"start_time":"2023-04-27T08:48:18.189688","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:06:28.042485Z","iopub.execute_input":"2023-04-29T20:06:28.042762Z","iopub.status.idle":"2023-04-29T20:06:28.216418Z","shell.execute_reply.started":"2023-04-29T20:06:28.042735Z","shell.execute_reply":"2023-04-29T20:06:28.215340Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preprocessing and Train/Validation Split\nThis section performs the data preprocessing step for the training set. It first initializes an empty array of shape (N, 299,299,3), where N is the number of images in the training set. It then loops over the training set image IDs, and for each image, it loads and preprocesses the image using the preprocess_image() function and stores it in the corresponding row of the x_train array. The preprocess_image() function resizes the image to (299,299), applies some data augmentation techniques, and normalizes the pixel values.\n\nAfter preprocessing the images, the code creates a one-hot encoded matrix of the target variable (diagnosis) using the pd.get_dummies() function and stores it in the y_train array. It then uses the train_test_split() function from Scikit-learn to split the preprocessed training data into training and validation sets with a 85:15 ratio.\n\nFinally, it saves the preprocessed images and labels, and frees up memory using the gc.collect() function.","metadata":{"papermill":{"duration":0.004492,"end_time":"2023-04-27T08:48:18.379586","exception":false,"start_time":"2023-04-27T08:48:18.375094","status":"completed"},"tags":[]}},{"cell_type":"code","source":"N = train_df.shape[0]\nx_train = np.empty((N, 299,299,3), dtype=np.uint8)\n\nfor i, image_id in enumerate(tqdm(train_df['id_code'])):\n    x_train[i, :, :, :] = preprocess_image(\n        f'../input/aptos2019-blindness-detection/train_images/{image_id}.png'\n    )\n    \ny_train = pd.get_dummies(train_df[['diagnosis']]).values\n\n# Save the preprocessed images and labels\noutput_dir = './'\nos.makedirs(output_dir, exist_ok=True)\nnp.save(os.path.join(output_dir, 'x_train.npy'), x_train)\npd.DataFrame(y_train).to_csv(os.path.join(output_dir, 'y_train.csv'), index=False)\n\n# Use garbage collector to release unreferenced memory\nimport gc\n\ndel train_df\ngc.collect()\n\n# Load the preprocessed images and labels if they exist\nif os.path.exists(os.path.join(output_dir, 'x_train.npy')):\n    x_train = np.load(os.path.join(output_dir, 'x_train.npy'))\n    \nif os.path.exists(os.path.join(output_dir, 'y_train.csv')):\n    y_train = pd.read_csv(os.path.join(output_dir, 'y_train.csv')).values\n\n# Train/validation split\nx_train, x_val, y_train, y_val = train_test_split(\n    x_train, y_train, test_size=0.15, random_state=2019)\n","metadata":{"papermill":{"duration":664.261323,"end_time":"2023-04-27T08:59:22.645552","exception":false,"start_time":"2023-04-27T08:48:18.384229","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:06:28.219595Z","iopub.execute_input":"2023-04-29T20:06:28.220257Z","iopub.status.idle":"2023-04-29T20:18:18.799920Z","shell.execute_reply.started":"2023-04-29T20:06:28.220212Z","shell.execute_reply":"2023-04-29T20:18:18.798800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ImageDataGenerator object creation and configuration\nThe section creates an object of ImageDataGenerator, which is a class from the Keras library used for image data preprocessing and augmentation.\n\nThe object is configured with several arguments, including horizontal and vertical flips, rotation range, zoom range, and shifts in width and height. These arguments specify the range and type of transformations to be applied to the images during training to increase the variety of training data and improve model accuracy.\n\nThe BATCH_SIZE parameter sets the number of images that will be used for training in each iteration.","metadata":{"papermill":{"duration":0.183984,"end_time":"2023-04-27T08:59:23.013718","exception":false,"start_time":"2023-04-27T08:59:22.829734","status":"completed"},"tags":[]}},{"cell_type":"code","source":"BATCH_SIZE = 15\n\ndata_generator = ImageDataGenerator(\n    horizontal_flip=True,\n    vertical_flip=True,\n    rotation_range=360,  # Increased rotation range\n    zoom_range=0.2,      # Adjusted zoom range\n    width_shift_range=0.1,  # Adjusted width shift range\n    height_shift_range=0.1,  # Adjusted height shift range\n    shear_range=0.1,        # Adjusted shear range\n    fill_mode='nearest'\n)","metadata":{"papermill":{"duration":0.194086,"end_time":"2023-04-27T08:59:23.388986","exception":false,"start_time":"2023-04-27T08:59:23.194900","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:18:18.801570Z","iopub.execute_input":"2023-04-29T20:18:18.801940Z","iopub.status.idle":"2023-04-29T20:18:18.807792Z","shell.execute_reply.started":"2023-04-29T20:18:18.801898Z","shell.execute_reply":"2023-04-29T20:18:18.806615Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Cohen Kappa Score Calculation Function\nThe cohen_kappa function defined below is a Python implementation of the Cohen Kappa score, which is a measure of inter-rater agreement for categorical items. In this case, the function is used to calculate the agreement between the actual labels of the images and the predicted labels generated by the model.\n\nThe function takes as input the true labels (y_true) and predicted labels (y_pred) in one-hot encoded format, and a parameter num_classes which specifies the number of classes being considered (default is 5).\n\nThe function starts by converting the one-hot encoded labels to class indices using the tf.argmax function. It then calculates the weight matrix w and the confusion matrix using the tf.math.confusion_matrix function.\n\nNext, the function calculates the expected matrix and the observed and expected weighted agreements. Finally, it returns 1 minus the ratio of observed weighted agreement to expected weighted agreement.\n\nThe Cohen Kappa score is commonly used in the evaluation of multi-class classification problems, particularly in medical diagnosis where inter-observer agreement is crucial.","metadata":{"papermill":{"duration":0.178114,"end_time":"2023-04-27T08:59:23.801337","exception":false,"start_time":"2023-04-27T08:59:23.623223","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def cohen_kappa(y_true, y_pred, num_classes=5):\n    y_true_classes = tf.cast(tf.argmax(y_true, axis=-1), dtype=tf.float32)\n    y_pred_classes = tf.cast(tf.argmax(y_pred, axis=-1), dtype=tf.float32)\n    \n    n = tf.cast(num_classes, dtype=tf.float32)\n    w = tf.reshape(tf.range(0, n), (1, -1)) - tf.reshape(tf.range(0, n), (-1, 1))\n    w = tf.square(w) / tf.square(n - 1)\n    \n    confusion_matrix = tf.math.confusion_matrix(y_true_classes, y_pred_classes, num_classes=num_classes, dtype=tf.float32)\n    sum_rows = tf.reduce_sum(confusion_matrix, axis=1)\n    sum_cols = tf.reduce_sum(confusion_matrix, axis=0)\n    \n    expected = tf.tensordot(sum_rows, sum_cols, axes=0) / tf.reduce_sum(sum_rows)\n    \n    w_obs = tf.reduce_sum(w * confusion_matrix) / tf.reduce_sum(confusion_matrix)\n    w_exp = tf.reduce_sum(w * expected) / tf.reduce_sum(expected)\n    \n    return 1.0 - w_obs / w_exp","metadata":{"papermill":{"duration":0.191687,"end_time":"2023-04-27T08:59:24.175784","exception":false,"start_time":"2023-04-27T08:59:23.984097","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:18:18.809335Z","iopub.execute_input":"2023-04-29T20:18:18.809955Z","iopub.status.idle":"2023-04-29T20:18:18.821710Z","shell.execute_reply.started":"2023-04-29T20:18:18.809918Z","shell.execute_reply":"2023-04-29T20:18:18.820739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating a DenseNet201 model for image classification\nThis section defines a convolutional neural network (CNN) model for image classification using the DenseNet201 architecture. The model takes in RGB images of dimensions 299x299 pixels, and includes a pre-trained DenseNet201 layer from the ImageNet dataset to extract features from the input image. The output from this layer is then passed through a global average pooling layer that computes the average value of each feature map.\n\nThe resulting feature vector is then passed through a dropout layer to reduce overfitting and a fully connected layer with a softmax activation function that outputs a probability score for each of the 5 classes of the Aptos 2019 Blindness Detection Challenge. The model also includes L1 and L2 regularization to prevent overfitting.\n\nThe model is compiled with the Adam optimizer with a learning rate of 0.0001, and the evaluation metrics of accuracy and Cohen's kappa. The loss function is the categorical cross-entropy with label smoothing.\n\nThe weights_path parameter allows the user to specify a path to pre-trained weights to use in the model. If no weights are specified, the DenseNet201 weights are initialized randomly.\n\nThis function is called later in the notebook to create the model, which is then trained and used to make predictions on new images.","metadata":{"papermill":{"duration":0.206739,"end_time":"2023-04-27T08:59:24.564270","exception":false,"start_time":"2023-04-27T08:59:24.357531","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def create_densenet201(weights_path=None):\n    densenet = DenseNet201(\n        weights=None,\n        include_top=False,\n        input_shape=(299, 299, 3)  # Increased input shape dimensions\n    )\n\n    if weights_path is not None:\n        densenet.load_weights(weights_path)\n\n    model = Sequential()\n    model.add(densenet)\n    model.add(layers.GlobalAveragePooling2D())\n    model.add(layers.Dropout(0.5))\n    model.add(layers.Dense(5, activation='softmax', \n                           kernel_regularizer=regularizers.l1_l2(l1=1e-5, l2=1e-4), \n                           bias_regularizer=regularizers.l2(1e-4), \n                           activity_regularizer=regularizers.l2(1e-5)))\n\n    optimizer = Adam(learning_rate=1e-4)\n    loss = CategoricalCrossentropy(label_smoothing=0.1)  # Added label smoothing\n    model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy', cohen_kappa])\n\n    return model","metadata":{"papermill":{"duration":0.189724,"end_time":"2023-04-27T08:59:24.938245","exception":false,"start_time":"2023-04-27T08:59:24.748521","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:18:18.823325Z","iopub.execute_input":"2023-04-29T20:18:18.823676Z","iopub.status.idle":"2023-04-29T20:18:18.834559Z","shell.execute_reply.started":"2023-04-29T20:18:18.823640Z","shell.execute_reply":"2023-04-29T20:18:18.833309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DenseNet201 Model Training\nThis section trains a DenseNet201 model using the data generated from the data generator. The target labels are one-hot encoded using the to_categorical() function from Keras. The weights for the pre-trained DenseNet201 model are loaded from a pre-defined path. The model is then created using the create_densenet201() function, passing in the pre-trained weights.\n\nA data generator is created using the training data and one-hot encoded labels. The validation data is split into a separate variable. Three callbacks are defined and added to a list: EarlyStopping, ReduceLROnPlateau, and ModelCheckpoint. These callbacks are used to monitor the validation loss, adjust the learning rate, and save the best model based on validation loss, respectively.\n\nThe fit() method is called on the model, passing in the data generator, steps per epoch, number of epochs, validation data, and callbacks. The history variable stores the training history, which includes the training and validation loss and accuracy. The trained model can then be used for prediction on the test data.![image.png](attachment:0d22904d-7144-409f-87d5-8e91a3e28eaf.png)","metadata":{"papermill":{"duration":0.181144,"end_time":"2023-04-27T08:59:25.303330","exception":false,"start_time":"2023-04-27T08:59:25.122186","status":"completed"},"tags":[]},"attachments":{"0d22904d-7144-409f-87d5-8e91a3e28eaf.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAlUAAAHBCAYAAABAGU56AAAABHNCSVQICAgIfAhkiAAAABl0RVh0U29mdHdhcmUAZ25vbWUtc2NyZWVuc2hvdO8Dvz4AAAAtdEVYdENyZWF0aW9uIFRpbWUAU2F0IDI5IEFwciAyMDIzIDA1OjA4OjI0IFBNIEVEVGmFC+MAACAASURBVHic7N17fM/1///x+9vOW9swtlmGYTnNmYQKzSHnQ774kHOf9CWZQ0l9EtLmUKjk9PmUiZDKpESIRPTJuah0craZNBtiY3v+/vDb+9vbhm1eeu9wu14u7z9ez9fz9Xo9Xq/3e9t9z9fhbTPGGAEAAOC2FHN2AQAAAIUBoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqiBJio2Nlc1ms788PT0VHBysFi1aKCYmRomJiVmWmTBhgmw2W6628+eff2rChAn64osvcrVcdtuqUKGCOnTokKv13MrSpUs1a9asbOfZbDZNmDDB0u1Z7fPPP1eDBg3k4+Mjm82mVatWZdvvyJEjDu93sWLFFBAQoHbt2mnHjh1/S60DBgxQhQoVHNrycoxPnTqlCRMmaN++fdYV9/9l/lwcOXLkhn3q1q2ru+++W+np6Tfs07RpU5UqVUppaWk52m7m+xMbG5vLiv8e2b13t+Nm7+GAAQN01113Wbat613/s3Cz180+BzmVl8/rDz/8oL59+6pixYry9PRUqVKlVK9ePT355JNKSUnJdQ3bt2/XhAkTdO7cuVwvi5sjVMHBwoULtWPHDm3YsEFvvvmm6tSpo6lTp6patWrauHGjQ9/HHnss13+A//zzT02cODHXoSov28qLm4WqHTt26LHHHrvjNeSVMUY9evSQm5ubVq9erR07dqhZs2Y3XWb48OHasWOHtm7dqpiYGO3fv18tWrTQ3r17/6aqHeXlGJ86dUoTJ068I6EqJwYPHqxTp07ps88+y3b+Tz/9pO3bt6tv375yd3f/m6u7M1544QXFxcVZtj5nvodlypTRjh07HF5169ZVxYoVs7SXKVPmtreX233du3ev6tevr++//17jx4/XunXrNG/ePLVv316fffaZ/vjjj1zXsH37dk2cOJFQdQe4OrsA5C8RERFq0KCBffqRRx7RyJEjdf/996tbt276+eefFRQUJEkqW7asypYte0fr+fPPP+Xt7f23bOtW7rvvPqdu/1ZOnTqlP/74Q127dlVkZGSOlilXrpx9v5o2barKlSsrMjJSc+bM0b///e9sl7l06ZI8PT1zPUqZE/n9GGenT58+evrpp/X222+rXbt2Wea//fbbkqRBgwb93aXdMZUqVXJ2CZbx8PDI8rnz8/NTWlpavvg8zpo1S8WKFdMXX3whX19fe3v37t310ksvyRjjxOpwPUaqcEvlypXTq6++qvPnz2v+/Pn29uxOyW3atEnNmzdXQECAvLy8VK5cOT3yyCP6888/deTIEZUuXVqSNHHiRPuQ+oABAxzWt2fPHnXv3l0lSpSw//K+2anGuLg41apVS56enqpYsaJef/11h/k3OoXzxRdfyGaz2UfNmjdvrjVr1ujo0aMOQ/6Zsjs1deDAAXXu3FklSpSQp6en6tSpo0WLFmW7nWXLlun5559XSEiI/Pz81LJlSx06dOjGB/4vtm3bpsjISPn6+srb21tNmjTRmjVr7PMnTJhgD51jx46VzWbL0+mZzD8iR48elfR/x279+vUaNGiQSpcuLW9vb6WmpkqS3nvvPTVu3Fg+Pj6666671KZNm2xHuWJjY1WlShV5eHioWrVqeuedd7LdfnbH+OTJk3r88ccVGhoqd3d3hYSEqHv37jp9+rS++OILNWzYUJI0cOBA+3v213Xs2rVLnTp1UsmSJeXp6am6detqxYoVWbb99ddfq2nTpvL09FRISIjGjRunK1eu3PKYlShRQl27dtXHH3+ss2fPOsxLT0/X4sWL1bBhQ9WsWVO//PKLBg4cqPDwcHl7e+vuu+9Wx44d9d13391yOzc65Zbdz4YxRnPmzFGdOnXk5eWlEiVKqHv37vrtt98c+u3du1cdOnRQYGCgPDw8FBISovbt2+vEiRO5rsVms+nJJ5/U4sWLVa1aNXl7e6t27dr65JNPbrqunLyHkvTLL7+oXbt2uuuuuxQaGqrRo0fbP4eZ0tLSNHnyZFWtWlUeHh4qXbq0Bg4cqDNnzty0hpxISUnRmDFjFBYWJnd3d919992KiorSxYsXHfq9//77atSokfz9/eXt7a2KFSvaA3VO9/Wvzp49Kz8/vxueAr3+vd+4caMiIyPl5+cnb29vNW3aVJ9//rl9/oQJE/T0009LksLCwuw15PbsAbJHqEKOtGvXTi4uLvryyy9v2OfIkSNq37693N3d9fbbb2vdunWaMmWKfHx8lJaWpjJlymjdunWSrp0yyRxSf+GFFxzW061bN1WuXFnvv/++5s2bd9O69u3bp6ioKI0cOVJxcXFq0qSJRowYoVdeeSXX+zhnzhw1bdpUwcHBDkP+N3Lo0CE1adJEBw8e1Ouvv66VK1eqevXqGjBggKZNm5al/3PPPaejR4/qP//5jxYsWKCff/5ZHTt2vOm1OJK0ZcsWPfTQQ0pOTtZbb72lZcuWydfXVx07dtR7770n6drp0ZUrV0r6v1N6eTk988svv0iSPfxmGjRokNzc3LR48WJ98MEHcnNzU3R0tP7xj3+oevXqWrFihRYvXqzz58/rgQce0Pfff29fNjY2VgMHDlS1atX04Ycf6l//+pdeeuklbdq06Zb1nDx5Ug0bNlRcXJxGjRqltWvXatasWfL391dSUpLq1aunhQsXSpL+9a9/2d+zzFOImzdvVtOmTXXu3DnNmzdPH330kerUqaOePXs6XK/0/fffKzIyUufOnVNsbKzmzZunvXv3avLkyTk6boMHD1ZaWpqWLFni0P7ZZ5/p1KlTGjx4sKRro4kBAQGaMmWK1q1bpzfffFOurq5q1KhRjgN2TgwZMkRRUVFq2bKlVq1apTlz5ujgwYNq0qSJTp8+LUm6ePGiWrVqpdOnT+vNN9/Uhg0bNGvWLJUrV07nz5/P03bXrFmj2bNna9KkSfrwww9VsmRJde3aNUuY+6tbvYeSdOXKFXXq1EmRkZH66KOPNGjQIM2cOVNTp06198nIyFDnzp01ZcoU9e7dW2vWrNGUKVO0YcMGNW/eXJcuXcrTPknXRsybNWumRYsW6amnntLatWs1duxYxcbGqlOnTvbRoh07dqhnz56qWLGili9frjVr1mj8+PG6evVqjvf1eo0bN1Z8fLz69OmjLVu23HQ/lixZotatW8vPz0+LFi3SihUrVLJkSbVp08YerB577DENHz5ckrRy5Up7DfXq1cvz8cFfGMAYs3DhQiPJ7Ny584Z9goKCTLVq1ezTL774ovnrR+iDDz4wksy+fftuuI4zZ84YSebFF1/MMi9zfePHj7/hvL8qX768sdlsWbbXqlUr4+fnZy5evOiwb4cPH3bot3nzZiPJbN682d7Wvn17U758+Wxrv77uXr16GQ8PD3Ps2DGHfm3btjXe3t7m3LlzDttp166dQ78VK1YYSWbHjh3Zbi/TfffdZwIDA8358+ftbVevXjURERGmbNmyJiMjwxhjzOHDh40kM3369Juu7699p06daq5cuWIuX75sdu/ebRo2bGgkmTVr1hhj/u/Y9evXz2H5Y8eOGVdXVzN8+HCH9vPnz5vg4GDTo0cPY4wx6enpJiQkxNSrV89epzHGHDlyxLi5uWU51tcf40GDBhk3Nzfz/fff33Bfdu7caSSZhQsXZplXtWpVU7duXXPlyhWH9g4dOpgyZcqY9PR0Y4wxPXv2NF5eXiYhIcHe5+rVq6Zq1arZfnaul5GRYcLCwkytWrUc2h955BHj7e1tkpOTs13u6tWrJi0tzYSHh5uRI0fa2zPfn7/uU//+/bP9bF7/s7Fjxw4jybz66qsO/Y4fP268vLzMM888Y4wxZteuXUaSWbVq1U33LTvZ1SLJBAUFmZSUFHtbQkKCKVasmImJibnp+m72Hvbv399IMitWrHBob9eunalSpYp9etmyZUaS+fDDD7Nd95w5c3K4d8Y0a9bM1KhRwz4dExNjihUrluX3Y+bvvE8//dQYY8wrr7xiJNl/9nO7r9m5fPmy6dKli5FkJBkXFxdTt25d8/zzz5vExER7v4sXL5qSJUuajh07Oiyfnp5uateube6991572/Tp03P0uUbuMVKFHDO3OHdfp04dubu76/HHH9eiRYtu+t/pzTzyyCM57lujRg3Vrl3boa13795KSUnRnj178rT9nNq0aZMiIyMVGhrq0D5gwAD9+eefWUa5OnXq5DBdq1YtSf93qi07Fy9e1H//+191797dYfjfxcVFffv21YkTJ25rhGPs2LFyc3OTp6en6tevr2PHjmn+/PlZrg26/j357LPPdPXqVfXr109Xr161vzw9PdWsWTP7qYRDhw7p1KlT6t27t8NpivLly6tJkya3rG/t2rVq0aKFqlWrlut9++WXX/Tjjz+qT58+kuRQZ7t27RQfH28/dps3b1ZkZKT9ekHp2jHu2bNnjrZls9k0cOBAffvtt9q9e7eka6dtPv74Yz3yyCPy8/Oz1xAdHa3q1avL3d1drq6ucnd3188//6wffvgh1/uYnU8++UQ2m02PPvqowz4HBwerdu3a9vemcuXKKlGihMaOHat58+Y5jC7mVYsWLRyu+wkKClJgYOBNP+M5YbPZ1LFjR4e2WrVqOaz3k08+UfHixdWxY0eH/a5Tp46Cg4Nv6/TWJ598ooiICNWpU8dh3W3atHE4dZZ5aq9Hjx5asWKFTp48medtZvLw8FBcXJy+//57zZw5U7169dKZM2f08ssvq1q1avbP8Pbt2/XHH3+of//+DjVmZGTo4Ycf1s6dO7OcqoT1CFXIkYsXL+rs2bMKCQm5YZ9KlSpp48aNCgwM1LBhw1SpUiVVqlRJr732Wq62lZs7bIKDg2/Ydv31LVY7e/ZstrVmHqPrtx8QEOAw7eHhIUk3Hc5PSkqSMSZX28mNESNGaOfOndq9e7d+/fVXxcfH6/HHH8/S7/rtZ55Catiwodzc3Bxe7733nn7//XeH2m72Pt3MmTNn8nyDQmaNY8aMyVLj0KFDJcmhzrzWmGngwIEqVqyY/fTOu+++q7S0NPupP0kaNWqUXnjhBXXp0kUff/yx/vvf/2rnzp2qXbv2bZ2e+qvTp0/LGKOgoKAs+/3111/b99nf319btmxRnTp19Nxzz6lGjRoKCQnRiy++mKNrybJz/WdcuvY5v9198/b2lqenZ5b1Xr582T59+vRpnTt3Tu7u7ln2OyEhwb7feXH69Gl9++23Wdbr6+srY4x93Q8++KBWrVpl/4ejbNmyioiI0LJly/K87UzVqlVTVFSUlixZomPHjmnGjBk6e/as/fKJzM979+7ds9Q5depUGWPydKcgcoe7/5Aja9asUXp6upo3b37Tfg888IAeeOABpaena9euXXrjjTcUFRWloKAg9erVK0fbys1dZQkJCTdsy/wFn/nL+PqLWm/nl2zm+uPj47O0nzp1SpJUqlSp21q/dO0i6GLFit2x7ZQtW9bhbs8buf49ydzmBx98oPLly99wucz34Gbv082ULl36lhdN30hmjePGjVO3bt2y7VOlShV7nXmtMVPZsmXVunVrLV26VK+++qoWLlyoypUr68EHH7T3WbJkifr166fo6GiHZX///XcVL178puv39PTM8hnOXPavSpUqJZvNpq1bt9qD+1/9ta1mzZpavny5jDH69ttvFRsbq0mTJsnLy0vPPvtsjvY7vyhVqpQCAgLs121e768jaHlZt5eXl/1OzuzmZ+rcubM6d+6s1NRUff3114qJiVHv3r1VoUIFNW7cOM81/JXNZtPIkSM1adIkHThwwKGGN95444Z3Lf51JBZ3BqEKt3Ts2DGNGTNG/v7+GjJkSI6WcXFxUaNGjVS1alW9++672rNnj3r16pWj0ZncOHjwoPbv3+9wCnDp0qXy9fW1X3iZeZfSt99+a/8jKkmrV6/Osr7c/FcdGRmpuLg4nTp1ymEE75133pG3t7clt2P7+PioUaNGWrlypV555RV5eXlJunZR7pIlS1S2bFndc889t72d3GrTpo1cXV3166+/3vR0bZUqVVSmTBktW7ZMo0aNsoezo0ePavv27Tcd+ZSktm3bavHixTp06JDDe/dXN/pMValSReHh4dq/f3+WEHO9Fi1aaPXq1Tp9+rT9D096err9RoCcGjx4sNatW6fx48dr3759evnll7PcQXp90FmzZo1OnjypypUr33TdFSpUUGJiokONaWlpWZ6P1aFDB02ZMkUnT55Ujx49clS3zWZT7dq1NXPmTMXGxt7xU+fXs+L3QocOHbR8+XKlp6erUaNGVpVmX3d0dLQCAgIUFhaWo2U8PDzUrFkzFS9eXJ999pn27t2rxo0b53pf4+Pjsx2pPnXqlFJSUlS/fn1J1x6JUrx4cX3//fd68sknb1lbbmpAzhGq4ODAgQP2c/GJiYnaunWrFi5cKBcXF8XFxWW5K+yv5s2bp02bNql9+/YqV66cLl++bP/PrmXLlpKu/bdYvnx5ffTRR4qMjFTJkiVVqlSpPD+dOSQkRJ06ddKECRNUpkwZLVmyRBs2bNDUqVPl7e0t6dopqipVqmjMmDG6evWqSpQoobi4OG3bti3L+mrWrKmVK1dq7ty5ql+/vooVK3bDkZwXX3xRn3zyiVq0aKHx48erZMmSevfdd7VmzRpNmzZN/v7+edqn68XExKhVq1Zq0aKFxowZI3d3d82ZM0cHDhzQsmXL7sjzom6lQoUKmjRpkp5//nn99ttvevjhh1WiRAmdPn1a33zzjXx8fDRx4kQVK1ZML730kh577DF17dpV//znP3Xu3DlNmDAhR6fWJk2apLVr1+rBBx/Uc889p5o1a+rcuXNat26dRo0apapVq6pSpUry8vLSu+++q2rVqumuu+5SSEiIQkJCNH/+fLVt21Zt2rTRgAEDdPfdd+uPP/7QDz/8oD179uj999+XdO1OrNWrV+uhhx7S+PHj5e3trTfffDPX16B06tRJpUqV0vTp0+Xi4qL+/fs7zO/QoYNiY2NVtWpV1apVS7t379b06dNzdIqzZ8+eGj9+vHr16qWnn35aly9f1uuvv57l7tGmTZvq8ccf18CBA7Vr1y49+OCD8vHxUXx8vLZt26aaNWvqf//3f/XJJ59ozpw56tKliypWrChjjFauXKlz586pVatWudrv23Wz9zCnevXqpXfffVft2rXTiBEjdO+998rNzU0nTpzQ5s2b1blzZ3Xt2jVP9UVFRenDDz/Ugw8+qJEjR6pWrVrKyMjQsWPHtH79eo0ePVqNGjXS+PHjdeLECUVGRqps2bI6d+6cXnvtNbm5udkfxJvbfX388cd17tw5PfLII4qIiJCLi4t+/PFHzZw5U8WKFdPYsWMlSXfddZfeeOMN9e/fX3/88Ye6d++uwMBAnTlzRvv379eZM2c0d+5cSdd+z0nSa6+9pv79+8vNzU1VqlS5rdE8/H9OvEge+UjmXV6ZL3d3dxMYGGiaNWtmoqOjHe4yyZTdXUddu3Y15cuXNx4eHiYgIMA0a9bMrF692mG5jRs3mrp16xoPDw8jyfTv399hfWfOnLnltoy5dvdf+/btzQcffGBq1Khh3N3dTYUKFcyMGTOyLP/TTz+Z1q1bGz8/P1O6dGkzfPhws2bNmix3//3xxx+me/fupnjx4sZmszlsU9nctfjdd9+Zjh07Gn9/f+Pu7m5q166d5a6ezLv/3n//fYf27O7wupGtW7eahx56yPj4+BgvLy9z3333mY8//jjb9eXm7r9b9b3VXaGrVq0yLVq0MH5+fsbDw8OUL1/edO/e3WzcuNGh33/+8x8THh5u3N3dzT333GPefvvtG95Bdv0xPn78uBk0aJAJDg42bm5uJiQkxPTo0cOcPn3a3mfZsmWmatWqxs3NLcs69u/fb3r06GECAwONm5ubCQ4ONg899JCZN2+ew3a++uorc9999xkPDw8THBxsnn76abNgwYJc3yU1cuTIbO/2NMaYpKQkM3jwYBMYGGi8vb3N/fffb7Zu3WqaNWtmmjVrZu93o8/Gp59+aurUqWO8vLxMxYoVzezZs7P92TDGmLfffts0atTI/pmpVKmS6devn9m1a5cxxpgff/zR/OMf/zCVKlUyXl5ext/f39x7770mNjb2lvt4o/du2LBhWfqWL1/e/jN+Mzd6D/v37298fHyy9M9uv69cuWJeeeUVU7t2bePp6WnuuusuU7VqVTNkyBDz888/37KGTNff/WeMMRcuXDD/+te/TJUqVYy7u7vx9/c3NWvWNCNHjrTfNfrJJ5+Ytm3bmrvvvtv+O7Rdu3Zm69atOdrX7Hz22Wdm0KBBpnr16sbf39+4urqaMmXKmG7dumV75/CWLVtM+/btTcmSJY2bm5u5++67Tfv27bP8/hk3bpwJCQkxxYoVy/J7EHlnM4bHsQIAANwu7v4DAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAI8/DOHMjIydOrUKfn6+jrlYYsAACD3jDE6f/68QkJCVKzYnR1LIlTl0KlTpxQaGursMgAAQB4cP348z1/QnlOEqhzKfHz/8ePH5efn5+RqAABATqSkpCg0NPRv+RoeQlUOZZ7y8/PzI1QBAFDA/B2X7nChOgAAgAUIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFjA1dkFAChaKjy7xtklFBhHprR3dgkAcsGpI1VffvmlOnbsqJCQENlsNq1atcphvjFGEyZMUEhIiLy8vNS8eXMdPHjQoU9qaqqGDx+uUqVKycfHR506ddKJEycc+iQlJalv377y9/eXv7+/+vbtq3Pnzt3x/QMAAEWHU0PVxYsXVbt2bc2ePTvb+dOmTdOMGTM0e/Zs7dy5U8HBwWrVqpXOnz9v7xMVFaW4uDgtX75c27Zt04ULF9ShQwelp6fb+/Tu3Vv79u3TunXrtG7dOu3bt099+/a94/sHAACKDpsxxji7CEmy2WyKi4tTly5dJF0bpQoJCVFUVJTGjh0r6dqoVFBQkKZOnaohQ4YoOTlZpUuX1uLFi9WzZ09J0qlTpxQaGqpPP/1Ubdq00Q8//KDq1avr66+/VqNGjSRJX3/9tRo3bqwff/xRVapUyVF9KSkp8vf3V3Jysvz8/O7AEQCKBk7/5Ryn/4Db93f+/c63F6ofPnxYCQkJat26tb3Nw8NDzZo10/bt2yVJu3fv1pUrVxz6hISEKCIiwt5nx44d8vf3twcqSbrvvvvk7+9v7wMAAHC78u2F6gkJCZKkoKAgh/agoCAdPXrU3sfd3V0lSpTI0idz+YSEBAUGBmZZf2BgoL1PdlJTU5WammqfTklJyduOAACAIiHfjlRlstlsDtPGmCxt17u+T3b9b7WemJgY+4Xt/v7+Cg0NzWXlAACgKMm3oSo4OFiSsowmJSYm2kevgoODlZaWpqSkpJv2OX36dJb1nzlzJsso2F+NGzdOycnJ9tfx48dva38AAEDhlm9DVVhYmIKDg7VhwwZ7W1pamrZs2aImTZpIkurXry83NzeHPvHx8Tpw4IC9T+PGjZWcnKxvvvnG3ue///2vkpOT7X2y4+HhIT8/P4cXAADAjTj1mqoLFy7ol19+sU8fPnxY+/btU8mSJVWuXDlFRUUpOjpa4eHhCg8PV3R0tLy9vdW7d29Jkr+/vwYPHqzRo0crICBAJUuW1JgxY1SzZk21bNlSklStWjU9/PDD+uc//6n58+dLkh5//HF16NAhx3f+AQAA3IpTQ9WuXbvUokUL+/SoUaMkSf3791dsbKyeeeYZXbp0SUOHDlVSUpIaNWqk9evXy9fX177MzJkz5erqqh49eujSpUuKjIxUbGysXFxc7H3effddPfXUU/a7BDt16nTDZ2MBAADkRb55TlV+x3OqAGvwnKqc4zlVwO3jOVUAAAAFDKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAAL5OtQdfXqVf3rX/9SWFiYvLy8VLFiRU2aNEkZGRn2PsYYTZgwQSEhIfLy8lLz5s118OBBh/WkpqZq+PDhKlWqlHx8fNSpUyedOHHi794dAABQiOXrUDV16lTNmzdPs2fP1g8//KBp06Zp+vTpeuONN+x9pk2bphkzZmj27NnauXOngoOD1apVK50/f97eJyoqSnFxcVq+fLm2bdumCxcuqEOHDkpPT3fGbgEAgELI1dkF3MyOHTvUuXNntW/fXpJUoUIFLVu2TLt27ZJ0bZRq1qxZev7559WtWzdJ0qJFixQUFKSlS5dqyJAhSk5O1ltvvaXFixerZcuWkqQlS5YoNDRUGzduVJs2bZyzcwAAoFDJ1yNV999/vz7//HP99NNPkqT9+/dr27ZtateunSTp8OHDSkhIUOvWre3LeHh4qFmzZtq+fbskaffu3bpylXHKAAAAIABJREFU5YpDn5CQEEVERNj7ZCc1NVUpKSkOLwAAgBvJ1yNVY8eOVXJysqpWrSoXFxelp6fr5Zdf1j/+8Q9JUkJCgiQpKCjIYbmgoCAdPXrU3sfd3V0lSpTI0idz+ezExMRo4sSJVu4OAAAoxPL1SNV7772nJUuWaOnSpdqzZ48WLVqkV155RYsWLXLoZ7PZHKaNMVnarnerPuPGjVNycrL9dfz48bzvCAAAKPTy9UjV008/rWeffVa9evWSJNWsWVNHjx5VTEyM+vfvr+DgYEnXRqPKlCljXy4xMdE+ehUcHKy0tDQlJSU5jFYlJiaqSZMmN9y2h4eHPDw87sRuAQCAQihfj1T9+eefKlbMsUQXFxf7IxXCwsIUHBysDRs22OenpaVpy5Yt9sBUv359ubm5OfSJj4/XgQMHbhqqAAAAciNfj1R17NhRL7/8ssqVK6caNWpo7969mjFjhgYNGiTp2mm/qKgoRUdHKzw8XOHh4YqOjpa3t7d69+4tSfL399fgwYM1evRoBQQEqGTJkhozZoxq1qxpvxsQAADgduXrUPXGG2/ohRde0NChQ5WYmKiQkBANGTJE48ePt/d55plndOnSJQ0dOlRJSUlq1KiR1q9fL19fX3ufmTNnytXVVT169NClS5cUGRmp2NhYubi4OGO3AABAIWQzxhhnF1EQpKSkyN/fX8nJyfLz83N2OUCBVeHZNc4uocA4MqW9s0sACry/8+93vr6mCgAAoKAgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAF8hSqDh8+bHUdAAAABVqeQlXlypXVokULLVmyRJcvX7a6JgAAgAInT6Fq//79qlu3rkaPHq3g4GANGTJE33zzjdW1AQAAFBh5ClURERGaMWOGTp48qYULFyohIUH333+/atSooRkzZujMmTNW1wkAAJCv3daF6q6ururatatWrFihqVOn6tdff9WYMWNUtmxZ9evXT/Hx8VbVCQAAkK/dVqjatWuXhg4dqjJlymjGjBkaM2aMfv31V23atEknT55U586draoTAAAgX8tTqJoxY4Zq1qypJk2a6NSpU3rnnXd09OhRTZ48WWFhYWratKnmz5+vPXv23HaBJ0+e1KOPPqqAgAB5e3urTp062r17t32+MUYTJkxQSEiIvLy81Lx5cx08eNBhHampqRo+fLhKlSolHx8fderUSSdOnLjt2gAAADLlKVTNnTtXvXv31rFjx7Rq1Sp16NBBxYo5rqpcuXJ66623bqu4pKQkNW3aVG5ublq7dq2+//57vfrqqypevLi9z7Rp0zRjxgzNnj1bO3fuVHBwsFq1aqXz58/b+0RFRSkuLk7Lly/Xtm3bdOHCBXXo0EHp6em3VR8AAEAmmzHGOLuIG3n22Wf11VdfaevWrdnON8YoJCREUVFRGjt2rKRro1JBQUGaOnWqhgwZouTkZJUuXVqLFy9Wz549JUmnTp1SaGioPv30U7Vp0yZHtaSkpMjf31/Jycny8/OzZgeBIqjCs2ucXUKBcWRKe2eXABR4f+ff7zyNVC1cuFDvv/9+lvb3339fixYtuu2iMq1evVoNGjTQ//zP/ygwMFB169bVv//9b/v8w4cPKyEhQa1bt7a3eXh4qFmzZtq+fbskaffu3bpy5YpDn5CQEEVERNj7ZCc1NVUpKSkOLwAAgBvJU6iaMmWKSpUqlaU9MDBQ0dHRt11Upt9++01z585VeHi4PvvsMz3xxBN66qmn9M4770iSEhISJElBQUEOywUFBdnnJSQkyN3dXSVKlLhhn+zExMTI39/f/goNDbVsvwAAQOGTp1B19OhRhYWFZWkvX768jh07dttFZcrIyFC9evUUHR2tunXrasiQIfrnP/+puXPnOvSz2WwO08aYLG3Xu1WfcePGKTk52f46fvx43ncEAAAUenkKVYGBgfr222+ztO/fv18BAQG3XVSmMmXKqHr16g5t1apVswe34OBgScoy4pSYmGgfvQoODlZaWpqSkpJu2Cc7Hh4e8vPzc3gBAADcSJ5CVa9evfTUU09p8+bNSk9PV3p6ujZt2qQRI0aoV69elhXXtGlTHTp0yKHtp59+Uvny5SVJYWFhCg4O1oYNG+zz09LStGXLFjVp0kSSVL9+fbm5uTn0iY+P14EDB+x9AAAAbpdrXhaaPHmyjh49qsjISLm6XltFRkaG+vXrZ+k1VSNHjlSTJk0UHR2tHj166JtvvtGCBQu0YMECSddO+0VFRSk6Olrh4eEKDw9XdHS0vL291bt3b0mSv7+/Bg8erNGjRysgIEAlS5bUmDFjVLNmTbVs2dKyWgEAQNGWp1Dl7u6u9957Ty+99JL2798vLy8v1axZ0z6CZJWGDRsqLi5O48aN06RJkxQWFqZZs2apT58+9j7PPPOMLl26pKFDhyopKUmNGjXS+vXr5evra+8zc+ZMubq6qkePHrp06ZIiIyMVGxsrFxcXS+sFAABFV75+TlV+wnOqAGvwnKqc4zlVwO37O/9+52mkKj09XbGxsfr888+VmJiojIwMh/mbNm2ypDgAAICCIk+hasSIEYqNjVX79u0VERFxy8cXAAAAFHZ5ClXLly/XihUr1K5dO6vrAQAAKJDy9EgFd3d3Va5c2epaAAAACqw8harRo0frtddeE9e4AwAAXJOn03/btm3T5s2btXbtWtWoUUNubm4O81euXGlJcQAAAAVFnkJV8eLF1bVrV6trAQAAKLDyFKoWLlxodR0AAAAFWp6uqZKkq1evauPGjZo/f77Onz8vSTp16pQuXLhgWXEAAAAFRZ5Gqo4ePaqHH35Yx44dU2pqqlq1aiVfX19NmzZNly9f1rx586yuEwAAIF/L00jViBEj1KBBAyUlJcnLy8ve3rVrV33++eeWFQcAAFBQ5Pnuv6+++kru7u4O7eXLl9fJkyctKQwAAKAgydNIVUZGhtLT07O0nzhxQr6+vrddFAAAQEGTp1DVqlUrzZo1yz5ts9l04cIFvfjii3x1DQAAKJLydPpv5syZatGihapXr67Lly+rd+/e+vnnn1WqVCktW7bM6hoBAADyvTyFqpCQEO3bt0/Lli3Tnj17lJGRocGDB6tPnz4OF64DAAAUFXkKVZLk5eWlQYMGadCgQVbWAwAAUCDlKVS98847N53fr1+/PBUDAABQUOUpVI0YMcJh+sqVK/rzzz/l7u4ub29vQhUAAChy8nT3X1JSksPrwoULOnTokO6//34uVAcAAEVSnr/773rh4eGaMmVKllEsAACAosCyUCVJLi4uOnXqlJWrBAAAKBDydE3V6tWrHaaNMYqPj9fs2bPVtGlTSwoDAAAoSPIUqrp06eIwbbPZVLp0aT300EN69dVXLSkMAACgIMlTqMrIyLC6DgAAgALN0muqAAAAiqo8jVSNGjUqx31nzJiRl00AAAAUKHkKVXv37tWePXt09epVValSRZL0008/ycXFRfXq1bP3s9ls1lQJAACQz+UpVHXs2FG+vr5atGiRSpQoIenaA0EHDhyoBx54QKNHj7a0SAAAgPwuT9dUvfrqq4qJibEHKkkqUaKEJk+ezN1/AACgSMpTqEpJSdHp06eztCcmJur8+fO3XRQAAEBBk6dQ1bVrVw0cOFAffPCBTpw4oRMnTuiDDz7Q4MGD1a1bN6trBAAAyPfydE3VvHnzNGbMGD366KO6cuXKtRW5umrw4MGaPn26pQUCAAAUBHkKVd7e3pozZ46mT5+uX3/9VcYYVa5cWT4+PlbXBwAAUCDc1sM/4+PjFR8fr3vuuUc+Pj4yxlhVFwAAQIGSp1B19uxZRUZG6p577lG7du0UHx8vSXrsscd4nAIAACiS8hSqRo4cKTc3Nx07dkze3t729p49e2rdunWWFQcAAFBQ5OmaqvXr1+uzzz5T2bJlHdrDw8N19OhRSwoDAAAoSPI0UnXx4kWHEapMv//+uzw8PG67KAAAgIImT6HqwQcf1DvvvGOfttlsysjI0PTp09WiRQvLigMAACgo8nT6b/r06WrevLl27dqltLQ0PfPMMzp48KD++OMPffXVV1bXCAAAkO/laaSqevXq+vbbb3XvvfeqVatWunjxorp166a9e/eqUqVKVtcIAACQ7+V6pOrKlStq3bq15s+fr4kTJ96JmgAAAAqcXI9Uubm56cCBA7LZbHeiHgAAgAIpT6f/+vXrp7feesvqWgAAAAqsPF2onpaWpv/85z/asGGDGjRokOU7/2bMmGFJcQAAAAVFrkLVb7/9pgoVKujAgQOqV6+eJOmnn35y6MNpQQAAUBTlKlSFh4crPj5emzdvlnTta2lef/11BQUF3ZHiAAAACopcXVNljHGYXrt2rS5evGhpQQAAAAVRni5Uz3R9yAIAACiqchWqbDZblmumuIYKAAAgl9dUGWM0YMAA+5cmX758WU888USWu/9WrlxpXYUAAAAFQK5CVf/+/R2mH330UUuLAQAAKKhyFaoWLlx4p+oAAAAo0G7rQnUAAABcQ6gCAACwAKEKAADAAoQqAAAACxSoUBUTEyObzaaoqCh7mzFGEyZMUEhIiLy8vNS8eXMdPHjQYbnU1FQNHz5cpUqVko+Pjzp16qQTJ0783eUDAIBCrMCEqp07d2rBggWqVauWQ/u0adM0Y8YMzZ49Wzt37lRwcLBatWql8+fP2/tERUUpLi5Oy5cv17Zt23ThwgV16NBB6enpf/duAACAQipXj1RwlgsXLqhPnz7697//rcmTJ9vbjTGaNWuWnn/+eXXr1k2StGjRIgUFBWnp0qUaMmSIkpOT9dZbb2nx4sVq2bKlJGnJkiUKDQ3Vxo0b1aZNG6fs019VeHaNs0soMI5Mae/sEgAAyFaBGKkaNmyY2rdvbw9FmQ4fPqyEhAS1bt3a3ubh4aFmzZpp+/btkqTdu3frypUrDn1CQkIUERFh75Od1NRUpaSkOLwAAABuJN+PVC1fvlx79uzRzp07s8xLSEiQJAUFBTm0BwUF6ejRo/Y+7u7uKlGiRJY+mctnJyYmRhMnTrzd8gEAQBGRr0eqjh8/rhEjRmjJkiXy9PS8Yb/rv9TZGHPLL3q+VZ9x48YpOTnZ/jp+/HjuigcAAEVKvg5Vu3fvVmJiourXry9XV1e5urpqy5Ytev311+Xq6mofobp+xCkxMdE+Lzg4WGlpaUpKSrphn+x4eHjIz8/P4QUAAHAj+TpURUZG6rvvvtO+ffvsrwYNGqhPnz7at2+fKlasqODgYG3YsMG+TFpamrZs2aImTZpIkurXry83NzeHPvHx8Tpw4IC9DwAAwO3K19dU+fr6KiIiwqHNx8dHAQEB9vaoqChFR0crPDxc4eHhio6Olre3t3r37i1J8vf31+DBgzV69GgFBASoZMmSGjNmjGrWrJnlwncAAIC8ytehKieeeeYZXbp0SUOHDlVSUpIaNWqk9evXy9fX195n5syZcnV1VY8ePXTp0iVFRkYqNjZWLi4uTqwcAAAUJjZjjHF2EQVBSkqK/P39lZycbPn1VTynKud4TlXBx+c95/i8A7fvTv79vl6+vqYKAACgoCBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFigwD+nCgBwazzKIud4lAXyipEqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsEC+DlUxMTFq2LChfH19FRgYqC5duujQoUMOfYwxmjBhgkJCQuTl5aXmzZvr4MGDDn1SU1M1fPhwlSpVSj4+PurUqZNOnDjxd+4KAAAo5PJ1qNqyZYuGDRumr7/+Whs2bNDVq1fVunVrXbx40d5n2rRpmjFjhmbPnq2dO3cqODhYrVq10vnz5+19oqKiFBcXp+XLl2vbtm26cOGCOnTooPT0dGfsFgAAKIRcnV3Azaxbt85heuHChQoMDNTu3bv14IMPyhijWbNm6fnnn1e3bt0kSYsWLVJQUJCWLl2qIUOGKDk5WW+99ZYWL16sli1bSpKWLFmi0NBQbdy4UW3atPnb9wsAABQ++Xqk6nrJycmSpJIlS0qSDh8+rISEBLVu3drex8PDQ82aNdP27dslSbt379aVK1cc+oSEhCgiIsLeJzupqalKSUlxeAEAANxIgQlVxhiNGjVK999/vyIiIiRJCQkJkqSgoCCHvkFBQfZ5CQkJcnd3V4kSJW7YJzsxMTHy9/e3v0JDQ63cHQAAUMgUmFD15JNP6ttvv9WyZcuyzLPZbA7Txpgsbde7VZ9x48YpOTnZ/jp+/HjeCgcAAEVCgQhVw4cP1+rVq7V582aVLVvW3h4cHCxJWUacEhMT7aNXwcHBSktLU1JS0g37ZMfDw0N+fn4OLwAAgBvJ16HKGKMnn3xSK1eu1KZNmxQWFuYwPywsTMHBwdqwYYO9LS0tTVu2bFGTJk0kSfXr15ebm5tDn/j4eB04cMDeBwAA4Hbl67v/hg0bpqVLl+qjjz6Sr6+vfUTK399fXl5estlsioqKUnR0tMLDwxUeHq7o6Gh5e3urd+/e9r6DBw/W6NGjFRAQoJIlS2rMmDGqWbOm/W5AAACA25WvQ9XcuXMlSc2bN3doX7hwoQYMGCBJeuaZZ3Tp0iUNHTpUSUlJatSokdavXy9fX197/5kzZ8rV1VU9evTQpUuXFBkZqdjYWLm4uPxduwIAAAq5fB2qjDG37GOz2TRhwgRNmDDhhn08PT31xhtv6I033rCwOgAAgP+Tr6+pAgAAKCgIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAUIVQAAABYgVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAAAFiBUAQAAWIBQBQAAYAFCFQAAgAVcnV0A4CwVnl3j7BIKjCNT2ju7BADI9xipAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALAAoQoAAMAChCoAAAALEKoAAAAsQKgCAACwAKEKAADAAoQqAAAACxCqAAAALECoAgAAsICrswsAAKCwqvDsGmeXUGAcmdLe2SXcNkaqAAAALECoAgAAsAChCgAAwAKEKgAAAAsQqgAAACxAqAIAALBAkQpVc+bMUVhYmDw9PVW/fn1t3brV2SUBAIBCosiEqvfee09RUVF6/vnntXfvXj3wwANq27atjh075uzSAABAIVBkQtWMGTM0ePBgPfbYY6pWrZpmzZql0NBQzZ0719mlAQCAQqBIhKq0tDTt3r1brVu3dmhv3bq1tm/f7qSqAABAYVIkvqbm999/V3p6uoKCghzag4KClJCQkO0yqampSk1NtU8nJydLklJSUiyvLyP1T8vXWVhZefw57jnHcXcOjrtzcNyd4078ff3reo0xd2T9f1UkQlUmm83mMG2MydKWKSYmRhMnTszSHhoaekdqQ874z3J2BUUTx905OO7OwXF3jjt93M+fPy9/f/87uo0iEapKlSolFxeXLKNSiYmJWUavMo0bN06jRo2yT2dkZOiPP/5QQEDADYNYYZKSkqLQ0FAdP35cfn5+zi6nyOC4OwfH3Tk47s5R1I67MUbnz59XSEjIHd9WkQhV7u7uql+/vjZs2KCuXbva2zds2KDOnTtnu4yHh4c8PDwc2ooXL35H68yP/Pz8isQPXX7DcXcOjrtzcNydoygd9zs9QpWpSIQqSRo1apT69u2rBg0aqHHjxlqwYIGOHTumJ554wtmlAQCAQqDIhKqePXvq7NmzmjRpkuLj4xUREaFPP/1U5cuXd3ZpAACgECgyoUqShg4dqqFDhzq7jALBw8NDL774YpZToLizOO7OwXF3Do67c3Dc7xyb+TvuMQQAACjkisTDPwEAAO40QhUAAIAFCFUAAAAWIFQBAABYgFCFLObMmaOwsDB5enqqfv362rp1q7NLKvS+/PJLdezYUSEhIbLZbFq1apWzSyr0YmJi1LBhQ/n6+iowMFBdunTRoUOHnF1WoTd37lzVqlXL/uDJxo0ba+3atc4uq8iJiYmRzWZTVFSUs0spVAhVcPDee+8pKipKzz//vPbu3asHHnhAbdu21bFjx5xdWqF28eJF1a5dW7Nnz3Z2KUXGli1bNGzYMH399dfasGGDrl69qtatW+vixYvOLq1QK1u2rKZMmaJdu3Zp165deuihh9S5c2cdPHjQ2aUVGTt37tSCBQtUq1YtZ5dS6PBIBTho1KiR6tWrp7lz59rbqlWrpi5duigmJsaJlRUdNptNcXFx6tKli7NLKVLOnDmjwMBAbdmyRQ8++KCzyylSSpYsqenTp2vw4MHOLqXQu3DhgurVq6c5c+Zo8uTJqlOnjmbN4hukrcJIFezS0tK0e/dutW7d2qG9devW2r59u5OqAv4eycnJkq79gcffIz09XcuXL9fFixfVuHFjZ5dTJAwbNkzt27dXy5YtnV1KoVSknqiOm/v999+Vnp6uoKAgh/agoCAlJCQ4qSrgzjPGaNSoUbr//vsVERHh7HIKve+++06NGzfW5cuXdddddykuLk7Vq1d3dlmF3vLly7Vnzx7t3LnT2aUUWoQqZGGz2RymjTFZ2oDC5Mknn9S3336rbdu2ObuUIqFKlSrat2+fzp07pw8//FD9+/fXli1bCFZ30PHjxzVixAitX79enp6ezi6n0CJUwa5UqVJycXHJMiqVmJiYZfQKKCyGDx+u1atX68svv1TZsmWdXU6R4O7ursqVK0uSGjRooJ07d+q1117T/PnznVxZ4bV7924lJiaqfv369rb09HR9+eWXmj17tlJTU+Xi4uLECgsHrqmCnbu7u+rXr68NGzY4tG/YsEFNmjRxUlXAnWGM0ZNPPqmVK1dq06ZNCgsLc3ZJRZYxRqmpqc4uo1CLjIzUd999p3379tlfDRo0UJ8+fbRv3z4ClUUYqYKDUaNGqW/fvmrQoIEaN26sBQsW6NixY3riiSecXVqhduHCBf3yyy/26cOHD2vfvn0qWbKkypUr58TKCq9hw4Zp6dKl+uijj+Tr62sfofX395eXl5eTqyu8nnvuObVt21ahoaE6f/68li9fri+++ELr1q1zdmmFmq+vb5brBX18fBQQEMB1hBYiVMFBz549dfbsWU2aNEnx8fGKiIjQp59+qvLlyzu7tEJt165datGihX161KhRkqT+/fsrNjbWSVUVbpmPDWnevLlD+8KFCzVgwIC/v6Ai4vTp0+rbt6/i4+Pl7++vWrVqad26dWrVqpWzSwNuG8+pAgAAsADXVAEAAFiAUAUAAGABQhUAAIAFCFUAAAAWIFQBAABYgFAFAABgAUIVAACABQhVAAoMm82mVatWObuMXPniiy9ks9l07tw5Z5cC4A4jVAFwugEDBshms8lms8nNzU1BQUFq1aqV3n77bWVkZNj7xcfHq23btk6sNPeaNGlif3o4gMKNUAUgX3j44YcVHx+vI0eOaO3atWrRooVGjBihDh066OrVq5Kk4OBgeXh4OLnS3HF3d1dwcLBsNpuzSwFwhxGqAOQLHh4eCg4O1t1336169erpueee00cffaS1a9fav//w+tN/Y8eO1T333CNvb29VrFhRL7zwgq5cueKw3smTJyswMFC+vr567LHH9Oyzz6pOnTr2+QMGDFCXLl30yiuvqEyZMgoICNCwYcMc1pOUlKR+/fqpRIkS8vb2Vtu2bfXzzz/b5x89elQdO3ZUiRIl5OPjoxo1aujTTz/9f+3dTUhUXRzH8e/T6CpHEDdOIaaoo0XYYkzQaCGEb8mAkgZRGXkhMMNQVPB1UbiSIEUUFwYRKYRKpIj5AoFQIQUOOJQiNLhQQdqIOkMyLcLhmUafhwHBGfl9VnPP+d/Dubsf55y5Fwjc/vuvWhEJb/qgsoiErNzcXDIyMhgeHqaysjKg32w28+LFC86cOYPD4cAwDMxmM/X19QC8evWKp0+f0tPTQ05ODoODg3R2dpKYmOg3zuzsLBaLhdnZWZaXlykvL+fSpUsYhgH8CV5LS0u8ffuW6OhoGhoaKCwsZHFxkcjISKqqqvB4PHz48IGpo9hBAAADL0lEQVTTp0+zuLhIVFTUgc8UTK2IhBeFKhEJaWlpaSwsLBzY19zc7Pt97tw5amtrGRoa8oWqrq4u7t+/z7179wBobW1lcnKSra0tv3FiYmLo7u7GZDKRlpZGUVER09PTGIbhC1Nzc3NkZ2cDf8JafHw8o6Oj3LhxA5fLRWlpKRcvXgQgKSnp0OcJplZEwou2/0QkpHm93kPPI71584YrV64QFxdHVFQULS0tuFwuX/+3b9+4fPmy3z1/XwNcuHABk8nku7ZYLGxsbADgdDqJiIggKyvL1x8bG4vVasXpdALw6NEjnjx5Qk5ODm1tbYeGwGBrRSS8KFSJSEhzOp0B23UAHz9+5ObNmxQUFPDu3Tu+fv1KU1MTHo/Hr+7vQOb1egPGioyMDLhn/1+HB9Xvt++PXVlZycrKCrdv38bhcGCz2ejq6jrwvmBqRSS8KFSJSMiamZnB4XBQWloa0Dc3N0dCQgJNTU3YbDZSUlL48eOHX43VauXz589+bfPz80HN4fz58/z69YtPnz752jY3N/n+/Tvp6em+tvj4eB48eMDw8DC1tbX09/cfOmYwtSISPnSmSkRCgtvtZm1tjb29PdbX15mYmKCjo4Pr169z586dgPrk5GRcLheDg4NkZmYyNjbGyMiIX011dTWGYWCz2cjOzmZoaIiFhYWgzjGlpKRgt9sxDIO+vj7MZjONjY2cPXsWu90OQE1NDQUFBaSmpvLz509mZmb8Ate/BVMrIuFFoUpEQsLExAQWi4WIiAhiYmLIyMjg+fPn3L17l1OnAhfV7XY7jx8/5uHDh7jdboqKimhpaaG9vd1Xc+vWLVZWVqirq2N3d5eysjIqKioCVq/+z8DAgO+dWR6Ph6tXrzI+Pu7bNtzb26OqqorV1VWio6PJz8/n2bNnB44VTK2IhJd/vIcdGBAROYGuXbtGXFwcL1++PO6piMgJo5UqETmxtre36e3tJS8vD5PJxOvXr5mamuL9+/fHPTUROYG0UiUiJ9bOzg7FxcV8+fIFt9uN1WqlubmZkpKS456aiJxAClUiIiIiR0CvVBARERE5AgpVIiIiIkdAoUpERETkCPwGahEo3xu2O70AAAAASUVORK5CYII="}}},{"cell_type":"code","source":"y_train_one_hot = to_categorical(y_train, num_classes=5)\ny_val_one_hot = to_categorical(y_val, num_classes=5)\n\nweights_path = '/kaggle/input/densenet201-weights/densenet201_weights_tf_dim_ordering_tf_kernels_notop.h5'\nmodel = create_densenet201(weights_path=weights_path)\n\ntrain_generator = data_generator.flow(x_train, y_train_one_hot, batch_size=BATCH_SIZE)\nvalidation_data = (x_val, y_val_one_hot)\n\n# Add early stopping, ReduceLROnPlateau, and model checkpoint callbacks\nearly_stopping = EarlyStopping(monitor='val_loss', patience=10, verbose=1, restore_best_weights=True)\nreduce_lr_on_plateau = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3, verbose=1)\nmodel_checkpoint = ModelCheckpoint(\"model.h5\", monitor='val_loss', verbose=1, save_best_only=True, mode='min')\ncallbacks = [early_stopping, reduce_lr_on_plateau, model_checkpoint]\n\nval_data_generator = ImageDataGenerator()\nval_generator = val_data_generator.flow(x_val, y_val_one_hot, batch_size=BATCH_SIZE)\n\nhistory = model.fit_generator(train_generator,\n                              steps_per_epoch=x_train.shape[0] / BATCH_SIZE,\n                              epochs=20,\n                              validation_data=val_generator,\n                              validation_steps=x_val.shape[0] / BATCH_SIZE,\n                              callbacks=callbacks)  # Add callbacks parameter here","metadata":{"papermill":{"duration":695.995179,"end_time":"2023-04-27T09:11:01.517309","exception":false,"start_time":"2023-04-27T08:59:25.522130","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:18:18.836122Z","iopub.execute_input":"2023-04-29T20:18:18.836771Z","iopub.status.idle":"2023-04-29T20:44:54.884860Z","shell.execute_reply.started":"2023-04-29T20:18:18.836732Z","shell.execute_reply":"2023-04-29T20:44:54.883878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Graph the Output\nThe first plot shows the accuracy of the model on the training and validation datasets as the number of epochs increases. The y-axis represents the accuracy score, while the x-axis represents the number of epochs. The blue line represents the accuracy on the training dataset, while the orange line represents the accuracy on the validation dataset.\n\nThe second plot shows the loss of the model on the training and validation datasets as the number of epochs increases. The y-axis represents the loss score, while the x-axis represents the number of epochs. The blue line represents the loss on the training dataset, while the orange line represents the loss on the validation dataset.","metadata":{"papermill":{"duration":0.234126,"end_time":"2023-04-27T09:11:02.011081","exception":false,"start_time":"2023-04-27T09:11:01.776955","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Plot training & validation accuracy values\nplt.plot(history.history['accuracy'])\nplt.plot(history.history['val_accuracy'])\nplt.title('Model accuracy')\nplt.ylabel('Accuracy')\nplt.xlabel('Epoch')\nplt.legend(['Train', 'Validation'], loc='upper left')\nplt.show()\n\n# Plot training & validation loss values\nplt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('Model loss')\nplt.ylabel('Loss')\nplt.xlabel('Epoch')\nplt.legend(['Train', 'Validation'], loc='upper left')\nplt.show()\n","metadata":{"papermill":{"duration":0.552912,"end_time":"2023-04-27T09:11:02.797485","exception":true,"start_time":"2023-04-27T09:11:02.244573","status":"failed"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:44:54.886612Z","iopub.execute_input":"2023-04-29T20:44:54.886977Z","iopub.status.idle":"2023-04-29T20:44:55.353215Z","shell.execute_reply.started":"2023-04-29T20:44:54.886938Z","shell.execute_reply":"2023-04-29T20:44:55.352234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction on Test Set\nThis section predicts the severity of diabetic retinopathy in the test set images.\n\nFirstly, it initializes the number of test images and creates an empty array of dimensions (N, 299, 299, 3), where N is the number of test images.\n\nNext, the code runs a for loop to iterate over all the test image IDs in the test_df dataframe, and preprocesses each image using the preprocess_image function. The output of this function is then stored in the empty x_test array created earlier, at index i.\n\nAfter all the test images have been preprocessed and stored in the x_test array, the model weights are loaded from the saved .h5 file using the load_weights() function. The model is then used to predict the probability of diabetic retinopathy in each of the test images using the predict() method, and the resulting probabilities are stored in the 'predictions' array.\n\nThe argmax() function is used to obtain the index of the maximum probability for each test image. These indices correspond to the predicted severity level for each image (0, 1, 2, 3, or 4).\n\nFinally, the predicted severity levels are stored in the predicted_labels array, which can be used to evaluate the performance of the model on the test set.","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]}},{"cell_type":"code","source":"N = test_df.shape[0]\nx_test = np.empty((N, 299, 299, 3), dtype=np.uint8)\n\nfor i, image_id in enumerate(tqdm(test_df['id_code'])):\n    x_test[i, :, :, :] = preprocess_image(\n        f'../input/aptos2019-blindness-detection/test_images/{image_id}.png',\n        desired_size=299\n    )\n\nmodel.load_weights(\"model.h5\")\npredictions = model.predict(x_test)\npredicted_labels = np.argmax(predictions, axis=1)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:44:55.356689Z","iopub.execute_input":"2023-04-29T20:44:55.356981Z","iopub.status.idle":"2023-04-29T20:49:32.247909Z","shell.execute_reply.started":"2023-04-29T20:44:55.356953Z","shell.execute_reply":"2023-04-29T20:49:32.246716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualizing the Distribution of Predicted Probabilities\nIn this section, a histogram is plotted to visualize the distribution of the predicted labels from the model on the test set. The plt.hist() function is used to create a histogram of the predicted labels, with the bins representing the different diagnosis levels (0-4). \n\nThis plot can be helpful in understanding the distribution of predicted labels and can provide insights into the performance of the model.\n","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]}},{"cell_type":"code","source":"plt.hist(predicted_labels, bins=[-0.5, 0.5, 1.5, 2.5, 3.5, 4.5], rwidth=0.8)\nplt.xlabel('Diagnosis')\nplt.ylabel('Frequency')\nplt.title('Distribution of Predicted Values in the Test Set')\nplt.xticks(range(5))\nplt.show()\n","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:51:56.679928Z","iopub.execute_input":"2023-04-29T20:51:56.680700Z","iopub.status.idle":"2023-04-29T20:51:56.870081Z","shell.execute_reply.started":"2023-04-29T20:51:56.680658Z","shell.execute_reply":"2023-04-29T20:51:56.869114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Save the submission\nThis section of the code creates a submission file for the Kaggle competition using the predictions generated by the trained model on the test data.","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]}},{"cell_type":"code","source":"test_df['diagnosis'] = predictions.argmax(axis=1)\ntest_df.to_csv('submission.csv', index=False)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2023-04-29T20:56:28.526462Z","iopub.execute_input":"2023-04-29T20:56:28.526844Z","iopub.status.idle":"2023-04-29T20:56:28.537862Z","shell.execute_reply.started":"2023-04-29T20:56:28.526808Z","shell.execute_reply":"2023-04-29T20:56:28.536827Z"},"trusted":true},"execution_count":null,"outputs":[]}]}