{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceType":"competition","sourceId":14774,"databundleVersionId":875431},{"sourceType":"datasetVersion","sourceId":5792201,"datasetId":3327349,"databundleVersionId":5868879},{"sourceType":"datasetVersion","sourceId":2822362,"datasetId":1725862,"databundleVersionId":2868791},{"sourceType":"datasetVersion","sourceId":5790616,"datasetId":3326528,"databundleVersionId":5867290},{"sourceType":"datasetVersion","sourceId":2822109,"datasetId":1725813,"databundleVersionId":2868531},{"sourceType":"datasetVersion","sourceId":952401,"datasetId":517172,"databundleVersionId":980293}],"dockerImageVersionId":30097,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### **This notebook contains a custom callback you may wish to copy and use  \nIt is a combination of the Keras callbacks Reduce Learning Rate on Plateau,  \nEarly Stopping and Model Checkpoint but eliminates some of the limitations  \nof each. In addition it provides an easier to read summary of the model's  \nperformance at the end of each epoch. It also provides a handy feature  \nthat enables you to set the number of epochs to train for until a message  \nasks if you wish to halt training on the current epoch by entering H or  \nto enter an integer which will determine how many more epochs to run  \nbefore the message appears again. This is very useful if you are training  \na model and decide the metrics are satisfactory and you want to end  \nthe model training early. Note the callback always returns your model  \nwith the weights set to those of the epoch which had the highest performance  \non the metric being monitored (accuracy or validation accuracy)  \nThe callback initially monitors training accuracy and will adjust the learning  \nrate based on that until the accuracy reaches a user specified threshold  \nlevel. Once that level of training accuracy is achieved the callback switches  \nto monitoring validation loss and adjusts the learning rate based on that.  \nthe callback is of the form:  \ncallbacks=[LRA(model, patience, stop_patience, threshold,factor, dwell,\n               model_name, freeze, batches,initial_epoch, epochs, ask_epoch )]    \n **where:**\n - **model** is your compiled model\n - **patience** is an integer that determines many consecutive epochs can occur before the learning rate\n   will be adjusted (similar to patience parameter in Reduce Learning Rate on Plateau)\n \n - **stop_patience** is an integer that determines hom many consecutive epochs for which the\n   learning rate was adjusted but no improvement in the monitored metric occurred before\n   training is halted(similar to patience parameter in early stopping)\n \n - **threshold** is a float that determines the level that training accuracy must achieve\n   before the callback switches over to monitoring validation loss. This  is useful for\n   cases where the validation loss in early epochs tends to vary widely and can cause\n   unwanted behavior when using the conventional Keras callbacks\n - **factor** is a float that determines the new learning rate by the equation lr=lr*factor.\n   (similar to the factor parameter in Reduce Learning Rate on Plateau)\n - **dwell** is a boolean. It is used in the callback as part of an experiment on training\n   models. If on a given epoch the metric being monitored fails to improve it means\n   your model has moved to a location on the surface of Nspace (where N is the number\n   of trainable parameters) that is NOT as favorable (poorer metric performance) than\n   the position in Nspace you were in for the previous epoch. If dwell is set to True\n   the callback loads the model with the weights from the previous (better metric value)\n   epoch. Why move to a worse place if the place you were in previously was better. Then\n   the learning rate is reduced for the next epoch of training. If dwell is set to false\n   this action does not take place.\n \n - **model_name** is a string\n - **freeze** is just a boolean\n   Note model_name and freeze are simply used to print out a header at the start of training.\n   For example is model_name is set to 'InceptionResnetV2' and freeze is set to False\n   the header would appear as 'Training InceptionResnetV2 traing all layers'. If freeze\n   is set to False the header would be 'Training InceptionRenetV2 base model layers frozen'\n   Parameter freese does not make the basemodel non trainable you must do that in\n   seperate code when building your model.\n - **batches** is an integer. It should be set to a value of \n   batches=int(number of traing samples/batch_size). During training the callback provides\n   information during an epoch of the form\n   'processing batch of batches  accuracy= accuracy  loss= loss where batch is the current \n    batch being processs, batches is as described above, accuracy is the current training\n    accuracy and loss is the current loss. Typically the message would appear as\n    processing batch 25 of 50  accuracy: 54%  loss: .04567. As each batch is processed\n    these values are changed.    \n - **initial_epoch** is an integer. Typically set this to zero Itis used in the information\n    printed out for each epoch. In the case where you train the model say with the\n    basemodel weights frozen say you train for 10 epochs. Then you want to fine tune\n    the model and train for more eppochs for the second training session you would\n    reinstantiate the callback and set initial_epoch=10.\n - **epochs** an integer value for the number of epochs to train\n - **ask_epoch** is either set to an integer value or None. If set to an integer it denotes\n    the epoch number at which user input is requested. If the user enter H training is\n    halted. If the user inters an integer it represents how many more epochs to run\n    before you are asked for the user input again. If ask_epoch is set to None the\n    user is NOT asked to provide any input. This feature is handy is when training your model\n    and the metrics are either unsatisfactory and you want to stop training, or for the case\n    where your metrics are satisfactory and there is no need to train any further. Note\n    you model is always set to the weights for the epoch that had the beset metric\n    performance. So if you halt the training you can still use the model for predictions.  \n      \n ### ** Example of Use:\n callbacks=[LRA(my_model, 2, 5, .9, .5, True, 'InceptionResnetV2', False, 80,20,10)]\n this implies:\n - your model is my_model\n - after 2 epochs of no improvement the learning rate will be reduced\n - after 5 consecutive adjustment of the leaarning rate with no metric improve training terminates\n - once the training accuracy reaches 90% the callback adjust learning rate based on validation loss\n - when the learning rate is adjust the new learning rate is .5 X learning rate\n - if the current epoch's metric value did not improve, the weights for the prior epoch are loaded\n   and the learning rate is reduced\n - the string name of your model to print in the header at the start of training\n - the header will note the entire model is being trained\n - the number of batches per epoch is 80\n - train for 20 epochs\n - after the tenth epoch you will be asked if you want to halt training by entering H or enter \n   an integer denoting how many more epochs to run before you will be prompted again\n                 \n","metadata":{"papermill":{"duration":0.019813,"end_time":"2021-05-21T20:58:39.646456","exception":false,"start_time":"2021-05-21T20:58:39.626643","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import backend as K\nfrom tensorflow.keras.layers import Dense, Activation,Dropout,Conv2D, MaxPooling2D,BatchNormalization, Flatten\nfrom tensorflow.keras.optimizers import Adam, Adamax\nfrom tensorflow.keras.metrics import categorical_crossentropy\nfrom tensorflow.keras import regularizers\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.models import Model, load_model, Sequential\nimport numpy as np\nimport pandas as pd\nimport shutil\nimport time\nimport cv2 as cv2\nfrom tqdm import tqdm\nfrom sklearn.model_selection import train_test_split\nimport matplotlib.pyplot as plt\nfrom matplotlib.pyplot import imshow\nimport os\nimport seaborn as sns\nsns.set_style('darkgrid')\nfrom PIL import Image\nfrom sklearn.metrics import confusion_matrix, classification_report\nfrom IPython.core.display import display, HTML","metadata":{"papermill":{"duration":5.693249,"end_time":"2021-05-21T20:58:45.358994","exception":false,"start_time":"2021-05-21T20:58:39.665745","status":"completed"},"tags":[],"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:10.748016Z","iopub.execute_input":"2023-05-27T17:51:10.748389Z","iopub.status.idle":"2023-05-27T17:51:16.874722Z","shell.execute_reply.started":"2023-05-27T17:51:10.748352Z","shell.execute_reply":"2023-05-27T17:51:16.873719Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Necessary utility modules and libraries\nimport os\nimport shutil\nimport pathlib\nimport random\nimport datetime\nimport cv2\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Plotting libraries \nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nfrom scipy.signal import gaussian, convolve2d\nimport seaborn as sns\n\n# Libraries for building the model\nimport tensorflow as tf\n#import tensorflow_hub as hub\nfrom tensorflow import keras\nfrom tensorflow.keras.layers import Input, Dense, Conv2D, Flatten, MaxPool2D, Dropout, Activation, GlobalAveragePooling2D, BatchNormalization, GlobalMaxPooling2D\nfrom tensorflow.keras.applications import DenseNet121, ResNet50, InceptionV3, Xception, VGG16\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras import backend\nfrom tensorflow.keras.regularizers import l2, l1\nfrom sklearn.metrics import classification_report, precision_recall_fscore_support, accuracy_score, confusion_matrix","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:16.876526Z","iopub.execute_input":"2023-05-27T17:51:16.876936Z","iopub.status.idle":"2023-05-27T17:51:16.925529Z","shell.execute_reply.started":"2023-05-27T17:51:16.876893Z","shell.execute_reply":"2023-05-27T17:51:16.924582Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Data loading","metadata":{"papermill":{"duration":0.018355,"end_time":"2021-05-21T20:58:45.477867","exception":false,"start_time":"2021-05-21T20:58:45.459512","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"classes = ['No_DR', 'Mild', 'Moderate', 'Severe', 'Proliferate_DR']","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:16.928063Z","iopub.execute_input":"2023-05-27T17:51:16.928754Z","iopub.status.idle":"2023-05-27T17:51:16.93399Z","shell.execute_reply.started":"2023-05-27T17:51:16.928708Z","shell.execute_reply":"2023-05-27T17:51:16.932586Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"eyepacs_df= pd.read_csv(\"/kaggle/input/eyepacspreprocess/trainLabels.csv\")\naptos_df= pd.read_csv(\"/kaggle/input/aptos2019-blindness-detection/train.csv\")\nddr_df= pd.read_csv(\"/kaggle/input/ddrdataset/DR_grading.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-05-27T17:51:16.936267Z","iopub.execute_input":"2023-05-27T17:51:16.936703Z","iopub.status.idle":"2023-05-27T17:51:17.016983Z","shell.execute_reply.started":"2023-05-27T17:51:16.936663Z","shell.execute_reply":"2023-05-27T17:51:17.015997Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ddr_df= pd.read_csv(\"/kaggle/input/ddrdataset/DR_grading.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-05-27T17:51:17.020412Z","iopub.execute_input":"2023-05-27T17:51:17.020822Z","iopub.status.idle":"2023-05-27T17:51:17.040308Z","shell.execute_reply.started":"2023-05-27T17:51:17.020782Z","shell.execute_reply":"2023-05-27T17:51:17.039344Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class_code = {0: \"No_DR\",\n              1: \"Mild\", \n              2: \"Moderate\",\n              3: \"Severe\",\n              4: \"Proliferate_DR\"}\neyepacs_df.rename(columns={\"image\": \"id_code\", \"level\": \"diagnosis\"}, inplace=True)\naptos_df.rename(columns={\"image\": \"id_code\", \"level\": \"diagnosis\"}, inplace=True)\n","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:17.044068Z","iopub.execute_input":"2023-05-27T17:51:17.04439Z","iopub.status.idle":"2023-05-27T17:51:17.054098Z","shell.execute_reply.started":"2023-05-27T17:51:17.04436Z","shell.execute_reply":"2023-05-27T17:51:17.052978Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for i in range(5):\n    if not os.path.exists('./'+class_code[i]):\n        os.mkdir('./'+class_code[i])","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:17.056115Z","iopub.execute_input":"2023-05-27T17:51:17.056943Z","iopub.status.idle":"2023-05-27T17:51:17.063516Z","shell.execute_reply.started":"2023-05-27T17:51:17.056904Z","shell.execute_reply":"2023-05-27T17:51:17.062694Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Loading EyePACS","metadata":{}},{"cell_type":"code","source":"dir_path= r'/kaggle/input/eyepacspreprocess/eyepacs_preprocess/eyepacs_preprocess'\n","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:17.066537Z","iopub.execute_input":"2023-05-27T17:51:17.067036Z","iopub.status.idle":"2023-05-27T17:51:17.073818Z","shell.execute_reply.started":"2023-05-27T17:51:17.066997Z","shell.execute_reply":"2023-05-27T17:51:17.07297Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def mapping_temp(df, root=dir_path):\n    class_code = {0: \"No_DR\",\n                  1: \"Mild\", \n                  2: \"Moderate\",\n                  3: \"Severe\",\n                  4: \"Proliferate_DR\"}\n    df['label'] = list(map(class_code.get, df['diagnosis']))\n    df['path'] = [i[1]['label']+'/'+i[1]['id_code']+'.jpeg' for i in df.iterrows()]\n    return df\n","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:17.075791Z","iopub.execute_input":"2023-05-27T17:51:17.076473Z","iopub.status.idle":"2023-05-27T17:51:17.085085Z","shell.execute_reply.started":"2023-05-27T17:51:17.076426Z","shell.execute_reply":"2023-05-27T17:51:17.084089Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dir_path= r'/kaggle/input/eyepacspreprocess/eyepacs_preprocess/eyepacs_preprocess'\neyepacs_df = mapping_temp(eyepacs_df)\neyepacs_df","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:17.087225Z","iopub.execute_input":"2023-05-27T17:51:17.087834Z","iopub.status.idle":"2023-05-27T17:51:20.600904Z","shell.execute_reply.started":"2023-05-27T17:51:17.087779Z","shell.execute_reply":"2023-05-27T17:51:20.599927Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Dropping the diagnosis column because the model assigns different codes for prediction\neyepacs_df.drop(['diagnosis'], axis=1, inplace=True)\n#df_temp","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:20.602337Z","iopub.execute_input":"2023-05-27T17:51:20.602735Z","iopub.status.idle":"2023-05-27T17:51:20.613614Z","shell.execute_reply.started":"2023-05-27T17:51:20.602694Z","shell.execute_reply":"2023-05-27T17:51:20.612722Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Read the first CSV file\ndf1 = pd.read_csv('/kaggle/input/badimages/bad_images_1.csv')\n\n# Read the second CSV file\ndf2 = pd.read_csv('/kaggle/input/badimages/bad_images_2.csv')\n\n# Concatenate the two dataframes based on the 'id_code' column\nconcatenated_df = pd.concat([df1, df2])","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Drop rows in df3 with the same id_code as in concatenated_df\neyepacs_df = eyepacs_df[~eyepacs_df['id_code'].isin(concatenated_df['id_code'])]","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport shutil\n# for i in df_temp.iloc[:5, :].iterrows():\n#     print(i[1][2])\nres = [[i[1][1], i[1][2]] for i in eyepacs_df.iterrows()]\nfor i in res:\n    des = './'+i[0]+'/'\n    src = dir_path+'/'+i[1].split('/')[1]\n    shutil.copy(src, des)","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:51:20.617326Z","iopub.execute_input":"2023-05-27T17:51:20.618519Z","iopub.status.idle":"2023-05-27T17:57:39.148569Z","shell.execute_reply.started":"2023-05-27T17:51:20.618471Z","shell.execute_reply":"2023-05-27T17:57:39.14751Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Loading APTOS","metadata":{}},{"cell_type":"code","source":"def mapping_temp(df, root=dir_path):\n    class_code = {0: \"No_DR\",\n                  1: \"Mild\", \n                  2: \"Moderate\",\n                  3: \"Severe\",\n                  4: \"Proliferate_DR\"}\n    df['label'] = list(map(class_code.get, df['diagnosis']))\n    df['path'] = [i[1]['label']+'/'+i[1]['id_code']+'.png' for i in df.iterrows()]\n    return df","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:57:39.150245Z","iopub.execute_input":"2023-05-27T17:57:39.150672Z","iopub.status.idle":"2023-05-27T17:57:39.160115Z","shell.execute_reply.started":"2023-05-27T17:57:39.150627Z","shell.execute_reply":"2023-05-27T17:57:39.159114Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dir_path= r'/kaggle/input/aptos2019-blindness-detection/train_images'\naptos_df = mapping_temp(aptos_df)\naptos_df","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:57:39.162461Z","iopub.execute_input":"2023-05-27T17:57:39.163087Z","iopub.status.idle":"2023-05-27T17:57:39.518154Z","shell.execute_reply.started":"2023-05-27T17:57:39.163046Z","shell.execute_reply":"2023-05-27T17:57:39.517175Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Dropping the diagnosis column because the model assigns different codes for prediction\naptos_df.drop(['diagnosis'], axis=1, inplace=True)\n#df_temp","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T17:57:39.519665Z","iopub.execute_input":"2023-05-27T17:57:39.520256Z","iopub.status.idle":"2023-05-27T17:57:39.527597Z","shell.execute_reply.started":"2023-05-27T17:57:39.520209Z","shell.execute_reply":"2023-05-27T17:57:39.526559Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport shutil\n# for i in df_temp.iloc[:5, :].iterrows():\n#     print(i[1][2])\nres = [[i[1][1], i[1][2]] for i in aptos_df.iterrows()]\nfor i in res:\n    des = './'+i[0]+'/'\n    src = dir_path+'/'+i[1].split('/')[1]\n    shutil.copy(src, des)","metadata":{"editable":false,"execution":{"iopub.status.busy":"2023-05-27T18:05:10.600351Z","iopub.execute_input":"2023-05-27T18:05:10.600778Z","iopub.status.idle":"2023-05-27T18:06:43.162038Z","shell.execute_reply.started":"2023-05-27T18:05:10.600736Z","shell.execute_reply":"2023-05-27T18:06:43.160954Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Loading DDR","metadata":{}},{"cell_type":"code","source":"def mapping_temp(df, root=dir_path):\n    class_code = {0: \"No_DR\",\n                  1: \"Mild\", \n                  2: \"Moderate\",\n                  3: \"Severe\",\n                  4: \"Proliferate_DR\"}\n    df['label'] = list(map(class_code.get, df['diagnosis']))\n    df['path'] = [i[1]['label']+'/'+i[1]['id_code'] for i in df.iterrows()]\n    return df","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:06:43.163936Z","iopub.execute_input":"2023-05-27T18:06:43.164339Z","iopub.status.idle":"2023-05-27T18:06:43.172291Z","shell.execute_reply.started":"2023-05-27T18:06:43.164294Z","shell.execute_reply":"2023-05-27T18:06:43.171128Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dir_path= r'/kaggle/input/ddrdataset/DR_grading/DR_grading'\nddr_df = mapping_temp(ddr_df)\nddr_df","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:06:43.174712Z","iopub.execute_input":"2023-05-27T18:06:43.175206Z","iopub.status.idle":"2023-05-27T18:06:44.466327Z","shell.execute_reply.started":"2023-05-27T18:06:43.175165Z","shell.execute_reply":"2023-05-27T18:06:44.465313Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Dropping the diagnosis column because the model assigns different codes for prediction\nddr_df.drop(['diagnosis'], axis=1, inplace=True)\n#df_temp","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:06:44.468335Z","iopub.execute_input":"2023-05-27T18:06:44.469043Z","iopub.status.idle":"2023-05-27T18:06:44.477154Z","shell.execute_reply.started":"2023-05-27T18:06:44.468993Z","shell.execute_reply":"2023-05-27T18:06:44.476063Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport shutil\n# for i in df_temp.iloc[:5, :].iterrows():\n#     print(i[1][2])\nres = [[i[1][1], i[1][2]] for i in ddr_df.iterrows()]\nfor i in res:\n    des = './'+i[0]+'/'\n    src = dir_path+'/'+i[1].split('/')[1]\n    shutil.copy(src, des)","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:06:44.478734Z","iopub.execute_input":"2023-05-27T18:06:44.47937Z","iopub.status.idle":"2023-05-27T18:09:14.595606Z","shell.execute_reply.started":"2023-05-27T18:06:44.479327Z","shell.execute_reply":"2023-05-27T18:09:14.594675Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_temp= pd.concat([eyepacs_df, aptos_df])\ndf_temp= pd.concat([df_temp, ddr_df])\ndf_temp","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:09:14.597259Z","iopub.execute_input":"2023-05-27T18:09:14.597868Z","iopub.status.idle":"2023-05-27T18:09:14.620871Z","shell.execute_reply.started":"2023-05-27T18:09:14.597818Z","shell.execute_reply":"2023-05-27T18:09:14.620076Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class_counts = df_temp['label'].value_counts()\nprint(class_counts)","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:09:14.622208Z","iopub.execute_input":"2023-05-27T18:09:14.622631Z","iopub.status.idle":"2023-05-27T18:09:14.641279Z","shell.execute_reply.started":"2023-05-27T18:09:14.62259Z","shell.execute_reply":"2023-05-27T18:09:14.640091Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def show_image_samples(gen ):\n    t_dict=gen.class_indices\n    classes=list(t_dict.keys())    \n    images,labels=next(gen) # get a sample batch from the generator \n    plt.figure(figsize=(20, 20))\n    length=len(labels)\n    if length<25:   #show maximum of 25 images\n        r=length\n    else:\n        r=25\n    for i in range(r):\n        plt.subplot(5, 5, i + 1)\n        image=images[i]/255\n        plt.imshow(image)\n        index=np.argmax(labels[i])\n        class_name=classes[index]\n        plt.title(class_name, color='blue', fontsize=16)\n        plt.axis('off')\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:09:14.643796Z","iopub.execute_input":"2023-05-27T18:09:14.644159Z","iopub.status.idle":"2023-05-27T18:09:14.654033Z","shell.execute_reply.started":"2023-05-27T18:09:14.644122Z","shell.execute_reply":"2023-05-27T18:09:14.653092Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Dark Bright filtering","metadata":{}},{"cell_type":"code","source":"import cv2\nimport numpy as np\n\ndef is_bright(image_path, dim=227, brightness_thresh=0.55, darkness_thresh=0.165):\n    # Read the image\n    image = cv2.imread(image_path)\n    # Resize image to dim x dim\n    image = cv2.resize(image, (dim, dim))\n    # Convert color space to LAB format and extract L channel\n    L, _, _ = cv2.split(cv2.cvtColor(image, cv2.COLOR_BGR2LAB))\n    # Normalize L channel by dividing all pixel values with the maximum pixel value\n    L = L / np.max(L)\n    # Check if the mean brightness exceeds the brightness threshold\n    if np.mean(L) > brightness_thresh:\n        return True\n    # Check if the mean brightness is below the darkness threshold\n    elif np.mean(L) < darkness_thresh:\n        return True\n    else:\n        return False\n\ndef filter_brightness(df):\n    filtered_df = df.copy()\n    indices_to_drop = []\n    for index, row in df.iterrows():\n        if is_bright(row['path']):\n            indices_to_drop.append(index)\n    filtered_df = filtered_df.drop(indices_to_drop)\n    filtered_df = filtered_df.reset_index(drop=True)\n    return filtered_df","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:12:06.583088Z","iopub.execute_input":"2023-05-27T18:12:06.583465Z","iopub.status.idle":"2023-05-27T18:12:06.592758Z","shell.execute_reply.started":"2023-05-27T18:12:06.583428Z","shell.execute_reply":"2023-05-27T18:12:06.591761Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_temp_dark_bright = filter_brightness(df_temp)\ndf_temp_dark_bright","metadata":{"execution":{"iopub.status.busy":"2023-05-27T18:12:07.353107Z","iopub.execute_input":"2023-05-27T18:12:07.353467Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"class_counts = df_temp_dark_bright['label'].value_counts()\nprint(class_counts)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_temp= df_temp_dark_bright","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Train Valid Test dataframe splitting","metadata":{"editable":false}},{"cell_type":"code","source":"df_no_DR = df_temp[df_temp['label']== 'No_DR' ]\ndf_classes = df_temp[df_temp['label']!= 'No_DR'] \n\n# split of class No_DR 70% 15% 15% \ntrain_class0, test_valid_class0 = train_test_split(df_no_DR, train_size=0.7, shuffle=True, random_state=123)\ntest_class0, valid_class0 = train_test_split(test_valid_class0, train_size=0.5, shuffle=True, random_state=123)\n\n# split of other classes 80% 15% 5% \ntrain_class, test_valid_class = train_test_split(df_classes, train_size=0.80, shuffle=True, random_state=123)\ntest_class, valid_class = train_test_split(test_valid_class, train_size=0.25, shuffle=True, random_state=123)\n\n# concatenate dataframes\ndf_trainLabels= pd.concat([train_class0, train_class])\nvalid_df= pd.concat([valid_class0, valid_class])\ntest_df= pd.concat([test_class0, test_class])\n\ndf_trainLabels.to_csv('train_data_without_aug.csv', index=False)\nvalid_df.to_csv('valid_data.csv', index=False)\ntest_df.to_csv('test_data.csv', index=False)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df=pd.read_csv('train_data.csv')\ndf","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### This is a very unbalanced data set for df,  limit  maximum number of samples in a class to 25802","metadata":{"editable":false}},{"cell_type":"markdown","source":"# Augmenation","metadata":{"editable":false}},{"cell_type":"markdown","source":"#### To balance the dataset we will create augmented images for classes with less than 3000 samples\n### first create directories to hold the augmented images","metadata":{"editable":false}},{"cell_type":"code","source":"working_dir = './'\naug_dir = os.path.join(working_dir, 'aug')\nif os.path.isdir(aug_dir):\n    shutil.rmtree(aug_dir)\nos.mkdir(aug_dir)\n\ntargets = {\n    'Proliferate_DR': 17500,\n    'Severe': 17500,\n    'Moderate': 17500,\n    'Mild': 17500,\n    'No_DR': 17500,\n}\n\ngen = ImageDataGenerator(horizontal_flip=True,  rotation_range=20, width_shift_range=.2,\n                              height_shift_range=.2, zoom_range=.2, )\n\ngroups = df.groupby('label')\n\nfor label in df['label'].unique():\n    group = groups.get_group(label)\n    sample_count = len(group)\n\n    if sample_count < targets[label]:\n        delta = targets[label] - sample_count\n        aug_img_count = 0\n        target_dir = os.path.join(aug_dir, label)\n\n        os.makedirs(target_dir, exist_ok=True)\n\n        aug_gen = gen.flow_from_dataframe(dataframe=group,  x_col='path', y_col=None, target_size=(224,224), class_mode=None,\n                                        batch_size=1, shuffle=False, save_to_dir=target_dir, save_prefix='aug-',\n                                        save_format='jpg')\n\n        while aug_img_count < delta:\n            images = next(aug_gen)\n            aug_img_count += len(images)\n            if aug_img_count >= delta:\n                break\n\naug_list = os.listdir(aug_dir)\n\nfor klass in aug_list:\n    classpath = os.path.join(aug_dir, klass)\n    flist = os.listdir(classpath)\n    print('klass: ', klass, '  file count: ', len(flist))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### create augmented images and store in the aug directories\n","metadata":{"editable":false}},{"cell_type":"code","source":"aug=r'./aug'\nauglist=os.listdir(aug)\nprint (auglist)\nfor klass in auglist:\n    classpath=os.path.join(aug, klass)\n    flist=os.listdir(classpath)\n    print('klass: ', klass, '  file count: ', len(flist))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Create the aux_df then concatenate with df ","metadata":{"editable":false}},{"cell_type":"code","source":"aug_fpaths=[]\naug_labels=[]\nclasslist=os.listdir(aug_dir)\nfor klass in classlist:\n    classpath=os.path.join(aug_dir, klass)     \n    flist=os.listdir(classpath)    \n    for f in flist:        \n        fpath=os.path.join(classpath,f)         \n        aug_fpaths.append(fpath)\n        aug_labels.append(klass)\nFseries=pd.Series(aug_fpaths, name='path')\nLseries=pd.Series(aug_labels, name='label')\naug_df=pd.concat([Fseries, Lseries], axis=1)\nndf=pd.concat([df,aug_df], axis=0).reset_index(drop=True)\n#ndf=df.sample(frac=1.0, replace=False, random_state=123, axis=0).reset_index(drop=True)\n\n\nprint (df['label'].value_counts()) \nprint(aug_df['label'].value_counts())\nprint (ndf['label'].value_counts())","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Preprocessing function","metadata":{"editable":false}},{"cell_type":"code","source":"def image_preprocessing(img):\n    # Read the image\n#   img = mpimg.imread(img_path)\n    img = img.astype(np.uint8)\n    \n    #  Extract the green channel of the image\n    b, g, r = cv2.split(img)\n    \n    #  Apply CLAHE to intensify the green channel extracted image\n    clh = cv2.createCLAHE(clipLimit=4.0, tileGridSize=(8,8))\n    r= clh.apply(r)\n    g= clh.apply(g)\n    b= clh.apply(b)\n    \n    # Convert enhanced image to grayscale\n    merged_bgr_green_fused = cv2.merge((b, g, r))\n    #img_bw = cv2.cvtColor(merged_bgr_green_fused, cv2.COLOR_BGR2GRAY)\n    \n    # Apply Gaussian blur to remove noise\n    img_blur = cv2.GaussianBlur(merged_bgr_green_fused, (5,5), 0)\n    \n    # Remove the isolated pixels using morphological cleaning operation.\n    kernel1 = np.ones((1, 1), np.uint8)\n    morph_open = cv2.morphologyEx(img_blur, cv2.MORPH_OPEN, kernel1)\n    \n    # Extract blood vessels using mean-C thresholding.\n    #thresh = cv2.adaptiveThreshold(morph_open, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 9, 5)\n    \n    # Applying morph_open operation\n    kernel2 = np.ones((2, 2), np.uint8)\n    morph_open2 = cv2.morphologyEx(morph_open, cv2.MORPH_OPEN, kernel2)\n    \n    # Stacking the image into 3 channels\n    #stacked_img = np.stack((morph_open2,)*3, axis=-1)\n    \n    return morph_open2.astype(\"float64\") #stacked_img\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Generators train, test, valid ","metadata":{"papermill":{"duration":0.023718,"end_time":"2021-05-21T20:58:54.520519","exception":false,"start_time":"2021-05-21T20:58:54.496801","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"train_df= ndf","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"height=224\nwidth=224\nchannels=3\nbatch_size=40\nimg_shape=(height, width, channels)\nimg_size=(height, width)\nlength=len(test_df)\ntest_batch_size=sorted([int(length/n) for n in range(1,length+1) if length % n ==0 and length/n<=80],reverse=True)[0]  \ntest_steps=int(length/test_batch_size)\nprint ( 'test batch size: ' ,test_batch_size, '  test steps: ', test_steps)\ndef scalar(img):\n    #img=img/127.5-1\n    return img \ntrgen=ImageDataGenerator(preprocessing_function=image_preprocessing, horizontal_flip=True)\ntvgen=ImageDataGenerator(preprocessing_function=image_preprocessing, horizontal_flip=True)\nsdir=r'../input/mars-surface-and-curiosity-image-set-nasa/Mars Surface and Curiosity Image/images'\ntrain_gen=trgen.flow_from_dataframe( train_df, x_col='path', y_col='label', target_size=img_size, class_mode='categorical',\n                                    color_mode='rgb', shuffle=True, batch_size=batch_size)\ntest_gen=tvgen.flow_from_dataframe( test_df, x_col='path', y_col='label', target_size=img_size, class_mode='categorical',\n                                    color_mode='rgb', shuffle=False, batch_size=test_batch_size)\nvalid_gen=tvgen.flow_from_dataframe( valid_df, x_col='path', y_col='label', target_size=img_size, class_mode='categorical',\n                                    color_mode='rgb', shuffle=True, batch_size=batch_size)\nclasses=list(train_gen.class_indices.keys())\nclass_count=len(classes)\n","metadata":{"papermill":{"duration":63.291308,"end_time":"2021-05-21T20:59:57.835586","exception":false,"start_time":"2021-05-21T20:58:54.544278","status":"completed"},"scrolled":true,"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_steps=int(len(train_gen.labels)/batch_size)","metadata":{"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### create function to show some image examples","metadata":{"papermill":{"duration":0.023999,"end_time":"2021-05-21T20:59:58.707982","exception":false,"start_time":"2021-05-21T20:59:58.683983","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"def show_image_samples(gen ):\n    t_dict=gen.class_indices\n    classes=list(t_dict.keys())    \n    images,labels=next(gen) # get a sample batch from the generator \n    plt.figure(figsize=(20, 20))\n    length=len(labels)\n    if length<25:   #show maximum of 25 images\n        r=length\n    else:\n        r=25\n    for i in range(r):\n        plt.subplot(5, 5, i + 1)\n        image=images[i]/255\n        plt.imshow(image)\n        index=np.argmax(labels[i])\n        class_name=classes[index]\n        plt.title(class_name, color='blue', fontsize=16)\n        plt.axis('off')\n    plt.show()","metadata":{"papermill":{"duration":0.03267,"end_time":"2021-05-21T20:59:58.764761","exception":false,"start_time":"2021-05-21T20:59:58.732091","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"show_image_samples(train_gen)","metadata":{"papermill":{"duration":3.031821,"end_time":"2021-05-21T21:00:01.820151","exception":false,"start_time":"2021-05-21T20:59:58.78833","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### define function to print text in RGB foreground and background colors","metadata":{"papermill":{"duration":0.067529,"end_time":"2021-05-21T21:00:01.954296","exception":false,"start_time":"2021-05-21T21:00:01.886767","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"def print_in_color(txt_msg,fore_tupple,back_tupple,):\n    #prints the text_msg in the foreground color specified by fore_tupple with the background specified by back_tupple \n    #text_msg is the text, fore_tupple is foregroud color tupple (r,g,b), back_tupple is background tupple (r,g,b)\n    rf,gf,bf=fore_tupple\n    rb,gb,bb=back_tupple\n    msg='{0}' + txt_msg\n    mat='\\33[38;2;' + str(rf) +';' + str(gf) + ';' + str(bf) + ';48;2;' + str(rb) + ';' +str(gb) + ';' + str(bb) +'m' \n    print(msg .format(mat), flush=True)\n    print('\\33[0m', flush=True) # returns default print color to back to black\n    return","metadata":{"papermill":{"duration":0.078026,"end_time":"2021-05-21T21:00:02.099684","exception":false,"start_time":"2021-05-21T21:00:02.021658","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" # The model","metadata":{"papermill":{"duration":0.067478,"end_time":"2021-05-21T21:00:02.234057","exception":false,"start_time":"2021-05-21T21:00:02.166579","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"model_name='EfficientNetB1'\nbase_model=tf.keras.applications.EfficientNetB1(include_top=False, weights=\"imagenet\",input_shape=img_shape, pooling='max') \nx=base_model.output\nx=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)\nx = Dense(256, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),\n                bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)\nx=Dropout(rate=.45, seed=123)(x)        \noutput=Dense(class_count, activation='softmax')(x)\nmodel=Model(inputs=base_model.input, outputs=output)\nmodel.compile(Adamax(lr=.001), loss='categorical_crossentropy', metrics=['accuracy']) ","metadata":{"papermill":{"duration":5.559742,"end_time":"2021-05-21T21:00:07.864272","exception":false,"start_time":"2021-05-21T21:00:02.30453","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### create a subclass of callbacks to control learning rate and print training results for each epoch","metadata":{"papermill":{"duration":0.067963,"end_time":"2021-05-21T21:00:08.001776","exception":false,"start_time":"2021-05-21T21:00:07.933813","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"  class LRA(keras.callbacks.Callback):\n    reset=False\n    count=0\n    stop_count=0\n    \n    def __init__(self,model, patience,stop_patience, threshold, factor, dwell, model_name, freeze,batches, initial_epoch,epochs, ask_epoch):\n        super(LRA, self).__init__()\n        self.epochs=epochs\n        self.ask_epoch=ask_epoch\n        self.model=model\n        self.patience=patience # specifies how many epochs without improvement before learning rate is adjusted\n        self.stop_patience=stop_patience\n        self.threshold=threshold # specifies training accuracy threshold when lr will be adjusted based on validation loss\n        self.factor=factor # factor by which to reduce the learning rate\n        self.dwell=dwell\n        self.lr=float(tf.keras.backend.get_value(model.optimizer.lr)) # get the initiallearning rate and save it in self.lr\n        self.highest_tracc=0.0 # set highest training accuracy to 0\n        self.lowest_vloss=np.inf # set lowest validation loss to infinity\n        #self.count=0 # initialize counter that counts epochs with no improvement\n        #self.stop_count=0 # initialize counter that counts how manytimes lr has been adjustd with no improvement  \n        self.initial_epoch=initial_epoch \n        self.batches=batches\n        #self.epochs=epochs\n        best_weights=self.model.get_weights() # set a class vaiable so weights can be loaded after training is completed        \n        msg=' '\n        if freeze==True:\n            msgs=f' Starting training using  base model { model_name} with weights frozen to imagenet weights initializing LRA callback'\n        else:\n            msgs=f' Starting training using base model { model_name} training all layers '            \n        print_in_color (msgs, (244, 252, 3), (55,65,80)) \n    def on_train_begin(self, logs=None):\n        msg='{0:^8s}{1:^10s}{2:^9s}{3:^9s}{4:^9s}{5:^9s}{6:^9s}{7:^10s}{8:^8s}'.format('Epoch', 'Loss', 'Accuracy',\n                                                                                              'V_loss','V_acc', 'LR', 'Next LR', 'Monitor', 'Duration')\n        print_in_color(msg, (244,252,3), (55,65,80)) \n    def on_train_end(self, logs=None):\n        model.set_weights(LRA.best_weights)\n        msg='Training is completed - model is set with weights for the epoch with the lowest loss'\n        print_in_color(msg, (0,255,0), (55,65,80)) \n        \n    def on_train_batch_end(self, batch, logs=None):\n        acc=logs.get('accuracy')* 100  # get training accuracy \n        loss=logs.get('loss')\n        msg='{0:20s}processing batch {1:4s} of {2:5s} accuracy= {3:8.3f}  loss: {4:8.5f}'.format(' ', str(batch), str(self.batches), acc, loss)\n        print(msg, '\\r', end='') # prints over on the same line to show running batch count\n        \n        \n    def on_epoch_begin(self,epoch, logs=None):\n        self.now= time.time()\n        \n    def on_epoch_end(self, epoch, logs=None):  # method runs on the end of each epoch\n        later=time.time()\n        duration=later-self.now \n        lr=float(tf.keras.backend.get_value(self.model.optimizer.lr)) # get the current learning rate\n        current_lr=lr\n        v_loss=logs.get('val_loss')  # get the validation loss for this epoch\n        acc=logs.get('accuracy')  # get training accuracy \n        v_acc=logs.get('val_accuracy')\n        loss=logs.get('loss')\n        #print ( '\\n',v_loss, self.lowest_vloss, acc, self.highest_tracc)\n        if acc < self.threshold: # if training accuracy is below threshold adjust lr based on training accuracy\n            monitor='accuracy'\n            if acc>self.highest_tracc: # training accuracy improved in the epoch                \n                self.highest_tracc=acc # set new highest training accuracy\n                LRA.best_weights=self.model.get_weights() # traing accuracy improved so save the weights\n                self.count=0 # set count to 0 since training accuracy improved\n                self.stop_count=0 # set stop counter to 0\n                if v_loss<self.lowest_vloss:\n                    self.lowest_vloss=v_loss\n                color= (0,255,0)\n                self.lr=lr\n            else: \n                # training accuracy did not improve check if this has happened for patience number of epochs\n                # if so adjust learning rate\n                if self.count>=self.patience -1:\n                    color=(245, 170, 66)\n                    self.lr= lr* self.factor # adjust the learning by factor\n                    tf.keras.backend.set_value(self.model.optimizer.lr, self.lr) # set the learning rate in the optimizer\n                    self.count=0 # reset the count to 0\n                    self.stop_count=self.stop_count + 1\n                    if self.dwell:\n                        self.model.set_weights(LRA.best_weights) # return to better point in N space                        \n                    else:\n                        if v_loss<self.lowest_vloss:\n                            self.lowest_vloss=v_loss                                    \n                else:\n                    self.count=self.count +1 # increment patience counter                    \n        else: # training accuracy is above threshold so adjust learning rate based on validation loss\n            monitor='val_loss'\n            if v_loss< self.lowest_vloss: # check if the validation loss improved \n                self.lowest_vloss=v_loss # replace lowest validation loss with new validation loss                \n                LRA.best_weights=self.model.get_weights() # validation loss improved so save the weights\n                self.count=0 # reset count since validation loss improved  \n                self.stop_count=0  \n                color=(0,255,0)\n                self.lr=lr\n            else: # validation loss did not improve\n                if self.count>=self.patience-1:\n                    color=(245, 170, 66)\n                    self.lr=self.lr * self.factor # adjust the learning rate                    \n                    self.stop_count=self.stop_count + 1 # increment stop counter because lr was adjusted \n                    self.count=0 # reset counter\n                    tf.keras.backend.set_value(self.model.optimizer.lr, self.lr) # set the learning rate in the optimizer\n                    if self.dwell:\n                        self.model.set_weights(LRA.best_weights) # return to better point in N space\n                else: \n                    self.count =self.count +1 # increment the patience counter                    \n                if acc>self.highest_tracc:\n                    self.highest_tracc= acc\n        msg=f'{str(epoch+1):^3s}/{str(self.epochs):4s} {loss:^9.3f}{acc*100:^9.3f}{v_loss:^9.5f}{v_acc*100:^9.3f}{current_lr:^9.5f}{self.lr:^9.5f}{monitor:^11s}{duration:^8.2f}'\n        print_in_color (msg,color, (55,65,80))\n        if self.stop_count> self.stop_patience - 1: # check if learning rate has been adjusted stop_count times with no improvement\n            msg=f' training has been halted at epoch {epoch + 1} after {self.stop_patience} adjustments of learning rate with no improvement'\n            print_in_color(msg, (0,255,255), (55,65,80))\n            self.model.stop_training = True # stop training\n        else: \n            if self.ask_epoch !=None:\n                if epoch + 1 >= self.ask_epoch:\n                    msg='enter H to halt training or an integer for number of epochs to run then ask again'\n                    print_in_color(msg, (0,255,255), (55,65,80))\n                    ans=input('')\n                    if ans=='H' or ans=='h':\n                        msg=f'training has been halted at epoch {epoch + 1} due to user input'\n                        print_in_color(msg, (0,255,255), (55,65,80))\n                        self.model.stop_training = True # stop training\n                    else:\n                        ans=int(ans)\n                        self.ask_epoch +=ans\n                           \n                   ","metadata":{"papermill":{"duration":0.095226,"end_time":"2021-05-21T21:00:08.166585","exception":false,"start_time":"2021-05-21T21:00:08.071359","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Train, instantiate the custom callback and train the model","metadata":{"papermill":{"duration":0.068299,"end_time":"2021-05-21T21:00:08.302663","exception":false,"start_time":"2021-05-21T21:00:08.234364","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"from tensorflow.keras.callbacks import ModelCheckpoint\n\n# Define the path and filename to save the model\ncheckpoint_path = \"model_checkpoint.h5\"\n\nepochs= 20\npatience= 1 # number of epochs to wait to adjust lr if monitored value does not improve\nstop_patience =4 # number of epochs to wait before stopping training if monitored value does not improve\nthreshold=.9 # if train accuracy is < threshhold adjust monitor accuracy, else monitor validation loss\nfactor=.3 # factor to reduce lr by\ndwell=True # experimental, if True and monitored metric does not improve on current epoch set  modelweights back to weights of previous epoch\nfreeze=False # if true free weights of  the base model\nask_epoch=31\nbatches=train_steps\n\ncallbacks=[LRA(model=model,patience=patience,stop_patience=stop_patience, threshold=threshold,\n                   factor=factor,dwell=dwell, model_name=model_name, freeze=freeze, batches=batches,initial_epoch=0,epochs=epochs, ask_epoch=ask_epoch )]\n\n\n# Define the ModelCheckpoint callback\ncheckpoint_callback = ModelCheckpoint(\n    filepath=checkpoint_path,\n    save_weights_only=False,\n    save_best_only=False,\n    save_freq=\"epoch\",\n    verbose=1,\n)\n\n# Train the model with the checkpoint callback\nhistory = model.fit(\n    x=train_gen,\n    epochs=epochs,\n    verbose=0,\n    callbacks=[checkpoint_callback, *callbacks],\n    validation_data=valid_gen,\n    validation_steps=None,\n    shuffle=False,\n    initial_epoch=0,\n)\n","metadata":{"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"","metadata":{"editable":false}},{"cell_type":"markdown","source":"### define function to plot the training data","metadata":{"papermill":{"duration":2.936894,"end_time":"2021-05-21T22:17:07.342801","exception":false,"start_time":"2021-05-21T22:17:04.405907","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"def tr_plot(tr_data, start_epoch):\n    #Plot the training and validation data\n    tacc=tr_data.history['accuracy']\n    tloss=tr_data.history['loss']\n    vacc=tr_data.history['val_accuracy']\n    vloss=tr_data.history['val_loss']\n    Epoch_count=len(tacc)+ start_epoch\n    Epochs=[]\n    for i in range (start_epoch ,Epoch_count):\n        Epochs.append(i+1)   \n    index_loss=np.argmin(vloss)#  this is the epoch with the lowest validation loss\n    val_lowest=vloss[index_loss]\n    index_acc=np.argmax(vacc)\n    acc_highest=vacc[index_acc]\n    plt.style.use('fivethirtyeight')\n    sc_label='best epoch= '+ str(index_loss+1 +start_epoch)\n    vc_label='best epoch= '+ str(index_acc + 1+ start_epoch)\n    fig,axes=plt.subplots(nrows=1, ncols=2, figsize=(20,8))\n    axes[0].plot(Epochs,tloss, 'r', label='Training loss')\n    axes[0].plot(Epochs,vloss,'g',label='Validation loss' )\n    axes[0].scatter(index_loss+1 +start_epoch,val_lowest, s=150, c= 'blue', label=sc_label)\n    axes[0].set_title('Training and Validation Loss')\n    axes[0].set_xlabel('Epochs')\n    axes[0].set_ylabel('Loss')\n    axes[0].legend()\n    axes[1].plot (Epochs,tacc,'r',label= 'Training Accuracy')\n    axes[1].plot (Epochs,vacc,'g',label= 'Validation Accuracy')\n    axes[1].scatter(index_acc+1 +start_epoch,acc_highest, s=150, c= 'blue', label=vc_label)\n    axes[1].set_title('Training and Validation Accuracy')\n    axes[1].set_xlabel('Epochs')\n    axes[1].set_ylabel('Accuracy')\n    axes[1].legend()\n    plt.tight_layout\n    #plt.style.use('fivethirtyeight')\n    plt.show()\n","metadata":{"papermill":{"duration":3.215798,"end_time":"2021-05-21T22:17:13.46258","exception":false,"start_time":"2021-05-21T22:17:10.246782","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### define function to generate the confusion matrix and classification report","metadata":{"papermill":{"duration":2.983235,"end_time":"2021-05-21T22:17:19.348807","exception":false,"start_time":"2021-05-21T22:17:16.365572","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"def print_info( test_gen, preds, print_code, save_dir, subject ):\n    class_dict=test_gen.class_indices\n    labels= test_gen.labels\n    file_names= test_gen.filenames \n    error_list=[]\n    true_class=[]\n    pred_class=[]\n    prob_list=[]\n    new_dict={}\n    error_indices=[]\n    y_pred=[]\n    for key,value in class_dict.items():\n        new_dict[value]=key             # dictionary {integer of class number: string of class name}\n    # store new_dict as a text fine in the save_dir\n    classes=list(new_dict.values())     # list of string of class names\n    dict_as_text=str(new_dict)\n    dict_name= subject + '-' +str(len(classes)) +'.txt'  \n    dict_path=os.path.join(save_dir,dict_name)    \n    with open(dict_path, 'w') as x_file:\n        x_file.write(dict_as_text)    \n    errors=0      \n    for i, p in enumerate(preds):\n        pred_index=np.argmax(p)        \n        true_index=labels[i]  # labels are integer values\n        if pred_index != true_index: # a misclassification has occurred\n            error_list.append(file_names[i])\n            true_class.append(new_dict[true_index])\n            pred_class.append(new_dict[pred_index])\n            prob_list.append(p[pred_index])\n            error_indices.append(true_index)            \n            errors=errors + 1\n        y_pred.append(pred_index)    \n    if print_code !=0:\n        if errors>0:\n            if print_code>errors:\n                r=errors\n            else:\n                r=print_code           \n            msg='{0:^28s}{1:^28s}{2:^28s}{3:^16s}'.format('Filename', 'Predicted Class' , 'True Class', 'Probability')\n            print_in_color(msg, (0,255,0),(55,65,80))\n            for i in range(r):                \n                split1=os.path.split(error_list[i])                \n                split2=os.path.split(split1[0])                \n                fname=split2[1] + '/' + split1[1]\n                msg='{0:^28s}{1:^28s}{2:^28s}{3:4s}{4:^6.4f}'.format(fname, pred_class[i],true_class[i], ' ', prob_list[i])\n                print_in_color(msg, (255,255,255), (55,65,60))\n                #print(error_list[i]  , pred_class[i], true_class[i], prob_list[i])               \n        else:\n            msg='With accuracy of 100 % there are no errors to print'\n            print_in_color(msg, (0,255,0),(55,65,80))\n    if errors>0:\n        plot_bar=[]\n        plot_class=[]\n        for  key, value in new_dict.items():        \n            count=error_indices.count(key) \n            if count!=0:\n                plot_bar.append(count) # list containg how many times a class c had an error\n                plot_class.append(value)   # stores the class \n        fig=plt.figure()\n        fig.set_figheight(len(plot_class)/3)\n        fig.set_figwidth(10)\n        plt.style.use('fivethirtyeight')\n        for i in range(0, len(plot_class)):\n            c=plot_class[i]\n            x=plot_bar[i]\n            plt.barh(c, x, )\n            plt.title( ' Errors by Class on Test Set')\n    y_true= np.array(labels)        \n    y_pred=np.array(y_pred)\n    if len(classes)<= 30:\n        # create a confusion matrix \n        cm = confusion_matrix(y_true, y_pred )        \n        length=len(classes)\n        if length<8:\n            fig_width=8\n            fig_height=8\n        else:\n            fig_width= int(length * .5)\n            fig_height= int(length * .5)\n        plt.figure(figsize=(fig_width, fig_height))\n        sns.heatmap(cm, annot=True, vmin=0, fmt='g', cmap='Blues', cbar=False)       \n        plt.xticks(np.arange(length)+.5, classes, rotation= 90)\n        plt.yticks(np.arange(length)+.5, classes, rotation=0)\n        plt.xlabel(\"Predicted\")\n        plt.ylabel(\"Actual\")\n        plt.title(\"Confusion Matrix\")\n        plt.show()\n    clr = classification_report(y_true, y_pred, target_names=classes)\n    print(\"Classification Report:\\n----------------------\\n\", clr)","metadata":{"papermill":{"duration":3.03999,"end_time":"2021-05-21T22:17:25.43058","exception":false,"start_time":"2021-05-21T22:17:22.39059","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.save('/kaggle/working/model_EfficientNetB1.h5')","metadata":{"papermill":{"duration":2.957254,"end_time":"2021-05-21T22:18:13.396108","exception":false,"start_time":"2021-05-21T22:18:10.438854","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Test","metadata":{"editable":false}},{"cell_type":"markdown","source":"### evaluate model on the test set then save the model","metadata":{"papermill":{"duration":2.985903,"end_time":"2021-05-21T22:17:31.319926","exception":false,"start_time":"2021-05-21T22:17:28.334023","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"tr_plot(history,0)\nsave_dir=r'./'\nsubject='plants'\nacc=model.evaluate( test_gen, batch_size=test_batch_size, verbose=1, steps=test_steps, return_dict=False)[1]*100\nmsg=f'accuracy on the test set is {acc:5.2f} %'\nprint_in_color(msg, (0,255,0),(55,65,80))\nsave_id=str (model_name +  '-' + subject +'-'+ str(acc)[:str(acc).rfind('.')+3] + '.h5')\nsave_loc=os.path.join(save_dir, save_id)\nmodel.save(save_loc)","metadata":{"papermill":{"duration":14.533987,"end_time":"2021-05-21T22:17:49.515069","exception":false,"start_time":"2021-05-21T22:17:34.981082","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### make predictions on test set and generate confusion matrix and classification report","metadata":{"papermill":{"duration":3.155539,"end_time":"2021-05-21T22:17:55.624417","exception":false,"start_time":"2021-05-21T22:17:52.468878","status":"completed"},"tags":[],"editable":false}},{"cell_type":"code","source":"print_code=0\npreds=model.predict(test_gen) \nprint_info( test_gen, preds, print_code, save_dir, subject )  ","metadata":{"papermill":{"duration":8.948725,"end_time":"2021-05-21T22:18:07.482198","exception":false,"start_time":"2021-05-21T22:17:58.533473","status":"completed"},"tags":[],"editable":false,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"editable":false},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"editable":false},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"editable":false},"outputs":[],"execution_count":null}]}