{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":13836,"databundleVersionId":1718836,"sourceType":"competition"},{"sourceId":1834160,"sourceType":"datasetVersion","datasetId":333968}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"papermill":{"default_parameters":{},"duration":12629.194758,"end_time":"2024-05-26T11:48:53.562581","environment_variables":{},"exception":true,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-05-26T08:18:24.367823","version":"2.5.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{}},"colab":{"provenance":[]}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import sys\npython = sys.executable # get python executable path\nprint(python)\n\nfor i in [\"numpy\",  \"scipy\", \"matplotlib\", \"pandas\", \"opencv-python\", \"opencv-contrib-python\", \"scikit-image\", \"scikit-learn\",]: # declare what packages we need\n  print(f\"Installing: {i}\")\n  ! $python -m pip install $i # install packages calling bash command from the notebook, e.g. \"/usr/bin/python -m pip install numpy\"\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom PIL import Image\nimport urllib.request\nfrom tensorflow import keras\nimport torch\nimport cv2","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2024-05-26T08:18:27.315146Z","iopub.status.busy":"2024-05-26T08:18:27.314747Z","iopub.status.idle":"2024-05-26T08:20:23.377756Z","shell.execute_reply":"2024-05-26T08:20:23.376864Z"},"papermill":{"duration":116.114344,"end_time":"2024-05-26T08:20:23.380206","exception":false,"start_time":"2024-05-26T08:18:27.265862","status":"completed"},"tags":[],"id":"a31cbb8c","outputId":"d43f9b60-8c0d-46a1-8b7e-7d3668b00ebc"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport torch.nn as nn\nfrom tqdm import tqdm","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:23.46735Z","iopub.status.busy":"2024-05-26T08:20:23.466265Z","iopub.status.idle":"2024-05-26T08:20:23.47147Z","shell.execute_reply":"2024-05-26T08:20:23.470575Z"},"papermill":{"duration":0.050223,"end_time":"2024-05-26T08:20:23.473378","exception":false,"start_time":"2024-05-26T08:20:23.423155","status":"completed"},"tags":[],"id":"3a5d4eaa"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Задача 1 Опция 1","metadata":{"papermill":{"duration":0.04159,"end_time":"2024-05-26T08:20:23.557287","exception":false,"start_time":"2024-05-26T08:20:23.515697","status":"completed"},"tags":[],"id":"8d124ae9"}},{"cell_type":"markdown","source":"## Подготовка всякого","metadata":{"papermill":{"duration":0.08883,"end_time":"2024-05-26T08:20:23.688554","exception":false,"start_time":"2024-05-26T08:20:23.599724","status":"completed"},"tags":[],"id":"020e3dd7"}},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/cassava-leaf-disease-classification/train.csv', sep=',')\ndf","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:23.773733Z","iopub.status.busy":"2024-05-26T08:20:23.772934Z","iopub.status.idle":"2024-05-26T08:20:23.81877Z","shell.execute_reply":"2024-05-26T08:20:23.81772Z"},"papermill":{"duration":0.090932,"end_time":"2024-05-26T08:20:23.82103","exception":false,"start_time":"2024-05-26T08:20:23.730098","status":"completed"},"tags":[],"id":"9aa262c2","outputId":"89d4e934-4ef7-4810-83f7-b23ad0d8a00c"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{"papermill":{"duration":0.043032,"end_time":"2024-05-26T08:20:23.908531","exception":false,"start_time":"2024-05-26T08:20:23.865499","status":"completed"},"tags":[],"id":"29057443"}},{"cell_type":"code","source":"y = df.label.to_list()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:23.99526Z","iopub.status.busy":"2024-05-26T08:20:23.99446Z","iopub.status.idle":"2024-05-26T08:20:23.999461Z","shell.execute_reply":"2024-05-26T08:20:23.998586Z"},"papermill":{"duration":0.050001,"end_time":"2024-05-26T08:20:24.001385","exception":false,"start_time":"2024-05-26T08:20:23.951384","status":"completed"},"tags":[],"id":"3cb9531a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_map = {\n    0: \"Cassava Bacterial Blight (CBB)\",\n    1: \"Cassava Brown Streak Disease (CBSD)\",\n    2: \"Cassava Green Mottle (CGM)\",\n    3: \"Cassava Mosaic Disease (CMD)\",\n    4: \"Healthy\"\n}","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:24.087271Z","iopub.status.busy":"2024-05-26T08:20:24.086909Z","iopub.status.idle":"2024-05-26T08:20:24.092712Z","shell.execute_reply":"2024-05-26T08:20:24.091946Z"},"papermill":{"duration":0.051306,"end_time":"2024-05-26T08:20:24.09461","exception":false,"start_time":"2024-05-26T08:20:24.043304","status":"completed"},"tags":[],"id":"d0a7289b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_ds, test_ds = keras.utils.image_dataset_from_directory(\n    directory=\"/kaggle/input/cassava-leaf-disease-classification/train_images\",\n    validation_split=0.2,\n    labels=y,\n    label_mode=\"int\",\n    subset=\"both\",\n    image_size=(600, 800),\n    batch_size=32,\n    seed=13\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:24.180545Z","iopub.status.busy":"2024-05-26T08:20:24.179704Z","iopub.status.idle":"2024-05-26T08:20:50.125923Z","shell.execute_reply":"2024-05-26T08:20:50.125023Z"},"papermill":{"duration":25.991732,"end_time":"2024-05-26T08:20:50.12833","exception":false,"start_time":"2024-05-26T08:20:24.136598","status":"completed"},"tags":[],"id":"6c182e53","outputId":"9a671882-af29-4e7f-b485-22dea6ac1b7a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transformation = keras.Sequential([\n    keras.layers.Resizing(75, 100),\n    keras.layers.Rescaling(1. / 255)\n])\nbase_train_ds = train_ds.map(lambda img, label: (transformation(img), label))\ntest_ds = test_ds.map(lambda img, label: (transformation(img), label))","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:50.216454Z","iopub.status.busy":"2024-05-26T08:20:50.215634Z","iopub.status.idle":"2024-05-26T08:20:50.285673Z","shell.execute_reply":"2024-05-26T08:20:50.284939Z"},"papermill":{"duration":0.116608,"end_time":"2024-05-26T08:20:50.287882","exception":false,"start_time":"2024-05-26T08:20:50.171274","status":"completed"},"tags":[],"id":"767fd912"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for data in base_train_ds:\n    X = data[0]\n    y = data[1]\n\n    fig, axs = plt.subplots(2, 2, figsize=(12, 8))\n\n    axs[0][0].imshow(X[0].numpy())\n    axs[0][0].set_title(labels_map[y[0].numpy()])\n\n    axs[0][1].imshow(X[1].numpy())\n    axs[0][1].set_title(labels_map[y[1].numpy()])\n\n    axs[1][0].imshow(X[2].numpy())\n    axs[1][0].set_title(labels_map[y[2].numpy()])\n\n    axs[1][1].imshow(X[3].numpy())\n    axs[1][1].set_title(labels_map[y[3].numpy()])\n\n    break","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:50.374999Z","iopub.status.busy":"2024-05-26T08:20:50.374293Z","iopub.status.idle":"2024-05-26T08:20:52.734801Z","shell.execute_reply":"2024-05-26T08:20:52.733901Z"},"papermill":{"duration":2.407728,"end_time":"2024-05-26T08:20:52.73845","exception":false,"start_time":"2024-05-26T08:20:50.330722","status":"completed"},"tags":[],"id":"8bde7f7d","outputId":"2e007438-51c3-449b-eebd-104036917c5a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"augmentation = keras.Sequential([\n    keras.layers.RandomFlip(\"horizontal\"),\n    keras.layers.RandomRotation(0.2),\n])\naugmentation_train_ds = base_train_ds.map(lambda img, label: (augmentation(img), label))","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:52.829671Z","iopub.status.busy":"2024-05-26T08:20:52.828751Z","iopub.status.idle":"2024-05-26T08:20:52.954Z","shell.execute_reply":"2024-05-26T08:20:52.952997Z"},"papermill":{"duration":0.173274,"end_time":"2024-05-26T08:20:52.956391","exception":false,"start_time":"2024-05-26T08:20:52.783117","status":"completed"},"tags":[],"id":"386b15d3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for data in augmentation_train_ds:\n    X = data[0]\n    y = data[1]\n\n    fig, axs = plt.subplots(2, 2, figsize=(12, 8))\n\n    axs[0][0].imshow(X[0].numpy())\n    axs[0][0].set_title(labels_map[y[0].numpy()])\n\n    axs[0][1].imshow(X[1].numpy())\n    axs[0][1].set_title(labels_map[y[1].numpy()])\n\n    axs[1][0].imshow(X[2].numpy())\n    axs[1][0].set_title(labels_map[y[2].numpy()])\n\n    axs[1][1].imshow(X[3].numpy())\n    axs[1][1].set_title(labels_map[y[3].numpy()])\n\n    break","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:53.047261Z","iopub.status.busy":"2024-05-26T08:20:53.046886Z","iopub.status.idle":"2024-05-26T08:20:55.223311Z","shell.execute_reply":"2024-05-26T08:20:55.222337Z"},"papermill":{"duration":2.225167,"end_time":"2024-05-26T08:20:55.226569","exception":false,"start_time":"2024-05-26T08:20:53.001402","status":"completed"},"tags":[],"id":"63419356","outputId":"531bba9c-a64c-4188-be04-0a56230c0a28"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table = pd.DataFrame(columns=['experiment', 'train_acc', 'test_acc'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:55.323662Z","iopub.status.busy":"2024-05-26T08:20:55.323281Z","iopub.status.idle":"2024-05-26T08:20:55.329362Z","shell.execute_reply":"2024-05-26T08:20:55.328422Z"},"papermill":{"duration":0.05714,"end_time":"2024-05-26T08:20:55.331398","exception":false,"start_time":"2024-05-26T08:20:55.274258","status":"completed"},"tags":[],"id":"edde57f8"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#df_table.loc[ len(df_table.index )] = ['model', 0.99, 0.99]","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:55.428711Z","iopub.status.busy":"2024-05-26T08:20:55.427847Z","iopub.status.idle":"2024-05-26T08:20:55.432111Z","shell.execute_reply":"2024-05-26T08:20:55.431142Z"},"papermill":{"duration":0.054034,"end_time":"2024-05-26T08:20:55.434023","exception":false,"start_time":"2024-05-26T08:20:55.379989","status":"completed"},"tags":[],"id":"1e0b4e60"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{"papermill":{"duration":0.04661,"end_time":"2024-05-26T08:20:55.528058","exception":false,"start_time":"2024-05-26T08:20:55.481448","status":"completed"},"tags":[],"id":"39aa8962"}},{"cell_type":"code","source":"from keras.callbacks import History\nhistory = History()\nepoch_number = 10","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:55.625304Z","iopub.status.busy":"2024-05-26T08:20:55.624818Z","iopub.status.idle":"2024-05-26T08:20:55.629163Z","shell.execute_reply":"2024-05-26T08:20:55.628326Z"},"papermill":{"duration":0.056029,"end_time":"2024-05-26T08:20:55.631061","exception":false,"start_time":"2024-05-26T08:20:55.575032","status":"completed"},"tags":[],"id":"18580faa"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+SGD","metadata":{"papermill":{"duration":0.048654,"end_time":"2024-05-26T08:20:55.727309","exception":false,"start_time":"2024-05-26T08:20:55.678655","status":"completed"},"tags":[],"id":"441a52dc"}},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:55.8243Z","iopub.status.busy":"2024-05-26T08:20:55.82372Z","iopub.status.idle":"2024-05-26T08:20:55.840141Z","shell.execute_reply":"2024-05-26T08:20:55.839215Z"},"papermill":{"duration":0.067482,"end_time":"2024-05-26T08:20:55.842234","exception":false,"start_time":"2024-05-26T08:20:55.774752","status":"completed"},"tags":[],"id":"d3e6d84f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\nmodel.compile(\n    optimizer=keras.optimizers.SGD(lr),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:55.940245Z","iopub.status.busy":"2024-05-26T08:20:55.939572Z","iopub.status.idle":"2024-05-26T08:20:55.960424Z","shell.execute_reply":"2024-05-26T08:20:55.95967Z"},"papermill":{"duration":0.072002,"end_time":"2024-05-26T08:20:55.962401","exception":false,"start_time":"2024-05-26T08:20:55.890399","status":"completed"},"tags":[],"id":"c4cc9848"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=base_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:20:56.059598Z","iopub.status.busy":"2024-05-26T08:20:56.058848Z","iopub.status.idle":"2024-05-26T08:34:35.99639Z","shell.execute_reply":"2024-05-26T08:34:35.995249Z"},"papermill":{"duration":819.988659,"end_time":"2024-05-26T08:34:35.998815","exception":false,"start_time":"2024-05-26T08:20:56.010156","status":"completed"},"tags":[],"id":"3187a6e8","outputId":"4e336d1f-9ca7-4a91-c2ef-04cac247cc83"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:36.987706Z","iopub.status.busy":"2024-05-26T08:34:36.986776Z","iopub.status.idle":"2024-05-26T08:34:37.163704Z","shell.execute_reply":"2024-05-26T08:34:37.162793Z"},"papermill":{"duration":0.663825,"end_time":"2024-05-26T08:34:37.166026","exception":false,"start_time":"2024-05-26T08:34:36.502201","status":"completed"},"tags":[],"id":"b1adc01d","outputId":"c3e51bbd-0f94-4bd0-b73e-5a5783289c1a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:38.22425Z","iopub.status.busy":"2024-05-26T08:34:38.223833Z","iopub.status.idle":"2024-05-26T08:34:38.466434Z","shell.execute_reply":"2024-05-26T08:34:38.465392Z"},"papermill":{"duration":0.824005,"end_time":"2024-05-26T08:34:38.4688","exception":false,"start_time":"2024-05-26T08:34:37.644795","status":"completed"},"tags":[],"id":"d8e046ac","outputId":"4e333bad-4cb3-4452-8189-e2e7db2dffba"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+SGD', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_SGD.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:39.430093Z","iopub.status.busy":"2024-05-26T08:34:39.429673Z","iopub.status.idle":"2024-05-26T08:34:39.479057Z","shell.execute_reply":"2024-05-26T08:34:39.478288Z"},"papermill":{"duration":0.532345,"end_time":"2024-05-26T08:34:39.481145","exception":false,"start_time":"2024-05-26T08:34:38.9488","status":"completed"},"tags":[],"id":"e3b28e5f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:40.49081Z","iopub.status.busy":"2024-05-26T08:34:40.490412Z","iopub.status.idle":"2024-05-26T08:34:40.500869Z","shell.execute_reply":"2024-05-26T08:34:40.499975Z"},"papermill":{"duration":0.494891,"end_time":"2024-05-26T08:34:40.502765","exception":false,"start_time":"2024-05-26T08:34:40.007874","status":"completed"},"tags":[],"id":"b2e6f2bb","outputId":"fbbb3383-6d13-4ac6-8a75-3f0be14d4052"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+ADAM","metadata":{"papermill":{"duration":0.479003,"end_time":"2024-05-26T08:34:41.462895","exception":false,"start_time":"2024-05-26T08:34:40.983892","status":"completed"},"tags":[],"id":"f99feddb"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:42.448279Z","iopub.status.busy":"2024-05-26T08:34:42.447545Z","iopub.status.idle":"2024-05-26T08:34:42.451972Z","shell.execute_reply":"2024-05-26T08:34:42.451052Z"},"papermill":{"duration":0.489551,"end_time":"2024-05-26T08:34:42.453933","exception":false,"start_time":"2024-05-26T08:34:41.964382","status":"completed"},"tags":[],"id":"47fc7744"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:43.461841Z","iopub.status.busy":"2024-05-26T08:34:43.461031Z","iopub.status.idle":"2024-05-26T08:34:43.477351Z","shell.execute_reply":"2024-05-26T08:34:43.476406Z"},"papermill":{"duration":0.497805,"end_time":"2024-05-26T08:34:43.479151","exception":false,"start_time":"2024-05-26T08:34:42.981346","status":"completed"},"tags":[],"id":"db9cff79"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\nmodel.compile(\n    optimizer=keras.optimizers.Adam(lr),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:44.493838Z","iopub.status.busy":"2024-05-26T08:34:44.493419Z","iopub.status.idle":"2024-05-26T08:34:44.504763Z","shell.execute_reply":"2024-05-26T08:34:44.503979Z"},"papermill":{"duration":0.547302,"end_time":"2024-05-26T08:34:44.50679","exception":false,"start_time":"2024-05-26T08:34:43.959488","status":"completed"},"tags":[],"id":"06307f4d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=base_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:34:45.584861Z","iopub.status.busy":"2024-05-26T08:34:45.583983Z","iopub.status.idle":"2024-05-26T08:47:11.585898Z","shell.execute_reply":"2024-05-26T08:47:11.584889Z"},"papermill":{"duration":747.462346,"end_time":"2024-05-26T08:47:12.546847","exception":false,"start_time":"2024-05-26T08:34:45.084501","status":"completed"},"tags":[],"id":"98547538","outputId":"d1f64b73-b346-4994-cab0-c581959e63a9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:14.380465Z","iopub.status.busy":"2024-05-26T08:47:14.380118Z","iopub.status.idle":"2024-05-26T08:47:14.612699Z","shell.execute_reply":"2024-05-26T08:47:14.611751Z"},"papermill":{"duration":1.150218,"end_time":"2024-05-26T08:47:14.614792","exception":false,"start_time":"2024-05-26T08:47:13.464574","status":"completed"},"tags":[],"id":"e3f1aad4","outputId":"3b9279e9-8757-417b-9e02-111c43d7fb05"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:16.510788Z","iopub.status.busy":"2024-05-26T08:47:16.510387Z","iopub.status.idle":"2024-05-26T08:47:16.738078Z","shell.execute_reply":"2024-05-26T08:47:16.737114Z"},"papermill":{"duration":1.146817,"end_time":"2024-05-26T08:47:16.740063","exception":false,"start_time":"2024-05-26T08:47:15.593246","status":"completed"},"tags":[],"id":"a03285a8","outputId":"a43bd84f-dc1a-47bb-d462-68b3ccb8eb68"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+ADAM', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_ADAM.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:18.659409Z","iopub.status.busy":"2024-05-26T08:47:18.658623Z","iopub.status.idle":"2024-05-26T08:47:18.704453Z","shell.execute_reply":"2024-05-26T08:47:18.703603Z"},"papermill":{"duration":0.970966,"end_time":"2024-05-26T08:47:18.706892","exception":false,"start_time":"2024-05-26T08:47:17.735926","status":"completed"},"tags":[],"id":"b95b4587"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:20.58486Z","iopub.status.busy":"2024-05-26T08:47:20.584479Z","iopub.status.idle":"2024-05-26T08:47:20.594675Z","shell.execute_reply":"2024-05-26T08:47:20.593775Z"},"papermill":{"duration":0.977941,"end_time":"2024-05-26T08:47:20.596809","exception":false,"start_time":"2024-05-26T08:47:19.618868","status":"completed"},"tags":[],"id":"8788bb9f","outputId":"2a2e0c1b-2c35-4e72-90cb-026dcaaeb981"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+RMS","metadata":{"papermill":{"duration":1.040973,"end_time":"2024-05-26T08:47:22.554362","exception":false,"start_time":"2024-05-26T08:47:21.513389","status":"completed"},"tags":[],"id":"1b981dbe"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:24.484519Z","iopub.status.busy":"2024-05-26T08:47:24.484132Z","iopub.status.idle":"2024-05-26T08:47:24.488926Z","shell.execute_reply":"2024-05-26T08:47:24.488008Z"},"papermill":{"duration":0.978899,"end_time":"2024-05-26T08:47:24.490959","exception":false,"start_time":"2024-05-26T08:47:23.51206","status":"completed"},"tags":[],"id":"4e2d9d12"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:26.459911Z","iopub.status.busy":"2024-05-26T08:47:26.459528Z","iopub.status.idle":"2024-05-26T08:47:26.477146Z","shell.execute_reply":"2024-05-26T08:47:26.476117Z"},"papermill":{"duration":1.074132,"end_time":"2024-05-26T08:47:26.479303","exception":false,"start_time":"2024-05-26T08:47:25.405171","status":"completed"},"tags":[],"id":"dffe6bdc"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\nmodel.compile(\n    optimizer=keras.optimizers.RMSprop(lr),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:28.374947Z","iopub.status.busy":"2024-05-26T08:47:28.374184Z","iopub.status.idle":"2024-05-26T08:47:28.38671Z","shell.execute_reply":"2024-05-26T08:47:28.385812Z"},"papermill":{"duration":0.947299,"end_time":"2024-05-26T08:47:28.389063","exception":false,"start_time":"2024-05-26T08:47:27.441764","status":"completed"},"tags":[],"id":"d0fe684a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=base_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T08:47:30.270862Z","iopub.status.busy":"2024-05-26T08:47:30.2699Z","iopub.status.idle":"2024-05-26T09:00:16.16427Z","shell.execute_reply":"2024-05-26T09:00:16.163287Z"},"papermill":{"duration":766.811693,"end_time":"2024-05-26T09:00:16.166194","exception":false,"start_time":"2024-05-26T08:47:29.354501","status":"completed"},"tags":[],"id":"924cca64","outputId":"fb193925-e07e-4a80-fe03-9f3a8eda3472"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:18.937154Z","iopub.status.busy":"2024-05-26T09:00:18.936208Z","iopub.status.idle":"2024-05-26T09:00:19.187807Z","shell.execute_reply":"2024-05-26T09:00:19.186854Z"},"papermill":{"duration":1.666405,"end_time":"2024-05-26T09:00:19.18986","exception":false,"start_time":"2024-05-26T09:00:17.523455","status":"completed"},"tags":[],"id":"11215d99","outputId":"b97a0e46-a915-41ce-c67d-d54e037695e9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:21.989084Z","iopub.status.busy":"2024-05-26T09:00:21.988697Z","iopub.status.idle":"2024-05-26T09:00:22.222957Z","shell.execute_reply":"2024-05-26T09:00:22.222027Z"},"papermill":{"duration":1.621629,"end_time":"2024-05-26T09:00:22.225038","exception":false,"start_time":"2024-05-26T09:00:20.603409","status":"completed"},"tags":[],"id":"4f3a5cd0","outputId":"5b4b2d8e-83f6-4b7c-fb7f-4fbb781da41b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+RMS', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_RMS.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:25.090461Z","iopub.status.busy":"2024-05-26T09:00:25.090035Z","iopub.status.idle":"2024-05-26T09:00:25.128915Z","shell.execute_reply":"2024-05-26T09:00:25.127897Z"},"papermill":{"duration":1.468532,"end_time":"2024-05-26T09:00:25.131121","exception":false,"start_time":"2024-05-26T09:00:23.662589","status":"completed"},"tags":[],"id":"25a870be"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:27.901436Z","iopub.status.busy":"2024-05-26T09:00:27.901034Z","iopub.status.idle":"2024-05-26T09:00:27.911345Z","shell.execute_reply":"2024-05-26T09:00:27.910467Z"},"papermill":{"duration":1.42317,"end_time":"2024-05-26T09:00:27.913273","exception":false,"start_time":"2024-05-26T09:00:26.490103","status":"completed"},"tags":[],"id":"8ac9f75f","outputId":"f558e572-6238-4fa8-cb9f-c722fa69f959"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+Aug+SGD","metadata":{"papermill":{"duration":1.412282,"end_time":"2024-05-26T09:00:30.718672","exception":false,"start_time":"2024-05-26T09:00:29.30639","status":"completed"},"tags":[],"id":"37e81a4d"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:33.578641Z","iopub.status.busy":"2024-05-26T09:00:33.57828Z","iopub.status.idle":"2024-05-26T09:00:33.582758Z","shell.execute_reply":"2024-05-26T09:00:33.581823Z"},"papermill":{"duration":1.432018,"end_time":"2024-05-26T09:00:33.584681","exception":false,"start_time":"2024-05-26T09:00:32.152663","status":"completed"},"tags":[],"id":"27757ce7"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:36.34613Z","iopub.status.busy":"2024-05-26T09:00:36.345708Z","iopub.status.idle":"2024-05-26T09:00:36.361884Z","shell.execute_reply":"2024-05-26T09:00:36.360966Z"},"papermill":{"duration":1.4346,"end_time":"2024-05-26T09:00:36.363849","exception":false,"start_time":"2024-05-26T09:00:34.929249","status":"completed"},"tags":[],"id":"04ec958a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\nmodel.compile(\n    optimizer=keras.optimizers.SGD(lr),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:39.30623Z","iopub.status.busy":"2024-05-26T09:00:39.305819Z","iopub.status.idle":"2024-05-26T09:00:39.315535Z","shell.execute_reply":"2024-05-26T09:00:39.314596Z"},"papermill":{"duration":1.556592,"end_time":"2024-05-26T09:00:39.31753","exception":false,"start_time":"2024-05-26T09:00:37.760938","status":"completed"},"tags":[],"id":"48ec03fb"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=augmentation_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:00:42.111542Z","iopub.status.busy":"2024-05-26T09:00:42.11119Z","iopub.status.idle":"2024-05-26T09:14:02.557858Z","shell.execute_reply":"2024-05-26T09:14:02.55683Z"},"papermill":{"duration":803.649154,"end_time":"2024-05-26T09:14:04.317059","exception":false,"start_time":"2024-05-26T09:00:40.667905","status":"completed"},"tags":[],"id":"1843e88f","outputId":"7da6039c-0e05-4f67-899f-cecdde2bbe79"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:08.021756Z","iopub.status.busy":"2024-05-26T09:14:08.021309Z","iopub.status.idle":"2024-05-26T09:14:08.274117Z","shell.execute_reply":"2024-05-26T09:14:08.273212Z"},"papermill":{"duration":2.100501,"end_time":"2024-05-26T09:14:08.276176","exception":false,"start_time":"2024-05-26T09:14:06.175675","status":"completed"},"tags":[],"id":"5208c882","outputId":"e9f4b2e4-90ec-4fd8-d17f-70828db2d5c7"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:12.019909Z","iopub.status.busy":"2024-05-26T09:14:12.019022Z","iopub.status.idle":"2024-05-26T09:14:12.249283Z","shell.execute_reply":"2024-05-26T09:14:12.248303Z"},"papermill":{"duration":2.111645,"end_time":"2024-05-26T09:14:12.251255","exception":false,"start_time":"2024-05-26T09:14:10.13961","status":"completed"},"tags":[],"id":"1123cd80","outputId":"fa34977e-2454-4d59-9a4c-a0a5921a0ecf"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+SGD+AUG', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_SGD_Aug.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:15.88826Z","iopub.status.busy":"2024-05-26T09:14:15.887352Z","iopub.status.idle":"2024-05-26T09:14:15.917811Z","shell.execute_reply":"2024-05-26T09:14:15.916876Z"},"papermill":{"duration":1.884458,"end_time":"2024-05-26T09:14:15.919928","exception":false,"start_time":"2024-05-26T09:14:14.03547","status":"completed"},"tags":[],"id":"97e50d8c"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:19.643444Z","iopub.status.busy":"2024-05-26T09:14:19.643045Z","iopub.status.idle":"2024-05-26T09:14:19.653194Z","shell.execute_reply":"2024-05-26T09:14:19.652286Z"},"papermill":{"duration":1.866078,"end_time":"2024-05-26T09:14:19.655216","exception":false,"start_time":"2024-05-26T09:14:17.789138","status":"completed"},"tags":[],"id":"943fdc13","outputId":"74cbd1bd-897f-4cfd-df8a-73717e4a5a63"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+Aug+ADAM","metadata":{"papermill":{"duration":1.799669,"end_time":"2024-05-26T09:14:23.393388","exception":false,"start_time":"2024-05-26T09:14:21.593719","status":"completed"},"tags":[],"id":"c8d78987"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:27.234033Z","iopub.status.busy":"2024-05-26T09:14:27.233275Z","iopub.status.idle":"2024-05-26T09:14:27.23777Z","shell.execute_reply":"2024-05-26T09:14:27.23678Z"},"papermill":{"duration":1.929048,"end_time":"2024-05-26T09:14:27.239848","exception":false,"start_time":"2024-05-26T09:14:25.3108","status":"completed"},"tags":[],"id":"84dc9d8b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:30.955401Z","iopub.status.busy":"2024-05-26T09:14:30.954497Z","iopub.status.idle":"2024-05-26T09:14:30.972689Z","shell.execute_reply":"2024-05-26T09:14:30.971651Z"},"papermill":{"duration":1.88723,"end_time":"2024-05-26T09:14:30.974695","exception":false,"start_time":"2024-05-26T09:14:29.087465","status":"completed"},"tags":[],"id":"a322f774"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\nmodel.compile(\n    optimizer=keras.optimizers.Adam(lr),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:34.615258Z","iopub.status.busy":"2024-05-26T09:14:34.614487Z","iopub.status.idle":"2024-05-26T09:14:34.623846Z","shell.execute_reply":"2024-05-26T09:14:34.622915Z"},"papermill":{"duration":1.862612,"end_time":"2024-05-26T09:14:34.62573","exception":false,"start_time":"2024-05-26T09:14:32.763118","status":"completed"},"tags":[],"id":"000de5fa"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=augmentation_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:14:38.385065Z","iopub.status.busy":"2024-05-26T09:14:38.384222Z","iopub.status.idle":"2024-05-26T09:28:00.230359Z","shell.execute_reply":"2024-05-26T09:28:00.229337Z"},"papermill":{"duration":806.002992,"end_time":"2024-05-26T09:28:02.521188","exception":false,"start_time":"2024-05-26T09:14:36.518196","status":"completed"},"tags":[],"id":"485564e1","outputId":"bbc7a67e-5f36-4a7c-f83b-8fb7ce054827"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:07.141783Z","iopub.status.busy":"2024-05-26T09:28:07.141349Z","iopub.status.idle":"2024-05-26T09:28:07.419011Z","shell.execute_reply":"2024-05-26T09:28:07.418093Z"},"papermill":{"duration":2.598059,"end_time":"2024-05-26T09:28:07.421189","exception":false,"start_time":"2024-05-26T09:28:04.82313","status":"completed"},"tags":[],"id":"f6b7e655","outputId":"586dc668-f3e4-43f1-fca1-574c803b1c56"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:11.992591Z","iopub.status.busy":"2024-05-26T09:28:11.992203Z","iopub.status.idle":"2024-05-26T09:28:12.22384Z","shell.execute_reply":"2024-05-26T09:28:12.222918Z"},"papermill":{"duration":2.548691,"end_time":"2024-05-26T09:28:12.225948","exception":false,"start_time":"2024-05-26T09:28:09.677257","status":"completed"},"tags":[],"id":"7a460679","outputId":"33e31e2d-04d6-4361-cbc4-66c42f8ba51d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+ADAM+AUG', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_ADAM_Aug.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:16.983949Z","iopub.status.busy":"2024-05-26T09:28:16.983051Z","iopub.status.idle":"2024-05-26T09:28:17.026195Z","shell.execute_reply":"2024-05-26T09:28:17.025188Z"},"papermill":{"duration":2.439291,"end_time":"2024-05-26T09:28:17.028434","exception":false,"start_time":"2024-05-26T09:28:14.589143","status":"completed"},"tags":[],"id":"5d8e92b5"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:21.558758Z","iopub.status.busy":"2024-05-26T09:28:21.557878Z","iopub.status.idle":"2024-05-26T09:28:21.56844Z","shell.execute_reply":"2024-05-26T09:28:21.567542Z"},"papermill":{"duration":2.317012,"end_time":"2024-05-26T09:28:21.570494","exception":false,"start_time":"2024-05-26T09:28:19.253482","status":"completed"},"tags":[],"id":"3233b537","outputId":"e48f9dd6-1752-458c-e3e9-dd9f8a87e580"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+Aug+RMS","metadata":{"papermill":{"duration":2.282226,"end_time":"2024-05-26T09:28:26.210416","exception":false,"start_time":"2024-05-26T09:28:23.92819","status":"completed"},"tags":[],"id":"a38903b5"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:30.816643Z","iopub.status.busy":"2024-05-26T09:28:30.816247Z","iopub.status.idle":"2024-05-26T09:28:30.820577Z","shell.execute_reply":"2024-05-26T09:28:30.819786Z"},"papermill":{"duration":2.297629,"end_time":"2024-05-26T09:28:30.822477","exception":false,"start_time":"2024-05-26T09:28:28.524848","status":"completed"},"tags":[],"id":"4de78346"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:35.390951Z","iopub.status.busy":"2024-05-26T09:28:35.390548Z","iopub.status.idle":"2024-05-26T09:28:35.406713Z","shell.execute_reply":"2024-05-26T09:28:35.405926Z"},"papermill":{"duration":2.286792,"end_time":"2024-05-26T09:28:35.408598","exception":false,"start_time":"2024-05-26T09:28:33.121806","status":"completed"},"tags":[],"id":"ec287397"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\nmodel.compile(\n    optimizer=keras.optimizers.RMSprop(lr),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:40.182865Z","iopub.status.busy":"2024-05-26T09:28:40.182472Z","iopub.status.idle":"2024-05-26T09:28:40.191821Z","shell.execute_reply":"2024-05-26T09:28:40.191116Z"},"papermill":{"duration":2.325687,"end_time":"2024-05-26T09:28:40.19375","exception":false,"start_time":"2024-05-26T09:28:37.868063","status":"completed"},"tags":[],"id":"39c6f75a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=augmentation_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:28:44.79365Z","iopub.status.busy":"2024-05-26T09:28:44.793255Z","iopub.status.idle":"2024-05-26T09:42:06.81466Z","shell.execute_reply":"2024-05-26T09:42:06.813618Z"},"papermill":{"duration":807.052469,"end_time":"2024-05-26T09:42:09.550835","exception":false,"start_time":"2024-05-26T09:28:42.498366","status":"completed"},"tags":[],"id":"208dea6b","outputId":"9cab5b49-a487-454c-8bce-68bf69670c5c"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:14.971284Z","iopub.status.busy":"2024-05-26T09:42:14.970884Z","iopub.status.idle":"2024-05-26T09:42:15.156779Z","shell.execute_reply":"2024-05-26T09:42:15.15586Z"},"papermill":{"duration":2.840205,"end_time":"2024-05-26T09:42:15.158779","exception":false,"start_time":"2024-05-26T09:42:12.318574","status":"completed"},"tags":[],"id":"0d7af714","outputId":"c548b0dc-89d8-4fcf-ba1d-cc981c7bf5b8"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:20.631385Z","iopub.status.busy":"2024-05-26T09:42:20.630467Z","iopub.status.idle":"2024-05-26T09:42:20.873241Z","shell.execute_reply":"2024-05-26T09:42:20.87229Z"},"papermill":{"duration":2.892997,"end_time":"2024-05-26T09:42:20.875313","exception":false,"start_time":"2024-05-26T09:42:17.982316","status":"completed"},"tags":[],"id":"16f2a6bf","outputId":"f11a5f59-d3b3-462b-c648-52f989f47702"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+RMS+AUG', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_RMS_Aug.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:26.463652Z","iopub.status.busy":"2024-05-26T09:42:26.462737Z","iopub.status.idle":"2024-05-26T09:42:26.502727Z","shell.execute_reply":"2024-05-26T09:42:26.501485Z"},"papermill":{"duration":2.699074,"end_time":"2024-05-26T09:42:26.505035","exception":false,"start_time":"2024-05-26T09:42:23.805961","status":"completed"},"tags":[],"id":"3fdafcc2"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:32.098154Z","iopub.status.busy":"2024-05-26T09:42:32.097747Z","iopub.status.idle":"2024-05-26T09:42:32.107831Z","shell.execute_reply":"2024-05-26T09:42:32.106992Z"},"papermill":{"duration":2.764215,"end_time":"2024-05-26T09:42:32.110189","exception":false,"start_time":"2024-05-26T09:42:29.345974","status":"completed"},"tags":[],"id":"bbce4f14","outputId":"2c61644d-6a19-4010-bb87-cf38715ce0c0"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+scheduler+SGD","metadata":{"papermill":{"duration":2.758527,"end_time":"2024-05-26T09:42:37.673502","exception":false,"start_time":"2024-05-26T09:42:34.914975","status":"completed"},"tags":[],"id":"ed94119e"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:43.185082Z","iopub.status.busy":"2024-05-26T09:42:43.184691Z","iopub.status.idle":"2024-05-26T09:42:43.188926Z","shell.execute_reply":"2024-05-26T09:42:43.188108Z"},"papermill":{"duration":2.768843,"end_time":"2024-05-26T09:42:43.190821","exception":false,"start_time":"2024-05-26T09:42:40.421978","status":"completed"},"tags":[],"id":"f8af083b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:48.648487Z","iopub.status.busy":"2024-05-26T09:42:48.648119Z","iopub.status.idle":"2024-05-26T09:42:48.664199Z","shell.execute_reply":"2024-05-26T09:42:48.663407Z"},"papermill":{"duration":2.829082,"end_time":"2024-05-26T09:42:48.666053","exception":false,"start_time":"2024-05-26T09:42:45.836971","status":"completed"},"tags":[],"id":"1d4d4ad9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\n\nscheduler = keras.optimizers.schedules.ExponentialDecay(\n    initial_learning_rate=lr,\n    decay_steps=122,\n    decay_rate=0.82\n)\nmodel.compile(\n    optimizer=keras.optimizers.SGD(scheduler),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:54.195715Z","iopub.status.busy":"2024-05-26T09:42:54.19533Z","iopub.status.idle":"2024-05-26T09:42:54.204802Z","shell.execute_reply":"2024-05-26T09:42:54.204013Z"},"papermill":{"duration":2.896823,"end_time":"2024-05-26T09:42:54.206631","exception":false,"start_time":"2024-05-26T09:42:51.309808","status":"completed"},"tags":[],"id":"e3404a5e"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=base_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:42:59.721743Z","iopub.status.busy":"2024-05-26T09:42:59.721351Z","iopub.status.idle":"2024-05-26T09:56:09.900015Z","shell.execute_reply":"2024-05-26T09:56:09.899011Z"},"papermill":{"duration":796.266288,"end_time":"2024-05-26T09:56:13.101138","exception":false,"start_time":"2024-05-26T09:42:56.83485","status":"completed"},"tags":[],"id":"ac92cb1f","outputId":"f4122bfa-9fa4-4198-dae3-b9f2d122324c"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:56:19.483972Z","iopub.status.busy":"2024-05-26T09:56:19.482834Z","iopub.status.idle":"2024-05-26T09:56:19.735917Z","shell.execute_reply":"2024-05-26T09:56:19.734987Z"},"papermill":{"duration":3.449934,"end_time":"2024-05-26T09:56:19.738175","exception":false,"start_time":"2024-05-26T09:56:16.288241","status":"completed"},"tags":[],"id":"94656418","outputId":"1aed9f73-8299-41cd-dc89-8d07741a2b06"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:56:26.21587Z","iopub.status.busy":"2024-05-26T09:56:26.215231Z","iopub.status.idle":"2024-05-26T09:56:26.398437Z","shell.execute_reply":"2024-05-26T09:56:26.397494Z"},"papermill":{"duration":3.373547,"end_time":"2024-05-26T09:56:26.400449","exception":false,"start_time":"2024-05-26T09:56:23.026902","status":"completed"},"tags":[],"id":"7a850df2","outputId":"13dbbefb-fcfc-4ac9-9c95-4c16cfa6e545"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+SGD+SCHED', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_SGD_Sched.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:56:33.053923Z","iopub.status.busy":"2024-05-26T09:56:33.053549Z","iopub.status.idle":"2024-05-26T09:56:33.084387Z","shell.execute_reply":"2024-05-26T09:56:33.08355Z"},"papermill":{"duration":3.337971,"end_time":"2024-05-26T09:56:33.086744","exception":false,"start_time":"2024-05-26T09:56:29.748773","status":"completed"},"tags":[],"id":"c5dfb3e3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:56:39.514771Z","iopub.status.busy":"2024-05-26T09:56:39.51402Z","iopub.status.idle":"2024-05-26T09:56:39.52435Z","shell.execute_reply":"2024-05-26T09:56:39.523497Z"},"papermill":{"duration":3.254588,"end_time":"2024-05-26T09:56:39.526424","exception":false,"start_time":"2024-05-26T09:56:36.271836","status":"completed"},"tags":[],"id":"301f6a64","outputId":"14066b81-7c9d-4ef0-9538-476832f2b21f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"papermill":{"duration":3.183988,"end_time":"2024-05-26T09:56:45.933634","exception":false,"start_time":"2024-05-26T09:56:42.749646","status":"completed"},"tags":[],"id":"ca6aa93a"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+scheduler+ADAM","metadata":{"papermill":{"duration":3.285608,"end_time":"2024-05-26T09:56:52.396655","exception":false,"start_time":"2024-05-26T09:56:49.111047","status":"completed"},"tags":[],"id":"e3da9a66"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:56:58.837059Z","iopub.status.busy":"2024-05-26T09:56:58.836322Z","iopub.status.idle":"2024-05-26T09:56:58.840818Z","shell.execute_reply":"2024-05-26T09:56:58.839871Z"},"papermill":{"duration":3.168555,"end_time":"2024-05-26T09:56:58.842707","exception":false,"start_time":"2024-05-26T09:56:55.674152","status":"completed"},"tags":[],"id":"e3e605f4"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:57:05.226194Z","iopub.status.busy":"2024-05-26T09:57:05.225772Z","iopub.status.idle":"2024-05-26T09:57:05.241795Z","shell.execute_reply":"2024-05-26T09:57:05.240871Z"},"papermill":{"duration":3.211909,"end_time":"2024-05-26T09:57:05.243768","exception":false,"start_time":"2024-05-26T09:57:02.031859","status":"completed"},"tags":[],"id":"7686b56f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\n\nscheduler = keras.optimizers.schedules.ExponentialDecay(\n    initial_learning_rate=lr,\n    decay_steps=122,\n    decay_rate=0.82\n)\nmodel.compile(\n    optimizer=keras.optimizers.Adam(scheduler),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:57:11.717487Z","iopub.status.busy":"2024-05-26T09:57:11.717099Z","iopub.status.idle":"2024-05-26T09:57:11.726303Z","shell.execute_reply":"2024-05-26T09:57:11.72556Z"},"papermill":{"duration":3.250454,"end_time":"2024-05-26T09:57:11.728183","exception":false,"start_time":"2024-05-26T09:57:08.477729","status":"completed"},"tags":[],"id":"316b86bd"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=base_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T09:57:18.105937Z","iopub.status.busy":"2024-05-26T09:57:18.105153Z","iopub.status.idle":"2024-05-26T10:10:29.677421Z","shell.execute_reply":"2024-05-26T10:10:29.676422Z"},"papermill":{"duration":798.52702,"end_time":"2024-05-26T10:10:33.441655","exception":false,"start_time":"2024-05-26T09:57:14.914635","status":"completed"},"tags":[],"id":"0b14e8f0","outputId":"14c05313-cc23-426c-8165-28bbba903f26"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:10:40.799924Z","iopub.status.busy":"2024-05-26T10:10:40.799526Z","iopub.status.idle":"2024-05-26T10:10:40.988549Z","shell.execute_reply":"2024-05-26T10:10:40.987579Z"},"papermill":{"duration":3.901897,"end_time":"2024-05-26T10:10:40.990594","exception":false,"start_time":"2024-05-26T10:10:37.088697","status":"completed"},"tags":[],"id":"7cb5faae","outputId":"36c86fec-6341-4747-cf2e-6e8c81f5b9d7"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:10:48.417633Z","iopub.status.busy":"2024-05-26T10:10:48.416731Z","iopub.status.idle":"2024-05-26T10:10:48.672627Z","shell.execute_reply":"2024-05-26T10:10:48.671671Z"},"papermill":{"duration":3.999835,"end_time":"2024-05-26T10:10:48.674783","exception":false,"start_time":"2024-05-26T10:10:44.674948","status":"completed"},"tags":[],"id":"3550a58f","outputId":"27c8b091-7fc4-42bc-ef48-cf50800c90b2"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+ADAM+SCHED', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_ADAM_Sched.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:10:56.141828Z","iopub.status.busy":"2024-05-26T10:10:56.140859Z","iopub.status.idle":"2024-05-26T10:10:56.184035Z","shell.execute_reply":"2024-05-26T10:10:56.18329Z"},"papermill":{"duration":3.751171,"end_time":"2024-05-26T10:10:56.186083","exception":false,"start_time":"2024-05-26T10:10:52.434912","status":"completed"},"tags":[],"id":"50e40a68"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:11:03.497567Z","iopub.status.busy":"2024-05-26T10:11:03.49679Z","iopub.status.idle":"2024-05-26T10:11:03.507694Z","shell.execute_reply":"2024-05-26T10:11:03.506766Z"},"papermill":{"duration":3.686093,"end_time":"2024-05-26T10:11:03.509603","exception":false,"start_time":"2024-05-26T10:10:59.82351","status":"completed"},"tags":[],"id":"92e0bab9","outputId":"59bc2ce6-ef72-4d8f-ebb8-07cc6adb39de"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+scheduler+RMS","metadata":{"papermill":{"duration":3.692582,"end_time":"2024-05-26T10:11:10.946771","exception":false,"start_time":"2024-05-26T10:11:07.254189","status":"completed"},"tags":[],"id":"d2d246ea"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:11:18.327326Z","iopub.status.busy":"2024-05-26T10:11:18.326926Z","iopub.status.idle":"2024-05-26T10:11:18.331564Z","shell.execute_reply":"2024-05-26T10:11:18.330593Z"},"papermill":{"duration":3.711727,"end_time":"2024-05-26T10:11:18.333618","exception":false,"start_time":"2024-05-26T10:11:14.621891","status":"completed"},"tags":[],"id":"bf015a8b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:11:25.841232Z","iopub.status.busy":"2024-05-26T10:11:25.840294Z","iopub.status.idle":"2024-05-26T10:11:25.85679Z","shell.execute_reply":"2024-05-26T10:11:25.85579Z"},"papermill":{"duration":3.718378,"end_time":"2024-05-26T10:11:25.858837","exception":false,"start_time":"2024-05-26T10:11:22.140459","status":"completed"},"tags":[],"id":"eda0bcf9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\n\nscheduler = keras.optimizers.schedules.ExponentialDecay(\n    initial_learning_rate=lr,\n    decay_steps=122,\n    decay_rate=0.82\n)\nmodel.compile(\n    optimizer=keras.optimizers.RMSprop(scheduler),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:11:33.239735Z","iopub.status.busy":"2024-05-26T10:11:33.238953Z","iopub.status.idle":"2024-05-26T10:11:33.248144Z","shell.execute_reply":"2024-05-26T10:11:33.24738Z"},"papermill":{"duration":3.705332,"end_time":"2024-05-26T10:11:33.250349","exception":false,"start_time":"2024-05-26T10:11:29.545017","status":"completed"},"tags":[],"id":"c69ff685"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=base_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:11:40.649939Z","iopub.status.busy":"2024-05-26T10:11:40.649549Z","iopub.status.idle":"2024-05-26T10:25:01.0636Z","shell.execute_reply":"2024-05-26T10:25:01.062595Z"},"papermill":{"duration":808.272055,"end_time":"2024-05-26T10:25:05.208762","exception":false,"start_time":"2024-05-26T10:11:36.936707","status":"completed"},"tags":[],"id":"29c3bb40","outputId":"0fe7fc56-edd1-4a47-df34-37be130eca1d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:25:13.730353Z","iopub.status.busy":"2024-05-26T10:25:13.72941Z","iopub.status.idle":"2024-05-26T10:25:13.987092Z","shell.execute_reply":"2024-05-26T10:25:13.98615Z"},"papermill":{"duration":4.419242,"end_time":"2024-05-26T10:25:13.989299","exception":false,"start_time":"2024-05-26T10:25:09.570057","status":"completed"},"tags":[],"id":"ad54f792","outputId":"8d39b75f-0df8-4122-f8fa-caed9df86ada"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:25:22.402626Z","iopub.status.busy":"2024-05-26T10:25:22.401924Z","iopub.status.idle":"2024-05-26T10:25:22.661381Z","shell.execute_reply":"2024-05-26T10:25:22.66043Z"},"papermill":{"duration":4.454482,"end_time":"2024-05-26T10:25:22.663528","exception":false,"start_time":"2024-05-26T10:25:18.209046","status":"completed"},"tags":[],"id":"85bcfd8a","outputId":"b3881cb5-c915-4661-b046-d88862605cb8"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+RMS+SCHED', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_RMS_Sched.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:25:31.075261Z","iopub.status.busy":"2024-05-26T10:25:31.07425Z","iopub.status.idle":"2024-05-26T10:25:31.112772Z","shell.execute_reply":"2024-05-26T10:25:31.111712Z"},"papermill":{"duration":4.316222,"end_time":"2024-05-26T10:25:31.115029","exception":false,"start_time":"2024-05-26T10:25:26.798807","status":"completed"},"tags":[],"id":"5c04c0df"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:25:39.701197Z","iopub.status.busy":"2024-05-26T10:25:39.700251Z","iopub.status.idle":"2024-05-26T10:25:39.712876Z","shell.execute_reply":"2024-05-26T10:25:39.71188Z"},"papermill":{"duration":4.2357,"end_time":"2024-05-26T10:25:39.715075","exception":false,"start_time":"2024-05-26T10:25:35.479375","status":"completed"},"tags":[],"id":"2cefecc5","outputId":"4bf18b43-7780-465b-d487-200e5c5f1cd7"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+scheduler+Aug+SGD","metadata":{"papermill":{"duration":4.213626,"end_time":"2024-05-26T10:25:48.14208","exception":false,"start_time":"2024-05-26T10:25:43.928454","status":"completed"},"tags":[],"id":"5192afe9"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:25:56.381436Z","iopub.status.busy":"2024-05-26T10:25:56.38054Z","iopub.status.idle":"2024-05-26T10:25:56.384966Z","shell.execute_reply":"2024-05-26T10:25:56.384091Z"},"papermill":{"duration":4.114943,"end_time":"2024-05-26T10:25:56.386994","exception":false,"start_time":"2024-05-26T10:25:52.272051","status":"completed"},"tags":[],"id":"097362b8"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:26:04.796762Z","iopub.status.busy":"2024-05-26T10:26:04.79598Z","iopub.status.idle":"2024-05-26T10:26:04.812276Z","shell.execute_reply":"2024-05-26T10:26:04.81139Z"},"papermill":{"duration":4.254436,"end_time":"2024-05-26T10:26:04.814179","exception":false,"start_time":"2024-05-26T10:26:00.559743","status":"completed"},"tags":[],"id":"6cfdb04c"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\n\nscheduler = keras.optimizers.schedules.ExponentialDecay(\n    initial_learning_rate=lr,\n    decay_steps=122,\n    decay_rate=0.82\n)\nmodel.compile(\n    optimizer=keras.optimizers.SGD(scheduler),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:26:13.176229Z","iopub.status.busy":"2024-05-26T10:26:13.175227Z","iopub.status.idle":"2024-05-26T10:26:13.185052Z","shell.execute_reply":"2024-05-26T10:26:13.184109Z"},"papermill":{"duration":4.229823,"end_time":"2024-05-26T10:26:13.187172","exception":false,"start_time":"2024-05-26T10:26:08.957349","status":"completed"},"tags":[],"id":"16d2295f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=augmentation_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:26:21.562936Z","iopub.status.busy":"2024-05-26T10:26:21.562007Z","iopub.status.idle":"2024-05-26T10:41:28.984023Z","shell.execute_reply":"2024-05-26T10:41:28.983008Z"},"papermill":{"duration":911.554798,"end_time":"2024-05-26T10:41:28.986157","exception":false,"start_time":"2024-05-26T10:26:17.431359","status":"completed"},"tags":[],"id":"17d390c9","outputId":"949ef1f2-b29a-45f8-99ec-b6759d94a73b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:41:38.214097Z","iopub.status.busy":"2024-05-26T10:41:38.213165Z","iopub.status.idle":"2024-05-26T10:41:38.46331Z","shell.execute_reply":"2024-05-26T10:41:38.462337Z"},"papermill":{"duration":4.898457,"end_time":"2024-05-26T10:41:38.465499","exception":false,"start_time":"2024-05-26T10:41:33.567042","status":"completed"},"tags":[],"id":"9a1f419f","outputId":"4caab3a3-9839-45e2-8d7e-279df484ef50"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:41:47.685705Z","iopub.status.busy":"2024-05-26T10:41:47.685328Z","iopub.status.idle":"2024-05-26T10:41:47.921201Z","shell.execute_reply":"2024-05-26T10:41:47.920273Z"},"papermill":{"duration":4.887008,"end_time":"2024-05-26T10:41:47.923329","exception":false,"start_time":"2024-05-26T10:41:43.036321","status":"completed"},"tags":[],"id":"a894d4a6","outputId":"9db8767d-3199-4baa-dd69-374da0ed85fe"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+SGD+SCHED+AUG', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_SGD_Sched_Aug.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:41:57.084024Z","iopub.status.busy":"2024-05-26T10:41:57.082808Z","iopub.status.idle":"2024-05-26T10:41:57.117613Z","shell.execute_reply":"2024-05-26T10:41:57.116536Z"},"papermill":{"duration":4.649323,"end_time":"2024-05-26T10:41:57.119913","exception":false,"start_time":"2024-05-26T10:41:52.47059","status":"completed"},"tags":[],"id":"53410479"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:42:06.357438Z","iopub.status.busy":"2024-05-26T10:42:06.356697Z","iopub.status.idle":"2024-05-26T10:42:06.367927Z","shell.execute_reply":"2024-05-26T10:42:06.367113Z"},"papermill":{"duration":4.549785,"end_time":"2024-05-26T10:42:06.36985","exception":false,"start_time":"2024-05-26T10:42:01.820065","status":"completed"},"tags":[],"id":"69df3f00","outputId":"49002c6d-d1fd-4823-c16c-756f775fe488"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+scheduler+AUG+ADAM","metadata":{"papermill":{"duration":4.665075,"end_time":"2024-05-26T10:42:15.580555","exception":false,"start_time":"2024-05-26T10:42:10.91548","status":"completed"},"tags":[],"id":"e6f4fca7"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:42:24.736041Z","iopub.status.busy":"2024-05-26T10:42:24.735648Z","iopub.status.idle":"2024-05-26T10:42:24.740402Z","shell.execute_reply":"2024-05-26T10:42:24.739354Z"},"papermill":{"duration":4.619627,"end_time":"2024-05-26T10:42:24.742342","exception":false,"start_time":"2024-05-26T10:42:20.122715","status":"completed"},"tags":[],"id":"f4982062"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:42:34.059693Z","iopub.status.busy":"2024-05-26T10:42:34.058886Z","iopub.status.idle":"2024-05-26T10:42:34.075201Z","shell.execute_reply":"2024-05-26T10:42:34.074331Z"},"papermill":{"duration":4.593567,"end_time":"2024-05-26T10:42:34.07718","exception":false,"start_time":"2024-05-26T10:42:29.483613","status":"completed"},"tags":[],"id":"675ed25a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\n\nscheduler = keras.optimizers.schedules.ExponentialDecay(\n    initial_learning_rate=lr,\n    decay_steps=122,\n    decay_rate=0.82\n)\nmodel.compile(\n    optimizer=keras.optimizers.Adam(scheduler),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:42:43.322608Z","iopub.status.busy":"2024-05-26T10:42:43.322233Z","iopub.status.idle":"2024-05-26T10:42:43.331578Z","shell.execute_reply":"2024-05-26T10:42:43.330871Z"},"papermill":{"duration":4.705979,"end_time":"2024-05-26T10:42:43.333483","exception":false,"start_time":"2024-05-26T10:42:38.627504","status":"completed"},"tags":[],"id":"2129b33a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=augmentation_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:42:52.403727Z","iopub.status.busy":"2024-05-26T10:42:52.403343Z","iopub.status.idle":"2024-05-26T10:56:09.427326Z","shell.execute_reply":"2024-05-26T10:56:09.426257Z"},"papermill":{"duration":806.841884,"end_time":"2024-05-26T10:56:14.700236","exception":false,"start_time":"2024-05-26T10:42:47.858352","status":"completed"},"tags":[],"id":"5573bcc2","outputId":"28fed342-10f7-4cd8-a002-058ff1f42d8a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:56:24.827864Z","iopub.status.busy":"2024-05-26T10:56:24.82696Z","iopub.status.idle":"2024-05-26T10:56:25.06701Z","shell.execute_reply":"2024-05-26T10:56:25.066113Z"},"papermill":{"duration":5.255439,"end_time":"2024-05-26T10:56:25.069039","exception":false,"start_time":"2024-05-26T10:56:19.8136","status":"completed"},"tags":[],"id":"3eda8a49","outputId":"18bb7d6e-3ef4-4005-b1fe-4265b43c2c82"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:56:35.205402Z","iopub.status.busy":"2024-05-26T10:56:35.204523Z","iopub.status.idle":"2024-05-26T10:56:35.4277Z","shell.execute_reply":"2024-05-26T10:56:35.426662Z"},"papermill":{"duration":5.368825,"end_time":"2024-05-26T10:56:35.429838","exception":false,"start_time":"2024-05-26T10:56:30.061013","status":"completed"},"tags":[],"id":"c54a11fb","outputId":"7df78da7-f385-4ef1-c45a-8b3eda78d51b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+ADAM+SCHED+AUG', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_ADAM_Sched_Aug.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:56:45.570459Z","iopub.status.busy":"2024-05-26T10:56:45.569807Z","iopub.status.idle":"2024-05-26T10:56:45.614159Z","shell.execute_reply":"2024-05-26T10:56:45.613382Z"},"papermill":{"duration":5.175382,"end_time":"2024-05-26T10:56:45.616471","exception":false,"start_time":"2024-05-26T10:56:40.441089","status":"completed"},"tags":[],"id":"dbaaf4f0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:56:55.727299Z","iopub.status.busy":"2024-05-26T10:56:55.726529Z","iopub.status.idle":"2024-05-26T10:56:55.737823Z","shell.execute_reply":"2024-05-26T10:56:55.736923Z"},"papermill":{"duration":5.030315,"end_time":"2024-05-26T10:56:55.739781","exception":false,"start_time":"2024-05-26T10:56:50.709466","status":"completed"},"tags":[],"id":"59902e99","outputId":"c5685df9-60e6-46bb-b75d-4a8270a57849"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model+scheduler+AUG+RMS","metadata":{"papermill":{"duration":5.134033,"end_time":"2024-05-26T10:57:05.844312","exception":false,"start_time":"2024-05-26T10:57:00.710279","status":"completed"},"tags":[],"id":"d0b1bc3a"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:57:16.046901Z","iopub.status.busy":"2024-05-26T10:57:16.045951Z","iopub.status.idle":"2024-05-26T10:57:16.050492Z","shell.execute_reply":"2024-05-26T10:57:16.04957Z"},"papermill":{"duration":5.143099,"end_time":"2024-05-26T10:57:16.052553","exception":false,"start_time":"2024-05-26T10:57:10.909454","status":"completed"},"tags":[],"id":"72598041"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = keras.Sequential([\n    keras.layers.Conv2D(filters=8, kernel_size=10, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=9, strides=4, padding='same'),\n\n    keras.layers.Conv2D(filters=4, kernel_size=6, strides=3, padding='same', activation='mish'),\n    keras.layers.BatchNormalization(),\n    keras.layers.MaxPooling2D(pool_size=3, strides=2, padding='same'),\n\n    keras.layers.Flatten(),\n\n    keras.layers.Dense(128, activation=\"relu\"),\n    keras.layers.Dense(5, activation=\"softmax\")\n])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:57:26.135329Z","iopub.status.busy":"2024-05-26T10:57:26.134543Z","iopub.status.idle":"2024-05-26T10:57:26.150957Z","shell.execute_reply":"2024-05-26T10:57:26.150057Z"},"papermill":{"duration":5.024009,"end_time":"2024-05-26T10:57:26.152997","exception":false,"start_time":"2024-05-26T10:57:21.128988","status":"completed"},"tags":[],"id":"b8775c87"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr = 0.001\n\nscheduler = keras.optimizers.schedules.ExponentialDecay(\n    initial_learning_rate=lr,\n    decay_steps=122,\n    decay_rate=0.82\n)\nmodel.compile(\n    optimizer=keras.optimizers.RMSprop(scheduler),\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:57:36.269567Z","iopub.status.busy":"2024-05-26T10:57:36.268705Z","iopub.status.idle":"2024-05-26T10:57:36.277835Z","shell.execute_reply":"2024-05-26T10:57:36.277111Z"},"papermill":{"duration":5.025819,"end_time":"2024-05-26T10:57:36.279727","exception":false,"start_time":"2024-05-26T10:57:31.253908","status":"completed"},"tags":[],"id":"0a5e68c1"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    x=augmentation_train_ds,\n    epochs=epoch_number,\n    validation_data=test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T10:57:46.516869Z","iopub.status.busy":"2024-05-26T10:57:46.515791Z","iopub.status.idle":"2024-05-26T11:11:13.77075Z","shell.execute_reply":"2024-05-26T11:11:13.769747Z"},"papermill":{"duration":812.400514,"end_time":"2024-05-26T11:11:13.772787","exception":false,"start_time":"2024-05-26T10:57:41.372273","status":"completed"},"tags":[],"id":"e6d71088","outputId":"cf126d11-2e70-4cf3-c1cb-c1d95f19a9b8"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:11:24.783101Z","iopub.status.busy":"2024-05-26T11:11:24.782706Z","iopub.status.idle":"2024-05-26T11:11:25.007749Z","shell.execute_reply":"2024-05-26T11:11:25.006834Z"},"papermill":{"duration":5.687105,"end_time":"2024-05-26T11:11:25.009872","exception":false,"start_time":"2024-05-26T11:11:19.322767","status":"completed"},"tags":[],"id":"0c991641","outputId":"1d1214dc-a8ec-4b8a-bd9e-bf2e5db020b3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:11:36.240522Z","iopub.status.busy":"2024-05-26T11:11:36.240158Z","iopub.status.idle":"2024-05-26T11:11:36.493535Z","shell.execute_reply":"2024-05-26T11:11:36.492617Z"},"papermill":{"duration":5.819964,"end_time":"2024-05-26T11:11:36.49577","exception":false,"start_time":"2024-05-26T11:11:30.675806","status":"completed"},"tags":[],"id":"adb751bb","outputId":"8df4ebb7-928f-44e9-c7aa-1a4d30405f85"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['model+RMS+SCHED+AUG', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('model_RMS_Sched_Aug.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:11:47.465601Z","iopub.status.busy":"2024-05-26T11:11:47.465183Z","iopub.status.idle":"2024-05-26T11:11:47.502824Z","shell.execute_reply":"2024-05-26T11:11:47.501866Z"},"papermill":{"duration":5.439411,"end_time":"2024-05-26T11:11:47.505157","exception":false,"start_time":"2024-05-26T11:11:42.065746","status":"completed"},"tags":[],"id":"72a755d8"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:11:58.544398Z","iopub.status.busy":"2024-05-26T11:11:58.543953Z","iopub.status.idle":"2024-05-26T11:11:58.55564Z","shell.execute_reply":"2024-05-26T11:11:58.554764Z"},"papermill":{"duration":5.620449,"end_time":"2024-05-26T11:11:58.557423","exception":false,"start_time":"2024-05-26T11:11:52.936974","status":"completed"},"tags":[],"id":"96e74587","outputId":"2cc4b615-5094-426e-c131-f0e078efc63d"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## VGG+SGD","metadata":{"papermill":{"duration":5.591111,"end_time":"2024-05-26T11:12:09.594439","exception":false,"start_time":"2024-05-26T11:12:04.003328","status":"completed"},"tags":[],"id":"1a919078"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:12:20.607787Z","iopub.status.busy":"2024-05-26T11:12:20.607399Z","iopub.status.idle":"2024-05-26T11:12:20.611806Z","shell.execute_reply":"2024-05-26T11:12:20.610889Z"},"papermill":{"duration":5.57615,"end_time":"2024-05-26T11:12:20.613656","exception":false,"start_time":"2024-05-26T11:12:15.037506","status":"completed"},"tags":[],"id":"33d57190"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"epoch_number=5","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:12:31.590259Z","iopub.status.busy":"2024-05-26T11:12:31.589845Z","iopub.status.idle":"2024-05-26T11:12:31.594242Z","shell.execute_reply":"2024-05-26T11:12:31.593366Z"},"papermill":{"duration":5.507049,"end_time":"2024-05-26T11:12:31.596445","exception":false,"start_time":"2024-05-26T11:12:26.089396","status":"completed"},"tags":[],"id":"77a3a2e2"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vgg = keras.applications.VGG19(weights='imagenet', include_top=True)\ninp = vgg.input\nnew_classification_layer = keras.layers.Dense(5, activation='softmax')\nout = new_classification_layer(vgg.layers[-2].output)\nmodel_new_vgg = keras.Model(inp, out)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:12:42.652196Z","iopub.status.busy":"2024-05-26T11:12:42.651287Z","iopub.status.idle":"2024-05-26T11:12:47.198739Z","shell.execute_reply":"2024-05-26T11:12:47.197901Z"},"papermill":{"duration":10.033983,"end_time":"2024-05-26T11:12:47.201143","exception":false,"start_time":"2024-05-26T11:12:37.16716","status":"completed"},"tags":[],"id":"e7947bf6","outputId":"8fa3418f-fade-4923-beb3-0987b3267abe"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transformation = keras.Sequential([\n    keras.layers.Resizing(224, 224),\n    keras.layers.Rescaling(1. / 255)\n])\nvgg_train_ds = train_ds.map(lambda img, label: (transformation(img), label))\nvgg_test_ds = test_ds.map(lambda img, label: (transformation(img), label))","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:12:58.336138Z","iopub.status.busy":"2024-05-26T11:12:58.335712Z","iopub.status.idle":"2024-05-26T11:12:58.375671Z","shell.execute_reply":"2024-05-26T11:12:58.3749Z"},"papermill":{"duration":5.598237,"end_time":"2024-05-26T11:12:58.377723","exception":false,"start_time":"2024-05-26T11:12:52.779486","status":"completed"},"tags":[],"id":"fd9fc2b2"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr=0.001\n\nmodel_new_vgg.compile(\n                      optimizer=keras.optimizers.SGD(lr),\n                      loss=\"sparse_categorical_crossentropy\",\n                      metrics=['accuracy'],\n                     )\nfor layer in model_new_vgg.layers:\n    layer.trainable = False\nmodel_new_vgg.layers[-1].trainable = True","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:13:09.56889Z","iopub.status.busy":"2024-05-26T11:13:09.568499Z","iopub.status.idle":"2024-05-26T11:13:09.580473Z","shell.execute_reply":"2024-05-26T11:13:09.579664Z"},"papermill":{"duration":5.459879,"end_time":"2024-05-26T11:13:09.582431","exception":false,"start_time":"2024-05-26T11:13:04.122552","status":"completed"},"tags":[],"id":"cbacd28f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_new_vgg.fit(\n    x=vgg_train_ds,\n    epochs=epoch_number,\n    validation_data=vgg_test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:13:20.64276Z","iopub.status.busy":"2024-05-26T11:13:20.641993Z","iopub.status.idle":"2024-05-26T11:21:33.415811Z","shell.execute_reply":"2024-05-26T11:21:33.414885Z"},"papermill":{"duration":504.13338,"end_time":"2024-05-26T11:21:39.286237","exception":false,"start_time":"2024-05-26T11:13:15.152857","status":"completed"},"tags":[],"id":"401b90bf","outputId":"647b8302-fdec-4df9-c7c4-3dd67c281393"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:21:50.781882Z","iopub.status.busy":"2024-05-26T11:21:50.781475Z","iopub.status.idle":"2024-05-26T11:21:51.038013Z","shell.execute_reply":"2024-05-26T11:21:51.037098Z"},"papermill":{"duration":6.06209,"end_time":"2024-05-26T11:21:51.040138","exception":false,"start_time":"2024-05-26T11:21:44.978048","status":"completed"},"tags":[],"id":"523caf47","outputId":"265cbeab-9206-4de7-ee2d-e2882bf2f94b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:22:02.512647Z","iopub.status.busy":"2024-05-26T11:22:02.511762Z","iopub.status.idle":"2024-05-26T11:22:02.689783Z","shell.execute_reply":"2024-05-26T11:22:02.688844Z"},"papermill":{"duration":5.984864,"end_time":"2024-05-26T11:22:02.691911","exception":false,"start_time":"2024-05-26T11:21:56.707047","status":"completed"},"tags":[],"id":"af8062de","outputId":"f5a25073-5988-4f57-fe8f-670f1b0b3dbe"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['VGG+SGD', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('VGG_SGD.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:22:14.313759Z","iopub.status.busy":"2024-05-26T11:22:14.313365Z","iopub.status.idle":"2024-05-26T11:22:14.350347Z","shell.execute_reply":"2024-05-26T11:22:14.349547Z"},"papermill":{"duration":6.004528,"end_time":"2024-05-26T11:22:14.352316","exception":false,"start_time":"2024-05-26T11:22:08.347788","status":"completed"},"tags":[],"id":"a1894aa0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:22:25.873646Z","iopub.status.busy":"2024-05-26T11:22:25.873262Z","iopub.status.idle":"2024-05-26T11:22:25.885482Z","shell.execute_reply":"2024-05-26T11:22:25.884587Z"},"papermill":{"duration":5.912426,"end_time":"2024-05-26T11:22:25.887318","exception":false,"start_time":"2024-05-26T11:22:19.974892","status":"completed"},"tags":[],"id":"3a42a802","outputId":"86f5cc3e-52e9-4869-aac6-a8cd29a46122"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## VGG+ADAM","metadata":{"papermill":{"duration":5.83513,"end_time":"2024-05-26T11:22:37.379837","exception":false,"start_time":"2024-05-26T11:22:31.544707","status":"completed"},"tags":[],"id":"1d1d1652"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:22:48.97268Z","iopub.status.busy":"2024-05-26T11:22:48.971878Z","iopub.status.idle":"2024-05-26T11:22:48.976268Z","shell.execute_reply":"2024-05-26T11:22:48.975352Z"},"papermill":{"duration":5.764487,"end_time":"2024-05-26T11:22:48.978127","exception":false,"start_time":"2024-05-26T11:22:43.21364","status":"completed"},"tags":[],"id":"e3269590"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vgg = keras.applications.VGG19(weights='imagenet', include_top=True)\ninp = vgg.input\nnew_classification_layer = keras.layers.Dense(5, activation='softmax')\nout = new_classification_layer(vgg.layers[-2].output)\nmodel_new_vgg = keras.Model(inp, out)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:23:00.561246Z","iopub.status.busy":"2024-05-26T11:23:00.560481Z","iopub.status.idle":"2024-05-26T11:23:02.802094Z","shell.execute_reply":"2024-05-26T11:23:02.80111Z"},"papermill":{"duration":8.024106,"end_time":"2024-05-26T11:23:02.804455","exception":false,"start_time":"2024-05-26T11:22:54.780349","status":"completed"},"tags":[],"id":"3b1790b4"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transformation = keras.Sequential([\n    keras.layers.Resizing(224, 224),\n    keras.layers.Rescaling(1. / 255)\n])\nvgg_train_ds = train_ds.map(lambda img, label: (transformation(img), label))\nvgg_test_ds = test_ds.map(lambda img, label: (transformation(img), label))","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:23:14.431886Z","iopub.status.busy":"2024-05-26T11:23:14.430897Z","iopub.status.idle":"2024-05-26T11:23:14.471703Z","shell.execute_reply":"2024-05-26T11:23:14.470947Z"},"papermill":{"duration":6.02542,"end_time":"2024-05-26T11:23:14.473723","exception":false,"start_time":"2024-05-26T11:23:08.448303","status":"completed"},"tags":[],"id":"fda801e4"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr=0.001\n\n\nmodel_new_vgg.compile(\n                      optimizer=keras.optimizers.Adam(lr),\n                      loss=\"sparse_categorical_crossentropy\",\n                      metrics=['accuracy'],\n                     )\nfor layer in model_new_vgg.layers:\n    layer.trainable = False\nmodel_new_vgg.layers[-1].trainable = True","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:23:25.77757Z","iopub.status.busy":"2024-05-26T11:23:25.776546Z","iopub.status.idle":"2024-05-26T11:23:25.787162Z","shell.execute_reply":"2024-05-26T11:23:25.786424Z"},"papermill":{"duration":5.668044,"end_time":"2024-05-26T11:23:25.788991","exception":false,"start_time":"2024-05-26T11:23:20.120947","status":"completed"},"tags":[],"id":"38162e6f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_new_vgg.fit(\n    x=vgg_train_ds,\n    epochs=epoch_number,\n    validation_data=vgg_test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:23:37.327564Z","iopub.status.busy":"2024-05-26T11:23:37.327169Z","iopub.status.idle":"2024-05-26T11:32:47.11458Z","shell.execute_reply":"2024-05-26T11:32:47.113528Z"},"papermill":{"duration":555.604421,"end_time":"2024-05-26T11:32:47.116722","exception":false,"start_time":"2024-05-26T11:23:31.512301","status":"completed"},"tags":[],"id":"7ff9696a","outputId":"5bc0152f-98f0-47b4-ecc3-25ccc63b9332"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:32:59.043022Z","iopub.status.busy":"2024-05-26T11:32:59.04263Z","iopub.status.idle":"2024-05-26T11:32:59.30348Z","shell.execute_reply":"2024-05-26T11:32:59.302477Z"},"papermill":{"duration":6.235369,"end_time":"2024-05-26T11:32:59.305626","exception":false,"start_time":"2024-05-26T11:32:53.070257","status":"completed"},"tags":[],"id":"3f82538b","outputId":"d080a5a6-44bc-43f2-fcbf-f11172272f54"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:33:11.229085Z","iopub.status.busy":"2024-05-26T11:33:11.228272Z","iopub.status.idle":"2024-05-26T11:33:11.42679Z","shell.execute_reply":"2024-05-26T11:33:11.425858Z"},"papermill":{"duration":6.070069,"end_time":"2024-05-26T11:33:11.428701","exception":false,"start_time":"2024-05-26T11:33:05.358632","status":"completed"},"tags":[],"id":"70f68fb0","outputId":"917d84bb-c04c-4936-d23f-e5ebb888ea83"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['VGG+ADAM', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('VGG_ADAM.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:33:23.624213Z","iopub.status.busy":"2024-05-26T11:33:23.62325Z","iopub.status.idle":"2024-05-26T11:33:23.660649Z","shell.execute_reply":"2024-05-26T11:33:23.659608Z"},"papermill":{"duration":6.289264,"end_time":"2024-05-26T11:33:23.662947","exception":false,"start_time":"2024-05-26T11:33:17.373683","status":"completed"},"tags":[],"id":"3b99c6d1"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:33:35.608207Z","iopub.status.busy":"2024-05-26T11:33:35.607435Z","iopub.status.idle":"2024-05-26T11:33:35.620205Z","shell.execute_reply":"2024-05-26T11:33:35.619213Z"},"papermill":{"duration":5.946595,"end_time":"2024-05-26T11:33:35.622995","exception":false,"start_time":"2024-05-26T11:33:29.6764","status":"completed"},"tags":[],"id":"7487742b","outputId":"e73c9ef2-bee2-48bd-9e18-824795fb0fb9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## VGG+RMS","metadata":{"papermill":{"duration":6.066773,"end_time":"2024-05-26T11:33:47.587027","exception":false,"start_time":"2024-05-26T11:33:41.520254","status":"completed"},"tags":[],"id":"cb95b67c"}},{"cell_type":"code","source":"history = History()","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:33:59.598112Z","iopub.status.busy":"2024-05-26T11:33:59.597284Z","iopub.status.idle":"2024-05-26T11:33:59.601846Z","shell.execute_reply":"2024-05-26T11:33:59.600904Z"},"papermill":{"duration":5.865451,"end_time":"2024-05-26T11:33:59.603744","exception":false,"start_time":"2024-05-26T11:33:53.738293","status":"completed"},"tags":[],"id":"09f54af0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vgg = keras.applications.VGG19(weights='imagenet', include_top=True)\ninp = vgg.input\nnew_classification_layer = keras.layers.Dense(5, activation='softmax')\nout = new_classification_layer(vgg.layers[-2].output)\nmodel_new_vgg = keras.Model(inp, out)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:34:11.499023Z","iopub.status.busy":"2024-05-26T11:34:11.4982Z","iopub.status.idle":"2024-05-26T11:34:13.800593Z","shell.execute_reply":"2024-05-26T11:34:13.79974Z"},"papermill":{"duration":8.166376,"end_time":"2024-05-26T11:34:13.802989","exception":false,"start_time":"2024-05-26T11:34:05.636613","status":"completed"},"tags":[],"id":"0c85e4ba"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr=0.001\n\n\nmodel_new_vgg.compile(\n                      optimizer=keras.optimizers.RMSprop(lr),\n                      loss=\"sparse_categorical_crossentropy\",\n                      metrics=['accuracy'],\n                     )\nfor layer in model_new_vgg.layers:\n    layer.trainable = False\nmodel_new_vgg.layers[-1].trainable = True","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:34:25.911055Z","iopub.status.busy":"2024-05-26T11:34:25.910662Z","iopub.status.idle":"2024-05-26T11:34:25.92145Z","shell.execute_reply":"2024-05-26T11:34:25.920662Z"},"papermill":{"duration":6.050198,"end_time":"2024-05-26T11:34:25.923372","exception":false,"start_time":"2024-05-26T11:34:19.873174","status":"completed"},"tags":[],"id":"b2a529e3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_new_vgg.fit(\n    x=vgg_train_ds,\n    epochs=epoch_number,\n    validation_data=vgg_test_ds,\n    callbacks=[history]\n)","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:34:37.91497Z","iopub.status.busy":"2024-05-26T11:34:37.914024Z","iopub.status.idle":"2024-05-26T11:42:05.34598Z","shell.execute_reply":"2024-05-26T11:42:05.34499Z"},"papermill":{"duration":453.351107,"end_time":"2024-05-26T11:42:05.348105","exception":false,"start_time":"2024-05-26T11:34:31.996998","status":"completed"},"tags":[],"id":"042b16b8","outputId":"5ca04648-f6ac-4ec4-c701-a29a29538a48"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:42:17.823563Z","iopub.status.busy":"2024-05-26T11:42:17.822782Z","iopub.status.idle":"2024-05-26T11:42:18.033238Z","shell.execute_reply":"2024-05-26T11:42:18.032345Z"},"papermill":{"duration":6.553158,"end_time":"2024-05-26T11:42:18.035295","exception":false,"start_time":"2024-05-26T11:42:11.482137","status":"completed"},"tags":[],"id":"d7f2c063","outputId":"13ce226f-a5e9-4b13-a6c5-da7c5a1435c6"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['val_loss'])","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:42:30.546025Z","iopub.status.busy":"2024-05-26T11:42:30.545272Z","iopub.status.idle":"2024-05-26T11:42:30.814006Z","shell.execute_reply":"2024-05-26T11:42:30.813094Z"},"papermill":{"duration":6.461137,"end_time":"2024-05-26T11:42:30.816031","exception":false,"start_time":"2024-05-26T11:42:24.354894","status":"completed"},"tags":[],"id":"4f568b3f","outputId":"be1f0c44-9f7a-4d77-c8eb-50c44ad64f5a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table.loc[ len(df_table.index )] = ['VGG+RMR', max(history.history['accuracy']), max(history.history['val_accuracy'])]\nmodel.save('VGG_RMR.keras')","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:42:43.227309Z","iopub.status.busy":"2024-05-26T11:42:43.2264Z","iopub.status.idle":"2024-05-26T11:42:43.263482Z","shell.execute_reply":"2024-05-26T11:42:43.262628Z"},"papermill":{"duration":6.16918,"end_time":"2024-05-26T11:42:43.265696","exception":false,"start_time":"2024-05-26T11:42:37.096516","status":"completed"},"tags":[],"id":"bfc63c57"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_table","metadata":{"execution":{"iopub.execute_input":"2024-05-26T11:42:55.71855Z","iopub.status.busy":"2024-05-26T11:42:55.717789Z","iopub.status.idle":"2024-05-26T11:42:55.728918Z","shell.execute_reply":"2024-05-26T11:42:55.728097Z"},"papermill":{"duration":6.138752,"end_time":"2024-05-26T11:42:55.73132","exception":false,"start_time":"2024-05-26T11:42:49.592568","status":"completed"},"tags":[],"id":"6c1f182c","outputId":"32c77290-c452-4b95-c2eb-b74dfeb11266"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2 Semantic segmentation","metadata":{"id":"QsAUmiakyPig"}},{"cell_type":"markdown","source":"## Подготовка всякого","metadata":{"id":"vh7VjMTLyPig"}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nfrom sklearn.model_selection import train_test_split\n\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import transforms as T\nimport torchvision\nimport torch.nn.functional as F\nfrom torch.autograd import Variable\n\nfrom PIL import Image\nimport cv2\nimport albumentations as A\n\nimport time\nimport os\nfrom tqdm.notebook import tqdm\n\n!pip install -q segmentation-models-pytorch\n!pip install -q torchsummary\n\nfrom torchsummary import summary\nimport segmentation_models_pytorch as smp","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:11:21.724502Z","iopub.execute_input":"2024-05-26T06:11:21.724953Z","iopub.status.idle":"2024-05-26T06:11:58.379363Z","shell.execute_reply.started":"2024-05-26T06:11:21.724912Z","shell.execute_reply":"2024-05-26T06:11:58.378193Z"},"id":"leIzK__TyPig","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_PATH = '/kaggle/input/semantic-drone-dataset/dataset/semantic_drone_dataset/original_images/'\nMASK_PATH = '/kaggle/input/semantic-drone-dataset/dataset/semantic_drone_dataset/label_images_semantic/'","metadata":{"execution":{"iopub.status.busy":"2024-05-26T05:55:35.729682Z","iopub.execute_input":"2024-05-26T05:55:35.730437Z","iopub.status.idle":"2024-05-26T05:55:35.741147Z","shell.execute_reply.started":"2024-05-26T05:55:35.730407Z","shell.execute_reply":"2024-05-26T05:55:35.740204Z"},"id":"YN9jNv5wyPig","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ndevice","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:23:09.844315Z","iopub.execute_input":"2024-05-26T06:23:09.844754Z","iopub.status.idle":"2024-05-26T06:23:09.878902Z","shell.execute_reply.started":"2024-05-26T06:23:09.844718Z","shell.execute_reply":"2024-05-26T06:23:09.877949Z"},"id":"cnlV_JF4yPig","outputId":"1e9015bb-3e9e-472c-8bba-30d80e5b1ff7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/semantic-drone-dataset/class_dict_seg.csv')\ndf","metadata":{"execution":{"iopub.status.busy":"2024-05-26T05:57:55.09243Z","iopub.execute_input":"2024-05-26T05:57:55.092703Z","iopub.status.idle":"2024-05-26T05:57:55.153714Z","shell.execute_reply.started":"2024-05-26T05:57:55.09268Z","shell.execute_reply":"2024-05-26T05:57:55.152738Z"},"id":"Sowk7rbKyPih","outputId":"602e292a-41bc-4fab-8741-b2fbba4fef56","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"CLASSES = df.name.to_list()\nCLASSES","metadata":{"execution":{"iopub.status.busy":"2024-05-26T05:57:55.156355Z","iopub.execute_input":"2024-05-26T05:57:55.156754Z","iopub.status.idle":"2024-05-26T05:57:55.164182Z","shell.execute_reply.started":"2024-05-26T05:57:55.156713Z","shell.execute_reply":"2024-05-26T05:57:55.163124Z"},"id":"A_8dHPsiyPih","outputId":"ab901e26-9e58-406c-ad43-2224747c57f7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"n_classes = len(CLASSES)\n\ndef create_df():\n    name = []\n    for dirname, _, filenames in os.walk(IMAGE_PATH):\n        for filename in filenames:\n            name.append(filename.split('.')[0])\n\n    return pd.DataFrame({'id': name}, index = np.arange(0, len(name)))\n\ndf_images = create_df()\nprint('Total Images: ', len(df_images))","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:05:23.26772Z","iopub.execute_input":"2024-05-26T06:05:23.268183Z","iopub.status.idle":"2024-05-26T06:05:23.390828Z","shell.execute_reply.started":"2024-05-26T06:05:23.26815Z","shell.execute_reply":"2024-05-26T06:05:23.389918Z"},"id":"9vNJ4g2MyPih","outputId":"2c614a6e-3de1-42c3-84bd-1e5507c3c2e6","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_trainval, X_test = train_test_split(df_images['id'].values, test_size=0.1, random_state=19)\nX_train, X_val = train_test_split(X_trainval, test_size=0.15, random_state=19)\n\nprint('Train Size   : ', len(X_train))\nprint('Val Size     : ', len(X_val))\nprint('Test Size    : ', len(X_test))","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:23:38.279131Z","iopub.execute_input":"2024-05-26T06:23:38.279893Z","iopub.status.idle":"2024-05-26T06:23:38.288987Z","shell.execute_reply.started":"2024-05-26T06:23:38.27986Z","shell.execute_reply":"2024-05-26T06:23:38.288247Z"},"id":"MsRGlewFyPih","outputId":"b7f78042-0a53-4b34-f0e3-9dadda60770d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img = Image.open(IMAGE_PATH + df_images['id'][100] + '.jpg')\nmask = Image.open(MASK_PATH + df_images['id'][100] + '.png')\nprint('Image Size', np.asarray(img).shape)\nprint('Mask Size', np.asarray(mask).shape)\n\n\nplt.imshow(img)\nplt.imshow(mask, alpha=0.6)\nplt.title('Picture with Mask Appplied')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:23:46.046155Z","iopub.execute_input":"2024-05-26T06:23:46.046553Z","iopub.status.idle":"2024-05-26T06:23:51.298252Z","shell.execute_reply.started":"2024-05-26T06:23:46.04652Z","shell.execute_reply":"2024-05-26T06:23:51.297361Z"},"id":"oF6Me_N9yPii","outputId":"5ad97ae6-3e0f-4186-e7d7-915b2c3532ec","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DroneDataset(Dataset):\n\n    def __init__(self, img_path, mask_path, X, mean, std, transform=None, patch=False):\n        self.img_path = img_path\n        self.mask_path = mask_path\n        self.X = X\n        self.transform = transform\n        self.patches = patch\n        self.mean = mean\n        self.std = std\n\n    def __len__(self):\n        return len(self.X)\n\n    def __getitem__(self, idx):\n        img = cv2.imread(self.img_path + self.X[idx] + '.jpg')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        mask = cv2.imread(self.mask_path + self.X[idx] + '.png', cv2.IMREAD_GRAYSCALE)\n\n        if self.transform is not None:\n            aug = self.transform(image=img, mask=mask)\n            img = Image.fromarray(aug['image'])\n            mask = aug['mask']\n\n        if self.transform is None:\n            img = Image.fromarray(img)\n\n        t = T.Compose([T.ToTensor(), T.Normalize(self.mean, self.std)])\n        img = t(img)\n        mask = torch.from_numpy(mask).long()\n\n        if self.patches:\n            img, mask = self.tiles(img, mask)\n\n        return img, mask\n\n    def tiles(self, img, mask):\n\n        img_patches = img.unfold(1, 512, 512).unfold(2, 768, 768)\n        img_patches  = img_patches.contiguous().view(3,-1, 512, 768)\n        img_patches = img_patches.permute(1,0,2,3)\n\n        mask_patches = mask.unfold(0, 512, 512).unfold(1, 768, 768)\n        mask_patches = mask_patches.contiguous().view(-1, 512, 768)\n\n        return img_patches, mask_patches","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:24:02.769098Z","iopub.execute_input":"2024-05-26T06:24:02.76946Z","iopub.status.idle":"2024-05-26T06:24:02.782345Z","shell.execute_reply.started":"2024-05-26T06:24:02.76943Z","shell.execute_reply":"2024-05-26T06:24:02.781208Z"},"id":"CJ2LGy8dyPii","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mean=[0.485, 0.456, 0.406]\nstd=[0.229, 0.224, 0.225]\n\nt_train = A.Compose([A.Resize(704, 1056, interpolation=cv2.INTER_NEAREST), A.HorizontalFlip(), A.VerticalFlip(),\n                     A.GridDistortion(p=0.2), A.RandomBrightnessContrast((0,0.5),(0,0.5)),\n                     A.GaussNoise()])\n\nt_val = A.Compose([A.Resize(704, 1056, interpolation=cv2.INTER_NEAREST), A.HorizontalFlip(),\n                   A.GridDistortion(p=0.2)])\n\ntrain_set = DroneDataset(IMAGE_PATH, MASK_PATH, X_train, mean, std, t_train, patch=False)\nval_set = DroneDataset(IMAGE_PATH, MASK_PATH, X_val, mean, std, t_val, patch=False)\n\nbatch_size= 3\n\ntrain_loader = DataLoader(train_set, batch_size=batch_size, shuffle=True)\nval_loader = DataLoader(val_set, batch_size=batch_size, shuffle=True)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:24:12.475032Z","iopub.execute_input":"2024-05-26T06:24:12.475404Z","iopub.status.idle":"2024-05-26T06:24:12.491511Z","shell.execute_reply.started":"2024-05-26T06:24:12.475376Z","shell.execute_reply":"2024-05-26T06:24:12.49038Z"},"id":"E770U0LYyPii","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def pixel_accuracy(output, mask):\n    with torch.no_grad():\n        output = torch.argmax(F.softmax(output, dim=1), dim=1)\n        correct = torch.eq(output, mask).int()\n        accuracy = float(correct.sum()) / float(correct.numel())\n    return accuracy","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:24:25.25465Z","iopub.execute_input":"2024-05-26T06:24:25.255308Z","iopub.status.idle":"2024-05-26T06:24:25.261299Z","shell.execute_reply.started":"2024-05-26T06:24:25.255276Z","shell.execute_reply":"2024-05-26T06:24:25.260286Z"},"id":"9v1JvZwlyPij","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def mIoU(pred_mask, mask, smooth=1e-10, n_classes=23):\n    with torch.no_grad():\n        pred_mask = F.softmax(pred_mask, dim=1)\n        pred_mask = torch.argmax(pred_mask, dim=1)\n        pred_mask = pred_mask.contiguous().view(-1)\n        mask = mask.contiguous().view(-1)\n\n        iou_per_class = []\n        for clas in range(0, n_classes): #loop per pixel class\n            true_class = pred_mask == clas\n            true_label = mask == clas\n\n            if true_label.long().sum().item() == 0: #no exist label in this loop\n                iou_per_class.append(np.nan)\n            else:\n                intersect = torch.logical_and(true_class, true_label).sum().float().item()\n                union = torch.logical_or(true_class, true_label).sum().float().item()\n\n                iou = (intersect + smooth) / (union +smooth)\n                iou_per_class.append(iou)\n        return np.nanmean(iou_per_class)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:24:28.048391Z","iopub.execute_input":"2024-05-26T06:24:28.048759Z","iopub.status.idle":"2024-05-26T06:24:28.057253Z","shell.execute_reply.started":"2024-05-26T06:24:28.04873Z","shell.execute_reply":"2024-05-26T06:24:28.056229Z"},"id":"4Rf34JBWyPij","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_lr(optimizer):\n    for param_group in optimizer.param_groups:\n        return param_group['lr']\n\ndef fit(epochs, model, train_loader, val_loader, criterion, optimizer, scheduler, patch=False):\n    torch.cuda.empty_cache()\n    train_losses = []\n    test_losses = []\n    val_iou = []; val_acc = []\n    train_iou = []; train_acc = []\n    lrs = []\n    min_loss = np.inf\n    decrease = 1 ; not_improve=0\n\n    model.to(device)\n    fit_time = time.time()\n    for e in range(epochs):\n        since = time.time()\n        running_loss = 0\n        iou_score = 0\n        accuracy = 0\n        model.train()\n        for i, data in enumerate(tqdm(train_loader)):\n            image_tiles, mask_tiles = data\n            if patch:\n                bs, n_tiles, c, h, w = image_tiles.size()\n\n                image_tiles = image_tiles.view(-1,c, h, w)\n                mask_tiles = mask_tiles.view(-1, h, w)\n\n            image = image_tiles.to(device); mask = mask_tiles.to(device);\n            output = model(image)\n            loss = criterion(output, mask)\n            iou_score += mIoU(output, mask)\n            accuracy += pixel_accuracy(output, mask)\n            loss.backward()\n            optimizer.step()\n            optimizer.zero_grad()\n\n            lrs.append(get_lr(optimizer))\n            scheduler.step()\n\n            running_loss += loss.item()\n\n        else:\n            model.eval()\n            test_loss = 0\n            test_accuracy = 0\n            val_iou_score = 0\n            with torch.no_grad():\n                for i, data in enumerate(tqdm(val_loader)):\n                    image_tiles, mask_tiles = data\n\n                    if patch:\n                        bs, n_tiles, c, h, w = image_tiles.size()\n\n                        image_tiles = image_tiles.view(-1,c, h, w)\n                        mask_tiles = mask_tiles.view(-1, h, w)\n\n                    image = image_tiles.to(device); mask = mask_tiles.to(device);\n                    output = model(image)\n                    val_iou_score +=  mIoU(output, mask)\n                    test_accuracy += pixel_accuracy(output, mask)\n                    loss = criterion(output, mask)\n                    test_loss += loss.item()\n\n            train_losses.append(running_loss/len(train_loader))\n            test_losses.append(test_loss/len(val_loader))\n\n\n            if min_loss > (test_loss/len(val_loader)):\n                print('Loss Decreasing.. {:.3f} >> {:.3f} '.format(min_loss, (test_loss/len(val_loader))))\n                min_loss = (test_loss/len(val_loader))\n                decrease += 1\n                if decrease % 5 == 0:\n                    print('saving model...')\n                    torch.save(model, 'Unet-Mobilenet_v2_mIoU-{:.3f}.pt'.format(val_iou_score/len(val_loader)))\n\n\n            if (test_loss/len(val_loader)) > min_loss:\n                not_improve += 1\n                min_loss = (test_loss/len(val_loader))\n                print(f'Loss Not Decrease for {not_improve} time')\n                if not_improve == 7:\n                    print('Loss not decrease for 7 times, Stop Training')\n                    break\n\n            val_iou.append(val_iou_score/len(val_loader))\n            train_iou.append(iou_score/len(train_loader))\n            train_acc.append(accuracy/len(train_loader))\n            val_acc.append(test_accuracy/ len(val_loader))\n            print(\"Epoch:{}/{}..\".format(e+1, epochs),\n                  \"Train Loss: {:.3f}..\".format(running_loss/len(train_loader)),\n                  \"Val Loss: {:.3f}..\".format(test_loss/len(val_loader)),\n                  \"Train mIoU:{:.3f}..\".format(iou_score/len(train_loader)),\n                  \"Val mIoU: {:.3f}..\".format(val_iou_score/len(val_loader)),\n                  \"Train Acc:{:.3f}..\".format(accuracy/len(train_loader)),\n                  \"Val Acc:{:.3f}..\".format(test_accuracy/len(val_loader)),\n                  \"Time: {:.2f}m\".format((time.time()-since)/60))\n\n    history = {'train_loss' : train_losses, 'val_loss': test_losses,\n               'train_miou' :train_iou, 'val_miou':val_iou,\n               'train_acc' :train_acc, 'val_acc':val_acc,\n               'lrs': lrs}\n    print('Total time: {:.2f} m' .format((time.time()- fit_time)/60))\n    return history","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:32.927151Z","iopub.execute_input":"2024-05-26T06:28:32.92798Z","iopub.status.idle":"2024-05-26T06:28:32.949916Z","shell.execute_reply.started":"2024-05-26T06:28:32.927949Z","shell.execute_reply":"2024-05-26T06:28:32.948929Z"},"id":"blb6shpqyPij","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_loss(history):\n    plt.plot(history['val_loss'], label='val', marker='o')\n    plt.plot( history['train_loss'], label='train', marker='o')\n    plt.title('Loss per epoch'); plt.ylabel('loss');\n    plt.xlabel('epoch')\n    plt.legend(), plt.grid()\n    plt.show()\n\ndef plot_score(history):\n    plt.plot(history['train_miou'], label='train_mIoU', marker='*')\n    plt.plot(history['val_miou'], label='val_mIoU',  marker='*')\n    plt.title('Score per epoch'); plt.ylabel('mean IoU')\n    plt.xlabel('epoch')\n    plt.legend(), plt.grid()\n    plt.show()\n\ndef plot_acc(history):\n    plt.plot(history['train_acc'], label='train_accuracy', marker='*')\n    plt.plot(history['val_acc'], label='val_accuracy',  marker='*')\n    plt.title('Accuracy per epoch'); plt.ylabel('Accuracy')\n    plt.xlabel('epoch')\n    plt.legend(), plt.grid()\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:39.023357Z","iopub.execute_input":"2024-05-26T06:28:39.023703Z","iopub.status.idle":"2024-05-26T06:28:39.032827Z","shell.execute_reply.started":"2024-05-26T06:28:39.023677Z","shell.execute_reply":"2024-05-26T06:28:39.031741Z"},"id":"gdi5WZwlyPik","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DroneTestDataset(Dataset):\n\n    def __init__(self, img_path, mask_path, X, transform=None):\n        self.img_path = img_path\n        self.mask_path = mask_path\n        self.X = X\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.X)\n\n    def __getitem__(self, idx):\n        img = cv2.imread(self.img_path + self.X[idx] + '.jpg')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        mask = cv2.imread(self.mask_path + self.X[idx] + '.png', cv2.IMREAD_GRAYSCALE)\n\n        if self.transform is not None:\n            aug = self.transform(image=img, mask=mask)\n            img = Image.fromarray(aug['image'])\n            mask = aug['mask']\n\n        if self.transform is None:\n            img = Image.fromarray(img)\n\n        mask = torch.from_numpy(mask).long()\n\n        return img, mask\n\n\nt_test = A.Resize(768, 1152, interpolation=cv2.INTER_NEAREST)\ntest_set = DroneTestDataset(IMAGE_PATH, MASK_PATH, X_test, transform=t_test)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:42.325401Z","iopub.execute_input":"2024-05-26T06:28:42.325767Z","iopub.status.idle":"2024-05-26T06:28:42.33511Z","shell.execute_reply.started":"2024-05-26T06:28:42.325739Z","shell.execute_reply":"2024-05-26T06:28:42.334225Z"},"id":"9PK68c2gyPik","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def predict_image_mask_miou(model, image, mask, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]):\n    model.eval()\n    t = T.Compose([T.ToTensor(), T.Normalize(mean, std)])\n    image = t(image)\n    model.to(device); image=image.to(device)\n    mask = mask.to(device)\n    with torch.no_grad():\n\n        image = image.unsqueeze(0)\n        mask = mask.unsqueeze(0)\n\n        output = model(image)\n        score = mIoU(output, mask)\n        masked = torch.argmax(output, dim=1)\n        masked = masked.cpu().squeeze(0)\n    return masked, score","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:46.404346Z","iopub.execute_input":"2024-05-26T06:28:46.405235Z","iopub.status.idle":"2024-05-26T06:28:46.412394Z","shell.execute_reply.started":"2024-05-26T06:28:46.405204Z","shell.execute_reply":"2024-05-26T06:28:46.411332Z"},"id":"puBRXJbxyPik","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def predict_image_mask_pixel(model, image, mask, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]):\n    model.eval()\n    t = T.Compose([T.ToTensor(), T.Normalize(mean, std)])\n    image = t(image)\n    model.to(device); image=image.to(device)\n    mask = mask.to(device)\n    with torch.no_grad():\n\n        image = image.unsqueeze(0)\n        mask = mask.unsqueeze(0)\n\n        output = model(image)\n        acc = pixel_accuracy(output, mask)\n        masked = torch.argmax(output, dim=1)\n        masked = masked.cpu().squeeze(0)\n    return masked, acc","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:48.71198Z","iopub.execute_input":"2024-05-26T06:28:48.713029Z","iopub.status.idle":"2024-05-26T06:28:48.719901Z","shell.execute_reply.started":"2024-05-26T06:28:48.712991Z","shell.execute_reply":"2024-05-26T06:28:48.718932Z"},"id":"Uj2NF79DyPik","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def miou_score(model, test_set):\n    score_iou = []\n    for i in tqdm(range(len(test_set))):\n        img, mask = test_set[i]\n        pred_mask, score = predict_image_mask_miou(model, img, mask)\n        score_iou.append(score)\n    return score_iou","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:52.149163Z","iopub.execute_input":"2024-05-26T06:28:52.149908Z","iopub.status.idle":"2024-05-26T06:28:52.154939Z","shell.execute_reply.started":"2024-05-26T06:28:52.149874Z","shell.execute_reply":"2024-05-26T06:28:52.153971Z"},"id":"1lPoKr0myPil","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def pixel_acc(model, test_set):\n    accuracy = []\n    for i in tqdm(range(len(test_set))):\n        img, mask = test_set[i]\n        pred_mask, acc = predict_image_mask_pixel(model, img, mask)\n        accuracy.append(acc)\n    return accuracy","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:28:57.356741Z","iopub.execute_input":"2024-05-26T06:28:57.357425Z","iopub.status.idle":"2024-05-26T06:28:57.36238Z","shell.execute_reply.started":"2024-05-26T06:28:57.357394Z","shell.execute_reply":"2024-05-26T06:28:57.361303Z"},"id":"nQiirLKoyPil","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Unet_mobilenet_v2","metadata":{"id":"B-KPT3YHyPim"}},{"cell_type":"code","source":"model = smp.Unet('mobilenet_v2', encoder_weights='imagenet', classes=n_classes, activation=None, encoder_depth=5, decoder_channels=[256, 128, 64, 32, 16])","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:35:25.584582Z","iopub.execute_input":"2024-05-26T06:35:25.585408Z","iopub.status.idle":"2024-05-26T06:35:25.766812Z","shell.execute_reply.started":"2024-05-26T06:35:25.585373Z","shell.execute_reply":"2024-05-26T06:35:25.766023Z"},"id":"4pLbabDKyPim","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"max_lr = 1e-3\nepoch = 15\nweight_decay = 1e-4\n\ncriterion = nn.CrossEntropyLoss()\noptimizer = torch.optim.AdamW(model.parameters(), lr=max_lr, weight_decay=weight_decay)\nsched = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr, epochs=epoch,\n                                            steps_per_epoch=len(train_loader))\n\nhistory = fit(epoch, model, train_loader, val_loader, criterion, optimizer, sched)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T06:35:27.327098Z","iopub.execute_input":"2024-05-26T06:35:27.327948Z","iopub.status.idle":"2024-05-26T07:33:55.879927Z","shell.execute_reply.started":"2024-05-26T06:35:27.32791Z","shell.execute_reply":"2024-05-26T07:33:55.878956Z"},"id":"xV594m6IyPim","outputId":"0e9bb416-57d1-4fcf-b1de-b52a038224e4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"torch.save(model, 'Unet-Mobilenet.pt')","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:33:55.8819Z","iopub.execute_input":"2024-05-26T07:33:55.882187Z","iopub.status.idle":"2024-05-26T07:33:55.97679Z","shell.execute_reply.started":"2024-05-26T07:33:55.882163Z","shell.execute_reply":"2024-05-26T07:33:55.976037Z"},"id":"3lTthNydyPim","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_loss(history)\nplot_score(history)\nplot_acc(history)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:33:55.981556Z","iopub.execute_input":"2024-05-26T07:33:55.98185Z","iopub.status.idle":"2024-05-26T07:33:56.669737Z","shell.execute_reply.started":"2024-05-26T07:33:55.981818Z","shell.execute_reply":"2024-05-26T07:33:56.668867Z"},"id":"WB8ITlXUyPim","outputId":"017d113f-0578-4c2b-f2e0-1fb1f3330dea","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mob_miou = miou_score(model, test_set)\nmob_acc = pixel_acc(model, test_set)\n","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:33:56.670831Z","iopub.execute_input":"2024-05-26T07:33:56.671113Z","iopub.status.idle":"2024-05-26T07:34:43.763167Z","shell.execute_reply.started":"2024-05-26T07:33:56.671088Z","shell.execute_reply":"2024-05-26T07:34:43.762199Z"},"id":"7AGNWnKiyPin","outputId":"2a1e08cc-4e9a-41f8-cb24-5b4a7b818993","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Test Set mIoU', np.mean(mob_miou))\nprint('Test Set Pixel Accuracy', np.mean(mob_acc))\n","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:34:43.764438Z","iopub.execute_input":"2024-05-26T07:34:43.764824Z","iopub.status.idle":"2024-05-26T07:34:43.770653Z","shell.execute_reply.started":"2024-05-26T07:34:43.764772Z","shell.execute_reply":"2024-05-26T07:34:43.769767Z"},"id":"g-Y-Ji-XyPin","outputId":"19ef8609-c317-4e1c-843f-676f5bddb066","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Unet_vgg19","metadata":{"id":"giWE0Ft2yPio"}},{"cell_type":"code","source":"model = smp.Unet('vgg19', encoder_weights='imagenet', classes=n_classes, activation=None, encoder_depth=5, decoder_channels=[256, 128, 64, 32, 16])","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:34:43.771718Z","iopub.execute_input":"2024-05-26T07:34:43.772017Z","iopub.status.idle":"2024-05-26T07:35:03.864318Z","shell.execute_reply.started":"2024-05-26T07:34:43.771993Z","shell.execute_reply":"2024-05-26T07:35:03.863507Z"},"id":"gO-ldMhvyPio","outputId":"ef0b3988-f2d9-4257-952f-98c2a463b31a","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"max_lr = 1e-3\nepoch = 5\nweight_decay = 1e-4\n\ncriterion = nn.CrossEntropyLoss()\noptimizer = torch.optim.AdamW(model.parameters(), lr=max_lr, weight_decay=weight_decay)\nsched = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr, epochs=epoch,\n                                            steps_per_epoch=len(train_loader))\n\nhistory = fit(epoch, model, train_loader, val_loader, criterion, optimizer, sched)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:35:03.865484Z","iopub.execute_input":"2024-05-26T07:35:03.865774Z","iopub.status.idle":"2024-05-26T07:57:23.503875Z","shell.execute_reply.started":"2024-05-26T07:35:03.865749Z","shell.execute_reply":"2024-05-26T07:57:23.502855Z"},"id":"uf7_xAF8yPio","outputId":"b141a8b7-a7a5-4e79-ec42-ba1b398afdbf","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"torch.save(model, 'Unet-Vgg19.pt')","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:57:23.505556Z","iopub.execute_input":"2024-05-26T07:57:23.505889Z","iopub.status.idle":"2024-05-26T07:57:23.675317Z","shell.execute_reply.started":"2024-05-26T07:57:23.505861Z","shell.execute_reply":"2024-05-26T07:57:23.674524Z"},"id":"rp0YSX1oyPio","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_loss(history)\nplot_score(history)\nplot_acc(history)","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:57:23.676564Z","iopub.execute_input":"2024-05-26T07:57:23.677031Z","iopub.status.idle":"2024-05-26T07:57:24.49106Z","shell.execute_reply.started":"2024-05-26T07:57:23.676989Z","shell.execute_reply":"2024-05-26T07:57:24.49006Z"},"id":"7DshyxNdyPio","outputId":"49b8bf10-00e1-4481-9993-f584bbcfedfe","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mob_miou = miou_score(model, test_set)\nmob_acc = pixel_acc(model, test_set)\n","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:57:24.493914Z","iopub.execute_input":"2024-05-26T07:57:24.494202Z","iopub.status.idle":"2024-05-26T07:58:10.747394Z","shell.execute_reply.started":"2024-05-26T07:57:24.494177Z","shell.execute_reply":"2024-05-26T07:58:10.746436Z"},"id":"loZouLD9yPio","outputId":"3590ecfc-99f6-4df6-9310-ee0a4788bf29","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Test Set mIoU', np.mean(mob_miou))\nprint('Test Set Pixel Accuracy', np.mean(mob_acc))\n","metadata":{"execution":{"iopub.status.busy":"2024-05-26T07:58:10.759013Z","iopub.execute_input":"2024-05-26T07:58:10.759302Z","iopub.status.idle":"2024-05-26T07:58:10.772632Z"},"id":"VGIvx0RQyPip","outputId":"71063100-d049-4c4a-cf99-0062124a6e82","trusted":true},"execution_count":null,"outputs":[]}]}