{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":13836,"databundleVersionId":1718836,"sourceType":"competition"}],"dockerImageVersionId":30699,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Approach : Inception - ResNet - v2","metadata":{}},{"cell_type":"code","source":"\n\nimport os\nimport shutil\nimport pandas as pd\nimport random\n\n# Path to the directory containing the images\nimage_dir = '/kaggle/input/cassava-leaf-disease-classification/train_images'\nsub_dir = '/kaggle/working/'\n\n# Path to the CSV file containing image mappings\ncsv_file = '/kaggle/input/cassava-leaf-disease-classification/train.csv'\n\n# Read the CSV file\ndf = pd.read_csv(csv_file)\n\n# Iterate over each row in the DataFrame\nfor index, row in df.iterrows():\n    # Get the image filename and label\n    image_id = row['image_id']\n    label = row['label']\n    \n    # Create subdirectory for the label if it doesn't exist\n    label_dir = os.path.join(sub_dir, str(label))\n    if not os.path.exists(label_dir):\n        os.makedirs(label_dir)\n    \n    # Move the image file to the corresponding subdirectory\n    src_path = os.path.join(image_dir, image_id)\n    dest_path = os.path.join(label_dir, image_id)\n    \n    # Check if label is \"3\" and transfer only 33% of such images\n    if label == 8 and random.random() < 0.33:\n        shutil.copy(src_path, dest_path)\n    elif label != 8:  # For other labels, transfer all images\n        shutil.copy(src_path, dest_path)\n","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:43:18.471776Z","iopub.execute_input":"2024-04-25T14:43:18.472420Z","iopub.status.idle":"2024-04-25T14:46:29.102129Z","shell.execute_reply.started":"2024-04-25T14:43:18.472392Z","shell.execute_reply":"2024-04-25T14:46:29.101326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install --upgrade pip setuptools\n# !pip cache purge\n#!pip install tensorflow-gpu==2.6.0\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#pip install tensorflow==2.15.0\n#!pip install tensorflow-gpu","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#!pip3 install tensorflow-gpu\n# print(\"Num GPUs Available: \", len(tf.config.experimental.list_physical_devices('GPU')))\n# import os\n#os.environ['CUDA_VISIBLE_DEVICES'] = '0'  # Use GPU 0\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import os\n\n# # Set the CUDA_VISIBLE_DEVICES environment variable to specify visible GPUs\n# os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0,1\"  # Specify GPU 0 and GPU 1\n\n# # Now import TensorFlow\n# import tensorflow as tf\n\n# # Verify that TensorFlow detects the specified GPUs\n# from tensorflow.python.client import device_lib\n# print(device_lib.list_local_devices())\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import tensorflow as tf\n\n# # Define the list of visible GPU devices\n# visible_devices = [\"0\", \"1\"]  # Assuming GPU 0 and GPU 1 are the ones you want to use\n\n# # Get the list of physical devices\n# physical_devices = tf.config.list_physical_devices('GPU')\n\n# # Filter the visible devices\n# visible_devices = [physical_devices[int(device)] for device in visible_devices]\n\n# # Set the session configuration\n# tf.config.experimental.set_visible_devices(visible_devices, 'GPU')\n\n# # Verify that TensorFlow now detects only the specified GPUs\n# from tensorflow.python.client import device_lib\n# print(device_lib.list_local_devices())\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#device = tf.test.gpu_device_name()\n#print(device)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# specify imports to be used in the notebook\nimport tensorflow as tf\nimport pandas as pd\nimport numpy as np\nimport os\n#from sklearn.preprocessing import StandardScaler\n","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:48:29.848294Z","iopub.execute_input":"2024-04-25T14:48:29.848671Z","iopub.status.idle":"2024-04-25T14:48:41.383260Z","shell.execute_reply.started":"2024-04-25T14:48:29.848641Z","shell.execute_reply":"2024-04-25T14:48:41.382485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# specify the training data directory\ndata_dir = '/kaggle/working/'","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:48:43.642237Z","iopub.execute_input":"2024-04-25T14:48:43.642947Z","iopub.status.idle":"2024-04-25T14:48:43.648200Z","shell.execute_reply.started":"2024-04-25T14:48:43.642910Z","shell.execute_reply":"2024-04-25T14:48:43.647110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# specify the classes of the plants\n#species = [0,1,2,3,4]","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Defining the Model","metadata":{}},{"cell_type":"code","source":"# function that returns the model with custom layers appended\ndef define_model(width, height):\n    \n    # define the input to the model\n    input_model = tf.keras.layers.Input(shape = (width, height, 3), name = 'image_input')\n    \n    # main model with Incpetion - ResNet - v2 layers\n    # omit the top layers as we are adding custom layers\n    # use transfer learning, with weights from Imagenet trained model\n    main_model = tf.keras.applications.inception_resnet_v2.InceptionResNetV2(include_top = False)(input_model)\n    \n    # flatten model to get appropriate dimensions\n    flattened_model = tf.keras.layers.Flatten()(main_model)\n    \n    # add custom dropout and dense layers\n    dropout_1 = tf.keras.layers.Dropout(0.5)(flattened_model)\n    dense_1 = tf.keras.layers.Dense(128, activation = 'relu', activity_regularizer=tf.keras.regularizers.l2(1e-5))(dropout_1)\n    dropout_2 = tf.keras.layers.Dropout(0.5)(dense_1)\n    \n    # output of model\n    output_model = tf.keras.layers.Dense(5, activation = \"softmax\", activity_regularizer=tf.keras.regularizers.l2(1e-5))(dropout_2)\n\n    model = tf.keras.models.Model(input_model,  output_model)\n    \n    # use Adam optimizer with model\n    optimizer = tf.keras.optimizers.Adam(learning_rate = 5e-4, beta_1 = 0.9, beta_2 = 0.999)\n    \n    # use categorical crossentropy loss since classification task\n    model.compile(loss=\"categorical_crossentropy\", optimizer = optimizer, metrics = [\"accuracy\"])\n    return model","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:48:47.756904Z","iopub.execute_input":"2024-04-25T14:48:47.757739Z","iopub.status.idle":"2024-04-25T14:48:47.766890Z","shell.execute_reply.started":"2024-04-25T14:48:47.757709Z","shell.execute_reply":"2024-04-25T14:48:47.765787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define appropriate callbacks\ndef training_callbacks():\n    \n    # save best model regularly\n    save_best_model = tf.keras.callbacks.ModelCheckpoint(filepath = 'model.keras',\n        monitor = 'loss', save_best_only = True, verbose = 1)\n    \n    # reduce learning rate when it stops decreasing\n    reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor = 'loss', factor = 0.4,\n                              patience = 5, min_lr = 1e-10, verbose = 1, cooldown = 1)\n    \n    # stop training early if no further improvement\n    early_stopping = tf.keras.callbacks.EarlyStopping(\n        monitor = 'loss', min_delta = 1e-2, patience = 8, verbose = 1,\n        mode = 'min', baseline = None, restore_best_weights = True\n    )\n\n    return [save_best_model, reduce_lr, early_stopping]","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:48:51.568912Z","iopub.execute_input":"2024-04-25T14:48:51.569267Z","iopub.status.idle":"2024-04-25T14:48:51.577881Z","shell.execute_reply.started":"2024-04-25T14:48:51.569240Z","shell.execute_reply":"2024-04-25T14:48:51.577074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Creating data generators, to feed into model","metadata":{}},{"cell_type":"code","source":"\n\ndef data_generators(data_dir, width, height, batch_size):\n    # Define data generator without data augmentation\n    \n    # No data augmentation for training data\n    data_generator = tf.keras.preprocessing.image.ImageDataGenerator(\n        validation_split=0.2  # Splitting training data for validation\n    )\n    \n    # Define training data generator\n    train_gen = data_generator.flow_from_directory(\n        directory=data_dir,\n        target_size=(width, height),\n        color_mode='rgb',\n        class_mode=\"categorical\",\n        batch_size=batch_size,\n        subset='training'\n    )\n    \n    # Define validation data generator\n    validation_gen = data_generator.flow_from_directory(\n        directory=data_dir,\n        target_size=(width, height),\n        color_mode='rgb',\n        class_mode=\"categorical\",\n        batch_size=batch_size,\n        subset='validation'\n    )\n\n    return train_gen, validation_gen\n\n\n# # create data generators\n# def data_generators(data_dir, width, height, batch_size):\n    \n#     # apply random transformations on training data\n#     train_data_generator = tf.keras.preprocessing.image.ImageDataGenerator(\n#         validation_split=0.2\n#     )\n#     aug_data_generator = tf.keras.preprocessing.image.ImageDataGenerator(\n#         rotation_range=360,\n#         shear_range=0.3,\n#         zoom_range=0.6,\n#         width_shift_range=0.4,\n#         height_shift_range=0.4,\n#         vertical_flip=True,\n#         horizontal_flip=True,\n#         validation_split=0.2,  # Use a validation split instead of validation_subset\n#     )\n    \n#     # Define training data generator\n#     train_gen = train_data_generator.flow_from_directory(\n#         directory=data_dir,\n#         target_size=(width, height),\n#         color_mode='rgb',\n#         class_mode=\"categorical\",\n#         batch_size=batch_size,\n#         subset='training',\n#         seed=0\n#     )\n    \n#     # Get augmented images from the same training data generator\n#     augmented_train_gen = aug_data_generator.flow_from_directory(\n#         directory=data_dir,\n#         target_size=(width, height),\n#         color_mode='rgb',\n#         class_mode=\"categorical\",\n#         batch_size=batch_size,\n#         subset='training',  # Ensure it's on the training subset\n#         seed=0\n#     )\n    \n#     # Combine original and augmented images for training\n#     train_gen = combine_generators(train_gen, augmented_train_gen)\n    \n#     # Define validation data generator\n#     validation_gen = train_data_generator.flow_from_directory(\n#         directory=data_dir,\n#         target_size=(width, height),\n#         color_mode='rgb',\n#         class_mode=\"categorical\",\n#         batch_size=batch_size,\n#         subset='validation',  # Subset should be 'validation' for validation generator\n#     )\n    \n#     return train_gen, validation_gen\n\n# # Function to combine original and augmented generators\n# # def combine_generators(generator1, generator2):\n# #     while True:\n# #         x1, y1 = next(generator1)\n# #         x2, y2 = next(generator2)\n# #         yield (x1, y1), (x2, y2)\n# def combine_generators(generator1, generator2):\n#     while True:\n#         # Get batches from each generator\n#         batch1 = next(generator1)\n#         batch2 = next(generator2)\n        \n#         # Combine the samples from both batches\n#         yield batch1\n#         yield batch2\n#         x1, y1 = batch1\n#         x2, y2 = batch2\n#         yield (np.concatenate([x1, x2], axis=0), np.concatenate([y1, y2], axis=0))\n\n","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:48:54.856299Z","iopub.execute_input":"2024-04-25T14:48:54.856675Z","iopub.status.idle":"2024-04-25T14:48:54.866290Z","shell.execute_reply.started":"2024-04-25T14:48:54.856644Z","shell.execute_reply":"2024-04-25T14:48:54.865355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\n\n# Path to the directory containing the images\ndata_dir = '/kaggle/working/'\n\n# List all subdirectories in the data directory\nsubdirectories = [d for d in os.listdir(data_dir) if os.path.isdir(os.path.join(data_dir, d))]\n\n# Print the list of subdirectories\nprint(\"Subdirectories found:\")\nprint(subdirectories)\n\nimport os\n\n# Path to the directory containing the images\ndata_dir = '/kaggle/working/'\n\n# List all subdirectories in the data directory\nsubdirectories = [d for d in os.listdir(data_dir) if os.path.isdir(os.path.join(data_dir, d))]\n\n# Check if '.virtual_documents' exists in the subdirectories list\nif '.virtual_documents' in subdirectories:\n    # Construct the path to the directory to be deleted\n    dir_to_delete = os.path.join(data_dir, '.virtual_documents')\n    \n    # Remove the directory\n    os.rmdir(dir_to_delete)\n    \n    print(f\"'{dir_to_delete}' directory deleted successfully.\")\nelse:\n    print(\"'.virtual_documents' directory not found.\")\n","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:49:00.050247Z","iopub.execute_input":"2024-04-25T14:49:00.050654Z","iopub.status.idle":"2024-04-25T14:49:00.059477Z","shell.execute_reply.started":"2024-04-25T14:49:00.050623Z","shell.execute_reply":"2024-04-25T14:49:00.058569Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training the model","metadata":{}},{"cell_type":"code","source":"# define training parameters\nheight = 299\nwidth = 299\nnum_epochs = 50\nbatch_size = 16\n","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:49:13.017438Z","iopub.execute_input":"2024-04-25T14:49:13.018464Z","iopub.status.idle":"2024-04-25T14:49:13.024124Z","shell.execute_reply.started":"2024-04-25T14:49:13.018426Z","shell.execute_reply":"2024-04-25T14:49:13.022897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define model and start training\nmodel = define_model(width, height)\ntrain_gen, validation_gen = data_generators(data_dir, width, height, batch_size)\n\n# the actual training\nwith tf.device('/device:GPU:0'):\n    #print(\"hi\")\n    model.fit(\n        train_gen,\n        callbacks = training_callbacks(),\n        epochs = num_epochs,\n        #batch_size=16,\n        \n        #steps_per_epoch = train_gen.samples // batch_size,\n        validation_data = validation_gen,\n        #validation_batch_size=16\n        #validation_steps = validation_gen.samples // batch_size,\n    )\n","metadata":{"execution":{"iopub.status.busy":"2024-04-25T14:49:16.385988Z","iopub.execute_input":"2024-04-25T14:49:16.386603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Making predictions on test data","metadata":{}},{"cell_type":"code","source":"# # Get model predictions\n# model_preds = model.predict(test_gen)\n\n# # Map predictions to classes using test_labels\n# classes = []\n# for data in range(len(test_gen)):\n#     batch_x, batch_y = next(test_gen)\n#     pred_index = model_preds[data].argmax(axis=-1)\n#     classes += [np.argmax(batch_y)]\n\n# # Calculate accuracy\n# accuracy = np.mean(np.argmax(model.predict(test_gen), axis=-1) == np.argmax(test_labels, axis=-1))\n# print(\"Accuracy on test set:\", accuracy)\n\n# # Get model predictions\n# model_preds = model.predict(test_gen)\n\n# # Map predictions to classes using test_labels\n# classes = []\n# for batch_x, batch_y in test_gen:\n#     pred_index = model.predict(batch_x).argmax(axis=-1)\n#     classes += [pred_index]\n\n# # Flatten the list of classes\n# classes = [item for sublist in classes for item in sublist]\n\n# # Calculate accuracy\n# accuracy = np.mean(np.array(classes) == np.argmax(test_labels, axis=-1))\n# print(\"Accuracy on test set:\", accuracy)\n","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Generate submission CSV\n# output_predictions = pd.DataFrame()\n# #output_predictions['file'] = test_filenames\n# output_predictions['disease'] = classes\n# output_predictions.to_csv('output.csv', index=False)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}