{"cells":[{"metadata":{},"cell_type":"markdown","source":"<center><img src=\"https://raw.githubusercontent.com/dimitreOliveira/MachineLearning/master/Kaggle/Understanding%20Clouds%20from%20Satellite%20Images/banner.png\" width=\"800\"></center>\n<h1><center>Understanding Clouds from Satellite Images</center></h1><p></p>\n<h2><center>Cloud Segmentation with utility scripts and Keras</center></h2>\n\n#### This kernel is to show the new feature on Kaggle, script notebooks. I'm not doing EDA here because I already did it on [my other kernel](https://www.kaggle.com/dimitreoliveira/understanding-clouds-eda-and-keras-u-net), the goal here is just to demonstrate how to use script notebook and how it can improve our work, making it faster and cleaner.\n#### I found this addition really cool as I always try to write clean and modular code, it always saves time later, this may be another push towards better software practices on data science projects.\n\nWhat you will find on the [script I made](https://www.kaggle.com/dimitreoliveira/cloud-images-segmentation-utillity-script):\n- All used dependencies\n- External repository codes (need internet option ON)\n- Seed function (to make model runs more reproducible)\n- Segmentation functions related to this competition\n- Multi-thread data process functions (to resize and apply transformations faster)\n- Model evaluation (training plots)\n- Model post-process (Set threshold and removing small masks)\n- Prediction evaluation (Generate metrics over predictions and sample evaluation)\n- Data generator\n- Learning rate schedulers\n\n##### If you have any request to update or add anything to the scripts please let me know in the comments."},{"metadata":{},"cell_type":"markdown","source":"### Dependencies"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":false,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"# Link for the script https://www.kaggle.com/dimitreoliveira/cloud-images-segmentation-utillity-script\nfrom cloud_images_segmentation_utillity_script import *\nfrom keras.models import load_model\n\n!pip install tta-wrapper --quiet\n\nseed = 0\nseed_everything(seed)\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Load data"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_kg_hide-input":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/understanding_cloud_organization/train.csv')\nsubmission = pd.read_csv('../input/understanding_cloud_organization/sample_submission.csv')\n\n# Preprocecss data\ntrain['image'] = train['Image_Label'].apply(lambda x: x.split('_')[0])\ntrain['label'] = train['Image_Label'].apply(lambda x: x.split('_')[1])\nsubmission['image'] = submission['Image_Label'].apply(lambda x: x.split('_')[0])\ntest = pd.DataFrame(submission['image'].unique(), columns=['image'])\n\n# Create one column for each mask\ntrain_df = pd.pivot_table(train, index=['image'], values=['EncodedPixels'], columns=['label'], aggfunc=np.min).reset_index()\ntrain_df.columns = ['image', 'Fish_mask', 'Flower_mask', 'Gravel_mask', 'Sugar_mask']\n\nprint('Compete set samples:', len(train_df))\nprint('Test samples:', len(submission))\n\ndisplay(train.head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train and validation split"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train, X_val = train_test_split(train_df, test_size=0.2, random_state=seed)\nX_train['set'] = 'train'\nX_val['set'] = 'validation'\ntest['set'] = 'test'\n\nprint('Train samples: ', len(X_train))\nprint('Validation samples: ', len(X_val))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model parameters"},{"metadata":{"trusted":true},"cell_type":"code","source":"BACKBONE = 'resnet18'\nBATCH_SIZE = 32\nEPOCHS = 10\nLEARNING_RATE = 3e-4\nHEIGHT = 384\nWIDTH = 480\nCHANNELS = 3\nN_CLASSES = 4\nES_PATIENCE = 5\nRLROP_PATIENCE = 3\nDECAY_DROP = 0.5\nmodel_path = 'uNet_%s_%sx%s.h5' % (BACKBONE, HEIGHT, WIDTH)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"preprocessing = sm.get_preprocessing(BACKBONE)\n\naugmentation = None\n\n#augmentation = albu.Compose([albu.HorizontalFlip(p=0.5),\n                            # albu.VerticalFlip(p=0.5),\n                            # albu.ShiftScaleRotate(rotate_limit=30, shift_limit=0.1, p=0.5)\n                            #])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Pre-process data"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_base_path = '../input/understanding_cloud_organization/train_images/'\ntest_base_path = '../input/understanding_cloud_organization/test_images/'\ntrain_images_dest_path = 'base_dir/train_images/'\nvalidation_images_dest_path = 'base_dir/validation_images/'\ntest_images_dest_path = 'base_dir/test_images/'\n\n# Making sure directories don't exist\nif os.path.exists(train_images_dest_path):\n    shutil.rmtree(train_images_dest_path)\nif os.path.exists(validation_images_dest_path):\n    shutil.rmtree(validation_images_dest_path)\nif os.path.exists(test_images_dest_path):\n    shutil.rmtree(test_images_dest_path)\n    \n# Creating train, validation and test directories\nos.makedirs(train_images_dest_path)\nos.makedirs(validation_images_dest_path)\nos.makedirs(test_images_dest_path)\n\ndef preprocess_data(df, HEIGHT=HEIGHT, WIDTH=WIDTH):\n    '''\n    This function needs to be defined here, because it will be called with no arguments, \n    and must have the default parameters from the beggining of the notebook (HEIGHT and WIDTH)\n    '''\n    df = df.reset_index()\n    for i in range(df.shape[0]):\n        item = df.iloc[i]\n        image_id = item['image']\n        item_set = item['set']\n        if item_set == 'train':\n            preprocess_image(image_id, train_base_path, train_images_dest_path, HEIGHT, WIDTH)\n        if item_set == 'validation':\n            preprocess_image(image_id, train_base_path, validation_images_dest_path, HEIGHT, WIDTH)\n        if item_set == 'test':\n            preprocess_image(image_id, test_base_path, test_images_dest_path, HEIGHT, WIDTH)\n\n# Pre-procecss train set\npre_process_set(X_train, preprocess_data)\n\n# Pre-procecss validation set\npre_process_set(X_val, preprocess_data)\n\n# Pre-procecss test set\npre_process_set(test, preprocess_data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data generator"},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"train_generator = DataGenerator(\n                  directory=train_images_dest_path,\n                  dataframe=X_train,\n                  target_df=train,\n                  batch_size=BATCH_SIZE,\n                  target_size=(HEIGHT, WIDTH),\n                  n_channels=CHANNELS,\n                  n_classes=N_CLASSES,\n                  preprocessing=preprocessing,\n                  augmentation=augmentation,\n                  seed=seed)\n\nvalid_generator = DataGenerator(\n                  directory=validation_images_dest_path,\n                  dataframe=X_val,\n                  target_df=train,\n                  batch_size=BATCH_SIZE, \n                  target_size=(HEIGHT, WIDTH),\n                  n_channels=CHANNELS,\n                  n_classes=N_CLASSES,\n                  preprocessing=preprocessing,\n                  seed=seed)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras import backend as K\nfrom keras.models import Model\nfrom keras.layers import Input, BatchNormalization, Dropout\nfrom keras.layers import Activation\nfrom keras.layers.convolutional import Conv2D, Conv2DTranspose\nfrom keras.layers.pooling import MaxPooling2D\nfrom keras.layers.merge import concatenate\nfrom keras.losses import binary_crossentropy\nfrom keras.optimizers import Adam, Nadam\nfrom keras.callbacks import Callback, ModelCheckpoint\n\n\ninputs = Input(shape=(HEIGHT, WIDTH, CHANNELS))\n\nc1 = Conv2D(8, (3, 3), activation=None, padding='same') (inputs)\nc1 = Conv2D(8, (3, 3), activation=None, padding='same') (c1)\nbn1 = BatchNormalization()(c1)\na1 = Activation('elu')(bn1)\np1 = MaxPooling2D((2, 2), padding='same') (a1)\nd1 = Dropout(0.8)(p1)\n\nc2 = Conv2D(16, (3, 3), activation=None, padding='same') (d1)\nc2 = Conv2D(16, (3, 3), activation=None, padding='same') (c2)\nbn2 = BatchNormalization()(c2)\na2 = Activation('elu')(bn2)\np2 = MaxPooling2D((2, 2), padding='same') (a2)\nd2 = Dropout(0.5)(p2)\n\nc3 = Conv2D(32, (3, 3), activation=None, padding='same') (d2)\nc3 = Conv2D(32, (3, 3), activation=None, padding='same') (c3)\nbn3 = BatchNormalization()(c3)\na3 = Activation('elu')(bn3)\np3 = MaxPooling2D((2, 2), padding='same') (a3)\nd3 = Dropout(0.5)(p3)\n\nc4 = Conv2D(64, (3, 3), activation=None, padding='same') (d3)\nc4 = Conv2D(64, (3, 3), activation=None, padding='same') (c4)\nbn4 = BatchNormalization()(c4)\na4 = Activation('elu')(bn4)\np4 = MaxPooling2D((2, 2), padding='same') (a4)\nd4 = Dropout(0.5)(p4)\n\nc5 = Conv2D(64, (3, 3), activation=None, padding='same') (d4)\nc5 = Conv2D(64, (3, 3), activation=None, padding='same') (c5)\nbn5 = BatchNormalization()(c5)\na5 = Activation('elu')(bn5)\np5 = MaxPooling2D((2, 2), padding='same') (a5)\nd5 = Dropout(0.5)(p5)\n\nc55 = Conv2D(128, (3, 3), activation=None, padding='same') (d5)\nc55 = Conv2D(128, (3, 3), activation=None, padding='same') (c55)\nbn55 = BatchNormalization()(c55)\na55 = Activation('elu')(bn55)\nd55 = Dropout(0.5)(a55)\n\nu6 = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same') (d55)\nu6 = concatenate([u6, a5])\nd6 = Dropout(0.5)(u6)\nc6 = Conv2D(64, (3, 3), activation=None, padding='same') (d6)\nc6 = Conv2D(64, (3, 3), activation=None, padding='same') (c6)\nbn6 = BatchNormalization()(c6)\na6 = Activation('elu')(bn6)\n\nu71 = Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same') (a6)\nu71 = concatenate([u71, a4])\nd71 = Dropout(0.5)(u71)\nc71 = Conv2D(32, (3, 3), activation=None, padding='same') (d71)\nc61 = Conv2D(32, (3, 3), activation=None, padding='same') (c71)\nbn61 = BatchNormalization()(c61)\na61 = Activation('elu')(bn61)\n\nu7 = Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same') (a61)\nu7 = concatenate([u7, a3])\nd7 = Dropout(0.5)(u7)\nc7 = Conv2D(32, (3, 3), activation=None, padding='same') (d7)\nc7 = Conv2D(32, (3, 3), activation=None, padding='same') (c7)\nbn7 = BatchNormalization()(c7)\na7 = Activation('elu')(bn7)\n\nu8 = Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same') (a7)\nu8 = concatenate([u8, a2])\nd8 = Dropout(0.5)(u8)\nc8 = Conv2D(16, (3, 3), activation=None, padding='same') (d8)\nc8 = Conv2D(16, (3, 3), activation=None, padding='same') (c8)\nbn8 = BatchNormalization()(c8)\na8 = Activation('elu')(bn8)\n\nu9 = Conv2DTranspose(8, (2, 2), strides=(2, 2), padding='same') (a8)\nu9 = concatenate([u9, a1], axis=3)\nd9 = Dropout(0.5)(u9)\nc9 = Conv2D(8, (3, 3), activation=None, padding='same') (d9)\nc9 = Conv2D(8, (3, 3), activation=None, padding='same') (c9)\nbn9 = BatchNormalization()(c9)\na9 = Activation('elu')(bn9)\n\noutputs = Conv2D(4, (1, 1), activation='sigmoid') (a9)\n\n\nmodel = Model(inputs=[inputs], outputs=[outputs])","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#model = sm.Unet(backbone_name=BACKBONE, \n               # encoder_weights='imagenet',\n               # classes=N_CLASSES,\n               # activation='sigmoid',\n                #input_shape=(HEIGHT, WIDTH, CHANNELS))\n\ncheckpoint = ModelCheckpoint(model_path, monitor='val_loss', mode='min', save_best_only=True, save_weights_only=True)\nes = EarlyStopping(monitor='val_loss', mode='min', patience=ES_PATIENCE, restore_best_weights=True, verbose=1)\nrlrop = ReduceLROnPlateau(monitor='val_loss', mode='min', patience=RLROP_PATIENCE, factor=DECAY_DROP, min_lr=1e-6, verbose=1)\n\nmetric_list = [dice_coef, sm.metrics.iou_score]\ncallback_list = [checkpoint, es, rlrop]\noptimizer = RAdam(learning_rate=LEARNING_RATE, warmup_proportion=0.1)\n\nmodel.compile(optimizer=optimizer, loss=sm.losses.bce_dice_loss, metrics=metric_list)\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"STEP_SIZE_TRAIN = len(X_train)//BATCH_SIZE\nSTEP_SIZE_VALID = len(X_val)//BATCH_SIZE\n\nhistory = model.fit_generator(generator=train_generator,\n                              steps_per_epoch=STEP_SIZE_TRAIN,\n                              validation_data=valid_generator,\n                              validation_steps=STEP_SIZE_VALID,\n                              callbacks=callback_list,\n                              epochs=EPOCHS,\n                              verbose=2).history","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model loss graph"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plot_metrics(history, metric_list=['loss', 'dice_coef', 'iou_score'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Load model trained longer\n# model = load_model('../input/cloud-seg-resnet18-trainedlonger/resnet18_trained_longer.h5', custom_objects={'RAdam':RAdam, 'binary_crossentropy_plus_dice_loss':sm.losses.bce_dice_loss, 'dice_coef':dice_coef, 'iou_score':sm.metrics.iou_score, 'f1-score':sm.metrics.f1_score})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Threshold and mask size tunning\n - Here we could use some kind of parameter search, but to simplify I'm using default values"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"class_names = ['Fish  ', 'Flower', 'Gravel', 'Sugar ']\nbest_tresholds = [.5, .5, .5, .35]\nbest_masks = [25000, 20000, 22500, 15000]\n\nfor index, name in enumerate(class_names):\n    print('%s treshold=%.2f mask size=%d' % (name, best_tresholds[index], best_masks[index]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model evaluation"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_metrics = get_metrics(model, train, X_train, train_images_dest_path, best_tresholds, best_masks, seed=seed, preprocessing=preprocessing, set_name='Train')\ndisplay(train_metrics)\n\nvalidation_metrics = get_metrics(model, train, X_val, validation_images_dest_path, best_tresholds, best_masks, seed=seed, preprocessing=preprocessing, set_name='Validation')\ndisplay(validation_metrics)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Apply model to test set"},{"metadata":{"trusted":true},"cell_type":"code","source":"from tta_wrapper import tta_segmentation\n\nmodel = tta_segmentation(model, h_flip=True, v_flip=True, h_shift=(-10, 10), v_shift=(-10, 10), merge='mean')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"test_df = []\n\nfor i in range(0, test.shape[0], 300):\n    batch_idx = list(range(i, min(test.shape[0], i + 300)))\n    batch_set = test[batch_idx[0]: batch_idx[-1]+1]\n    \n    test_generator = DataGenerator(\n                      directory=test_images_dest_path,\n                      dataframe=batch_set,\n                      target_df=submission,\n                      batch_size=1, \n                      target_size=(HEIGHT, WIDTH),\n                      n_channels=CHANNELS,\n                      n_classes=N_CLASSES,\n                      preprocessing=preprocessing,\n                      seed=seed,\n                      mode='predict',\n                      shuffle=False)\n    \n    preds = model.predict_generator(test_generator)\n\n    for index, b in enumerate(batch_idx):\n        filename = test['image'].iloc[b]\n        image_df = submission[submission['image'] == filename].copy()\n        pred_masks = preds[index, ].round().astype(int)\n        pred_rles = build_rles(pred_masks, reshape=(350, 525))\n        image_df['EncodedPixels'] = pred_rles\n\n        ### Post procecssing\n        pred_masks_post = preds[index, ].astype('float32') \n        for class_index in range(N_CLASSES):\n            pred_mask = pred_masks_post[...,class_index]\n            pred_mask = post_process(pred_mask, threshold=best_tresholds[class_index], min_size=best_masks[class_index])\n            pred_masks_post[...,class_index] = pred_mask\n\n        pred_rles_post = build_rles(pred_masks_post, reshape=(350, 525))\n        image_df['EncodedPixels_post'] = pred_rles_post\n        ###\n        \n        test_df.append(image_df)\n\nsub_df = pd.concat(test_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inspecting some of the validation set predictions\n\n## Without post-processing"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Choose 50 samples at random\nimages_to_inspect = np.random.choice(X_val['image'].unique(), 50, replace=False)\ninspect_set = train[train['image'].isin(images_to_inspect)].copy()\ninspect_set_temp = []\n\ninspect_generator = DataGenerator(\n                    directory=validation_images_dest_path,\n                    dataframe=inspect_set,\n                    target_df=train,\n                    batch_size=1, \n                    target_size=(HEIGHT, WIDTH),\n                    n_channels=CHANNELS,\n                    n_classes=N_CLASSES,\n                    preprocessing=preprocessing,\n                    seed=seed,\n                    mode='fit',\n                    shuffle=False)\n\npreds = model.predict_generator(inspect_generator)\n\nfor index, b in enumerate(range(len(preds))):\n    filename = inspect_set['image'].iloc[b]\n    image_df = inspect_set[inspect_set['image'] == filename].copy()\n    pred_masks = preds[index, ].round().astype(int)\n    pred_rles = build_rles(pred_masks, reshape=(350, 525))\n    image_df['EncodedPixels_pred'] = pred_rles\n    \n    ### Post procecssing\n    pred_masks_post = preds[index, ].astype('float32') \n    for class_index in range(N_CLASSES):\n        pred_mask = pred_masks_post[...,class_index]\n        pred_mask = post_process(pred_mask, threshold=best_tresholds[class_index], min_size=best_masks[class_index])\n        pred_masks_post[...,class_index] = pred_mask\n\n    pred_rles_post = build_rles(pred_masks_post, reshape=(350, 525))\n    image_df['EncodedPixels_pred_post'] = pred_rles_post\n    ###\n    inspect_set_temp.append(image_df)\n\n\ninspect_set = pd.concat(inspect_set_temp)\ninspect_predictions(inspect_set, images_to_inspect, validation_images_dest_path, pred_col='EncodedPixels_pred')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## With post-processing"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"inspect_predictions(inspect_set, images_to_inspect, validation_images_dest_path, pred_col='EncodedPixels_pred_post')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inspecting some of the test set predictions\n\n## Without post-process"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Choose 5 samples at random\nimages_to_inspect_test =  np.random.choice(sub_df['image'].unique(), 4, replace=False)\ninspect_predictions(sub_df, images_to_inspect_test, test_images_dest_path)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def get_score(model, target_df, df, df_images_dest_path, tresholds, min_mask_sizes, N_CLASSES=4, seed=0, preprocessing=None, set_name='Complete set'):\n    class_names = ['Fish', 'Flower', 'Gravel', 'Sugar']\n    metrics = []\n\n    for class_name in class_names:\n        metrics.append([class_name, 0, 0])\n\n    metrics_df = pd.DataFrame(metrics, columns=['Class', 'Dice', 'Dice Post'])\n    \n    for i in range(0, df.shape[0], 300):\n        batch_idx = list(range(i, min(df.shape[0], i + 300)))\n        batch_set = df[batch_idx[0]: batch_idx[-1]+1]\n        ratio = len(batch_set) / len(df)\n\n        generator = DataGenerator(\n                      directory=df_images_dest_path,\n                      dataframe=batch_set,\n                      target_df=target_df,\n                      batch_size=len(batch_set), \n                      target_size=model.input_shape[1:3],\n                      n_channels=model.input_shape[3],\n                      n_classes=N_CLASSES,\n                      preprocessing=preprocessing,\n                      seed=seed,\n                      mode='fit',\n                      shuffle=False)\n\n        x, y = generator.__getitem__(0)\n        preds = model.predict(x)\n        \n        for class_index in range(N_CLASSES):\n            class_score = []\n            class_score_post = []\n            mask_class = y[..., class_index]\n            pred_class = preds[..., class_index]\n            for index in range(len(batch_idx)):\n                sample_mask = mask_class[index, ]\n                sample_pred = pred_class[index, ]\n                sample_pred_post = post_process(sample_pred, threshold=tresholds[class_index], min_size=min_mask_sizes[class_index])\n                if (sample_mask.sum() == 0) & (sample_pred.sum() == 0):\n                    dice_score = 1.\n                else:\n                    dice_score = dice_coefficient(sample_pred, sample_mask)\n                if (sample_mask.sum() == 0) & (sample_pred_post.sum() == 0):\n                    dice_score_post = 1.\n                else:\n                    dice_score_post = dice_coefficient(sample_pred_post, sample_mask)\n                class_score.append(dice_score)\n                class_score_post.append(dice_score_post)\n           # metrics_df.loc[metrics_df['Class'] == class_names[class_index], 'Dice'] += np.mean(class_score) * ratio\n           # metrics_df.loc[metrics_df['Class'] == class_names[class_index], 'Dice Post'] += np.mean(class_score_post) * ratio\n\n   # metrics_df = metrics_df.append({'Class':set_name, 'Dice':np.mean(metrics_df['Dice'].values), 'Dice Post':np.mean(metrics_df['Dice Post'].values)}, ignore_index=True).set_index('Class')\n    \n  #  return metrics_df\n    return class_score,class_score_post","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Cleaning created directories\nif os.path.exists(train_images_dest_path):\n    shutil.rmtree(train_images_dest_path)\nif os.path.exists(validation_images_dest_path):\n    shutil.rmtree(validation_images_dest_path)\nif os.path.exists(test_images_dest_path):\n    shutil.rmtree(test_images_dest_path)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"}},"nbformat":4,"nbformat_minor":1}