{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## summary \n**This notebook is written for new kaggler**\n\n**I can't guarantee that all comments are interpreted correctly, so please point out if there are mistakes**\n\nOriginal code is here [https://www.kaggle.com/code/tanakar/2-5d-segmentaion-baseline-training](http://)\n* 2.5d segmentation\n    *  segmentation_models_pytorch \n    *  Unet\n* use only 6 slices in the middle\n* slide inference","metadata":{}},{"cell_type":"markdown","source":"sklearn.metrics.classification is an evaluation metrics module for classification problems in the scikit-learn library.This module provides a series of functions to calculate the accuracy, precision, recall, F1 value and other metrics of a classification model to evaluate the performance of the model.These metrics can help us understand the classification ability of the model, so that we can optimize the parameters and algorithms of the model and improve the prediction accuracy of the model.","metadata":{}},{"cell_type":"markdown","source":"Pickle is a module in Python for serializing and deserializing Python objects.With Pickle, we can convert a Python object to a byte stream and then save it to a file or transfer it over the network.Conversely, we can also deserialize byte streams to Python objects.Pickle is a persistent storage method in Python that makes it easy to save and restore data.","metadata":{}},{"cell_type":"markdown","source":"Autocast and GradScaler are both tools in PyTorch for accelerating model training and reducing memory footprint.\nAutocast is an automatic mixed precision tool that automatically converts floating point numbers to half-precision floating point numbers during the forward and backward propagation of the model. This helps reduce GPU memory usage and speed up model training. When using Autocast, the model and optimizer need to be wrapped in the torch.cuda.amp.autocast() context manager.GradScaler is a gradient scaling tool for scaling the value of the gradient during training. This helps to solve the problem of disappearing or exploding gradients and improves model stability and training results. When using GradScaler, the gradient needs to be multiplied by a scaling factor before back-propagation is performed. The scaling factor can be dynamically adjusted according to the value of the gradient to ensure the stability of the gradient. ","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import roc_auc_score, accuracy_score, f1_score, log_loss\nimport pickle\nfrom torch.utils.data import DataLoader\nfrom torch.cuda.amp import autocast, GradScaler\nimport warnings\nimport sys\nimport pandas as pd\nimport os\nimport gc\nimport sys\nimport math\nimport time\nimport random\nimport shutil\nfrom pathlib import Path\nfrom contextlib import contextmanager\nfrom collections import defaultdict, Counter\nimport cv2\n\nimport scipy as sp\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\nfrom tqdm.auto import tqdm\nfrom functools import partial\n\nimport argparse\nimport importlib\nimport torch\nimport torch.nn as nn\nfrom torch.optim import Adam, SGD, AdamW\n\nimport datetime","metadata":{"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sys.path.append('/kaggle/input/pretrainedmodels/pretrainedmodels-0.7.4')\n# sys.path.append('/kaggle/input/efficientnet-pytorch/EfficientNet-PyTorch-master')\n# sys.path.append('/kaggle/input/timm-pytorch-image-models/pytorch-image-models-master')\n# sys.path.append('/kaggle/input/segmentation-models-pytorch/segmentation_models.pytorch-master')","metadata":{"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install segmentation_models_pytorch","metadata":{"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import segmentation_models_pytorch as smp","metadata":{"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For segmentation_models_pytorch, it is a PyTorch-based deep learning library for image segmentation tasks.\n\nIt supports many popular segmentation models, such as UNet, LinkNet, FPN, etc., and provides many pre-trained models and datasets to facilitate users to quickly build and train their own models.","metadata":{}},{"cell_type":"code","source":"!pip install warmup_scheduler","metadata":{"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nfrom torch.utils.data import DataLoader, Dataset\nimport cv2\nimport torch\nimport os\nimport albumentations as A\nfrom albumentations.pytorch import ToTensorV2\nfrom albumentations import ImageOnlyTransform","metadata":{"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## config","metadata":{"tags":[]}},{"cell_type":"code","source":"class CFG:\n    # ============== comp exp name =============\n    comp_name = 'vesuvius'\n\n    # comp_dir_path = './'\n    comp_dir_path = '/kaggle/input/'\n    comp_folder_name = 'vesuvius-challenge-ink-detection'\n    # comp_dataset_path = f'{comp_dir_path}datasets/{comp_folder_name}/'\n    comp_dataset_path = f'{comp_dir_path}{comp_folder_name}/'\n    \n    exp_name = 'vesuvius_2d_slide_exp002'\n\n    # ============== pred target =============\n    target_size = 1\n\n    # ============== model cfg =============\n    # Image segmentation model Pre-training parameters for convolutional networks\n    model_name = 'Unet'\n    #backbone = 'efficientnet-b0'\n    backbone = 'resnext101_32x4d'\n    #backbone = 'resnext50_32x4d'\n    #backbone = 'resnet50'\n\n    # There are 65 \"channels\",   \n    # a three-dimensional image of a certain dimension split into multiple two-dimensional images, \n    # the ink will be immersed in the deeper the image, the larger the information more complete\n    in_chans = 3 # 64\n    # ============== training cfg =============\n    # Size modification of input image or mask image\n    size = 224\n    tile_size = 224\n    # The number of small images cut out and the position of each small image can be controlled by setting CFG.stride.\n    stride = tile_size // 2\n\n    train_batch_size = 16 # 32\n    valid_batch_size = train_batch_size * 2\n    use_amp = True\n\n    # Learning rate regulator\n    scheduler = 'GradualWarmupSchedulerV2'\n    # scheduler = 'CosineAnnealingLR'\n    epochs = 10 # 30\n\n    '''\n    warmup_factor is a scaling factor that controls the rate of learning rate increase. It is usually used at the beginning of training to accelerate the learning rate increase if the model weights are not yet accurate enough.\n    lr is the learning rate size, and in this equation, the learning rate is divided by 10 with the effect of warmup_factor.\n    This is because at the beginning of the training, the learning rate should be relatively small so that the model can converge better, and as the training proceeds, the learning rate will gradually increase so that the weight space can be explored better.\n    Thus, with this formula, a relatively small learning rate can be used at the beginning of training and gradually increased to the appropriate size.\n    '''\n    # adamW \n    warmup_factor = 10\n    # lr = 1e-4 / warmup_factor\n    lr = 1e-4 / warmup_factor\n\n    # ============== fold =============\n    # k-fold cross-validation This method has the advantage of making better use of the data while reducing errors due to the chance of data division.\n    # The disadvantage is that k-times training and validation are required and the computational cost is high.\n    # This seems to mean that each of the three folders is used as a validation set\n    valid_id = 1\n\n    '''\n    In this example, metric_direction is set to 'maximize', indicating that the metric we want to optimize should be as large as possible.\n    '''\n    # objective_cv = 'binary'  # 'binary', 'multiclass', 'regression'\n    metric_direction = 'maximize'  # maximize, 'minimize'\n    # metrics = 'dice_coef'\n\n    # ============== fixed =============\n    pretrained = True\n    inf_weight = 'best'  # 'best'\n\n    min_lr = 1e-6\n    weight_decay = 1e-6\n    max_grad_norm = 1000\n\n    print_freq = 50\n    num_workers = 10\n\n    # Fixed seeds make code reproducible\n    # Randomness in deep learning is caused by factors such as weight initialization, random sampling, etc. These randomness can make the model more expressive and generalizable.\n    # However, since deep learning models are usually very large, their training requires a lot of time and computational resources. Therefore, it is useful to fix random seeds in order to make the experiments reproducible.\n    # Using a fixed random seed ensures that the same sequence of random numbers is generated using the same random number generator each time the experiment is run.\n    # This allows researchers to get the same results when running experiments on different machines, thus facilitating the comparison and validation of results.\n    # In addition, using a fixed random seed also makes the model's behavior more predictable during training, thus helping researchers to better understand the model's performance and behavior.\n    seed = 42\n\n    # ============== set dataset path =============\n    print('set dataset path')\n\n    outputs_path = f'/kaggle/working/outputs/{comp_name}/{exp_name}/'\n\n    submission_dir = outputs_path + 'submissions/'\n    submission_path = submission_dir + f'submission_{exp_name}.csv'\n\n    model_dir = outputs_path + f'{comp_name}-models/'\n\n    figures_dir = outputs_path + 'figures/'\n\n    log_dir = outputs_path + 'logs/'\n    log_path = log_dir + f'{exp_name}.txt'\n\n    # ============== augmentation =============\n    # Data Enhancement\n    train_aug_list = [\n        # A.RandomResizedCrop(\n        #     size, size, scale=(0.85, 1.0)),\n        A.Resize(size, size),\n        A.HorizontalFlip(p=0.5),\n        A.VerticalFlip(p=0.5),\n        A.RandomBrightnessContrast(p=0.75),\n        A.ShiftScaleRotate(p=0.75),\n        A.OneOf([\n                A.GaussNoise(var_limit=[10, 50]),\n                A.GaussianBlur(),\n                A.MotionBlur(),\n                ], p=0.4),\n        A.GridDistortion(num_steps=5, distort_limit=0.3, p=0.5),\n        A.CoarseDropout(max_holes=1, max_width=int(size * 0.3), max_height=int(size * 0.3), \n                        mask_fill_value=0, p=0.5),\n        # A.Cutout(max_h_size=int(size * 0.6),\n        #          max_w_size=int(size * 0.6), num_holes=1, p=1.0),\n        A.Normalize(\n            mean= [0] * in_chans,\n            std= [1] * in_chans\n        ),\n        ToTensorV2(transpose_mask=True),\n    ]\n\n    valid_aug_list = [\n        A.Resize(size, size),\n        A.Normalize(\n            mean= [0] * in_chans,\n            std= [1] * in_chans\n        ),\n        ToTensorV2(transpose_mask=True),\n    ]\n\n","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.19323Z","iopub.status.busy":"2023-04-04T05:50:38.192696Z","iopub.status.idle":"2023-04-04T05:50:38.211206Z","shell.execute_reply":"2023-04-04T05:50:38.209636Z","shell.execute_reply.started":"2023-04-04T05:50:38.193193Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## helper\n\n    Computes and stores the average and current value\n\n    Specifically, it serves to call the update method to update the value of the instance each time the average needs to be calculated, and to call the avg method to return the average when the average needs to be calculated.","metadata":{}},{"cell_type":"code","source":"class AverageMeter(object):\n    \"\"\"Computes and stores the average and current value\"\"\"\n\n    # Initialize the property values of the instance\n    def __init__(self):\n        self.reset()\n\n    # Reset the property value of the instance to 0\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    # Update the property value of the instance to the given value\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.213481Z","iopub.status.busy":"2023-04-04T05:50:38.212989Z","iopub.status.idle":"2023-04-04T05:50:38.223704Z","shell.execute_reply":"2023-04-04T05:50:38.222663Z","shell.execute_reply.started":"2023-04-04T05:50:38.213439Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# log\n    accepts one argument log_file log file path\n    The function initializes a logger object, which is used to record logging information when the program is running. This is implemented as follows:","metadata":{}},{"cell_type":"code","source":"def init_logger(log_file):\n    from logging import getLogger, INFO, FileHandler, Formatter, StreamHandler\n    logger = getLogger(__name__)\n    logger.setLevel(INFO)\n    handler1 = StreamHandler()\n    handler1.setFormatter(Formatter(\"%(message)s\"))\n    handler2 = FileHandler(filename=log_file)\n    handler2.setFormatter(Formatter(\"%(message)s\"))\n    logger.addHandler(handler1)\n    logger.addHandler(handler2)\n    return logger\n\ndef set_seed(seed=None, cudnn_deterministic=True):\n    if seed is None:\n        seed = 42\n\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = cudnn_deterministic\n    torch.backends.cudnn.benchmark = False","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.226059Z","iopub.status.busy":"2023-04-04T05:50:38.225626Z","iopub.status.idle":"2023-04-04T05:50:38.235593Z","shell.execute_reply":"2023-04-04T05:50:38.234144Z","shell.execute_reply.started":"2023-04-04T05:50:38.225965Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"    This function creates these directories using the os.makedirs() method and does not overwrite the directories if they already exist, i.e. the exist_ok=True argument means that no errors will be reported.\n    The purpose of this function is to create these directories when needed for use in subsequent code.","metadata":{}},{"cell_type":"code","source":"def make_dirs(cfg):\n    for dir in [cfg.model_dir, cfg.figures_dir, cfg.submission_dir, cfg.log_dir]:\n        os.makedirs(dir, exist_ok=True)","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.241784Z","iopub.status.busy":"2023-04-04T05:50:38.241459Z","iopub.status.idle":"2023-04-04T05:50:38.247843Z","shell.execute_reply":"2023-04-04T05:50:38.246748Z","shell.execute_reply.started":"2023-04-04T05:50:38.241756Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Initialization functions","metadata":{}},{"cell_type":"code","source":"def cfg_init(cfg, mode='train'):\n    set_seed(cfg.seed)\n    # set_env_name()\n    # set_dataset_path(cfg)\n\n    if mode == 'train':\n        make_dirs(cfg)","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.24971Z","iopub.status.busy":"2023-04-04T05:50:38.249241Z","iopub.status.idle":"2023-04-04T05:50:38.257126Z","shell.execute_reply":"2023-04-04T05:50:38.255307Z","shell.execute_reply.started":"2023-04-04T05:50:38.249675Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cfg_init(CFG)\n\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\nLogger = init_logger(log_file=CFG.log_path)\n\nLogger.info('\\n\\n-------- exp_info -----------------')\n# Logger.info(datetime.datetime.now().strftime('%Y年%m月%d日 %H:%M:%S'))","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.258747Z","iopub.status.busy":"2023-04-04T05:50:38.258387Z","iopub.status.idle":"2023-04-04T05:50:38.326583Z","shell.execute_reply":"2023-04-04T05:50:38.325136Z","shell.execute_reply.started":"2023-04-04T05:50:38.258711Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## image, mask","metadata":{}},{"cell_type":"markdown","source":"    The purpose of this code is to find a window of length CFG.in_chans, so that the subscript of the center point of the window is mid.\n    Specifically, first divide mid by 2, then subtract half of CFG.in_chans from the result to get the start subscript of the window, then add half of CFG.in_chans to mid to get the end subscript of the window.\n    Then add half of CFG.in_chans to mid to get the end of the window.\n    Finally, use the range function to generate a sequence of integers idxs from start to end-1.\n    \n  **Set the number of images to be read from 0-65, and adjust the number of input \"channels\" by changing in_chans**","metadata":{}},{"cell_type":"code","source":"def read_image_mask(fragment_id):\n\n    images = []\n\n    # idxs = range(65)\n    mid = 65 // 2\n    start = mid - CFG.in_chans // 2\n    end = mid + CFG.in_chans // 2\n    idxs = range(start, end)\n\n    for i in tqdm(idxs):\n        \n        image = cv2.imread(CFG.comp_dataset_path + f\"train/{fragment_id}/surface_volume/{i:02}.tif\", 0)\n\n        pad0 = (CFG.tile_size - image.shape[0] % CFG.tile_size)\n        pad1 = (CFG.tile_size - image.shape[1] % CFG.tile_size)\n\n        image = np.pad(image, [(0, pad0), (0, pad1)], constant_values=0)\n\n        images.append(image)\n    images = np.stack(images, axis=2)\n\n    mask = cv2.imread(CFG.comp_dataset_path + f\"train/{fragment_id}/inklabels.png\", 0)\n    mask = np.pad(mask, [(0, pad0), (0, pad1)], constant_values=0)\n\n    mask = mask.astype('float32')\n    mask /= 255.0\n    \n    return images, mask","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.329105Z","iopub.status.busy":"2023-04-04T05:50:38.328402Z","iopub.status.idle":"2023-04-04T05:50:38.33872Z","shell.execute_reply":"2023-04-04T05:50:38.337763Z","shell.execute_reply.started":"2023-04-04T05:50:38.329062Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_train_valid_dataset():\n    train_images = []\n    train_masks = []\n\n    valid_images = []\n    valid_masks = []\n    valid_xyxys = []\n\n    for fragment_id in range(1, 4):\n\n        image, mask = read_image_mask(fragment_id)\n\n        '''\n        This code is generating a set of image crop coordinates that are used to split a large size image into smaller pieces for easier processing. Where image is the original image and CFG is a set of constant parameters.\n        Specifically, the parameters of the range function in the code set the step size and range of the crop coordinates.\n        image.shape[0] and image.shape[1] denote the height and width of the image, respectively. Subtracting CFG.tile_size is to ensure that the segmented chunks are all squares with CFG.tile_size as the side length.\n        The final x1_list and y1_list are the coordinates of all the generated images, which are used for subsequent processing.\n        '''\n        x1_list = list(range(0, image.shape[1]-CFG.tile_size+1, CFG.stride))\n        y1_list = list(range(0, image.shape[0]-CFG.tile_size+1, CFG.stride))\n\n        # where y1_list and x1_list are the list of starting coordinates in the width and height directions of the large image, respectively, and CFG.tile_size is the size of each small image block.\n        # If fragment_id is equal to CFG.valid_id, the current image block and mask are stored in the list of valid_images and valid_masks\n        # and store the start and end coordinates corresponding to the current image block into the valid_xyxys list; otherwise store them into the train_images and train_masks lists.\n        for y1 in y1_list:\n            for x1 in x1_list:\n                y2 = y1 + CFG.tile_size\n                x2 = x1 + CFG.tile_size\n                # xyxys.append((x1, y1, x2, y2))\n        \n                # Place the set folder with the corresponding label in the validation data set, folder 1 or folder 2 or folder 3\n                if fragment_id == CFG.valid_id:\n                    valid_images.append(image[y1:y2, x1:x2])\n                    valid_masks.append(mask[y1:y2, x1:x2, None])\n\n                    valid_xyxys.append([x1, y1, x2, y2])\n                # unspecified data sets into the training set, 2 training sets 1 validation set\n                else:\n                    train_images.append(image[y1:y2, x1:x2])\n                    train_masks.append(mask[y1:y2, x1:x2, None])\n\n    return train_images, train_masks, valid_images, valid_masks, valid_xyxys","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.341555Z","iopub.status.busy":"2023-04-04T05:50:38.3407Z","iopub.status.idle":"2023-04-04T05:50:38.351622Z","shell.execute_reply":"2023-04-04T05:50:38.350835Z","shell.execute_reply.started":"2023-04-04T05:50:38.341513Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_images, train_masks, valid_images, valid_masks, valid_xyxys = get_train_valid_dataset()","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:50:38.354134Z","iopub.status.busy":"2023-04-04T05:50:38.353277Z","iopub.status.idle":"2023-04-04T05:51:20.631165Z","shell.execute_reply":"2023-04-04T05:51:20.630108Z","shell.execute_reply.started":"2023-04-04T05:50:38.354042Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"valid_xyxys = np.stack(valid_xyxys)","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:20.63304Z","iopub.status.busy":"2023-04-04T05:51:20.632692Z","iopub.status.idle":"2023-04-04T05:51:20.64927Z","shell.execute_reply":"2023-04-04T05:51:20.647916Z","shell.execute_reply.started":"2023-04-04T05:51:20.632997Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## dataset","metadata":{}},{"cell_type":"code","source":"import numpy as np\nfrom torch.utils.data import DataLoader, Dataset\nimport cv2\nimport torch\nimport os\nimport albumentations as A\nfrom albumentations.pytorch import ToTensorV2\nfrom albumentations import ImageOnlyTransform","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:20.651743Z","iopub.status.busy":"2023-04-04T05:51:20.651337Z","iopub.status.idle":"2023-04-04T05:51:20.658373Z","shell.execute_reply":"2023-04-04T05:51:20.656822Z","shell.execute_reply.started":"2023-04-04T05:51:20.651703Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_transforms(data, cfg):\n    if data == 'train':\n        aug = A.Compose(cfg.train_aug_list)\n    elif data == 'valid':\n        aug = A.Compose(cfg.valid_aug_list)\n\n    # print(aug)\n    return aug\n\nclass CustomDataset(Dataset):\n    def __init__(self, images, cfg, labels=None, transform=None):\n        self.images = images\n        self.cfg = cfg\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self):\n        # return len(self.df)\n        return len(self.images)\n\n    def __getitem__(self, idx):\n        image = self.images[idx]\n        label = self.labels[idx]\n\n        if self.transform:\n            data = self.transform(image=image, mask=label)\n            image = data['image']\n            label = data['mask']\n\n        return image, label","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:20.661414Z","iopub.status.busy":"2023-04-04T05:51:20.660178Z","iopub.status.idle":"2023-04-04T05:51:20.670557Z","shell.execute_reply":"2023-04-04T05:51:20.669713Z","shell.execute_reply.started":"2023-04-04T05:51:20.66137Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training set and validation set data processing","metadata":{}},{"cell_type":"code","source":"\ntrain_dataset = CustomDataset(\n    train_images, CFG, labels=train_masks, transform=get_transforms(data='train', cfg=CFG))\nvalid_dataset = CustomDataset(\n    valid_images, CFG, labels=valid_masks, transform=get_transforms(data='valid', cfg=CFG))\n\ntrain_loader = DataLoader(train_dataset,\n                          batch_size=CFG.train_batch_size,\n                          shuffle=True,\n                          num_workers=CFG.num_workers, pin_memory=True, drop_last=True,\n                          )\nvalid_loader = DataLoader(valid_dataset,\n                          batch_size=CFG.valid_batch_size,\n                          shuffle=False,\n                          num_workers=CFG.num_workers, pin_memory=True, drop_last=False)\n","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:20.673078Z","iopub.status.busy":"2023-04-04T05:51:20.67192Z","iopub.status.idle":"2023-04-04T05:51:20.683545Z","shell.execute_reply":"2023-04-04T05:51:20.682452Z","shell.execute_reply.started":"2023-04-04T05:51:20.67304Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dataset[0][0].shape","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:20.685727Z","iopub.status.busy":"2023-04-04T05:51:20.684968Z","iopub.status.idle":"2023-04-04T05:51:20.73428Z","shell.execute_reply":"2023-04-04T05:51:20.733329Z","shell.execute_reply.started":"2023-04-04T05:51:20.685689Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nplot_dataset = CustomDataset(\n    train_images, CFG, labels=train_masks)\n\ntransform = CFG.train_aug_list\ntransform = A.Compose(\n    [t for t in transform if not isinstance(t, (A.Normalize, ToTensorV2))])\n\n\nplot_count = 0\nfor i in range(1000):\n\n    image, mask = plot_dataset[i]\n    data = transform(image=image, mask=mask)\n    aug_image = data['image']\n    aug_mask = data['mask']\n\n    if mask.sum() == 0:\n        continue\n\n    fig, axes = plt.subplots(1, 4, figsize=(15, 8))\n    axes[0].imshow(image[..., 0], cmap=\"gray\")\n    axes[1].imshow(mask, cmap=\"gray\")\n    axes[2].imshow(aug_image[..., 0], cmap=\"gray\")\n    axes[3].imshow(aug_mask, cmap=\"gray\")\n    \n    plt.savefig(CFG.figures_dir + f'aug_fold_{CFG.valid_id}_{plot_count}.png')\n\n    plot_count += 1\n    if plot_count == 5:\n        break","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:20.736127Z","iopub.status.busy":"2023-04-04T05:51:20.735684Z","iopub.status.idle":"2023-04-04T05:51:26.656584Z","shell.execute_reply":"2023-04-04T05:51:26.655392Z","shell.execute_reply.started":"2023-04-04T05:51:20.736089Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del plot_dataset\ngc.collect()","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:26.658688Z","iopub.status.busy":"2023-04-04T05:51:26.658231Z","iopub.status.idle":"2023-04-04T05:51:26.872679Z","shell.execute_reply":"2023-04-04T05:51:26.871379Z","shell.execute_reply.started":"2023-04-04T05:51:26.65865Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## model","metadata":{}},{"cell_type":"markdown","source":"   **Unet convolutional network construction**\n\nThis code is initializing the image segmentation task using the Unet model from the segmentation_models_pytorch library.","metadata":{}},{"cell_type":"code","source":"class CustomModel(nn.Module):\n    def __init__(self, cfg, weight=None):\n        super().__init__()\n        self.cfg = cfg\n        \n        # The encoder_name parameter is used to specify which pre-trained encoder model to use. Here, a pre-trained model is used, so the encoder_name parameter is specified.\n        # The advantage of this is that the feature extraction capability of the existing pre-trained model can be used to accelerate the training of the model and improve the accuracy of the model. Also, different pre-trained models can be selected according to actual needs to achieve better results.\n        # We can use the pre-training weights of the image classification model to initialize the convolutional layer of the UNet network, thus improving the performance and generalization ability of the model\n        \n        # encoder_name: indicates the name of the pre-trained model used\n        # encoder_weights: indicates the weights of the pre-trained model used, usually imagenet is used.\n        # in_channels: indicates the number of channels of the input image, e.g., 3 channels for RGB images\n        # classes: the number of classifications, usually the number of pixel classifications for image segmentation tasks.\n        # activation: indicates the activation function, usually None (no activation function is used) or sigmoid (the output is between 0 and 1).\n        self.encoder = smp.Unet(\n            encoder_name=cfg.backbone, \n            encoder_weights=weight,\n            in_channels=cfg.in_chans,\n            classes=cfg.target_size,\n            activation=None,\n        )\n\n    # After the initialization, the input image can be fed into the model for forward propagation to obtain the corresponding segmentation results.\n    def forward(self, image):\n        output = self.encoder(image)\n        # output = output.squeeze(-1)\n        return output\n\n#weight=\"imagenet\"\ndef build_model(cfg, weight=\"ssl\"):\n    print('model_name', cfg.model_name)\n    print('backbone', cfg.backbone)\n\n    model = CustomModel(cfg, weight)\n\n    return model","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:26.876524Z","iopub.status.busy":"2023-04-04T05:51:26.875667Z","iopub.status.idle":"2023-04-04T05:51:26.88457Z","shell.execute_reply":"2023-04-04T05:51:26.88351Z","shell.execute_reply.started":"2023-04-04T05:51:26.876478Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## scheduler","metadata":{}},{"cell_type":"markdown","source":"[https://www.kaggle.com/code/underwearfitting/single-fold-training-of-resnet200d-lb0-965](http://)\n\nIts main role is to gradually increase the learning rate at the beginning of training to help the model converge faster. It is a subclass of the GradualWarmupScheduler class with the addition of an after_scheduler parameter for using other learning rate scheduling methods after the warm-up is over.\n    In each epoch, the get_lr() method will return the current learning rate.\n    At the beginning of training, the learning rate will be gradually increased from the initial value to the maximum value, after which the learning rate returned by after_scheduler (another learning rate update method) will be used. If after_scheduler is not provided, the current learning rate will continue to be used.\n","metadata":{}},{"cell_type":"code","source":"\nimport torch.nn as nn\nimport torch\nimport math\nimport time\nimport numpy as np\nimport torch\n\nfrom torch.optim.lr_scheduler import CosineAnnealingWarmRestarts, CosineAnnealingLR, ReduceLROnPlateau\nfrom warmup_scheduler import GradualWarmupScheduler\n\n\nclass GradualWarmupSchedulerV2(GradualWarmupScheduler):\n    def __init__(self, optimizer, multiplier, total_epoch, after_scheduler=None):\n        super(GradualWarmupSchedulerV2, self).__init__(\n            optimizer, multiplier, total_epoch, after_scheduler)\n\n    def get_lr(self):\n        if self.last_epoch > self.total_epoch:\n            if self.after_scheduler:\n                if not self.finished:\n                    self.after_scheduler.base_lrs = [\n                        base_lr * self.multiplier for base_lr in self.base_lrs]\n                    self.finished = True\n                return self.after_scheduler.get_lr()\n            return [base_lr * self.multiplier for base_lr in self.base_lrs]\n        if self.multiplier == 1.0:\n            return [base_lr * (float(self.last_epoch) / self.total_epoch) for base_lr in self.base_lrs]\n        else:\n            return [base_lr * ((self.multiplier - 1.) * self.last_epoch / self.total_epoch + 1.) for base_lr in self.base_lrs]\n\n","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:26.88703Z","iopub.status.busy":"2023-04-04T05:51:26.886749Z","iopub.status.idle":"2023-04-04T05:51:26.901457Z","shell.execute_reply":"2023-04-04T05:51:26.900426Z","shell.execute_reply.started":"2023-04-04T05:51:26.887004Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This code is a function to get the learning rate scheduler with input parameters of configuration (cfg) and optimizer (optimizer). Two learning rate schedulers are used in this function, CosineAnnealingLR and GradualWarmupSchedulerV2.\n\nAmong them, CosineAnnealingLR is a cosine annealing learning rate scheduler, whose function is to gradually reduce the learning rate during training to make the model converge more stably.\n\nGradualWarmupSchedulerV2 is a learning rate preheating scheduler, whose role is to gradually increase the learning rate at the beginning of training to avoid the model from falling into a local optimum solution at the beginning and failing to jump out.\n\nIn this function, we first define a CosineAnnealingLR scheduler, setting it to the total number of training rounds (cfg.epochs) as cycles and a minimum learning rate of 1e-7.\n\nThen, we use this scheduler as the after_scheduler parameter of GradualWarmupSchedulerV2, and also use it as the parameter of the input optimizer, multiplier indicates the learning rate multiplier during warm-up, and total_epoch indicates the number of warm-up rounds.\n\nFinally, the GradualWarmupSchedulerV2 scheduler is returned as the output.","metadata":{}},{"cell_type":"code","source":"def get_scheduler(cfg, optimizer):\n    scheduler_cosine = torch.optim.lr_scheduler.CosineAnnealingLR(\n        optimizer, cfg.epochs, eta_min=1e-7)\n    scheduler = GradualWarmupSchedulerV2(\n        optimizer, multiplier=10, total_epoch=1, after_scheduler=scheduler_cosine)\n\n    return scheduler\n\ndef scheduler_step(scheduler, avg_val_loss, epoch):\n    scheduler.step(epoch)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This line of code uses the AdamW optimizer, which takes the parameters of the model as input and uses the learning rate of lr from CFG. AdamW is a variant of the Adam optimizer that is used to update the model parameters when training a neural network.\n\nIt updates the parameters by weighted averaging the gradients of the parameters to minimize the loss function.AdamW also uses a regularization method called weight decay to prevent the model from overfitting the training data.","metadata":{}},{"cell_type":"code","source":"model = build_model(CFG)\nmodel.to(device)\n\noptimizer = AdamW(model.parameters(), lr=CFG.lr)\nscheduler = get_scheduler(CFG, optimizer)","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:26.903907Z","iopub.status.busy":"2023-04-04T05:51:26.902941Z","iopub.status.idle":"2023-04-04T05:51:30.20362Z","shell.execute_reply":"2023-04-04T05:51:30.202562Z","shell.execute_reply.started":"2023-04-04T05:51:26.903869Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## loss","metadata":{}},{"cell_type":"markdown","source":"Dice Loss is a commonly used loss function for image segmentation, which is based on the binary cross-entropy loss function and the Dice coefficient.\nThe Dice coefficient is used to evaluate the similarity between the predicted segmentation result and the true label, which is defined as twice the ratio of the intersection size of the two sets to their concatenation size.\nAnd Dice Loss is defined as 1 minus the Dice coefficient as the loss function, i.e., Dice Loss = 1 - (2 * (the intersection size of the predicted segmentation result and the true label) / (the size of the predicted segmentation result + the size of the true label)).\nDuring the training process, the optimizer minimizes the Dice Loss so as to maximize the similarity between the predicted segmentation result and the true labels.\n\n\nBCELoss is a Binary Cross Entropy Loss function, which is usually used in binary classification problems.\nIt measures the performance of the model by calculating the difference between the model prediction results and the true labels. In the\nbinary cross entropy loss function, for each sample, we denote its true label as 0 or 1, and the prediction result is also a probability value between 0 and 1.","metadata":{}},{"cell_type":"code","source":"\nDiceLoss = smp.losses.DiceLoss(mode='binary')\nBCELoss = smp.losses.SoftBCEWithLogitsLoss()\n\nalpha = 0.5\nbeta = 1 - alpha","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:30.205674Z","iopub.status.busy":"2023-04-04T05:51:30.205266Z","iopub.status.idle":"2023-04-04T05:51:30.214065Z","shell.execute_reply":"2023-04-04T05:51:30.212997Z","shell.execute_reply.started":"2023-04-04T05:51:30.205636Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is a loss function that uses the Tversky index, a metric used to evaluate the degree of similarity between two sets. In this case\nit is used to assess the degree of similarity between the predicted and true results of the model.\nalpha and beta are hyperparameters that are used to adjust the weights of the Tversky index. This loss function is used for the binary classification problem.\nThe log_loss parameter determines whether the logarithmic loss function is used.\nsmp refers to the Segmentation Models PyTorch library, which is a deep learning library for image segmentation tasks.","metadata":{}},{"cell_type":"code","source":"TverskyLoss = smp.losses.TverskyLoss(\n    mode='binary', log_loss=False, alpha=alpha, beta=beta)\n\ndef criterion(y_pred, y_true):\n    # return 0.5 * BCELoss(y_pred, y_true) + 0.5 * DiceLoss(y_pred, y_true)\n    return BCELoss(y_pred, y_true)\n    # return 0.5 * BCELoss(y_pred, y_true) + 0.5 * TverskyLoss(y_pred, y_true)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## train, val","metadata":{}},{"cell_type":"code","source":"def train_fn(train_loader, model, criterion, optimizer, device):\n    model.train()\n    '''\n    GradScaler is a tool in PyTorch for mixed-accuracy training that automatically scales gradient values\n    to avoid gradient underflow when computing at FP16 precision. In the configuration, GradScaler is enabled if AMP (Automatic Mixed Precision) is used.\n    '''\n    scaler = GradScaler(enabled=CFG.use_amp)\n    '''\n    This is a tool class for calculating the average loss. When training a neural network, it is common to calculate the losses for each batch (batches) and add them up to a total loss value.\n    In order to get the average loss, the total loss value needs to be divided by the number of batches.\n    The AverageMeter class encapsulates this process by making it easy to record the total loss value and the number of batches and to calculate the average loss value.\n    After each batch, the total loss value and batch count can be updated by calling the update() method of this class, and finally the average loss value can be obtained by calling the avg property of this class.\n    '''\n    losses = AverageMeter()\n\n    for step, (images, labels) in tqdm(enumerate(train_loader), total=len(train_loader)):\n        images = images.to(device)\n        labels = labels.to(device)\n        batch_size = labels.size(0)\n\n        with autocast(CFG.use_amp):\n            y_preds = model(images)\n            loss = criterion(y_preds, labels)\n\n        # Backpropagation and gradient calculation for the loss function\n        # scaler.scale(loss) is gradient scaling using PyTorch's GradScaler, this is to prevent gradient explosion or gradient disappearance during backpropagation. backward() is to backpropagate the parameters of the model and calculate the gradient.\n        losses.update(loss.item(), batch_size)\n        scaler.scale(loss).backward()\n        # This code is used to perform gradient cropping. During the training of a deep learning model, the gradient values may become very large, which can lead to instability of the model. To avoid this, we can use the gradient cropping method to keep the gradient value within an acceptable range.\n        grad_norm = torch.nn.utils.clip_grad_norm_(\n            model.parameters(), CFG.max_grad_norm)\n\n        # Specifically, scaler.step(optimizer) is the optimizer's gradient update on mixed precision\n        # scaler.update() is used to update the scaling factor inside the scaler.\n        # optimizer.zero_grad() is used to clear the gradient information in the optimizer so that the gradient can be recalculated in the next iteration.\n        scaler.step(optimizer)\n        scaler.update()\n        optimizer.zero_grad()\n\n    return losses.avg\n\ndef valid_fn(valid_loader, model, criterion, device, valid_xyxys, valid_mask_gt):\n    mask_pred = np.zeros(valid_mask_gt.shape)\n    mask_count = np.zeros(valid_mask_gt.shape)\n\n    model.eval()\n    losses = AverageMeter()\n\n    for step, (images, labels) in tqdm(enumerate(valid_loader), total=len(valid_loader)):\n        images = images.to(device)\n        labels = labels.to(device)\n        batch_size = labels.size(0)\n\n        with torch.no_grad():\n            y_preds = model(images)\n            loss = criterion(y_preds, labels)\n        losses.update(loss.item(), batch_size)\n\n        '''\n        This code is used to generate the prediction masks, predicting the mask value for each pixel point based on the model's output, and then assigning it to the corresponding region of the mask.\n        where y_preds is the output of the model, which is mapped to between [0,1] as the mask value by the sigmoid function;\n        valid_xyxys is the coordinate and size information of the validation set images, and start_idx and end_idx are the corresponding start and end indexes of the currently processed batch in valid_xyxys;\n        mask_pred and mask_count are the generated masks and the number of times each pixel point is assigned. Finally, the mask value is divided by the number of assignments to get the average mask value of each pixel point.\n        '''       \n        # make whole mask\n        y_preds = torch.sigmoid(y_preds).to('cpu').numpy()\n        start_idx = step*CFG.valid_batch_size\n        end_idx = start_idx + batch_size\n        for i, (x1, y1, x2, y2) in enumerate(valid_xyxys[start_idx:end_idx]):\n            mask_pred[y1:y2, x1:x2] += y_preds[i].squeeze(0)\n            mask_count[y1:y2, x1:x2] += np.ones((CFG.tile_size, CFG.tile_size))\n\n    print(f'mask_count_min: {mask_count.min()}')\n    mask_pred /= mask_count\n    return losses.avg, mask_pred","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:30.216151Z","iopub.status.busy":"2023-04-04T05:51:30.21568Z","iopub.status.idle":"2023-04-04T05:51:30.23155Z","shell.execute_reply":"2023-04-04T05:51:30.230509Z","shell.execute_reply.started":"2023-04-04T05:51:30.21611Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## metrics","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import fbeta_score\n\ndef fbeta_numpy(targets, preds, beta=0.5, smooth=1e-5):\n    \"\"\"\n    https://www.kaggle.com/competitions/vesuvius-challenge-ink-detection/discussion/397288\n    \"\"\"\n    y_true_count = targets.sum()\n    ctp = preds[targets==1].sum()\n    cfp = preds[targets==0].sum()\n    beta_squared = beta * beta\n\n    c_precision = ctp / (ctp + cfp + smooth)\n    c_recall = ctp / (y_true_count + smooth)\n    dice = (1 + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall + smooth)\n\n    return dice","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:30.23806Z","iopub.status.busy":"2023-04-04T05:51:30.237696Z","iopub.status.idle":"2023-04-04T05:51:30.248748Z","shell.execute_reply":"2023-04-04T05:51:30.247706Z","shell.execute_reply.started":"2023-04-04T05:51:30.23803Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The purpose of this code is to calculate the F-beta score between a given mask and a predicted mask and return the best threshold and the best F-beta score.\nIt calculates the F-beta score by spreading the mask and the prediction mask into a one-dimensional array and looping over a series of thresholds.\nThe optimal threshold is the threshold that maximizes the F-beta score, and the best F-beta score is the F-beta score calculated at the optimal threshold.","metadata":{}},{"cell_type":"code","source":"def calc_fbeta(mask, mask_pred):\n    mask = mask.astype(int).flatten()\n    mask_pred = mask_pred.flatten()\n\n    best_th = 0\n    best_dice = 0\n    for th in np.array(range(10, 50+1, 5)) / 100:\n        \n        # dice = fbeta_score(mask, (mask_pred >= th).astype(int), beta=0.5)\n        dice = fbeta_numpy(mask, (mask_pred >= th).astype(int), beta=0.5)\n        print(f'th: {th}, fbeta: {dice}')\n\n        if dice > best_dice:\n            best_dice = dice\n            best_th = th\n    \n    Logger.info(f'best_th: {best_th}, fbeta: {best_dice}')\n    return best_dice, best_th","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This code defines a function called calc_cv that takes two arguments: mask_gt and mask_pred. These two arguments represent the true mask (i.e., ground truth) and the model prediction mask, respectively.\nThe function internally calls the calc_fbeta function to calculate the best Dice coefficient and the best threshold, and returns them as a tuple.\nThe Dice coefficient is a commonly used measure of the similarity of two masks and takes values from 0 to 1, with higher values indicating higher similarity. The threshold is a parameter needed to convert the mask into a binarized image.","metadata":{}},{"cell_type":"code","source":"def calc_cv(mask_gt, mask_pred):\n    best_dice, best_th = calc_fbeta(mask_gt, mask_pred)\n\n    return best_dice, best_th","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## main","metadata":{}},{"cell_type":"code","source":"fragment_id = CFG.valid_id\n\nvalid_mask_gt = cv2.imread(CFG.comp_dataset_path + f\"train/{fragment_id}/inklabels.png\", 0)\nvalid_mask_gt = valid_mask_gt / 255 \n'''\nThe purpose of this code is to zero-fill the valid_mask_gt array so that the number of rows is a multiple of CFG.tile_size.\nSpecifically, pad0 and pad1 represent the number of zeros to be filled in the first and second dimensions of the valid_mask_gt array, respectively, \nso that both valid_mask_gt.shape[0] and valid_mask_gt.shape[1] are multiples of CFG.tile_size.\n'''\n\npad0 = (CFG.tile_size - valid_mask_gt.shape[0] % CFG.tile_size)\npad1 = (CFG.tile_size - valid_mask_gt.shape[1] % CFG.tile_size)\n\nvalid_mask_gt = np.pad(valid_mask_gt, [(0, pad0), (0, pad1)], constant_values=0)\n\n\n\n'''\n这行代码使用了NumPy中的`pad`函数，将`valid_mask_gt`数组在两个维度上进行了填充，\n以便与另一个数组进行操作时具有相同的形状。具体来说，`[(0, pad0), (0, pad1)]`表示在第一个维度上不进行填充（前面填0个，后面填0个），\n在第二个维度上填充`pad1`个0在后面，填充`pad0`个0在前面。\n这样做的目的是将`valid_mask_gt`数组的形状扩展到与另一个数组相同，以便进行一些操作，例如相加、相减等。\n'''\n","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:30.250925Z","iopub.status.busy":"2023-04-04T05:51:30.250303Z","iopub.status.idle":"2023-04-04T05:51:30.765669Z","shell.execute_reply":"2023-04-04T05:51:30.764621Z","shell.execute_reply.started":"2023-04-04T05:51:30.250887Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nfold = CFG.valid_id\n'''\nThis code initializes the variables for the best score based on the direction of the evaluation metric. If the evaluation indicator is of the \"minimization\" type, the initial value should be positive infinity (np.inf);\nIf the evaluation indicator is of the \"maximize\" type, then the initial value should be negative one (-1). This ensures that the value of the best score can be updated if a better score emerges during the subsequent evaluation.\n'''\nif CFG.metric_direction == 'minimize':\n    best_score = np.inf\nelif CFG.metric_direction == 'maximize':\n    best_score = -1\n\nbest_loss = np.inf\n\nfor epoch in range(CFG.epochs):\n\n    start_time = time.time()\n\n    # train\n    avg_loss = train_fn(train_loader, model, criterion, optimizer, device)\n\n    # eval\n    avg_val_loss, mask_pred = valid_fn(\n        valid_loader, model, criterion, device, valid_xyxys, valid_mask_gt)\n\n    scheduler_step(scheduler, avg_val_loss, epoch)\n\n    best_dice, best_th = calc_cv(valid_mask_gt, mask_pred)\n\n    # score = avg_val_loss\n    score = best_dice\n    elapsed = time.time() - start_time\n\n    Logger.info(\n        f'Epoch {epoch+1} - avg_train_loss: {avg_loss:.4f}  avg_val_loss: {avg_val_loss:.4f}  time: {elapsed:.0f}s')\n    # Logger.info(f'Epoch {epoch+1} - avgScore: {avg_score:.4f}')\n    Logger.info(\n        f'Epoch {epoch+1} - avgScore: {score:.4f}')\n\n    '''\n    This code is used to update the best score based on the direction of the evaluation metric (whether it is minimized or maximized). If the direction of the evaluation metric is minimize, the best score is updated to the new score if the new score is lower than the current best score.\n    If the direction of the evaluation metric is maximization, the best score is updated to the new score if the new score is higher than the current best score. This allows tracking the best model during training and using that model during evaluation.\n    '''\n    if CFG.metric_direction == 'minimize':\n        update_best = score < best_score\n    elif CFG.metric_direction == 'maximize':\n        update_best = score > best_score\n\n    if update_best:\n        best_loss = avg_val_loss\n        best_score = score\n\n        Logger.info(\n            f'Epoch {epoch+1} - Save Best Score: {best_score:.4f} Model')\n        Logger.info(\n            f'Epoch {epoch+1} - Save Best Loss: {best_loss:.4f} Model')\n        \n        torch.save({'model': model.state_dict(),\n                    'preds': mask_pred},\n                    CFG.model_dir + f'{CFG.model_name}_fold{fold}_best.pth')","metadata":{"execution":{"iopub.execute_input":"2023-04-04T05:51:30.767837Z","iopub.status.busy":"2023-04-04T05:51:30.767438Z","iopub.status.idle":"2023-04-04T06:51:14.117016Z","shell.execute_reply":"2023-04-04T06:51:14.115452Z","shell.execute_reply.started":"2023-04-04T05:51:30.767795Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"check_point = torch.load(\n    CFG.model_dir + f'{CFG.model_name}_fold{fold}_{CFG.inf_weight}.pth', map_location=torch.device('cpu'))","metadata":{"execution":{"iopub.execute_input":"2023-04-04T06:51:14.122575Z","iopub.status.busy":"2023-04-04T06:51:14.122159Z","iopub.status.idle":"2023-04-04T06:51:17.552043Z","shell.execute_reply":"2023-04-04T06:51:17.550364Z","shell.execute_reply.started":"2023-04-04T06:51:14.122524Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mask_pred = check_point['preds']","metadata":{"execution":{"iopub.execute_input":"2023-04-04T06:51:17.554159Z","iopub.status.busy":"2023-04-04T06:51:17.553755Z","iopub.status.idle":"2023-04-04T06:51:17.591872Z","shell.execute_reply":"2023-04-04T06:51:17.590665Z","shell.execute_reply.started":"2023-04-04T06:51:17.554119Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"best_dice, best_th  = calc_fbeta(valid_mask_gt, mask_pred)","metadata":{"execution":{"iopub.execute_input":"2023-04-04T06:51:17.594773Z","iopub.status.busy":"2023-04-04T06:51:17.593945Z","iopub.status.idle":"2023-04-04T06:51:23.216932Z","shell.execute_reply":"2023-04-04T06:51:23.215813Z","shell.execute_reply.started":"2023-04-04T06:51:17.594736Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(1, 3, figsize=(15, 8))\naxes[0].imshow(valid_mask_gt)\naxes[1].imshow(mask_pred)\naxes[2].imshow((mask_pred>=best_th).astype(int))","metadata":{"execution":{"iopub.execute_input":"2023-04-04T06:51:23.218989Z","iopub.status.busy":"2023-04-04T06:51:23.218597Z","iopub.status.idle":"2023-04-04T06:51:37.221509Z","shell.execute_reply":"2023-04-04T06:51:37.220561Z","shell.execute_reply.started":"2023-04-04T06:51:23.21895Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.hist(mask_pred.flatten(), bins=20)","metadata":{"execution":{"iopub.execute_input":"2023-04-04T06:51:37.223885Z","iopub.status.busy":"2023-04-04T06:51:37.22286Z","iopub.status.idle":"2023-04-04T06:51:38.516812Z","shell.execute_reply":"2023-04-04T06:51:38.515375Z","shell.execute_reply.started":"2023-04-04T06:51:37.223843Z"}},"execution_count":null,"outputs":[]}]}