The approach is just a fun experiment, implemented in tensorflow/keras. This idea probably can generate a more meaningful modeling approach. Please consider that before looking into this code.
\n
\n\n---\n\n**Version 4+ Update**\n- Update latest TF `2.12` compatible\n- **Multi-GPU** and **TPU-VM** Support\n\n**Version 3+ Update**\n- Update latest TF `2.6` compatible. \n- Fix some bugs (`steps`, `epochs`)\n\n**Version 2 Update**\n\n- Fix some bugs \n- Train with more epoch \n\n---\n\n# About \n\nThe idea is discussed [here, method 1](https://www.kaggle.com/c/siim-covid19-detection/discussion/245323), that is to **add an extras supervison** to the classificaiton mdoel by adding the **segmentation loss**. Basically, we make the use of the bounding box information to crop the anomaly region from the chest x-ray samples and further use those cropped samples as an additional targets for the classifier model at **cetain layer**. Here is the general overview of this approach. \n\n\n\n
\n Figure 1: Cropped ROI Segment for the Additional Supervision to the Classifier.\n
","metadata":{}},{"cell_type":"markdown","source":"For input image of shape `512`, we will get `16` dimention of `n` depth feature maps at the last layer of base model. As the segmentation target would be one channel and downsampled to match last layer feature maps, we need to add a simple **mask head**. ","metadata":{}},{"cell_type":"code","source":"import os, math, warnings\nimport psutil, random \nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'\nwarnings.simplefilter(action='ignore', category=FutureWarning)\n\nimport numpy as np \nimport pandas as pd \nimport matplotlib.pyplot as plt\n\nimport cv2; print(cv2.__version__)\nimport tensorflow as tf; print(tf.__version__)\nfrom tensorflow import keras\nfrom tensorflow.python.client import device_lib","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:36.071275Z","iopub.execute_input":"2023-04-28T19:17:36.071771Z","iopub.status.idle":"2023-04-28T19:17:45.263867Z","shell.execute_reply.started":"2023-04-28T19:17:36.071726Z","shell.execute_reply":"2023-04-28T19:17:45.261888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def set_tpu(mixed_precision=True):\n try: \n tpu = tf.distribute.cluster_resolver.TPUClusterResolver.connect() \n \n if mixed_precision:\n keras.mixed_precision.set_global_policy(\n \"mixed_bfloat16\"\n )\n else:\n keras.mixed_precision.set_global_policy(\n keras.backend.floatx()\n )\n \n tf.config.set_soft_device_placement(True)\n strategy = tf.distribute.TPUStrategy(tpu)\n physical_devices = tf.config.list_logical_devices('TPU')\n return (strategy, physical_devices)\n except:\n return False","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:45.266024Z","iopub.execute_input":"2023-04-28T19:17:45.267063Z","iopub.status.idle":"2023-04-28T19:17:45.274570Z","shell.execute_reply.started":"2023-04-28T19:17:45.267022Z","shell.execute_reply":"2023-04-28T19:17:45.273370Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def set_cpu_gpus(mixed_precision=True, set_jit=False):\n try: \n # printed out the detected devices\n list_ld = device_lib.list_local_devices()\n for dev in list_ld: \n print(dev.name,dev.memory_limit)\n \n # get the lisf of physical devices\n physical_devices = tf.config.list_physical_devices(\n 'GPU' if len(list_ld) - 1 else 'CPU'\n )\n \n # For GPU devices, configure related stuff\n if 'GPU' in physical_devices[-1]:\n tf.config.optimizer.set_jit(set_jit)\n \n if mixed_precision:\n keras.mixed_precision.set_global_policy(\n \"mixed_float16\"\n )\n else:\n keras.mixed_precision.set_global_policy(\n keras.backend.floatx()\n )\n \n for pd in physical_devices:\n tf.config.experimental.set_memory_growth(\n pd, True\n )\n \n strategy = tf.distribute.MirroredStrategy()\n return (strategy, physical_devices)\n except: \n raise ValueError('No Device Detected!')","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:45.277076Z","iopub.execute_input":"2023-04-28T19:17:45.277345Z","iopub.status.idle":"2023-04-28T19:17:45.296261Z","shell.execute_reply.started":"2023-04-28T19:17:45.277320Z","shell.execute_reply":"2023-04-28T19:17:45.295233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mxp = True\njit = True\n\nstrategy, physical_devices = set_tpu(mixed_precision=mxp) or set_cpu_gpus(mixed_precision=mxp, set_jit=jit)\nphysical_devices, tf.__version__","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:45.300304Z","iopub.execute_input":"2023-04-28T19:17:45.300601Z","iopub.status.idle":"2023-04-28T19:17:49.344716Z","shell.execute_reply.started":"2023-04-28T19:17:45.300574Z","shell.execute_reply":"2023-04-28T19:17:49.343736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preprocess\n\nHere is the quick data pre-process to make the data suitable for the training data. For extensive data data analysis, please refer to the following scripts.\n\n- [SIIM Covid-19: Box Detect & .dcm metadata](https://www.kaggle.com/andradaolteanu/siim-covid-19-box-detect-dcm-metadata)\n- [SIIM COVID-19 Detection: EDA + Data Augmentation](https://www.kaggle.com/ruchi798/siim-covid-19-detection-eda-data-augmentation)","metadata":{}},{"cell_type":"code","source":"study_df = pd.read_csv('../input/siim-covid19-detection/train_study_level.csv'); print(study_df.shape)\nstudy_df['StudyInstanceUID'] = study_df['id'].apply(lambda x: x.replace('_study', ''))\ndel study_df['id']\n\ndef hot_to_sparse(row):\n return(row.index[row.apply(lambda x: x==1)][0])\nstudy_df['diagnosis'] = study_df.apply(lambda row:hot_to_sparse(row), axis=1)\ncls = {\n 'Typical Appearance':1, \n 'Negative for Pneumonia':2, \n 'Indeterminate Appearance':3, \n 'Atypical Appearance':4, \n}\nstudy_df['sparse_gt'] = study_df.diagnosis.map(cls) \n\nimage_df = pd.read_csv('../input/siim-covid19-detection/train_image_level.csv'); print(image_df.shape)\ntrain = image_df.merge(study_df, on='StudyInstanceUID')\ntrain['id'] = train['id'].apply(lambda x: x.replace('_image', ''))\ndisplay(train.head()); print(train.shape)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:49.346411Z","iopub.execute_input":"2023-04-28T19:17:49.347010Z","iopub.status.idle":"2023-04-28T19:17:50.534567Z","shell.execute_reply.started":"2023-04-28T19:17:49.346972Z","shell.execute_reply":"2023-04-28T19:17:50.533348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ROI Segment: Cropped Bounding Box","metadata":{}},{"cell_type":"code","source":"def vis(path1, path2, n_images, is_random=True, figsize=(16, 16)):\n '''\n https://gist.github.com/innat/00de7561033ba373745d425c6da7bf8c\n '''\n image_names = os.listdir(path1)\n masks_names = os.listdir(path2)\n \n for i in range(n_images):\n if is_random:\n image_name = random.choice(masks_names)\n masks_name = image_name\n else:\n image_name = masks_names[i]\n masks_name = masks_names[i]\n \n img = cv2.resize(cv2.imread(os.path.join(path1, image_name)), (512, 512))\n msk = cv2.resize(cv2.imread(os.path.join(path2, masks_name)), (512, 512))\n \n plt.figure(figsize=(20,20))\n plt.subplot(121); plt.imshow(img);\n plt.subplot(122); plt.imshow(msk);\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:50.536204Z","iopub.execute_input":"2023-04-28T19:17:50.536903Z","iopub.status.idle":"2023-04-28T19:17:50.546689Z","shell.execute_reply.started":"2023-04-28T19:17:50.536864Z","shell.execute_reply":"2023-04-28T19:17:50.545732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"base_path = '../input/covid19-detection-890pxpng-study'\nTRAIN_IMG_PATH = os.path.join(base_path, 'train/')\nTRAIN_MSK_PATH = os.path.join(base_path, 'ROI Mask/')\nvis(TRAIN_IMG_PATH, TRAIN_MSK_PATH, 5, is_random=True)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:50.548326Z","iopub.execute_input":"2023-04-28T19:17:50.548724Z","iopub.status.idle":"2023-04-28T19:17:54.229444Z","shell.execute_reply.started":"2023-04-28T19:17:50.548687Z","shell.execute_reply":"2023-04-28T19:17:54.228389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import StratifiedKFold\n\nskf = StratifiedKFold(n_splits=5, shuffle=True, random_state=101)\nfor index, (train_index, val_index) in enumerate(\n skf.split(X=train.index, y=train.sparse_gt)\n):\n train.loc[val_index, 'fold'] = index\n \nprint(train.groupby(['fold', train.sparse_gt]).size())","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:54.230962Z","iopub.execute_input":"2023-04-28T19:17:54.231528Z","iopub.status.idle":"2023-04-28T19:17:54.913804Z","shell.execute_reply.started":"2023-04-28T19:17:54.231476Z","shell.execute_reply":"2023-04-28T19:17:54.912637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Covid-19 `Keras-Sequence` Data Generator","metadata":{}},{"cell_type":"code","source":"import albumentations as A \n\n# For Validation \ndef albu_transforms_train(data_resize): \n return A.Compose([\n A.Resize(data_resize, data_resize)\n # Can be added more augmentation ... \n ], p=1.)\n\n\n# For Validation \ndef albu_transforms_valid(data_resize): \n return A.Compose([\n A.Resize(data_resize, data_resize)\n ], p=1.)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:54.915471Z","iopub.execute_input":"2023-04-28T19:17:54.915980Z","iopub.status.idle":"2023-04-28T19:17:55.935964Z","shell.execute_reply.started":"2023-04-28T19:17:54.915939Z","shell.execute_reply":"2023-04-28T19:17:55.934765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Covid19Generator(keras.utils.Sequence):\n def __init__(\n self, \n img_path, \n msk_path, \n data, \n batch_size, \n random_state, \n idim, mdim, \n shuffle=True, \n transform=None, \n is_train=False\n ):\n self.idim = idim\n self.mdim = mdim \n self.data = data\n self.shuffle = shuffle\n self.img_path = img_path\n self.msk_path = msk_path\n self.is_train = is_train\n self.augment = transform\n self.batch_size = batch_size\n self.random_state = random_state\n self.list_idx = data.index.values\n self.label = self.data[\n [\n 'Negative for Pneumonia', 'Typical Appearance', \n 'Indeterminate Appearance', 'Atypical Appearance']\n ] if self.is_train else np.nan\n self.on_epoch_end()\n \n def __len__(self):\n batch_len = int(np.floor(len(self.list_idx) / self.batch_size))\n if batch_len*self.batch_size < len(self.data):\n batch_len += 1\n return batch_len\n \n def __getitem__(self, index):\n batch_idx = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n idx = [self.list_idx[k] for k in batch_idx]\n \n Data = np.zeros((self.batch_size,) + self.idim + (3,), dtype=\"float32\")\n Mask = np.zeros((self.batch_size,) + self.mdim + (1,), dtype=\"float32\")\n Target = np.zeros((self.batch_size, 4), dtype = np.float32)\n\n for i, k in enumerate(idx):\n # load the image file using cv2\n image = cv2.imread(self.img_path + self.data['id'][k] + '.png')[:, :, [2, 1, 0]]\n mask = cv2.imread(self.msk_path + self.data['id'][k] + '.png', 0)\n \n try:\n mask = cv2.resize(mask, self.mdim)[:, :, np.newaxis]\n except:\n mask = np.zeros_like(cv2.resize(image[:,:,:1], self.mdim))[:, :, np.newaxis]\n \n res = self.augment(image=image)\n image = res['image']\n \n # mask normalization must\n mask = np.where(mask == 0, 0, 1)\n # mask = mask.astype(np.float32)/255.0 \n\n # assign \n if self.is_train:\n Data[i,] = image\n Mask[i,] = mask\n Target[i,] = self.label.iloc[k,].values \n else:\n Data[i,] = image \n \n inps = {'input': Data}\n outs = {'clss': Target, 'segg': Mask}\n return inps, outs\n \n def on_epoch_end(self):\n self.indices = np.arange(len(self.list_idx))\n if self.shuffle:\n np.random.seed(self.random_state)\n np.random.shuffle(self.indices)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:55.940301Z","iopub.execute_input":"2023-04-28T19:17:55.940645Z","iopub.status.idle":"2023-04-28T19:17:55.956893Z","shell.execute_reply.started":"2023-04-28T19:17:55.940614Z","shell.execute_reply":"2023-04-28T19:17:55.955807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Viz**","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt \nfrom pylab import rcParams\n\n# helper function to plot sample \ndef plot_imgs(dataset_show, row, col):\n rcParams['figure.figsize'] = 20,10\n for i in range(row):\n f, ax = plt.subplots(1,col)\n for p in range(col):\n idx = np.random.randint(0, len(dataset_show))\n img, label = dataset_show[idx]\n ax[p].grid(False)\n ax[p].imshow(label['segg'][0], cmap='gray')\n ax[p].set_title(label['clss'][0])\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:55.958574Z","iopub.execute_input":"2023-04-28T19:17:55.959175Z","iopub.status.idle":"2023-04-28T19:17:55.977076Z","shell.execute_reply.started":"2023-04-28T19:17:55.959136Z","shell.execute_reply":"2023-04-28T19:17:55.976073Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fold = 0\nimg_size = 512\nmsk_sizze = 16\nbatch_size = 32 * strategy.num_replicas_in_sync","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:55.978413Z","iopub.execute_input":"2023-04-28T19:17:55.978888Z","iopub.status.idle":"2023-04-28T19:17:55.984620Z","shell.execute_reply.started":"2023-04-28T19:17:55.978850Z","shell.execute_reply":"2023-04-28T19:17:55.983364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fold_generator(fold):\n # for way one - data generator\n train_labels = train[train.fold != fold].reset_index(drop=True)\n val_labels = train[train.fold == fold].reset_index(drop=True)\n\n train_generator = Covid19Generator(\n TRAIN_IMG_PATH, TRAIN_MSK_PATH,\n train_labels, \n batch_size, \n 1234, \n (img_size, img_size), \n (msk_sizze, msk_sizze),\n shuffle = True, \n is_train = True,\n transform = albu_transforms_train(img_size)\n )\n \n val_generator = Covid19Generator(\n TRAIN_IMG_PATH, TRAIN_MSK_PATH,\n val_labels, \n batch_size, \n 1234, \n (img_size, img_size), \n (msk_sizze, msk_sizze),\n shuffle = False, \n is_train = True,\n transform = albu_transforms_valid(img_size)\n )\n\n return train_generator, val_generator, train_labels, val_labels\n\n\n# first fold \ntrain_gen, val_gen, train_len, val_len = fold_generator(fold)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:55.986131Z","iopub.execute_input":"2023-04-28T19:17:55.987172Z","iopub.status.idle":"2023-04-28T19:17:56.002198Z","shell.execute_reply.started":"2023-04-28T19:17:55.987134Z","shell.execute_reply":"2023-04-28T19:17:56.001077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_imgs(train_gen, 5, 4) # plotting only 16x mask","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:17:56.003618Z","iopub.execute_input":"2023-04-28T19:17:56.004034Z","iopub.status.idle":"2023-04-28T19:18:48.800316Z","shell.execute_reply.started":"2023-04-28T19:17:56.003998Z","shell.execute_reply":"2023-04-28T19:18:48.799307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model \n\nFor demonstration purpose, we pick `EfficientNet` model with image shape `512 x 512`. And for segmentation prediction maps, we will pick layer `top_activation` and pass it to **maks head** branch further processsing. \n\n```python\nfor layer in applications.EfficientNetB1(input_shape=(512, 512, 3), weights=None).layers[-10:]:\n print(layer.name, layer.output_shape)\n \n[out]:\nblock7b_project_conv (None, 16, 16, 320)\nblock7b_project_bn (None, 16, 16, 320)\nblock7b_drop (None, 16, 16, 320)\nblock7b_add (None, 16, 16, 320)\ntop_conv (None, 16, 16, 1280)\ntop_bn (None, 16, 16, 1280)\ntop_activation (None, 16, 16, 1280)\n\n# moved out for top free\navg_pool (None, 1280)\ntop_dropout (None, 1280)\npredictions (None, 1000)\n```","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras import Model \nfrom tensorflow.keras import Sequential \nfrom tensorflow.keras import Input \nfrom tensorflow.keras import layers \nfrom tensorflow.keras import applications \n\nclass CovidNet(Model):\n def __init__(self):\n super(CovidNet, self).__init__()\n self.base = applications.EfficientNetB1(\n input_shape=(512, 512, 3),\n include_top=False,\n weights='imagenet'\n )\n \n # desired model \n self.base = Model(\n [self.base.inputs], \n [\n self.base.get_layer('top_activation').output, \n self.base.output\n ]\n )\n \n # tail / head for the classifier \n self.tail = Sequential(\n [\n layers.GlobalAveragePooling2D(),\n layers.Dropout(0.2),\n layers.BatchNormalization(),\n layers.Dense(4),\n layers.Softmax()\n ]\n )\n \n # tail / head for the mask \n self.msk = Sequential(\n [\n layers.Conv2D(\n filters=512, \n kernel_size=(1, 1), \n strides=(1, 1), \n padding=\"same\"\n ),\n layers.ReLU(),\n layers.BatchNormalization(),\n layers.Conv2D(filters=1, kernel_size=(1,1), padding=\"same\")\n ]\n )\n\n # feed-forwarding \n def call(self, inputs, training=None, **kwargs):\n segg, clss = self.base(inputs['input'])\n\n return {\n 'clss': self.tail(clss), \n 'segg': self.msk(segg)\n }\n \n\nwith strategy.scope():\n model = CovidNet()\n model.build(input_shape={'input': (None, 512, 512, 3)})\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:18:48.802653Z","iopub.execute_input":"2023-04-28T19:18:48.803360Z","iopub.status.idle":"2023-04-28T19:18:57.448369Z","shell.execute_reply.started":"2023-04-28T19:18:48.803318Z","shell.execute_reply":"2023-04-28T19:18:57.447255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"code","source":"# https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/schedules\nfrom tensorflow.keras.optimizers.schedules import LearningRateSchedule, ExponentialDecay\n\nclass WarmupLearningRateSchedule(LearningRateSchedule):\n \"\"\"Provides a variety of learning rate decay schedules with warm up.\"\"\"\n\n def __init__(\n self,\n initial_lr,\n steps_per_epoch=None,\n lr_decay_type='exponential',\n decay_factor=0.97,\n decay_epochs=2.4,\n total_steps=None,\n warmup_epochs=5,\n minimal_lr=0\n ):\n super(WarmupLearningRateSchedule, self).__init__()\n self.initial_lr = initial_lr\n self.steps_per_epoch = steps_per_epoch\n self.lr_decay_type = lr_decay_type\n self.decay_factor = decay_factor\n self.decay_epochs = decay_epochs\n self.total_steps = total_steps\n self.warmup_epochs = warmup_epochs\n self.minimal_lr = minimal_lr\n\n def __call__(self, step):\n if self.lr_decay_type == 'exponential':\n assert self.steps_per_epoch is not None\n decay_steps = self.steps_per_epoch * self.decay_epochs\n lr = ExponentialDecay(\n self.initial_lr, \n decay_steps, \n self.decay_factor, staircase=True\n )(step)\n elif self.lr_decay_type == 'cosine':\n assert self.total_steps is not None\n lr = 0.5 * self.initial_lr * (\n 1 + tf.cos(np.pi * tf.cast(step, tf.float32) / self.total_steps))\n \n elif self.lr_decay_type == 'linear':\n assert self.total_steps is not None\n lr = (1.0 - tf.cast(step, tf.float32) / self.total_steps) * self.initial_lr\n elif self.lr_decay_type == 'constant':\n lr = self.initial_lr\n else:\n assert False, 'Unknown lr_decay_type : %s' % self.lr_decay_type\n\n if self.minimal_lr:\n lr = tf.math.maximum(lr, self.minimal_lr)\n\n if self.warmup_epochs:\n warmup_steps = int(self.warmup_epochs * self.steps_per_epoch)\n warmup_lr = (\n self.initial_lr * tf.cast(step, tf.float32) /\n tf.cast(warmup_steps, tf.float32))\n lr = tf.cond(step < warmup_steps, lambda: warmup_lr, lambda: lr)\n\n return lr\n\n def get_config(self):\n return {\n 'initial_lr': self.initial_lr,\n 'steps_per_epoch': self.steps_per_epoch,\n 'lr_decay_type': self.lr_decay_type,\n 'decay_factor': self.decay_factor,\n 'decay_epochs': self.decay_epochs,\n 'total_steps': self.total_steps,\n 'warmup_epochs': self.warmup_epochs,\n 'minimal_lr': self.minimal_lr,\n }","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:18:57.450164Z","iopub.execute_input":"2023-04-28T19:18:57.450901Z","iopub.status.idle":"2023-04-28T19:18:57.465370Z","shell.execute_reply.started":"2023-04-28T19:18:57.450862Z","shell.execute_reply":"2023-04-28T19:18:57.464322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"steps_per_epoch = np.ceil(float(len(train_len)) / batch_size) \nvalidation_steps = np.ceil(float(len(val_gen)) / batch_size) \nepochs = 20\n\nlr_sched = 'cosine'\nlr_base = 0.016\nlr_min=0\nlr_decay_epoch = 2.4\nlr_warmup_epoch = 5\nlr_decay_factor = 0.97\n\nscaled_lr = lr_base * (batch_size / 256.0)\nscaled_lr_min = lr_min * (batch_size / 256.0)\ntotal_steps = steps_per_epoch * epochs\n\nlearning_rate = WarmupLearningRateSchedule(\n scaled_lr,\n steps_per_epoch=steps_per_epoch,\n decay_epochs=lr_decay_epoch,\n warmup_epochs=lr_warmup_epoch,\n decay_factor=lr_decay_factor,\n lr_decay_type=lr_sched,\n total_steps=total_steps,\n minimal_lr=scaled_lr_min\n)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:18:57.467029Z","iopub.execute_input":"2023-04-28T19:18:57.467463Z","iopub.status.idle":"2023-04-28T19:18:57.479922Z","shell.execute_reply.started":"2023-04-28T19:18:57.467424Z","shell.execute_reply":"2023-04-28T19:18:57.478823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Compile and Run ","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras import losses \nfrom tensorflow.keras import metrics\nfrom tensorflow.keras import optimizers\n\nwith strategy.scope():\n # bind all\n model.compile(\n loss = {\n 'clss': losses.CategoricalCrossentropy(\n label_smoothing=0, from_logits=False\n ),\n 'segg': losses.BinaryCrossentropy(\n from_logits=True\n )\n },\n\n metrics = {\n 'clss': [\n metrics.AUC(curve='ROC'),\n metrics.SpecificityAtSensitivity(0.70, name='@sensitivity')\n ]\n },\n\n optimizer = optimizers.Adam(learning_rate)\n )","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:18:57.481271Z","iopub.execute_input":"2023-04-28T19:18:57.482334Z","iopub.status.idle":"2023-04-28T19:18:57.588715Z","shell.execute_reply.started":"2023-04-28T19:18:57.482296Z","shell.execute_reply":"2023-04-28T19:18:57.587769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# list of call backs \nfrom tensorflow.keras import callbacks\n\ncallback_list = [\n callbacks.ModelCheckpoint(\n filepath='model.{epoch:02d}-{val_loss:.4f}.h5', \n save_freq='epoch', \n verbose=0, \n monitor='val_loss', \n save_weights_only=True, \n save_best_only=True\n ) \n]","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:18:57.589992Z","iopub.execute_input":"2023-04-28T19:18:57.590354Z","iopub.status.idle":"2023-04-28T19:18:57.598068Z","shell.execute_reply.started":"2023-04-28T19:18:57.590318Z","shell.execute_reply":"2023-04-28T19:18:57.595008Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# fitter \nmodel.fit(\n train_gen, \n steps_per_epoch=steps_per_epoch,\n validation_steps=validation_steps,\n validation_data=val_gen, \n callbacks=callback_list, \n workers=psutil.cpu_count() - 1, \n verbose=1,\n epochs=epochs\n)","metadata":{"execution":{"iopub.status.busy":"2023-04-28T19:18:57.599716Z","iopub.execute_input":"2023-04-28T19:18:57.600147Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Resources \n- [Mask Spatial Supervision](https://www.kaggle.com/ipythonx/blending-mask-with-x-ray-for-spatial-supervision/notebook?scriptVersionId=66218468): Try to experiemnt by blending with bounding box mask information, (method 1 + method 2).\n- [Add Attention Mechansim](https://www.kaggle.com/ipythonx/tf-keras-ranzcr-multi-attention-efficientnet): Try to add attention function to build hybrid model. \n- [Out-of-Fold Evaluation](https://www.kaggle.com/ipythonx/optimizing-metrics-out-of-fold-weights-ensemble): Compute the oof each fold and compare the match and non-match prediction in order to emphasize on the weak or minor cases. ","metadata":{}}]}