{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## RSNA-MICCAI Brain Tumor Radiogenomic Classification\nThis notebook can be used to finetune a pretrained model on the RSNA-MICCAI dataset. It uses MedicalNet[[1]](https://github.com/Tencent/MedicalNet) as a starting point.\n\n## Table of Contents\n1. [Setup](#setup) \n    1. [Configuration](#config) \n    2. [Imports](#imports) \n    3. [Helpers](#helpers) \n    4. [System setup](#env) \n2. [Model Definition](#model)\n3. [Dataset](#data)\n4. [Trainer](#trainer)\n5. [Training](#training)\n\n### Trusting CV vs LB\nIn short: Don't trust CV. It is still unclear if our models can even find a meaningful signal to predict the MGMT value. I have not found a valid cross validation scheme as of yet. This is refelected in the subpar learning curve.\n\n### Notes\nThe competition data consists of MRI scans in 4 different expressions (FLAIR, T1w, T1Gd, T2), each of which can have different orientations and resolutions. If we want to train a model that leverages all of this information, we would most likely have to superimpose these scans to preserve the spacial information. A promising approach for this was already introduced by @boojum[[2]](https://www.kaggle.com/boojum/connecting-voxel-spaces). However we are only interested in a baseline for now, so we will only use the FLAIR scans and also discard additional information from the DICOM files. This allows us to use the converted png dataset by MICCAI [[3]](https://www.kaggle.com/jonathanbesomi/rsna-miccai-png), giving us a simple and fast preprocessing pipeline.\n\n**Thank you to all the domain experts for your valuable insights.**\n\n\n### EDA\nThere are several excellent kernels out there. I mainly referred to these kernels and discussions:\n - https://www.kaggle.com/ayuraj/brain-tumor-eda-and-interactive-viz-with-w-b\n - https://www.kaggle.com/ihelon/brain-tumor-eda-with-animations-and-modeling\n - https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/253736\n - https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/253488\n \n\n## TODO\nThere is a lot of room for improvement, for example:\n - data augmentation\n - preprocessing, e.g. center cropping\n - adding T1w, T1Gd, T2 channels\n - architecture choices\n - hyperparameter optimization (this notebook uses default params)\n\n## References\n[1]\nChen, Sihong and Ma, Kai and Zheng, Yefeng (2019). \nMed3D: Transfer Learning for 3D Medical Image Analysis.\narXiv preprint arXiv:1904.00625\n\n[2] https://www.kaggle.com/boojum/connecting-voxel-spaces\n\n[3] https://www.kaggle.com/jonathanbesomi/rsna-miccai-png\n","metadata":{}},{"cell_type":"markdown","source":"## Setup <a class=\"anchor\" id=\"setup\"></a>\n### Configuration <a class=\"anchor\" id=\"config\"></a>","metadata":{}},{"cell_type":"code","source":"class CFG:\n    # system\n    seed=42\n    no_cuda=False\n    \n    # model\n    model_name = 'resnet_34_23dataset'\n    use_pretrained = True\n    \n    # cohort to use\n    cohort = 'FLAIR'\n    \n    img_size=256\n    \nclass TrainerConfig:\n    num_epochs = 15\n    batch_size = 8\n    gradient_accumulation_steps = 1\n\n    # optimizer\n    lr = 3e-2\n    warm_up_ratio = 0.1\n    weight_decay = 0.0\n\n    # log every log_steps to wandb\n    log_steps = 20\n\n    # environment\n    device = 'cuda'","metadata":{"execution":{"iopub.status.busy":"2021-08-13T05:17:56.782651Z","iopub.execute_input":"2021-08-13T05:17:56.783358Z","iopub.status.idle":"2021-08-13T05:17:56.795296Z","shell.execute_reply.started":"2021-08-13T05:17:56.783262Z","shell.execute_reply":"2021-08-13T05:17:56.793722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Imports <a class=\"anchor\" id=\"imports\"></a>\n\nAdd MedicalNet to sys path and import libraries","metadata":{}},{"cell_type":"code","source":"# add medicalnet to path\nimport sys\nimport os\nos.system('cp -r ../input/medicalnet-with-weights/MedicalNet MedicalNet')\nos.system('touch MedicalNet/__init__.py')\nsys.path.append('/kaggle/working/MedicalNet')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-08-13T05:17:56.79747Z","iopub.execute_input":"2021-08-13T05:17:56.798028Z","iopub.status.idle":"2021-08-13T05:18:01.045458Z","shell.execute_reply.started":"2021-08-13T05:17:56.79797Z","shell.execute_reply":"2021-08-13T05:18:01.043812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# system\nimport sys\nimport os\nimport random\nimport time\n\n# data processing\nimport numpy as np\nimport pandas as pd\n\n# dl librarires\n## torch\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import DataLoader, Dataset\n\n## transformers\nfrom transformers import get_cosine_schedule_with_warmup\nfrom transformers.optimization import AdamW\n\n## sklearn\nfrom sklearn import model_selection\nfrom sklearn.metrics import roc_auc_score\n\n# wandb for logging\nimport wandb\nfrom kaggle_secrets import UserSecretsClient\n\n# plotting\nimport cv2\nimport matplotlib.pyplot as plt\nfrom IPython import display\n\n# med\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n\nimport MedicalNet.model","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-08-13T05:18:01.048672Z","iopub.execute_input":"2021-08-13T05:18:01.049267Z","iopub.status.idle":"2021-08-13T05:18:10.805197Z","shell.execute_reply.started":"2021-08-13T05:18:01.049182Z","shell.execute_reply":"2021-08-13T05:18:10.804045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Helpers <a class=\"anchor\" id=\"helpers\"></a>\nSome helper functions. `Struct` is only needed for MedicalNet  \n\nFunctions: `seed_everything`, `get_folds`\n\nClasses: `Struct`","metadata":{}},{"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n    \n# MedicalNet expect a class to hold hyperparameters\nclass Struct:\n    def __init__(self, entries):\n        self.__dict__.update(entries)\n\n\n# create 5 stratified folds, optionally pass rng generator \ndef get_folds(df, rng=42):\n    # read df\n    if isinstance(df, str):\n        train_df = pd.read_csv(df)\n    elif isinstance(df, pd.DataFrame):\n        train_df = df\n    else:\n        print(f\"Didn't understand data type for df: {type(df)}\")\n\n    \n\n    # shuffle before split\n    train_df = train_df.sample(frac=1, random_state=rng).reset_index(drop=True)\n\n    # get stratified splits with sklearn\n    kf = model_selection.StratifiedKFold(n_splits=5)\n    # train_df.loc[:, 'fold'] = 0\n    for fold_idx, (_, indices_test) in enumerate(kf.split(X=train_df.BraTS21ID, y=train_df.MGMT_value)):\n        train_df.loc[indices_test, 'fold'] = fold_idx\n    return train_df","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-08-13T05:18:10.807432Z","iopub.execute_input":"2021-08-13T05:18:10.807863Z","iopub.status.idle":"2021-08-13T05:18:10.818939Z","shell.execute_reply.started":"2021-08-13T05:18:10.807815Z","shell.execute_reply":"2021-08-13T05:18:10.817644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### System setup <a class=\"anchor\" id=\"env\"></a>\nInitialize random seed, setup cuda and set environment for wandb.\n\nVariables: `DEVICE`, `config_wandb`, `wandb_group_name`","metadata":{}},{"cell_type":"code","source":"# system setup\nseed_everything(CFG.seed)\n\nDEVICE = torch.device(\"cuda\") if torch.cuda.is_available() and not CFG.no_cuda else torch.device(\"cpu\")\n\n# setup some wandb params\n# login wandb\nuser_secrets = UserSecretsClient()\nwandb_api = user_secrets.get_secret(\"wandb_api\")\nwandb.login(key=wandb_api)\n\n# config for logging\nconfig_wandb = {\n    'learning_rate': TrainerConfig.lr,\n    'num_epochs': TrainerConfig.num_epochs,\n    'batch_size': TrainerConfig.batch_size,\n    'warm_up_ratio': TrainerConfig.warm_up_ratio,\n    'weight_decay': TrainerConfig.weight_decay,\n    'architecture': CFG.model_name,\n}\nwandb_group_name = \"exp\" + wandb.util.generate_id()","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-08-13T05:18:10.82043Z","iopub.execute_input":"2021-08-13T05:18:10.820836Z","iopub.status.idle":"2021-08-13T05:18:14.838603Z","shell.execute_reply.started":"2021-08-13T05:18:10.820799Z","shell.execute_reply":"2021-08-13T05:18:14.837796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model Definition <a class=\"anchor\" id=\"model\"></a>\nTake a pretrained segemntation model from MedicalNet, add a global pooling layer and a linear layer and *et voila*, we have a classifier.\n\nDict: `model_pretrained_params`\n\nClass: `MedicalNetWithHead`","metadata":{}},{"cell_type":"code","source":"# model specific args\nmodel_pretrained_params = {\n    'resnet_10': {'model_depth': 10, 'resnet_shortcut': 'B'},\n    'resnet_10_23dataset': {'model_depth': 10, 'resnet_shortcut': 'B'},\n    'resnet_18': {'model_depth': 18, 'resnet_shortcut': 'A'},\n    'resnet_18_23dataset': {'model_depth': 18, 'resnet_shortcut': 'A'},\n    'resnet_34': {'model_depth': 34, 'resnet_shortcut': 'A'},\n    'resnet_34_23dataset': {'model_depth': 34, 'resnet_shortcut': 'A'},\n    'resnet_50': {'model_depth': 50, 'resnet_shortcut': 'B'},\n    'resnet_50_23dataset': {'model_depth': 50, 'resnet_shortcut': 'B'},\n    'resnet_101': {'model_depth': 101, 'resnet_shortcut': 'B'},\n    'resnet_152': {'model_depth': 152, 'resnet_shortcut': 'B'},\n    'resnet_200': {'model_depth': 200, 'resnet_shortcut': 'B'},\n}\n# consistent args\nopts = {\n    'model': 'resnet',\n    'input_W': 256,\n    'input_H': 256,\n    'input_D': 64,\n    'no_cuda': CFG.no_cuda,\n    'n_seg_classes': 1,\n    'phase': 'train',\n    'pretrain_path': None,\n    'gpu_id': [1],\n}\n\n# merge modelspecific args and global args\nfor model_name, model_dict in model_pretrained_params.items():\n    model_pretrained_params[model_name] = Struct({**model_dict, **opts})\n    \n    \n# MedicalNet with a global pooling head\nclass MedicalNetWithHead(nn.Module):\n    def __init__(self, model_name, pretrain_path=None):\n        super().__init__()\n        self.model_name = model_name\n        model, parameters = MedicalNet.model.generate_model(model_pretrained_params[model_name])\n        self.medical_net = model\n        self.drop_in = nn.Dropout(p=0.1)\n        \n        # init model with pretrained weights\n        if not pretrain_path and CFG.use_pretrained:\n            self.init_model()\n            \n        # use simple pooling for now\n        self.pool = nn.AdaptiveAvgPool3d(1)\n        \n    def forward(self, x):\n        x = self.medical_net(self.drop_in(x))\n        out = self.pool(x)\n        return out\n            \n    def init_model(self):\n        net_dict = self.medical_net.state_dict()\n        # load pretrain\n        pretrain = torch.load(f'../input/medicalnet-with-weights/MedicalNet_pytorch_files2/pretrain/{self.model_name}.pth', map_location=DEVICE)\n        pretrain_dict = {k: v for k, v in pretrain['state_dict'].items() if k in net_dict.keys()}\n        net_dict.update(pretrain_dict)\n        self.medical_net.load_state_dict(net_dict)\n        print(\"loaded pretrained weights\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-08-13T05:18:14.841312Z","iopub.execute_input":"2021-08-13T05:18:14.841859Z","iopub.status.idle":"2021-08-13T05:18:14.854579Z","shell.execute_reply.started":"2021-08-13T05:18:14.841813Z","shell.execute_reply":"2021-08-13T05:18:14.853334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Dataset <a class=\"anchor\" id=\"data\"></a>\nSimple dataset without augmenations. We would like to have the same depth for each MRI scan with dimensions DxWxH, however the sampling rate varies. If there are less than the required number of depth channels, the dataset randomly repeats layers. \n\nClasses: `BTRCDataset`","metadata":{}},{"cell_type":"code","source":"class BTRCDataset(torch.utils.data.Dataset):\n    def __init__(self, df, data_dir,  cohort='FLAIR'):\n        self.df = df\n        self.data_dir = data_dir\n        self.cohort = cohort\n        \n        \n    def __getitem__(self, idx):\n        # get sample info\n        sample_id, target = self.df.loc[idx].values\n        \n        # get sample path. combination of dir, padded id and cohort\n        sample_dir = os.path.join(self.data_dir, f'{sample_id:05d}', self.cohort)\n        sample_files = os.listdir(sample_dir)\n        \n        # take subset of available images if n_images > 64\n        if len(sample_files) > 64:\n            sample_files = np.random.choice(sample_files, size=64, replace=False)\n        \n        # sort samples\n        sample_files = sorted(sample_files, key=lambda x: int(x[6:-4]))\n        \n        # load images\n        imgs = [self.read_img(os.path.join(sample_dir, path)) for path in sample_files]\n        imgs = np.stack(imgs)\n        \n        # resample images if not enough samples are available\n        if len(sample_files) < 64:\n            indices = sorted(np.random.choice(len(sample_files), size=64, replace=True))\n            \n            imgs = np.stack(imgs[indices])\n        \n        imgs = np.stack(imgs)\n            \n        return torch.tensor(imgs, dtype=torch.float32).unsqueeze(0), torch.tensor(target, dtype=torch.float32)\n        \n    def __len__(self):\n        return len(self.df)\n    \n    def read_img(self, path):\n        if path.endswith('dcm'):\n            img = self.read_dicom(path)\n        elif path.endswith('png'):\n            img = self.read_png(path)\n        else:\n            print('unknown file format')\n        img = cv2.resize(img, (CFG.img_size, CFG.img_size))\n        return img\n    \n    @staticmethod\n    def read_png(path):\n        img = cv2.imread(path, cv2.IMREAD_UNCHANGED)\n        return img\n    \n    @staticmethod\n    def read_dicom(path):\n        dicom = pydicom.read_file(path)\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n        if dicom.PhotometricInterpretation == \"MONOCHROME1\":\n            data = np.amax(data) - data\n        data = data - np.min(data)\n        data = data / np.max(data)\n        data = (data * 255).astype(np.uint8)\n        return data\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-08-13T05:18:14.85643Z","iopub.execute_input":"2021-08-13T05:18:14.857241Z","iopub.status.idle":"2021-08-13T05:18:14.873078Z","shell.execute_reply.started":"2021-08-13T05:18:14.857188Z","shell.execute_reply":"2021-08-13T05:18:14.872016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Trainer <a class=\"anchor\" id=\"trainer\"></a>\nDoes the heavy lifting for training.\n\nClasses: `AverageMeter`, `Trainer`","metadata":{}},{"cell_type":"code","source":"class AverageMeter(object):\n    \"\"\"\n    Computes and stores the average and current value\n    Copied from: https://github.com/pytorch/examples/blob/master/imagenet/main.py\n    \"\"\"\n\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count\n\n\nclass Trainer:\n    def __init__(self, cfg: type(TrainerConfig),\n                 model: torch.nn.Module,\n                 model_path: str,\n                 dataset_train: Dataset = None,\n                 dataset_val: Dataset = None,\n                 wandb_run: wandb.sdk.wandb_run.Run = None):\n        self.cfg = cfg\n        self.model = model\n        self.best_model = None\n        self.model_path = model_path\n        self.wandb_run = wandb_run\n\n        # datasets\n        self.dataset_train = dataset_train\n        self.dataset_eval = dataset_val\n\n        # dataloaders, train/eval is optional\n        kwargs_dataloader = {'batch_size': self.cfg.batch_size, 'num_workers': 2}\n        if self.dataset_eval is not None:\n            self.dataloader_train = DataLoader(self.dataset_train, shuffle=True, **kwargs_dataloader)\n        if self.dataset_eval is not None:\n            self.dataloader_eval = DataLoader(self.dataset_eval, shuffle=False, **kwargs_dataloader)\n\n        # setup loss\n        self.loss_fnc = torch.nn.BCEWithLogitsLoss()\n\n        # if train set is provided, setup training\n        if dataset_train is not None:\n            # init optimizer\n            #self.optimizer = AdamW(self.model.parameters(), self.cfg.lr, weight_decay=self.cfg.weight_decay)\n            self.optimizer = torch.optim.SGD(self.model.parameters(), self.cfg.lr, weight_decay=self.cfg.weight_decay, momentum=0.9)\n\n            # setup lr scheduler\n            _n_steps = cfg.num_epochs * len(self.dataloader_train)\n            self.scheduler = get_cosine_schedule_with_warmup(\n                optimizer=self.optimizer,\n                num_warmup_steps=_n_steps * cfg.warm_up_ratio,\n                num_training_steps=_n_steps,\n            )\n            # call optimizer once so we can properly init the lr in step_train()\n            self.optimizer.zero_grad()\n            self.optimizer.step()\n            \n        else:\n            self.optimizer = None\n            self.scheduler = None\n            \n        self.epoch = 0\n\n    def train(self):\n        print(f'Training model for {self.cfg.num_epochs} epochs.')\n        print('Epoch | train_loss | eval_loss | train_auc | val_auc')\n        train_log = pd.DataFrame(columns=['epoch', 'train_loss', 'eval_loss', 'train_auc', 'eval_auc', 'lr'])\n        \n        best_loss = 1e3\n\n        for epoch in range(self.cfg.num_epochs):\n            train_loss, train_auc = self.step_train()\n            eval_loss, eval_auc = self.step_eval(return_predictions=False)\n            \n            log_item = {\n                'epoch': epoch,\n                'step': epoch*len(self.dataloader_train),\n                'train_loss': train_loss,\n                'eval_loss': eval_loss,\n                'train_auc': train_auc,\n                'eval_auc': eval_auc,\n                'lr': self.optimizer.param_groups[0]['lr']\n            }\n            self.wandb_run.log(log_item)\n            train_log = train_log.append(log_item, ignore_index=True)\n            print(f\"{epoch: <6}|{train_loss: >12.3f}|{eval_loss: >11.3f}|{train_auc: >11.3f}|{eval_auc: >8.3f}\")\n      \n            # checkpointing\n            if eval_loss < best_loss:\n                torch.save(self.model, self.model_path)\n                best_loss = eval_loss\n            self.epoch += 1\n        best_epoch = train_log.eval_loss.idxmin()\n        print(\"Training done. Best model at epoch {} with eval_loss {:3.2f} and auc {:3.2f}\".format(\n            train_log.loc[best_epoch, 'epoch'],\n            train_log.loc[best_epoch, 'eval_loss'],\n            train_log.loc[best_epoch, 'eval_auc']\n        ))\n        return train_log\n\n    def step_train(self):\n        self.model.train()\n        \n        # setup logging\n        loss_agg = AverageMeter()\n        targets = []\n        predictions = []\n        \n        # train one epoch\n        for batch_idx, (x, y) in enumerate(self.dataloader_train):\n            \n            x = x.to(self.cfg.device)\n            y = y.to(self.cfg.device)\n            \n            # forward pass  \n            logits = self.model(x)\n            loss = self.loss_fnc(logits.view(-1), y)\n\n            # backward pass\n            loss.backward()\n            if (batch_idx+1) % self.cfg.gradient_accumulation_steps == 0 or (batch_idx+1) == len(self.dataloader_train):\n                self.optimizer.step()\n                self.optimizer.zero_grad()\n                self.scheduler.step()\n\n            # update loss meter\n            loss_agg.update(loss.item(), self.cfg.batch_size)\n\n            # save preds/targets for roc computation\n            predictions.append(torch.sigmoid(logits.view(-1)).detach().cpu().squeeze().numpy())\n            targets.append(y.detach().cpu().squeeze().numpy())\n\n            # log every cfg.log_steps steps\n            if batch_idx > 0 and batch_idx % self.cfg.log_steps == 0:\n                log_item = {\n                    'step': self.epoch*len(self.dataloader_train) + batch_idx,\n                    'train_loss': loss_agg.avg,\n                    'train_auc': roc_auc_score(np.hstack(targets), np.hstack(predictions)),\n                    'lr': self.optimizer.param_groups[0]['lr']\n                }\n                self.wandb_run.log(log_item)\n            \n        # compute auc for whole epoch\n        auc = roc_auc_score(np.hstack(targets), np.hstack(predictions))\n        return loss_agg.avg, auc\n\n    @torch.no_grad()\n    def step_eval(self, return_predictions=False):\n        self.model.eval()\n        loss_agg = AverageMeter()\n        predictions = []\n        targets = []\n        for batch_idx, (x, y) in enumerate(self.dataloader_eval):\n            x = x.to(self.cfg.device)\n            y = y.to(self.cfg.device)\n            logits = self.model(x)\n            loss = self.loss_fnc(logits.view(-1), y)\n\n            # update loss meter\n            loss_agg.update(loss.item(), self.cfg.batch_size)\n\n            # optionally return predictions\n            # save preds/targets for roc computation\n            predictions.append(torch.sigmoid(logits.view(-1)).detach().cpu().squeeze().numpy())\n            targets.append(y.detach().cpu().squeeze().numpy())\n\n        # compute auc\n        targets = np.hstack(targets)\n        predictions = np.hstack(predictions)\n        auc = roc_auc_score(targets, predictions)\n\n        # setup output\n        if return_predictions:\n            out = (loss_agg.avg, auc, predictions)\n        else:\n            out = (loss_agg.avg, auc)\n        return out\n    ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-08-13T05:18:14.875653Z","iopub.execute_input":"2021-08-13T05:18:14.875962Z","iopub.status.idle":"2021-08-13T05:18:14.90474Z","shell.execute_reply.started":"2021-08-13T05:18:14.875932Z","shell.execute_reply":"2021-08-13T05:18:14.903324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training <a class=\"anchor\" id=\"training\"></a>\nTrain k models and save the best epochs for each. Logging is done in console and with wandb.","metadata":{}},{"cell_type":"code","source":"# load training dataframe\ntrain_df = pd.read_csv('../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv')\n\n# drop 3 samples, see https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/262046\ntrain_df = train_df[~train_df.BraTS21ID.isin([109, 123, 709])]\n\n# setup cross validation\ntrain_df = get_folds(train_df, rng=CFG.seed)\ntrain_df.to_csv('folds.csv')\n\n# train each fold\nfor fold_idx in range(int(train_df['fold'].max())+1):\n    # datasets\n    df_train = train_df.loc[train_df.fold != fold_idx, ['BraTS21ID', 'MGMT_value']].reset_index(drop=True)\n    df_eval = train_df.loc[train_df.fold == fold_idx, ['BraTS21ID', 'MGMT_value']].reset_index(drop=True)\n    dataset_train = BTRCDataset(df_train, data_dir='../input/rsna-miccai-png/train', cohort=CFG.cohort)\n    dataset_eval = BTRCDataset(df_eval, data_dir='../input/rsna-miccai-png/train', cohort=CFG.cohort)\n\n    # init new model\n    model = MedicalNetWithHead(model_name=CFG.model_name).to(DEVICE)\n    \n    # setup wandb\n    wandb_run = wandb.init(project=\"kaggle-BTRC\", config=config_wandb, group=wandb_group_name, name=f\"fold{fold_idx}\", job_type=\"finetuning\")\n    wandb_run.define_metric(\"step\")\n    wandb_run.define_metric(\"*\", step_metric=\"step\", step_sync=True)\n    \n    # setup trainer\n    trainer = Trainer(cfg=TrainerConfig,\n                      model=model,\n                      model_path=f'model_fold{fold_idx}.torch',\n                      dataset_train=dataset_train,\n                      dataset_val=dataset_eval,\n                      wandb_run=wandb_run)\n\n    # train the model\n    train_log = trainer.train()\n    train_log.to_csv(f\"fold{fold_idx}_log.csv\")\n    wandb_run.finish()\n    ","metadata":{"execution":{"iopub.status.busy":"2021-08-13T05:18:14.906734Z","iopub.execute_input":"2021-08-13T05:18:14.907178Z","iopub.status.idle":"2021-08-13T05:18:16.650578Z","shell.execute_reply.started":"2021-08-13T05:18:14.907119Z","shell.execute_reply":"2021-08-13T05:18:16.647533Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer = Trainer(cfg=TrainerConfig,\n                      model=nn.Linear(10,10),\n                      model_path=f'model_fold{fold_idx}.torch',\n                      dataset_train=dataset_train,\n                      dataset_val=dataset_eval,\n                      wandb_run=None)","metadata":{"execution":{"iopub.status.busy":"2021-08-13T05:20:33.189368Z","iopub.execute_input":"2021-08-13T05:20:33.191679Z","iopub.status.idle":"2021-08-13T05:20:33.198009Z","shell.execute_reply.started":"2021-08-13T05:20:33.191633Z","shell.execute_reply":"2021-08-13T05:20:33.196934Z"},"trusted":true},"execution_count":null,"outputs":[]}]}