{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

\n\n
\n

RSNA | EDA + Visual + DeepUnderstanding + W&B

\n\n\n
","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"
\n

Contents

","metadata":{}},{"cell_type":"markdown","source":"1. [Competition Overview](#competition-overview) \n2. [Understanding MRI](#understanding-mri)\n3. [Libraries](#libraries) \n4. [Weights and Biases](#weights-and-biases)\n5. [Global Config](#global-config)\n6. [Load Datasets](#load-datasets) \n7. [Tabular Exploration](#tabular-exploration) \n8. [Model](#model)\n9. [wandb System Metrics](#wandb-system-metrics)\n9. [References](#references) ","metadata":{"execution":{"iopub.status.busy":"2021-07-28T08:06:43.601074Z","iopub.execute_input":"2021-07-28T08:06:43.601533Z","iopub.status.idle":"2021-07-28T08:06:43.61593Z","shell.execute_reply.started":"2021-07-28T08:06:43.601421Z","shell.execute_reply":"2021-07-28T08:06:43.614653Z"}}},{"cell_type":"markdown","source":"
\n

If you find this notebook useful, do give me an upvote, it helps to keep up my motivation. This notebook will be updated frequently so keep checking for furthur developments.

","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Competition Overview

","metadata":{}},{"cell_type":"markdown","source":"## Description","metadata":{}},{"cell_type":"markdown","source":"The Radiological Society of North America (RSNA) has teamed up with the Medical Image Computing and Computer Assisted Intervention Society (the MICCAI Society) to improve diagnosis and treatment planning for patients with glioblastoma. \n\nIn this competition you will predict the genetic subtype of glioblastoma using MRI (magnetic resonance imaging) scans to train and test your model to detect for the presence of MGMT promoter methylation.\n\nIf successful, you'll help brain cancer patients receive less invasive diagnoses and treatments. The introduction of new and customized treatment strategies before surgery has the potential to improve the management, survival, and prospects of patients with brain cancer.","metadata":{}},{"cell_type":"markdown","source":"## Evaluation Criteria","metadata":{}},{"cell_type":"markdown","source":"Submissions are evaluated on the [area under the ROC curve](http://en.wikipedia.org/wiki/Receiver_operating_characteristic) between the predicted probability and the observed target.","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Understanding MRI

","metadata":{}},{"cell_type":"markdown","source":"**Magnetic resonance imaging (MRI)** is one of the most commonly used tests in neurology and neurosurgery. MRI provides exquisite detail of brain, spinal cord and vascular anatomy, and has the advantage of being able to visualize anatomy in all three planes: axial, sagittal and coronal (see the example image below).\n\n","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"MRI has an advantage over CT in being able to detect flowing blood and cryptic vascular malformations. It can also detect demyelinating disease, and has no beam-hardening artifacts such as can be seen with CT. \n\nThus, the posterior fossa is more easily visualized on MRI than CT. Imaging is also performed without any ionizing radiation.","metadata":{}},{"cell_type":"markdown","source":"## MRI Imaging Sequences\n\nThe most common MRI sequences are **T1-weighted** and **T2-weighted** scans. \n\n- **T1-weighted** images are produced by using short TE and TR times. The contrast and brightness of the image are predominately determined by T1 properties of tissue. \n\n- **T2-weighted** images are produced by using longer TE and TR times. In these images, the contrast and brightness are predominately determined by the T2 properties of tissue.\n\nIn general, T1- and T2-weighted images can be easily differentiated by looking the CSF. **CSF** is dark on T1-weighted imaging and bright on T2-weighted imaging.\n\nA third commonly used sequence is the **Fluid Attenuated Inversion Recovery (Flair)**. The Flair sequence is similar to a T2-weighted image except that the TE and TR times are very long. By doing so, abnormalities remain bright but normal CSF fluid is attenuated and made dark. This sequence is very sensitive to pathology and makes the differentiation between CSF and an abnormality much easier.","metadata":{}},{"cell_type":"markdown","source":"## Comparison of T1 vs T2 vs Flair (Brain)\n
","metadata":{}},{"cell_type":"markdown","source":"## Comparison of T1 vs T1 with Gadolinium\n
","metadata":{}},{"cell_type":"markdown","source":"## Comparison of Flair vs Diffusion Weighted\n
","metadata":{}},{"cell_type":"markdown","source":"## Comparison of T1 vs T2 - Spine\n
","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Libraries

","metadata":{}},{"cell_type":"code","source":"import json\nimport glob\nimport random\nimport collections\n\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nimport cv2\n\nimport numpy as np\nimport pandas as pd\n\nimport os\nimport plotly.express as px\n\nimport matplotlib\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns\nimport cv2\n\n#Text Color\nfrom termcolor import colored\n\npackage_path = \"../input/efficientnet-pytorch/EfficientNet-PyTorch/EfficientNet-PyTorch-master/\"\nimport sys \nsys.path.append(package_path)\n\nimport time\n\nimport torch\nfrom torch import nn\nfrom torch.utils import data as torch_data\nfrom sklearn import model_selection as sk_model_selection\nfrom torch.nn import functional as torch_functional\nimport efficientnet_pytorch\n\nfrom sklearn.model_selection import StratifiedKFold\n\n# W&B for experiment tracking\nimport wandb\nwandb.login()","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:37.335340Z","iopub.execute_input":"2021-08-14T19:35:37.335729Z","iopub.status.idle":"2021-08-14T19:35:51.930322Z","shell.execute_reply.started":"2021-08-14T19:35:37.335642Z","shell.execute_reply":"2021-08-14T19:35:51.929383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Weights and Biases

","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"**Weights & Biases** is the machine learning platform for developers to build better models faster. \n\nYou can use W&B's lightweight, interoperable tools to \n- quickly track experiments, \n- version and iterate on datasets, \n- evaluate model performance, \n- reproduce models, \n- visualize results and spot regressions, \n- and share findings with colleagues. \n\nSet up W&B in 5 minutes, then quickly iterate on your machine learning pipeline with the confidence that your datasets and models are tracked and versioned in a reliable system of record.\n\nIn this notebook I will use Weights and Biases's amazing features to perform wonderful visualizations seamlessly. ","metadata":{}},{"cell_type":"markdown","source":"\n
\n

Global Config

","metadata":{}},{"cell_type":"code","source":"class config:\n DIRECTORY_PATH = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification\"\n TRAIN_LABELS_PATH = DIRECTORY_PATH + \"/train_labels.csv\"\n \n# wandb config\nWANDB_CONFIG = {\n 'competition': 'rsna-miccai-brain', \n '_wandb_kernel': 'neuracort'\n}","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:55.372903Z","iopub.execute_input":"2021-08-14T19:35:55.373245Z","iopub.status.idle":"2021-08-14T19:35:55.377976Z","shell.execute_reply.started":"2021-08-14T19:35:55.373207Z","shell.execute_reply":"2021-08-14T19:35:55.376816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def set_seed(seed):\n random.seed(seed)\n os.environ[\"PYTHONHASHSEED\"] = str(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n if torch.cuda.is_available():\n torch.cuda.manual_seed_all(seed)\n torch.backends.cudnn.deterministic = True\n\n\nset_seed(42)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:55.649689Z","iopub.execute_input":"2021-08-14T19:35:55.650011Z","iopub.status.idle":"2021-08-14T19:35:55.699694Z","shell.execute_reply.started":"2021-08-14T19:35:55.649981Z","shell.execute_reply":"2021-08-14T19:35:55.698757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Load Datasets

","metadata":{}},{"cell_type":"markdown","source":"## Data Files\n\n- **train/** - folder containing the training files, with each top-level folder representing a subject\n\n- **train_labels.csv** - file containing the target MGMT_value for each subject in the training data (e.g. the presence of MGMT promoter methylation)\n\n- **test/** - the test files, which use the same structure as train/; your task is to predict the MGMT_value for each subject in the test data. NOTE: the total size of the rerun test set (Public and Private) is ~5x the size of the Public test set\n\n- **sample_submission.csv** - a sample submission file in the correct format","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(config.TRAIN_LABELS_PATH)\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:56.734028Z","iopub.execute_input":"2021-08-14T19:35:56.734439Z","iopub.status.idle":"2021-08-14T19:35:56.765035Z","shell.execute_reply.started":"2021-08-14T19:35:56.734406Z","shell.execute_reply":"2021-08-14T19:35:56.764226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Tabular Exploration

","metadata":{}},{"cell_type":"markdown","source":"## Basic Tabular Details","metadata":{}},{"cell_type":"code","source":"# Function to print stylized text\ndef style_text(text, text_color = 'yellow', attributes = ['bold'], data = False):\n \"\"\"\n Function to stylize print Text using Colored by Termcolor\n \n parameters: text(str) - Input Text to be Stylized\n text_color(str) - Color of text\n attributes(list of strings) - Attributes to be applied on text\n data - To be printed with text \n \"\"\"\n if data:\n print(colored(text, text_color, attrs = attributes), data)\n \n else:\n print(colored(text, text_color, attrs = attributes))","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:58.285119Z","iopub.execute_input":"2021-08-14T19:35:58.285504Z","iopub.status.idle":"2021-08-14T19:35:58.291128Z","shell.execute_reply.started":"2021-08-14T19:35:58.285470Z","shell.execute_reply":"2021-08-14T19:35:58.290223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Data shape\nstyle_text(\"No. of Rows in train_df: \", data = train_df.shape[0])\nstyle_text(\"No. of Columns in train_df: \", data = train_df.shape[1])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:58.655433Z","iopub.execute_input":"2021-08-14T19:35:58.655770Z","iopub.status.idle":"2021-08-14T19:35:58.661168Z","shell.execute_reply.started":"2021-08-14T19:35:58.655735Z","shell.execute_reply":"2021-08-14T19:35:58.659963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Missing Values\nstyle_text(\"Missing Values in train_df:\")\nprint(train_df.isnull().sum())","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:59.003793Z","iopub.execute_input":"2021-08-14T19:35:59.004107Z","iopub.status.idle":"2021-08-14T19:35:59.010649Z","shell.execute_reply.started":"2021-08-14T19:35:59.004078Z","shell.execute_reply":"2021-08-14T19:35:59.009624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Thus, there are no missing values in the `train_df` dataset.","metadata":{}},{"cell_type":"code","source":"#Dataset Info\nstyle_text(\"Info about train_df:\")\ntrain_df.info()","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:35:59.814073Z","iopub.execute_input":"2021-08-14T19:35:59.814424Z","iopub.status.idle":"2021-08-14T19:35:59.831559Z","shell.execute_reply.started":"2021-08-14T19:35:59.814392Z","shell.execute_reply":"2021-08-14T19:35:59.830425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## CountPlot for MGMT Value","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(5, 5))\nsns.countplot(data=train_df, x=\"MGMT_value\");","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:00.605866Z","iopub.execute_input":"2021-08-14T19:36:00.606186Z","iopub.status.idle":"2021-08-14T19:36:00.750143Z","shell.execute_reply.started":"2021-08-14T19:36:00.606156Z","shell.execute_reply":"2021-08-14T19:36:00.749193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

Model

","metadata":{"execution":{"iopub.status.busy":"2021-08-14T18:20:34.203972Z","iopub.execute_input":"2021-08-14T18:20:34.204548Z","iopub.status.idle":"2021-08-14T18:20:34.226074Z","shell.execute_reply.started":"2021-08-14T18:20:34.204424Z","shell.execute_reply":"2021-08-14T18:20:34.224367Z"}}},{"cell_type":"markdown","source":"Credits to the model goes to [\nYaroslav Isaienkov](https://www.kaggle.com/ihelon)\n\nI will take his model a step ahead and incorporate Weights and Biases in it with the explanation on how to utilise it.","metadata":{}},{"cell_type":"markdown","source":"**Wandb Step 1:** In the first step we need to initialize wandb with the name of a `project` where we want to save our runs.","metadata":{}},{"cell_type":"code","source":"wandb.init(project='brain-tumor-viz', config=WANDB_CONFIG)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:02.614877Z","iopub.execute_input":"2021-08-14T19:36:02.615204Z","iopub.status.idle":"2021-08-14T19:36:08.890814Z","shell.execute_reply.started":"2021-08-14T19:36:02.615172Z","shell.execute_reply":"2021-08-14T19:36:08.889995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_dicom(path):\n dicom = pydicom.read_file(path)\n data = dicom.pixel_array\n data = data - np.min(data)\n if np.max(data) != 0:\n data = data / np.max(data)\n data = (data * 255).astype(np.uint8)\n return data","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:08.893675Z","iopub.execute_input":"2021-08-14T19:36:08.893938Z","iopub.status.idle":"2021-08-14T19:36:08.900199Z","shell.execute_reply.started":"2021-08-14T19:36:08.893907Z","shell.execute_reply":"2021-08-14T19:36:08.899357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv\")\ndf_train, df_valid = sk_model_selection.train_test_split(\n df, \n test_size=0.2, \n random_state=42, \n stratify=train_df[\"MGMT_value\"],\n)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:08.902388Z","iopub.execute_input":"2021-08-14T19:36:08.902960Z","iopub.status.idle":"2021-08-14T19:36:08.917969Z","shell.execute_reply.started":"2021-08-14T19:36:08.902899Z","shell.execute_reply":"2021-08-14T19:36:08.917142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DataRetriever(torch_data.Dataset):\n def __init__(self, paths, targets):\n self.paths = paths\n self.targets = targets\n \n def __len__(self):\n return len(self.paths)\n \n def __getitem__(self, index):\n _id = self.paths[index]\n patient_path = f\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/{str(_id).zfill(5)}/\"\n channels = []\n for t in (\"FLAIR\", \"T1w\", \"T1wCE\"): # \"T2w\"\n t_paths = sorted(\n glob.glob(os.path.join(patient_path, t, \"*\")), \n key=lambda x: int(x[:-4].split(\"-\")[-1]),\n )\n # start, end = int(len(t_paths) * 0.475), int(len(t_paths) * 0.525)\n x = len(t_paths)\n if x < 10:\n r = range(x)\n else:\n d = x // 10\n r = range(d, x - d, d)\n \n channel = []\n # for i in range(start, end + 1):\n for i in r:\n channel.append(cv2.resize(load_dicom(t_paths[i]), (256, 256)) / 255)\n channel = np.mean(channel, axis=0)\n channels.append(channel)\n \n y = torch.tensor(self.targets[index], dtype=torch.float)\n \n return {\"X\": torch.tensor(channels).float(), \"y\": y}","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:08.919595Z","iopub.execute_input":"2021-08-14T19:36:08.919894Z","iopub.status.idle":"2021-08-14T19:36:08.930924Z","shell.execute_reply.started":"2021-08-14T19:36:08.919867Z","shell.execute_reply":"2021-08-14T19:36:08.929971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data_retriever = DataRetriever(\n df_train[\"BraTS21ID\"].values, \n df_train[\"MGMT_value\"].values, \n)\n\nvalid_data_retriever = DataRetriever(\n df_valid[\"BraTS21ID\"].values, \n df_valid[\"MGMT_value\"].values,\n)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:08.932389Z","iopub.execute_input":"2021-08-14T19:36:08.933016Z","iopub.status.idle":"2021-08-14T19:36:08.943936Z","shell.execute_reply.started":"2021-08-14T19:36:08.932976Z","shell.execute_reply":"2021-08-14T19:36:08.943050Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(16, 6))\nfor i in range(3):\n plt.subplot(1, 3, i + 1)\n plt.imshow(train_data_retriever[100][\"X\"].numpy()[i], cmap=\"gray\")","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:08.945413Z","iopub.execute_input":"2021-08-14T19:36:08.946169Z","iopub.status.idle":"2021-08-14T19:36:09.996058Z","shell.execute_reply.started":"2021-08-14T19:36:08.946125Z","shell.execute_reply":"2021-08-14T19:36:09.995205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Model(nn.Module):\n def __init__(self):\n super().__init__()\n self.net = efficientnet_pytorch.EfficientNet.from_name(\"efficientnet-b0\")\n checkpoint = torch.load(\"../input/efficientnet-pytorch/efficientnet-b0-08094119.pth\")\n self.net.load_state_dict(checkpoint)\n n_features = self.net._fc.in_features\n self.net._fc = nn.Linear(in_features=n_features, out_features=1, bias=True)\n \n def forward(self, x):\n out = self.net(x)\n return out","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:09.997212Z","iopub.execute_input":"2021-08-14T19:36:09.997755Z","iopub.status.idle":"2021-08-14T19:36:10.005168Z","shell.execute_reply.started":"2021-08-14T19:36:09.997712Z","shell.execute_reply":"2021-08-14T19:36:10.004205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class LossMeter:\n def __init__(self):\n self.avg = 0\n self.n = 0\n\n def update(self, val):\n self.n += 1\n # incremental update\n self.avg = val / self.n + (self.n - 1) / self.n * self.avg\n\n \nclass AccMeter:\n def __init__(self):\n self.avg = 0\n self.n = 0\n \n def update(self, y_true, y_pred):\n y_true = y_true.cpu().numpy().astype(int)\n y_pred = y_pred.cpu().numpy() >= 0\n last_n = self.n\n self.n += len(y_true)\n true_count = np.sum(y_true == y_pred)\n # incremental update\n self.avg = true_count / self.n + last_n / self.n * self.avg","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:10.007433Z","iopub.execute_input":"2021-08-14T19:36:10.007938Z","iopub.status.idle":"2021-08-14T19:36:10.018924Z","shell.execute_reply.started":"2021-08-14T19:36:10.007900Z","shell.execute_reply":"2021-08-14T19:36:10.018019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Wandb Step 2:** In this example we are going to log the Training and Validation losses. To do this we need to instruct wandb to `watch` the `model`","metadata":{}},{"cell_type":"code","source":"class Trainer:\n def __init__(\n self, \n model, \n device, \n optimizer, \n criterion, \n loss_meter, \n score_meter\n ):\n self.model = model\n self.device = device\n self.optimizer = optimizer\n self.criterion = criterion\n self.loss_meter = loss_meter\n self.score_meter = score_meter\n \n self.best_valid_score = -np.inf\n self.n_patience = 0\n \n self.messages = {\n \"epoch\": \"[Epoch {}: {}] loss: {:.5f}, score: {:.5f}, time: {} s\",\n \"checkpoint\": \"The score improved from {:.5f} to {:.5f}. Save model to '{}'\",\n \"patience\": \"\\nValid score didn't improve last {} epochs.\"\n }\n \n def fit(self, epochs, train_loader, valid_loader, save_path, patience): \n for n_epoch in range(1, epochs + 1):\n self.info_message(\"EPOCH: {}\", n_epoch)\n \n train_loss, train_score, train_time = self.train_epoch(train_loader)\n valid_loss, valid_score, valid_time = self.valid_epoch(valid_loader)\n \n self.info_message(\n self.messages[\"epoch\"], \"Train\", n_epoch, train_loss, train_score, train_time\n )\n \n self.info_message(\n self.messages[\"epoch\"], \"Valid\", n_epoch, valid_loss, valid_score, valid_time\n )\n\n if True:\n# if self.best_valid_score < valid_score:\n self.info_message(\n self.messages[\"checkpoint\"], self.best_valid_score, valid_score, save_path\n )\n self.best_valid_score = valid_score\n self.save_model(n_epoch, save_path)\n self.n_patience = 0\n else:\n self.n_patience += 1\n \n if self.n_patience >= patience:\n self.info_message(self.messages[\"patience\"], patience)\n break\n \n def train_epoch(self, train_loader):\n \n wandb.watch(model) # Use wandb.watch() to provide the model to be logged upon\n\n self.model.train()\n t = time.time()\n train_loss = self.loss_meter()\n train_score = self.score_meter()\n \n for step, batch in enumerate(train_loader, 1):\n X = batch[\"X\"].to(self.device)\n targets = batch[\"y\"].to(self.device)\n self.optimizer.zero_grad()\n outputs = self.model(X).squeeze(1)\n \n loss = self.criterion(outputs, targets)\n wandb.log({\"train_loss\": loss}) # Use wandb.log() to log desired metrics \n \n loss.backward()\n\n train_loss.update(loss.detach().item())\n train_score.update(targets, outputs.detach())\n\n self.optimizer.step()\n \n _loss, _score = train_loss.avg, train_score.avg\n message = 'Train Step {}/{}, train_loss: {:.5f}, train_score: {:.5f}'\n self.info_message(message, step, len(train_loader), _loss, _score, end=\"\\r\")\n \n return train_loss.avg, train_score.avg, int(time.time() - t)\n \n def valid_epoch(self, valid_loader):\n self.model.eval()\n t = time.time()\n valid_loss = self.loss_meter()\n valid_score = self.score_meter()\n\n for step, batch in enumerate(valid_loader, 1):\n with torch.no_grad():\n X = batch[\"X\"].to(self.device)\n targets = batch[\"y\"].to(self.device)\n\n outputs = self.model(X).squeeze(1)\n loss = self.criterion(outputs, targets)\n wandb.log({\"valid_loss\": loss}) # Use wandb.log() to log desired metrics \n\n valid_loss.update(loss.detach().item())\n valid_score.update(targets, outputs)\n \n _loss, _score = valid_loss.avg, valid_score.avg\n message = 'Valid Step {}/{}, valid_loss: {:.5f}, valid_score: {:.5f}'\n self.info_message(message, step, len(valid_loader), _loss, _score, end=\"\\r\")\n \n return valid_loss.avg, valid_score.avg, int(time.time() - t)\n \n def save_model(self, n_epoch, save_path):\n torch.save(\n {\n \"model_state_dict\": self.model.state_dict(),\n \"optimizer_state_dict\": self.optimizer.state_dict(),\n \"best_valid_score\": self.best_valid_score,\n \"n_epoch\": n_epoch,\n },\n save_path,\n )\n \n @staticmethod\n def info_message(message, *args, end=\"\\n\"):\n print(message.format(*args), end=end)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:10.020575Z","iopub.execute_input":"2021-08-14T19:36:10.020953Z","iopub.status.idle":"2021-08-14T19:36:10.043719Z","shell.execute_reply.started":"2021-08-14T19:36:10.020915Z","shell.execute_reply":"2021-08-14T19:36:10.042902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\ntrain_data_retriever = DataRetriever(\n df_train[\"BraTS21ID\"].values, \n df_train[\"MGMT_value\"].values, \n)\n\nvalid_data_retriever = DataRetriever(\n df_valid[\"BraTS21ID\"].values, \n df_valid[\"MGMT_value\"].values,\n)\n\ntrain_loader = torch_data.DataLoader(\n train_data_retriever,\n batch_size=8,\n shuffle=True,\n num_workers=8,\n)\n\nvalid_loader = torch_data.DataLoader(\n valid_data_retriever, \n batch_size=8,\n shuffle=False,\n num_workers=8,\n)\n\nmodel = Model()\nmodel.to(device)\n\noptimizer = torch.optim.Adam(model.parameters(), lr=0.001)\ncriterion = torch_functional.binary_cross_entropy_with_logits\n\ntrainer = Trainer(\n model, \n device, \n optimizer, \n criterion, \n LossMeter, \n AccMeter\n)\n\nhistory = trainer.fit(\n 1, \n train_loader, \n valid_loader, \n f\"best-model-0.pth\", \n 100,\n)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:36:10.045046Z","iopub.execute_input":"2021-08-14T19:36:10.045397Z","iopub.status.idle":"2021-08-14T19:37:27.434973Z","shell.execute_reply.started":"2021-08-14T19:36:10.045363Z","shell.execute_reply":"2021-08-14T19:37:27.434030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And that's it, with just 2 Steps you have successfully integrated wandb to your project. Now you can go to your dashboard and check the logged metrics. You can even follow the same method to log the hyperparameters.","metadata":{}},{"cell_type":"code","source":"models = []\nfor i in range(1):\n model = Model()\n model.to(device)\n \n checkpoint = torch.load(f\"best-model-{i}.pth\")\n model.load_state_dict(checkpoint[\"model_state_dict\"])\n model.eval()\n \n models.append(model)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:27.436916Z","iopub.execute_input":"2021-08-14T19:37:27.437307Z","iopub.status.idle":"2021-08-14T19:37:27.730510Z","shell.execute_reply.started":"2021-08-14T19:37:27.437219Z","shell.execute_reply":"2021-08-14T19:37:27.729656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DataRetriever(torch_data.Dataset):\n def __init__(self, paths):\n self.paths = paths\n \n def __len__(self):\n return len(self.paths)\n \n def __getitem__(self, index):\n _id = self.paths[index]\n patient_path = f\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/test/{str(_id).zfill(5)}/\"\n channels = []\n for t in (\"FLAIR\", \"T1w\", \"T1wCE\"): # \"T2w\"\n t_paths = sorted(\n glob.glob(os.path.join(patient_path, t, \"*\")), \n key=lambda x: int(x[:-4].split(\"-\")[-1]),\n )\n # start, end = int(len(t_paths) * 0.475), int(len(t_paths) * 0.525)\n x = len(t_paths)\n if x < 10:\n r = range(x)\n else:\n d = x // 10\n r = range(d, x - d, d)\n \n channel = []\n # for i in range(start, end + 1):\n for i in r:\n channel.append(cv2.resize(load_dicom(t_paths[i]), (256, 256)) / 255)\n channel = np.mean(channel, axis=0)\n channels.append(channel)\n \n return {\"X\": torch.tensor(channels).float(), \"id\": _id}","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:27.731881Z","iopub.execute_input":"2021-08-14T19:37:27.732211Z","iopub.status.idle":"2021-08-14T19:37:27.747965Z","shell.execute_reply.started":"2021-08-14T19:37:27.732175Z","shell.execute_reply":"2021-08-14T19:37:27.746877Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.read_csv(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/sample_submission.csv\")\n\ntest_data_retriever = DataRetriever(\n submission[\"BraTS21ID\"].values, \n)\n\ntest_loader = torch_data.DataLoader(\n test_data_retriever,\n batch_size=4,\n shuffle=False,\n num_workers=8,\n)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:27.751302Z","iopub.execute_input":"2021-08-14T19:37:27.751592Z","iopub.status.idle":"2021-08-14T19:37:27.774195Z","shell.execute_reply.started":"2021-08-14T19:37:27.751564Z","shell.execute_reply":"2021-08-14T19:37:27.773331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred = []\nids = []\n\nfor e, batch in enumerate(test_loader):\n print(f\"{e}/{len(test_loader)}\", end=\"\\r\")\n with torch.no_grad():\n tmp_pred = np.zeros((batch[\"X\"].shape[0], ))\n for model in models:\n tmp_res = torch.sigmoid(model(batch[\"X\"].to(device))).cpu().numpy().squeeze()\n tmp_pred += tmp_res\n y_pred.extend(tmp_pred)\n ids.extend(batch[\"id\"].numpy().tolist())","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:31.148537Z","iopub.execute_input":"2021-08-14T19:37:31.148859Z","iopub.status.idle":"2021-08-14T19:37:42.398428Z","shell.execute_reply.started":"2021-08-14T19:37:31.148827Z","shell.execute_reply":"2021-08-14T19:37:42.397381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.DataFrame({\"BraTS21ID\": ids, \"MGMT_value\": y_pred})\nsubmission.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:42.400364Z","iopub.execute_input":"2021-08-14T19:37:42.400752Z","iopub.status.idle":"2021-08-14T19:37:42.413401Z","shell.execute_reply.started":"2021-08-14T19:37:42.400705Z","shell.execute_reply":"2021-08-14T19:37:42.412331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:42.415342Z","iopub.execute_input":"2021-08-14T19:37:42.415782Z","iopub.status.idle":"2021-08-14T19:37:42.433896Z","shell.execute_reply.started":"2021-08-14T19:37:42.415740Z","shell.execute_reply":"2021-08-14T19:37:42.432679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

wandb System Metrics

","metadata":{}},{"cell_type":"markdown","source":"Yet another interesting fact is that you can view your hardware utilization too in the wandb dashboard. I am putting up some examples here for reference. These can be viewed in my [project page](https://wandb.ai/ishandutta/brain-tumor-viz/runs/1ipijldy/overview?workspace=user-ishandutta) as well.","metadata":{}},{"cell_type":"code","source":"# Store all wandb image paths in a list\n\nwandb_img_paths = []\nfolder_path = \"../input/wandb-rsna/wandb_1.png\"\n\nfor i in range(1, 8):\n path = \"../input/wandb-rsna/wandb_\" + str(i) + \".png\"\n wandb_img_paths.append(path)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:46.293912Z","iopub.execute_input":"2021-08-14T19:37:46.294256Z","iopub.status.idle":"2021-08-14T19:37:46.300550Z","shell.execute_reply.started":"2021-08-14T19:37:46.294212Z","shell.execute_reply":"2021-08-14T19:37:46.299630Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_img(img_path):\n \"\"\"\n Function which takes an image path and displays it.\n \n params: img_path(str): Path of Image to be displayed\n \"\"\"\n\n fig = matplotlib.pyplot.gcf()\n fig.set_size_inches(25.5, 17.5)\n\n img = cv2.imread(img_path)\n\n plt.axis('off')\n plt.imshow(img)","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:52.542653Z","iopub.execute_input":"2021-08-14T19:37:52.542985Z","iopub.status.idle":"2021-08-14T19:37:52.549099Z","shell.execute_reply.started":"2021-08-14T19:37:52.542954Z","shell.execute_reply":"2021-08-14T19:37:52.548215Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[0])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:53.556537Z","iopub.execute_input":"2021-08-14T19:37:53.556901Z","iopub.status.idle":"2021-08-14T19:37:53.945128Z","shell.execute_reply.started":"2021-08-14T19:37:53.556868Z","shell.execute_reply":"2021-08-14T19:37:53.944130Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[1])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:58.085411Z","iopub.execute_input":"2021-08-14T19:37:58.085729Z","iopub.status.idle":"2021-08-14T19:37:58.625498Z","shell.execute_reply.started":"2021-08-14T19:37:58.085700Z","shell.execute_reply":"2021-08-14T19:37:58.624701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[2])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:37:59.634179Z","iopub.execute_input":"2021-08-14T19:37:59.634556Z","iopub.status.idle":"2021-08-14T19:37:59.996820Z","shell.execute_reply.started":"2021-08-14T19:37:59.634522Z","shell.execute_reply":"2021-08-14T19:37:59.996039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[3])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:38:00.411687Z","iopub.execute_input":"2021-08-14T19:38:00.412011Z","iopub.status.idle":"2021-08-14T19:38:00.777547Z","shell.execute_reply.started":"2021-08-14T19:38:00.411981Z","shell.execute_reply":"2021-08-14T19:38:00.776530Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[4])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:38:00.975993Z","iopub.execute_input":"2021-08-14T19:38:00.976326Z","iopub.status.idle":"2021-08-14T19:38:01.521482Z","shell.execute_reply.started":"2021-08-14T19:38:00.976275Z","shell.execute_reply":"2021-08-14T19:38:01.520513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[5])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:38:01.807088Z","iopub.execute_input":"2021-08-14T19:38:01.807461Z","iopub.status.idle":"2021-08-14T19:38:02.167961Z","shell.execute_reply.started":"2021-08-14T19:38:01.807428Z","shell.execute_reply":"2021-08-14T19:38:02.167190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_img(wandb_img_paths[6])","metadata":{"execution":{"iopub.status.busy":"2021-08-14T19:38:02.479106Z","iopub.execute_input":"2021-08-14T19:38:02.479482Z","iopub.status.idle":"2021-08-14T19:38:02.838996Z","shell.execute_reply.started":"2021-08-14T19:38:02.479449Z","shell.execute_reply":"2021-08-14T19:38:02.838163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n

References

","metadata":{}},{"cell_type":"markdown","source":"> **[01]** [Magnetic Resonance Imaging (MRI) of the Brain and Spine: Basics](https://case.edu/med/neurology/NR/MRI%20Basics.htm) \n> **[02]** [\n> Brain Tumor EDA and Interactive Viz with W&B](https://www.kaggle.com/ayuraj/brain-tumor-eda-and-interactive-viz-with-w-b) \n> **[03]** [šŸ§ Brain TumoršŸ§  - EDA with Animations and Modeling](https://www.kaggle.com/ihelon/brain-tumor-eda-with-animations-and-modeling/data)","metadata":{}},{"cell_type":"markdown","source":"
\n

This notebook will be updated frequently so keep checking for furthur developments.

","metadata":{}},{"cell_type":"markdown","source":"### Connect with me on [LinkedIn](https://www.linkedin.com/in/ishandutta0098) :-)","metadata":{}}]}