{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-competitions/kaggle/29653/logos/header.png)","metadata":{}},{"cell_type":"markdown","source":"# RSNA-MICCAI Brain Tumor Radiogenomic Classification - Exploratory Data Analysis and Modeling\n\n\n### Predict the status of a genetic biomarker important for brain cancer treatment\n\nQuick Exploratory Data Analysis for [RSNA-MICCAI Brain Tumor Radiogenomic Classification](https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification) challenge    \n","metadata":{}},{"cell_type":"markdown","source":"### The work uses some ideas :\n\n- https://www.kaggle.com/ihelon/brain-tumor-eda-with-animations-and-modeling\n- https://www.kaggle.com/rluethy/efficientnet3d-with-one-mri-type\n- https://www.kaggle.com/smoschou55/dicom-to-2d-resized-axial-pngs-256x256-x36 \n- https://www.kaggle.com/abhranta/transfer-learning-pytorch-png-brain-tumor","metadata":{}},{"cell_type":"markdown","source":"## If you find this kernel interesting, please drop an upvote. It motivates me to produce more quality content :)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Files Provided\n**train/** - folder containing the training files, with each top-level folder representing a subject  \n**train_labels.csv** - file containing the target MGMT_value for each subject in the training data (e.g. the presence of MGMT promoter methylation)   \n**test/** - the test files, which use the same structure as train/; your task is to predict the MGMT_value for each subject in the test data. NOTE: the total size of the rerun test set (Public and Private) is ~5x the size of the Public test set   \n**sample_submission.csv** - a sample submission file in the correct formatcontrast (T1Gd)\nT2-weighted (T2)","metadata":{}},{"cell_type":"markdown","source":"## The exact mpMRI scans included are:\n\n- Fluid Attenuated Inversion Recovery (FLAIR)\n- T1-weighted pre-contrast (T1w)\n- T1-weighted post-contrast (T1Gd)\n- T2-weighted (T2)","metadata":{}},{"cell_type":"markdown","source":"## Exact folder structure:\n\n\n```\nTraining/Validation/Testing\n│\n└─── 00000\n│   │\n│   └─── FLAIR\n│   │   │ Image-1.dcm\n│   │   │ Image-2.dcm\n│   │   │ ...\n│   │   \n│   └─── T1w\n│   │   │ Image-1.dcm\n│   │   │ Image-2.dcm\n│   │   │ ...\n│   │   \n│   └─── T1wCE\n│   │   │ Image-1.dcm\n│   │   │ Image-2.dcm\n│   │   │ ...\n│   │   \n│   └─── T2w\n│   │   │ Image-1.dcm\n│   │   │ Image-2.dcm\n│   │   │ .....\n│   \n└─── 00001\n│   │ ...\n│   \n│ ...   \n│   \n└─── 00002\n│   │ ...\n```","metadata":{}},{"cell_type":"markdown","source":"# Quick Defination of Some Terms","metadata":{}},{"cell_type":"markdown","source":"## What is MRI ?","metadata":{}},{"cell_type":"markdown","source":"MRI\nMagnetic resonance imaging (MRI) is a medical imaging technique used in radiology to form pictures of the anatomy and the physiological processes of the body. MRI scanners use strong magnetic fields, magnetic field gradients, and radio waves to generate images of the organs in the body. MRI does not involve X-rays or the use of ionizing radiation, which distinguishes it from CT and PET scans. MRI is a medical application of nuclear magnetic resonance (NMR) which can also be used for imaging in other NMR applications, such as NMR spectroscopy.","metadata":{}},{"cell_type":"markdown","source":"<img src = \"https://th.bing.com/th/id/R.1a075468649f3edcdd9eddd1d2912311?rik=g%2bzRhNm9n5Rzug&riu=http%3a%2f%2fupload.wikimedia.org%2fwikipedia%2fcommons%2fd%2fde%2fNPH_MRI_274.gif&ehk=L5YlZU3h38C7qw7m1kLoMa17XdpwSmG7uyTrfFhljzc%3d&risl=1&pid=ImgRaw&r=0\">","metadata":{}},{"cell_type":"markdown","source":"## What are genetic biomarkers ?\nA biomarker is a biological molecule found in blood, other body fluids, or tissues that is a sign of a normal or abnormal process, or of a condition or disease. A biomarker may be used to see how well the body responds to a treatment for a disease or condition. Also called molecular marker and signature molecule.\n\nBiomarkers are indicators of a process going on inside a body. When doctors suspect that something abnormal is going on in the body, they look for signs. These biomarkers are the red flags that pop up to indicate that something is going on.","metadata":{}},{"cell_type":"code","source":"# Lets get to Topic","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:09:45.730054Z","iopub.execute_input":"2021-08-24T15:09:45.730406Z","iopub.status.idle":"2021-08-24T15:09:45.734111Z","shell.execute_reply.started":"2021-08-24T15:09:45.730351Z","shell.execute_reply":"2021-08-24T15:09:45.733061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 1. Import Libraries","metadata":{}},{"cell_type":"code","source":"import os\nimport json\nimport glob\nimport random\nimport collections\n\nimport numpy as np\nimport pandas as pd\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:02.622945Z","iopub.execute_input":"2021-08-24T15:30:02.623314Z","iopub.status.idle":"2021-08-24T15:30:03.637234Z","shell.execute_reply.started":"2021-08-24T15:30:02.623226Z","shell.execute_reply":"2021-08-24T15:30:03.636435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv\")\ntrain_df","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:06.530959Z","iopub.execute_input":"2021-08-24T15:30:06.531291Z","iopub.status.idle":"2021-08-24T15:30:06.554246Z","shell.execute_reply.started":"2021-08-24T15:30:06.531261Z","shell.execute_reply":"2021-08-24T15:30:06.553332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Exploratory Data Analysis","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(5, 5))\nsns.countplot(data=train_df, x=\"MGMT_value\");","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:17.929256Z","iopub.execute_input":"2021-08-24T15:30:17.929608Z","iopub.status.idle":"2021-08-24T15:30:18.04879Z","shell.execute_reply.started":"2021-08-24T15:30:17.929578Z","shell.execute_reply":"2021-08-24T15:30:18.047705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Load the Images","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:19.623845Z","iopub.execute_input":"2021-08-24T15:30:19.624164Z","iopub.status.idle":"2021-08-24T15:30:19.627816Z","shell.execute_reply.started":"2021-08-24T15:30:19.624136Z","shell.execute_reply":"2021-08-24T15:30:19.626972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_dicom(path):\n    dicom = pydicom.read_file(path)\n    data = dicom.pixel_array\n    data = data - np.min(data)\n    if np.max(data) != 0:\n        data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n    return data\n\n\ndef visualize_sample(\n    brats21id, \n    slice_i,\n    mgmt_value,\n    types=(\"FLAIR\", \"T1w\", \"T1wCE\", \"T2w\")\n):\n    plt.figure(figsize=(16, 5))\n    patient_path = os.path.join(\n        \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/\", \n        str(brats21id).zfill(5),\n    )\n    for i, t in enumerate(types, 1):\n        t_paths = sorted(\n            glob.glob(os.path.join(patient_path, t, \"*\")), \n            key=lambda x: int(x[:-4].split(\"-\")[-1]),\n        )\n        data = load_dicom(t_paths[int(len(t_paths) * slice_i)])\n        plt.subplot(1, 4, i)\n        plt.imshow(data, cmap=\"gray\")\n        plt.title(f\"{t}\", fontsize=16)\n        plt.axis(\"off\")\n\n    plt.suptitle(f\"MGMT_value: {mgmt_value}\", fontsize=16)\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:20.833269Z","iopub.execute_input":"2021-08-24T15:30:20.833607Z","iopub.status.idle":"2021-08-24T15:30:20.842214Z","shell.execute_reply.started":"2021-08-24T15:30:20.833576Z","shell.execute_reply":"2021-08-24T15:30:20.841435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in random.sample(range(train_df.shape[0]), 5):\n    _brats21id = train_df.iloc[i][\"BraTS21ID\"]\n    _mgmt_value = train_df.iloc[i][\"MGMT_value\"]\n    visualize_sample(brats21id=_brats21id, mgmt_value=_mgmt_value, slice_i=0.5)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:23.606171Z","iopub.execute_input":"2021-08-24T15:30:23.606553Z","iopub.status.idle":"2021-08-24T15:30:25.742178Z","shell.execute_reply.started":"2021-08-24T15:30:23.60652Z","shell.execute_reply":"2021-08-24T15:30:25.741317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from matplotlib import animation, rc\nrc('animation', html='jshtml')\n\n\ndef create_animation(ims):\n    fig = plt.figure(figsize=(6, 6))\n    plt.axis('off')\n    im = plt.imshow(ims[0], cmap=\"gray\")\n\n    def animate_func(i):\n        im.set_array(ims[i])\n        return [im]\n\n    return animation.FuncAnimation(fig, animate_func, frames = len(ims), interval = 1000//24)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:30.925996Z","iopub.execute_input":"2021-08-24T15:30:30.926345Z","iopub.status.idle":"2021-08-24T15:30:30.932542Z","shell.execute_reply.started":"2021-08-24T15:30:30.926314Z","shell.execute_reply":"2021-08-24T15:30:30.931728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_dicom_line(path):\n    t_paths = sorted(\n        glob.glob(os.path.join(path, \"*\")), \n        key=lambda x: int(x[:-4].split(\"-\")[-1]),\n    )\n    images = []\n    for filename in t_paths:\n        data = load_dicom(filename)\n        if data.max() == 0:\n            continue\n        images.append(data)\n        \n    return images","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:32.10943Z","iopub.execute_input":"2021-08-24T15:30:32.109833Z","iopub.status.idle":"2021-08-24T15:30:32.121457Z","shell.execute_reply.started":"2021-08-24T15:30:32.109794Z","shell.execute_reply":"2021-08-24T15:30:32.120406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Animation 1","metadata":{}},{"cell_type":"code","source":"images = load_dicom_line(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/FLAIR\")\ncreate_animation(images)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:30:41.256502Z","iopub.execute_input":"2021-08-24T15:30:41.256827Z","iopub.status.idle":"2021-08-24T15:30:59.087306Z","shell.execute_reply.started":"2021-08-24T15:30:41.256797Z","shell.execute_reply":"2021-08-24T15:30:59.086343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Animation 2","metadata":{}},{"cell_type":"code","source":"images = load_dicom_line(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/T2w\")\ncreate_animation(images)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:31:09.43069Z","iopub.execute_input":"2021-08-24T15:31:09.431024Z","iopub.status.idle":"2021-08-24T15:31:26.813195Z","shell.execute_reply.started":"2021-08-24T15:31:09.430995Z","shell.execute_reply":"2021-08-24T15:31:26.812391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Plotting 3D MRI scans","metadata":{}},{"cell_type":"code","source":"'''\nTo plot them, we need to rasterizd images into a point cloud with reduced dimensionality.\nPassing each scanned pixel into our visualization would net us more than a million points,\nso we need to\n1) resize every image to 128x128 \n2) downsample space without tumor for brevity.\n'''","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tarfile\nimport plotly\nfrom plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:31:34.424813Z","iopub.execute_input":"2021-08-24T15:31:34.42513Z","iopub.status.idle":"2021-08-24T15:31:34.442204Z","shell.execute_reply.started":"2021-08-24T15:31:34.4251Z","shell.execute_reply":"2021-08-24T15:31:34.441317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def extract_task1_files(root=\"./data\"):\n    tar = tarfile.open(\"../input/brats-2021-task1/BraTS2021_Training_Data.tar\")\n    tar.extractall(root)\n    tar.close()","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:25:48.392836Z","iopub.execute_input":"2021-08-24T15:25:48.393151Z","iopub.status.idle":"2021-08-24T15:25:48.399209Z","shell.execute_reply.started":"2021-08-24T15:25:48.393118Z","shell.execute_reply":"2021-08-24T15:25:48.398385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"extract_task1_files()\n","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:25:50.570148Z","iopub.execute_input":"2021-08-24T15:25:50.570467Z","iopub.status.idle":"2021-08-24T15:28:44.31196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import nibabel as nib\nimport os\nimport albumentations as A\nimport numpy as np\n\n\nclass ImageReader:\n    def __init__(self, root:str, img_size:int=256, normalize:bool=False, single_class:bool=False):\n        pad_size = 256 if img_size > 256 else 224\n        self.resize = A.Compose(\n            [\n                A.PadIfNeeded(min_height=pad_size, min_width=pad_size, value=0),\n                A.Resize(img_size, img_size)\n            ]\n        )\n        self.normalize=normalize\n        self.single_class=single_class\n        self.root=root\n        \n    def read_file(self, path:str) -> dict:\n        scan_type = path.split('_')[-1]\n        raw_image = nib.load(path).get_fdata()\n        raw_mask = nib.load(path.replace(scan_type, 'seg.nii.gz')).get_fdata()\n        processed_frames, processed_masks = [], []\n        for frame_idx in range(raw_image.shape[2]):\n            frame = raw_image[:, :, frame_idx]\n            mask = raw_mask[:, :, frame_idx]\n            if self.normalize:\n                if frame.max() > 0:\n                    frame = frame/frame.max()\n                frame = frame.astype(np.float32)\n            else:\n                frame = frame.astype(np.uint8)\n            resized = self.resize(image=frame, mask=mask)\n            processed_frames.append(resized['image'])\n            processed_masks.append(1*(resized['mask'] > 0) if self.single_class else resized['mask'])\n        return {\n            'scan': np.stack(processed_frames, 0),\n            'segmentation': np.stack(processed_masks, 0),\n            'orig_shape': raw_image.shape\n        }\n    \n    def load_patient_scan(self, idx:int, scan_type:str='flair') -> dict:\n        patient_id = str(idx).zfill(5)\n        scan_filename = f'{self.root}/BraTS2021_{patient_id}/BraTS2021_{patient_id}_{scan_type}.nii.gz'\n        return self.read_file(scan_filename)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:31:40.55429Z","iopub.execute_input":"2021-08-24T15:31:40.554659Z","iopub.status.idle":"2021-08-24T15:31:41.470568Z","shell.execute_reply.started":"2021-08-24T15:31:40.554627Z","shell.execute_reply":"2021-08-24T15:31:41.469732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\nimport numpy as np\n\n\ndef generate_3d_scatter(\n    x:np.array, y:np.array, z:np.array, colors:np.array,\n    size:int=3, opacity:float=0.2, scale:str='Teal',\n    hover:str='skip', name:str='MRI'\n) -> go.Scatter3d:\n    return go.Scatter3d(\n        x=x, y=y, z=z,\n        mode='markers', hoverinfo=hover,\n        marker = dict(\n            size=size, opacity=opacity,\n            color=colors, colorscale=scale\n        ),\n        name=name\n    )\n\n\nclass ImageViewer3d():\n    def __init__(\n        self, reader:ImageReader, mri_downsample:int=10, mri_colorscale:str='Ice'\n    ) -> None:\n        self.reader = reader\n        self.mri_downsample = mri_downsample\n        self.mri_colorscale = mri_colorscale\n\n    def load_clean_mri(self, image:np.array, orig_dim:int) -> dict:\n        shape_offset = image.shape[1]/orig_dim\n        z, x, y = (image > 0).nonzero()\n        # only (1/mri_downsample) is sampled for the resulting image\n        x, y, z = x[::self.mri_downsample], y[::self.mri_downsample], z[::self.mri_downsample]\n        colors = image[z, x, y]\n        return dict(x=x/shape_offset, y=y/shape_offset, z=z, colors=colors)\n    \n    def load_tumor_segmentation(self, image:np.array, orig_dim:int) -> dict:\n        tumors = {}\n        shape_offset = image.shape[1]/orig_dim\n        # 1/1, 1/3 and 1/5 pixels for tumor tissue classes 1(core), 2(invaded) and 4(enhancing)\n        sampling = {\n            1: 1, 2: 3, 4: 5\n        }\n        for class_idx in sampling:\n            z, x, y = (image == class_idx).nonzero()\n            x, y, z = x[::sampling[class_idx]], y[::sampling[class_idx]], z[::sampling[class_idx]]\n            tumors[class_idx] = dict(\n                x=x/shape_offset, y=y/shape_offset, z=z,\n                colors=class_idx/4\n            )\n        return tumors\n    \n    def collect_patient_data(self, scan:dict) -> tuple:\n        clean_mri = self.load_clean_mri(scan['scan'], scan['orig_shape'][0])\n        tumors = self.load_tumor_segmentation(scan['segmentation'], scan['orig_shape'][0])\n        markers_created = clean_mri['x'].shape[0] + sum(tumors[class_idx]['x'].shape[0] for class_idx in tumors)\n        return [\n            generate_3d_scatter(**clean_mri, scale=self.mri_colorscale, opacity=0.3, hover='skip', name='Brain MRI'),\n            generate_3d_scatter(**tumors[1], opacity=0.8, hover='all', name='Necrotic tumor core'),\n            generate_3d_scatter(**tumors[2], opacity=0.4, hover='all', name='Peritumoral invaded tissue'),\n            generate_3d_scatter(**tumors[4], opacity=0.4, hover='all', name='GD-enhancing tumor'),\n        ], markers_created\n    \n    def get_3d_scan(self, patient_idx:int, scan_type:str='flair') -> go.Figure:\n        scan = self.reader.load_patient_scan(patient_idx, scan_type)\n        data, num_markers = self.collect_patient_data(scan)\n        fig = go.Figure(data=data)\n        fig.update_layout(\n            title=f\"[Patient id:{patient_idx}] brain MRI scan ({num_markers} points)\",\n            legend_title=\"Pixel class (click to enable/disable)\",\n            font=dict(\n                family=\"Courier New, monospace\",\n                size=14,\n            ),\n            margin=dict(\n                l=0,r=0,b=0,t=30\n            ),\n            legend=dict(itemsizing='constant')\n        )\n        return fig","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:31:43.226689Z","iopub.execute_input":"2021-08-24T15:31:43.227067Z","iopub.status.idle":"2021-08-24T15:31:43.264131Z","shell.execute_reply.started":"2021-08-24T15:31:43.227022Z","shell.execute_reply":"2021-08-24T15:31:43.263385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"reader = ImageReader('./data', img_size=128, normalize=True, single_class=False)\nviewer = ImageViewer3d(reader, mri_downsample=25)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:31:44.944255Z","iopub.execute_input":"2021-08-24T15:31:44.944607Z","iopub.status.idle":"2021-08-24T15:31:44.949067Z","shell.execute_reply.started":"2021-08-24T15:31:44.944577Z","shell.execute_reply":"2021-08-24T15:31:44.947773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = viewer.get_3d_scan(0, 't1')\nplotly.offline.iplot(fig)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:31:47.39641Z","iopub.execute_input":"2021-08-24T15:31:47.396758Z","iopub.status.idle":"2021-08-24T15:31:48.647564Z","shell.execute_reply.started":"2021-08-24T15:31:47.39673Z","shell.execute_reply":"2021-08-24T15:31:48.64678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = viewer.get_3d_scan(9, 'flair')\nplotly.offline.iplot(fig)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:33:51.167371Z","iopub.execute_input":"2021-08-24T15:33:51.167716Z","iopub.status.idle":"2021-08-24T15:33:51.546234Z","shell.execute_reply.started":"2021-08-24T15:33:51.167682Z","shell.execute_reply":"2021-08-24T15:33:51.545256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Feature Engineering","metadata":{}},{"cell_type":"code","source":"from skimage.morphology import binary_closing\nimport plotly.express as px\n\ndata = reader.load_patient_scan(0)\n\nimage = data['scan'][60]\nmasked_image = 1 * (image > 0)\nfilled_image = 1 * binary_closing(image)\n\npx.imshow(\n    np.array([image, masked_image, filled_image]),\n    facet_col=0, title=\"Different image masking - none, threshold and binary closing\",\n)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:35:01.947166Z","iopub.execute_input":"2021-08-24T15:35:01.947526Z","iopub.status.idle":"2021-08-24T15:35:02.266207Z","shell.execute_reply.started":"2021-08-24T15:35:01.947492Z","shell.execute_reply":"2021-08-24T15:35:02.265423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_approx_pixel_count(scan:np.array, close:bool=False, mask:bool=False, mask_idx:int=-1) -> int:\n    slice_areas = []\n    for slice_idx in range(scan.shape[0]):\n        if close:\n            mri = 1 * binary_closing(scan[slice_idx, :, :])\n        elif mask_idx >= 0:\n            mri = 1 * (scan[slice_idx, :, :] == mask_idx)\n        elif mask:\n            mri = 1 * (scan[slice_idx, :, :] > 0)\n        else:\n            raise ValueError('Masking mechanism should be specified')\n        mri_area = mri.sum()\n        slice_areas.append(mri_area)\n    return np.sum(slice_areas)\n\nget_approx_pixel_count(data['segmentation'], mask=True) / get_approx_pixel_count(data['scan'], mask=True)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:35:31.215615Z","iopub.execute_input":"2021-08-24T15:35:31.215928Z","iopub.status.idle":"2021-08-24T15:35:31.242272Z","shell.execute_reply.started":"2021-08-24T15:35:31.215899Z","shell.execute_reply":"2021-08-24T15:35:31.24133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_centroid(scan:np.array, mask_idx:int=1) -> list:\n    z, x, y = (scan == mask_idx).nonzero()\n    x, y, z = np.median(x), np.median(y), np.median(z)\n    return [x/scan.shape[1], y/scan.shape[2], z/scan.shape[0]]\n\nget_centroid(data['segmentation'], 4), get_centroid(data['segmentation'], 1)","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:36:46.844985Z","iopub.execute_input":"2021-08-24T15:36:46.845411Z","iopub.status.idle":"2021-08-24T15:36:46.884401Z","shell.execute_reply.started":"2021-08-24T15:36:46.845375Z","shell.execute_reply":"2021-08-24T15:36:46.883605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\ndf = pd.read_csv('../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv')\ntargets = dict(zip(df.BraTS21ID, df.MGMT_value))","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:36:55.575683Z","iopub.execute_input":"2021-08-24T15:36:55.576012Z","iopub.status.idle":"2021-08-24T15:36:55.586758Z","shell.execute_reply.started":"2021-08-24T15:36:55.575983Z","shell.execute_reply":"2021-08-24T15:36:55.585957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\nfeatures = []\nfor patient_idx in targets:\n    try:\n        data = reader.load_patient_scan(patient_idx)\n        scan_px = get_approx_pixel_count(data['scan'], mask=True)\n        tumor_px = get_approx_pixel_count(data['segmentation'], mask=True)\n        dimension = np.product(data['scan'].shape)\n        patient_features = [patient_idx, targets[patient_idx]]\n        patient_features.extend([scan_px/dimension, tumor_px/dimension, tumor_px/scan_px])\n        patient_features.extend(get_centroid(data['segmentation'], 4))\n        features.append(patient_features)\n    except FileNotFoundError:\n        continue","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:37:11.609853Z","iopub.execute_input":"2021-08-24T15:37:11.610171Z","iopub.status.idle":"2021-08-24T15:39:42.873106Z","shell.execute_reply.started":"2021-08-24T15:37:11.610141Z","shell.execute_reply":"2021-08-24T15:39:42.872188Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame(features, columns=['idx', 'target', 'scan_pct', 'tumor_pct', 'tumor_ratio', 'x', 'y', 'z']).set_index('idx')\ndf","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:40:00.110566Z","iopub.execute_input":"2021-08-24T15:40:00.110899Z","iopub.status.idle":"2021-08-24T15:40:00.135859Z","shell.execute_reply.started":"2021-08-24T15:40:00.110869Z","shell.execute_reply":"2021-08-24T15:40:00.134738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.histogram(df, x=\"tumor_pct\", color=\"target\", marginal=\"box\", nbins=100, barmode='relative')\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:40:18.323782Z","iopub.execute_input":"2021-08-24T15:40:18.324171Z","iopub.status.idle":"2021-08-24T15:40:18.47781Z","shell.execute_reply.started":"2021-08-24T15:40:18.324128Z","shell.execute_reply":"2021-08-24T15:40:18.476876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2021-08-24T15:55:28.71207Z","iopub.execute_input":"2021-08-24T15:55:28.712406Z","iopub.status.idle":"2021-08-24T15:55:28.718185Z","shell.execute_reply.started":"2021-08-24T15:55:28.71237Z","shell.execute_reply":"2021-08-24T15:55:28.717295Z"},"trusted":true},"execution_count":null,"outputs":[]}]}