{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](http://storage.googleapis.com/kaggle-competitions/kaggle/29653/logos/header.png?t=2021-07-07-17-26-56)\n# Interactive Task1 EDA\nIn this notebook, you will find\n* How useful are Task1 scans for Task2 (this competition).\n* How to build interactive figures with Plotly.\n* That Task2 is not about finding tumors.\n","metadata":{}},{"cell_type":"code","source":"import tarfile\nimport plotly\nfrom plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot\n\ninit_notebook_mode(connected=True)\n\ndef extract_task1_files(root=\"./data\"):\n    tar = tarfile.open(\"../input/brats-2021-task1/BraTS2021_Training_Data.tar\")\n    tar.extractall(root)\n    tar.close()","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:20:10.364788Z","iopub.execute_input":"2023-05-26T00:20:10.365145Z","iopub.status.idle":"2023-05-26T00:20:10.373188Z","shell.execute_reply.started":"2023-05-26T00:20:10.365112Z","shell.execute_reply":"2023-05-26T00:20:10.372170Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"extract_task1_files()","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:20:10.375511Z","iopub.execute_input":"2023-05-26T00:20:10.375987Z","iopub.status.idle":"2023-05-26T00:22:06.737470Z","shell.execute_reply.started":"2023-05-26T00:20:10.375930Z","shell.execute_reply":"2023-05-26T00:22:06.736594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 1. Introduction","metadata":{}},{"cell_type":"markdown","source":"RSNA-MICCAI Brain Tumor Radiogenomic Classification is a second part to the RSNA-ASNR-MICCAI BraTS 2021 challenge. First part focuses on multiclass segmentation, this one (Task 2) - on binary classification.\n\n> The RSNA-ASNR-MICCAI BraTS 2021 challenge utilizes ... mpMRI scans, and focuses on (**Task 1**) the evaluation of state-of-the-art methods for the **segmentation** of intrinsically heterogeneous **brain glioblastoma sub-regions** in mpMRI scans.\n>\n> Furthermore, this year's challenge also focuses on (**Task 2**) the evaluation of **classification** methods to predict the **MGMT promoter methylation status** at pre-operative baseline scans.\n\nTask 1 sub-regions are defined as\n\n> ... the **GD-enhancing tumor** (ET — label **4**), the peritumoral edematous/**invaded tissue** (ED — label **2**), and the **necrotic tumor core** (NCR — label **1**).\n\nBoth tasks share matching patient ids. Can task1 scans be used in task2?","metadata":{}},{"cell_type":"markdown","source":"# 2. Plotting 3D MRI scans","metadata":{}},{"cell_type":"markdown","source":"First, let's take a peek at how task1 scans look in 3D. To plot them, we need to rasterize stacked images into a point cloud with reduced dimensionality. Passing each scanned pixel into our visualization would net us more than a million points, so we need to 1) resize every image to 128x128 and 2) downsample space without tumor for brevity.","metadata":{}},{"cell_type":"code","source":"import nibabel as nib\nimport os\nimport albumentations as A\nimport numpy as np\n\n\nclass ImageReader:\n    def __init__(\n        self, root:str, img_size:int=256,\n        normalize:bool=False, single_class:bool=False\n    ) -> None:\n        pad_size = 256 if img_size > 256 else 224\n        self.resize = A.Compose(\n            [\n                A.PadIfNeeded(min_height=pad_size, min_width=pad_size, value=0),\n                A.Resize(img_size, img_size)\n            ]\n        )\n        self.normalize=normalize\n        self.single_class=single_class\n        self.root=root\n        \n    def read_file(self, path:str) -> dict:\n        scan_type = path.split('_')[-1]\n        raw_image = nib.load(path).get_fdata()\n        raw_mask = nib.load(path.replace(scan_type, 'seg.nii.gz')).get_fdata()\n        processed_frames, processed_masks = [], []\n        for frame_idx in range(raw_image.shape[2]):\n            frame = raw_image[:, :, frame_idx]\n            mask = raw_mask[:, :, frame_idx]\n            resized = self.resize(image=frame, mask=mask)\n            processed_frames.append(resized['image'])\n            processed_masks.append(\n                1*(resized['mask'] > 0) if self.single_class else resized['mask']\n            )\n        scan_data = np.stack(processed_frames, 0)\n        if self.normalize:\n            if scan_data.max() > 0:\n                scan_data = scan_data/scan_data.max()\n            scan_data = scan_data.astype(np.float32)\n        return {\n            'scan': scan_data,\n            'segmentation': np.stack(processed_masks, 0),\n            'orig_shape': raw_image.shape\n        }\n    \n    def load_patient_scan(self, idx:int, scan_type:str='flair') -> dict:\n        patient_id = str(idx).zfill(5)\n        scan_filename = f'{self.root}/BraTS2021_{patient_id}/BraTS2021_{patient_id}_{scan_type}.nii.gz'\n        return self.read_file(scan_filename)\n            ","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:06.739838Z","iopub.execute_input":"2023-05-26T00:22:06.740191Z","iopub.status.idle":"2023-05-26T00:22:09.174470Z","shell.execute_reply.started":"2023-05-26T00:22:06.740158Z","shell.execute_reply":"2023-05-26T00:22:09.173521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A 3D point cloud is visualized by utilizing the Plotly library. Generating a trace (plotly.graph_objects.Scatter3d) per tissue type allows us to simultaneously show different point clouds with different opacities on a single 3D graph (plotly.graph_objects.Figure).\nThe resulting figure is interactive. Try to rotate it or disable overlaying tumor tissue types.","metadata":{}},{"cell_type":"code","source":"import plotly.graph_objects as go\nimport numpy as np\n\n\ndef generate_3d_scatter(\n    x:np.array, y:np.array, z:np.array, colors:np.array,\n    size:int=3, opacity:float=0.2, scale:str='Teal',\n    hover:str='skip', name:str='MRI'\n) -> go.Scatter3d:\n    return go.Scatter3d(\n        x=x, y=y, z=z,\n        mode='markers', hoverinfo=hover,\n        marker = dict(\n            size=size, opacity=opacity,\n            color=colors, colorscale=scale\n        ),\n        name=name\n    )\n\n\nclass ImageViewer3d():\n    def __init__(\n        self, reader:ImageReader,\n        mri_downsample:int=10, mri_colorscale:str='Ice'\n    ) -> None:\n        self.reader = reader\n        self.mri_downsample = mri_downsample\n        self.mri_colorscale = mri_colorscale\n\n    def load_clean_mri(self, image:np.array, orig_dim:int) -> dict:\n        shape_offset = image.shape[1]/orig_dim\n        z, x, y = (image > 0).nonzero()\n        # only (1/mri_downsample) is sampled for the resulting image\n        x, y, z = x[::self.mri_downsample], y[::self.mri_downsample], z[::self.mri_downsample]\n        colors = image[z, x, y]\n        return dict(x=x/shape_offset, y=y/shape_offset, z=z, colors=colors)\n    \n    def load_tumor_segmentation(self, image:np.array, orig_dim:int) -> dict:\n        tumors = {}\n        shape_offset = image.shape[1]/orig_dim\n        # 1/1, 1/3 and 1/5 pixels for tumor tissue classes 1(core), 2(invaded) and 4(enhancing)\n        sampling = {\n            1: 1, 2: 3, 4: 5\n        }\n        for class_idx in sampling:\n            z, x, y = (image == class_idx).nonzero()\n            x, y, z = x[::sampling[class_idx]], y[::sampling[class_idx]], z[::sampling[class_idx]]\n            tumors[class_idx] = dict(\n                x=x/shape_offset, y=y/shape_offset, z=z,\n                colors=class_idx/4\n            )\n        return tumors\n    \n    def collect_patient_data(self, scan:dict) -> tuple:\n        clean_mri = self.load_clean_mri(scan['scan'], scan['orig_shape'][0])\n        tumors = self.load_tumor_segmentation(scan['segmentation'], scan['orig_shape'][0])\n        markers_created = clean_mri['x'].shape[0] + sum(tumors[class_idx]['x'].shape[0] for class_idx in tumors)\n        return [\n            generate_3d_scatter(\n                **clean_mri, scale=self.mri_colorscale, opacity=0.4,\n                hover='skip', name='Brain MRI'\n            ),\n            generate_3d_scatter(\n                **tumors[1], opacity=0.8,\n                hover='all', name='Necrotic tumor core'\n            ),\n            generate_3d_scatter(\n                **tumors[2], opacity=0.4,\n                hover='all', name='Peritumoral invaded tissue'\n            ),\n            generate_3d_scatter(\n                **tumors[4], opacity=0.4,\n                hover='all', name='GD-enhancing tumor'\n            ),\n        ], markers_created\n    \n    def get_3d_scan(self, patient_idx:int, scan_type:str='flair') -> go.Figure:\n        scan = self.reader.load_patient_scan(patient_idx, scan_type)\n        data, num_markers = self.collect_patient_data(scan)\n        fig = go.Figure(data=data)\n        fig.update_layout(\n            title=f\"[Patient id:{patient_idx}] brain MRI scan ({num_markers} points)\",\n            legend_title=\"Pixel class (click to enable/disable)\",\n            font=dict(\n                family=\"Courier New, monospace\",\n                size=14,\n            ),\n            margin=dict(\n                l=0, r=0, b=0, t=30\n            ),\n            legend=dict(itemsizing='constant')\n        )\n        return fig","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:09.175795Z","iopub.execute_input":"2023-05-26T00:22:09.176152Z","iopub.status.idle":"2023-05-26T00:22:09.219678Z","shell.execute_reply.started":"2023-05-26T00:22:09.176119Z","shell.execute_reply":"2023-05-26T00:22:09.218571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"reader = ImageReader('./data', img_size=128, normalize=True, single_class=False)\nviewer = ImageViewer3d(reader, mri_downsample=20)","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:09.220940Z","iopub.execute_input":"2023-05-26T00:22:09.221289Z","iopub.status.idle":"2023-05-26T00:22:09.226467Z","shell.execute_reply.started":"2023-05-26T00:22:09.221255Z","shell.execute_reply":"2023-05-26T00:22:09.225388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Positive scan: a tumor is present.","metadata":{}},{"cell_type":"code","source":"fig = viewer.get_3d_scan(0, 't1')\nplotly.offline.iplot(fig)","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:09.228069Z","iopub.execute_input":"2023-05-26T00:22:09.228689Z","iopub.status.idle":"2023-05-26T00:22:11.184487Z","shell.execute_reply.started":"2023-05-26T00:22:09.228644Z","shell.execute_reply":"2023-05-26T00:22:11.183081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Negative scan: a tumor is present too.","metadata":{}},{"cell_type":"code","source":"fig = viewer.get_3d_scan(9, 'flair')\nplotly.offline.iplot(fig)","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:11.187047Z","iopub.execute_input":"2023-05-26T00:22:11.187396Z","iopub.status.idle":"2023-05-26T00:22:12.126428Z","shell.execute_reply.started":"2023-05-26T00:22:11.187363Z","shell.execute_reply":"2023-05-26T00:22:12.124324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you can see, we're not looking at whether a tumor is present on an MRI scan, but rather *classifying a type of this tumor* (with or without MGMT promoter methylation).","metadata":{}},{"cell_type":"markdown","source":"# 3. Feature engineering","metadata":{}},{"cell_type":"markdown","source":"Let's collect a simple set of features - centroids for tumor cores and overall tumor size relative to a full MRI scan.","metadata":{}},{"cell_type":"code","source":"from skimage.morphology import binary_closing\nimport plotly.express as px\n\ndata = reader.load_patient_scan(0)\n\nimage = data['scan'][40]\nmasked_image = 1 * (image > 0)\nfilled_image = 1 * binary_closing(image)\n\npx.imshow(\n    np.array([image, masked_image, filled_image]),\n    facet_col=0, title=\"Different image masking - none, threshold and binary closing\",\n)","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:12.128023Z","iopub.execute_input":"2023-05-26T00:22:12.128331Z","iopub.status.idle":"2023-05-26T00:22:14.085727Z","shell.execute_reply.started":"2023-05-26T00:22:12.128301Z","shell.execute_reply":"2023-05-26T00:22:14.084670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Tumor to all tissue ratio can be (approximately) calculated as (sum of tumor pixels/sum of tissue pixels)","metadata":{}},{"cell_type":"code","source":"def get_approx_pixel_count(scan:np.array, close:bool=False, mask:bool=False, mask_idx:int=-1) -> int:\n    slice_areas = []\n    for slice_idx in range(scan.shape[0]):\n        if close:\n            mri = 1 * binary_closing(scan[slice_idx, :, :])\n        elif mask_idx >= 0:\n            mri = 1 * (scan[slice_idx, :, :] == mask_idx)\n        elif mask:\n            mri = 1 * (scan[slice_idx, :, :] > 0)\n        else:\n            raise ValueError('Masking mechanism should be specified')\n        mri_area = mri.sum()\n        slice_areas.append(mri_area)\n    return np.sum(slice_areas)\n\nget_approx_pixel_count(data['segmentation'], mask=True) / get_approx_pixel_count(data['scan'], mask=True)","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:14.086946Z","iopub.execute_input":"2023-05-26T00:22:14.087246Z","iopub.status.idle":"2023-05-26T00:22:14.121032Z","shell.execute_reply.started":"2023-05-26T00:22:14.087216Z","shell.execute_reply":"2023-05-26T00:22:14.119666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_centroid(scan:np.array, mask_idx:int=1) -> list:\n    z, x, y = (scan == mask_idx).nonzero()\n    x, y, z = np.median(x), np.median(y), np.median(z)\n    return [x/scan.shape[1], y/scan.shape[2], z/scan.shape[0]]\n\nget_centroid(data['segmentation'], 4), get_centroid(data['segmentation'], 1)","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:14.122489Z","iopub.execute_input":"2023-05-26T00:22:14.122833Z","iopub.status.idle":"2023-05-26T00:22:14.155766Z","shell.execute_reply.started":"2023-05-26T00:22:14.122800Z","shell.execute_reply":"2023-05-26T00:22:14.154846Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Putting everything into one DataFrame.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\ndf = pd.read_csv('../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv')\ntargets = dict(zip(df.BraTS21ID, df.MGMT_value))","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:14.157095Z","iopub.execute_input":"2023-05-26T00:22:14.157410Z","iopub.status.idle":"2023-05-26T00:22:14.183654Z","shell.execute_reply.started":"2023-05-26T00:22:14.157381Z","shell.execute_reply":"2023-05-26T00:22:14.182612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\nfeatures = []\nfor patient_idx in targets:\n    try:\n        data = reader.load_patient_scan(patient_idx)\n        scan_px = get_approx_pixel_count(data['scan'], mask=True)\n        tumor_px = get_approx_pixel_count(data['segmentation'], mask=True)\n        core_px = get_approx_pixel_count(data['segmentation'], mask_idx=4)\n        dimension = np.product(data['scan'].shape)\n        patient_features = [patient_idx, targets[patient_idx]]\n        patient_features.extend([scan_px/dimension, tumor_px/dimension, tumor_px/scan_px, core_px/tumor_px])\n        patient_features.extend(get_centroid(data['segmentation'], 4))\n        features.append(patient_features)\n    except FileNotFoundError:\n        continue","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:22:14.185016Z","iopub.execute_input":"2023-05-26T00:22:14.185334Z","iopub.status.idle":"2023-05-26T00:25:42.354674Z","shell.execute_reply.started":"2023-05-26T00:22:14.185303Z","shell.execute_reply":"2023-05-26T00:25:42.353192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame(\n    features, columns=['idx', 'target', 'scan_pct', 'tumor_pct', 'tumor_ratio', 'core_ratio', 'x', 'y', 'z']\n).set_index('idx')\n\ndf","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:42.356954Z","iopub.execute_input":"2023-05-26T00:25:42.357386Z","iopub.status.idle":"2023-05-26T00:25:42.402627Z","shell.execute_reply.started":"2023-05-26T00:25:42.357339Z","shell.execute_reply":"2023-05-26T00:25:42.401706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Is there a difference between 1 and 0 classes? Let's look at the tumor volume percent:","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(\n    df, x=\"tumor_pct\", color=\"target\", marginal=\"box\",\n    nbins=100, barmode='relative'\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:42.404335Z","iopub.execute_input":"2023-05-26T00:25:42.405082Z","iopub.status.idle":"2023-05-26T00:25:42.619209Z","shell.execute_reply.started":"2023-05-26T00:25:42.405027Z","shell.execute_reply":"2023-05-26T00:25:42.618138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Or tumor to matter ratio:","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(\n    df, x=\"tumor_ratio\", color=\"target\", marginal=\"box\",\n    nbins=100, barmode='relative'\n)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:42.620742Z","iopub.execute_input":"2023-05-26T00:25:42.621134Z","iopub.status.idle":"2023-05-26T00:25:42.757275Z","shell.execute_reply.started":"2023-05-26T00:25:42.621097Z","shell.execute_reply":"2023-05-26T00:25:42.756230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There appears to be a slight difference. Is it significant enough to help in task one?","metadata":{}},{"cell_type":"markdown","source":"# 4. Models","metadata":{}},{"cell_type":"markdown","source":"If there is indeed such a difference then a simple model would pick it up. Let's build an ensemble of KNN, decision tree and logistic regression classifiers.","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nX, y = df.drop('target', axis=1).values, df['target'].values\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15, shuffle=True)\nprint(f'train: {X_train.shape[0]} scans, test: {X_test.shape[0]} scans')","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:42.758548Z","iopub.execute_input":"2023-05-26T00:25:42.758856Z","iopub.status.idle":"2023-05-26T00:25:42.937009Z","shell.execute_reply.started":"2023-05-26T00:25:42.758827Z","shell.execute_reply":"2023-05-26T00:25:42.935882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression, RidgeClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.metrics import roc_auc_score, log_loss\n\nmodels = [\n    RidgeClassifier(),\n    DecisionTreeClassifier(),\n    KNeighborsClassifier(n_neighbors=60),\n]\n\ndef train_ensemble(models:list, data:tuple, print_individual:bool=False):\n    X_train, X_test, y_train, y_test = data\n    full_preds = []\n    for clf in models:\n        clf.fit(X_train, y_train)\n        try:\n            preds = clf.predict_proba(X_test)[:, 1]\n        except AttributeError:\n            preds = clf.predict(X_test)\n        full_preds.append(preds)\n        if print_individual:\n            print(\n                clf.__class__.__name__.ljust(25), '  scores:',\n                roc_auc_score(y_test, preds), 'ROC AUC; ',\n                log_loss(y_test, preds), 'loss.'\n            )\n    return full_preds\n\nensemble_predictions = train_ensemble(models, (X_train, X_test, y_train, y_test), True)\nprint('average:'.rjust(35), roc_auc_score(y_test, np.mean(ensemble_predictions, 0)), 'ROC AUC')","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:42.938520Z","iopub.execute_input":"2023-05-26T00:25:42.938982Z","iopub.status.idle":"2023-05-26T00:25:43.183145Z","shell.execute_reply.started":"2023-05-26T00:25:42.938946Z","shell.execute_reply":"2023-05-26T00:25:43.181977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**0.56 - 0.65** ROC AUC score with simple features - not great, not terrible (and not very representative of our data).","metadata":{}},{"cell_type":"code","source":"split_scores = []\nfor _ in range(50):\n    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, shuffle=True)\n    models = [RidgeClassifier(), DecisionTreeClassifier(), KNeighborsClassifier(n_neighbors=60)]\n    ensemble_predictions = train_ensemble(models, (X_train, X_test, y_train, y_test), False)\n    split_scores.append(roc_auc_score(y_test, np.mean(ensemble_predictions, 0)))\nprint('mean', np.mean(split_scores), 'max', np.max(split_scores))","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:43.184587Z","iopub.execute_input":"2023-05-26T00:25:43.184911Z","iopub.status.idle":"2023-05-26T00:25:43.772067Z","shell.execute_reply.started":"2023-05-26T00:25:43.184858Z","shell.execute_reply":"2023-05-26T00:25:43.770734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**0.56 - 0.6** ROC AUC average for different random seeds with 0.75-0.25 train-test split.","metadata":{}},{"cell_type":"code","source":"split_scores = []\nfor _ in range(50):\n    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, shuffle=True)\n    y_test = np.random.permutation(y_test)\n    models = [RidgeClassifier(), DecisionTreeClassifier(), KNeighborsClassifier(n_neighbors=60)]\n    ensemble_predictions = train_ensemble(models, (X_train, X_test, y_train, y_test), False)\n    split_scores.append(roc_auc_score(y_test, np.mean(ensemble_predictions, 0)))\nprint('mean', np.mean(split_scores), 'max', np.max(split_scores))","metadata":{"execution":{"iopub.status.busy":"2023-05-26T00:25:43.773517Z","iopub.execute_input":"2023-05-26T00:25:43.773867Z","iopub.status.idle":"2023-05-26T00:25:44.364739Z","shell.execute_reply.started":"2023-05-26T00:25:43.773834Z","shell.execute_reply":"2023-05-26T00:25:44.363664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Worse results for randomly shuffled test labels. Nonetheless, a random model can still score up to 0.7 ROC AUC with a (un)lucky seed.","metadata":{}},{"cell_type":"markdown","source":"# 5. Conclusion\nIt is possible to achieve a ~0.64 local validation ROC AUC Task2 score just with the simple features derived from Task1 data. In theory, using Task1 scans in Task2 should improve model quality. However, pure Task2 image encoders could be learning to find the same features on their own.\n\nUnfortunately, not all patient ids from Task2 are present in Task1, so one-to-one conversion is not straightforward. Nevertheless, a pretrained segmentation model can be used instead.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}