{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.14","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":71549,"databundleVersionId":8561470,"sourceType":"competition"},{"sourceId":8727685,"sourceType":"datasetVersion","datasetId":5236783},{"sourceId":8728463,"sourceType":"datasetVersion","datasetId":5238658},{"sourceId":8761420,"sourceType":"datasetVersion","datasetId":5264030},{"sourceId":9231709,"sourceType":"datasetVersion","datasetId":5583775},{"sourceId":9235934,"sourceType":"datasetVersion","datasetId":5586514},{"sourceId":9238129,"sourceType":"datasetVersion","datasetId":5587999},{"sourceId":9238150,"sourceType":"datasetVersion","datasetId":5588015},{"sourceId":9273089,"sourceType":"datasetVersion","datasetId":5611997},{"sourceId":9306154,"sourceType":"datasetVersion","datasetId":5635376},{"sourceId":9365806,"sourceType":"datasetVersion","datasetId":5679443},{"sourceId":9409759,"sourceType":"datasetVersion","datasetId":5577143},{"sourceId":184402550,"sourceType":"kernelVersion"},{"sourceId":192715298,"sourceType":"kernelVersion"},{"sourceId":193161758,"sourceType":"kernelVersion"},{"sourceId":193298353,"sourceType":"kernelVersion"},{"sourceId":193335312,"sourceType":"kernelVersion"},{"sourceId":193338638,"sourceType":"kernelVersion"},{"sourceId":193417638,"sourceType":"kernelVersion"},{"sourceId":100132,"sourceType":"modelInstanceVersion","modelInstanceId":64905,"modelId":89293},{"sourceId":111113,"sourceType":"modelInstanceVersion","modelInstanceId":85952,"modelId":84065}],"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"One of the [outstanding members](https://www.kaggle.com/cdeotte) of the Kaggle community, grandmaster, gives a link to his [work](https://www.kaggle.com/code/cdeotte/top-solutions-ensemble-0-947), where he shows the refinement of the prediction using an experimental example. In order to try to refine our predictions at the [RSNA 2024 Lumbar Spine Degenerative Classification](https://www.kaggle.com/competitions/rsna-2024-lumbar-spine-degenerative-classification/leaderboard) competition\n\nThis approach has proven itself in the following competitions: 1. [ISIC 2024 - Skin Cancer Detection with 3D-TBP](https://www.kaggle.com/competitions/isic-2024-challenge/code?competitionId=63056&sortBy=scoreDescending&excludeNonAccessedDatasources=true), 2. [RSNA 2024 Lumbar Spine Degenerative Classification](https://www.kaggle.com/competitions/rsna-2024-lumbar-spine-degenerative-classification/code?competitionId=71549&sortBy=scoreAscending&excludeNonAccessedDatasources=true), 3. [NeurIPS - Ariel Data Challenge 2024](https://www.kaggle.com/competitions/ariel-data-challenge-2024/code?competitionId=70367&sortBy=scoreDescending&excludeNonAccessedDatasources=true), 4. [Child Mind Institute — Problematic Internet Use](https://www.kaggle.com/competitions/child-mind-institute-problematic-internet-use/code?competitionId=81933&sortBy=scoreDescending&excludeNonAccessedDatasources=true), 5. [BrisT1D Blood Glucose Prediction Competition](https://www.kaggle.com/competitions/brist1d/code?competitionId=82611&sortBy=scoreAscending&excludeNonAccessedDatasources=true), 6. [Eedi - Mining Misconceptions in Mathematics](https://www.kaggle.com/competitions/eedi-mining-misconceptions-in-mathematics/code?competitionId=82695&sortBy=scoreDescending&excludeNonAccessedDatasources=true), 7. [Connect X](https://www.kaggle.com/competitions/connectx/leaderboard?)\n\nAnd accordingly in the following notebooks: 1. [ISIC_2024 | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/isic-2024-ensemble-of-solutions), 2. [RSNA | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/rsna-ensemble-of-solutions), 3. [Ariel | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/ariel-ensemble-of-solutions), 4. [CMI | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/cmi-ensemble-of-solutions), 5. [BrisT1D | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/brist1d-ensemble-of-solutions), 6. [Eedi | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/eedi-ensemble-of-solutions), 7. [agents Connect X](https://www.kaggle.com/code/vyacheslavbolotin/agents-connect-x) \n\n**[RSNA | Ensemble of solutions](https://www.kaggle.com/code/vyacheslavbolotin/rsna-ensemble-solutions)** Leaderboard=0.5?\n\n1. **0.56** Japan [[Inference] RSNA2024](https://www.kaggle.com/code/takashimanaoya/inference-rsna2024) by expert [Naoya](https://www.kaggle.com/takashimanaoya)\n2. **0.56** Gliser [[Inference] RSNA2024-Normalize-Change](https://www.kaggle.com/code/drqingyang/inference-rsna2024-normalize-change) by contributor [Yang](https://www.kaggle.com/drqingyang)\n3. **0.57** Türkiye [84% Accuracy in Spine Classification Using CNN](https://www.kaggle.com/code/regisvargas/fork-of-neurips-ariel-2024-starter-5be123) by contributor [AIMED](https://www.kaggle.com/ceng49)\n4. **0.59** USA [RSNA2024 LSDC DenseNet Submission](https://www.kaggle.com/code/hugowjd/rsna2024-lsdc-densenet-submission) by expert [Jiadi Wang](https://www.kaggle.com/hugowjd)\n5. **0.56** Egypt [RSNA Lumbar Spine Prediction](https://www.kaggle.com/code/marwanashref/rsna-lumbar-spine-prediction) by expert [Marwan Ashref](https://www.kaggle.com/marwanashref)\n6. **0.57** Japan [RSNA[INF]edgenext_base_x512[LB.57]](https://www.kaggle.com/code/hideyukizushi/rsna-inf-edgenext-base-x512-lb-57) by expert [yukiZ](https://www.kaggle.com/hideyukizushi)\n7. **0.54** Germany [LSDC Yolo Approach](https://www.kaggle.com/code/namgalielei/lsdc-yolo-approach) by master [Liam Nguyen](https://www.kaggle.com/namgalielei)\n8. **0.55** Japan [LB0.55 | Inference by tempature0.9 | RSNA2024](https://www.kaggle.com/code/taikimori/lb0-55-inference-by-tempature0-9-rsna2024) by expert [Taimo](https://www.kaggle.com/taikimori)\n9. **0.51** USA [3D Vision Transformer Single-Stage (Inference)](https://www.kaggle.com/code/vsahin/3d-vision-transformer-single-stage-inference) by contributor [Victor S](https://www.kaggle.com/vsahin)\n\n\n#### options\n\n- option 1 -> LB=0.57 work (1,2)\n- option 2 -> LB=0.57 work (2,3) \n- option 3 -> LB= ?.?? work (3,4) \n- option 4 -> LB=1.30 work (1,2,4)\n- option 5 -> LB=0.57 work (2,3) + weights(0.45+0.55)\n- option 6 -> LB=0.57 work (2,3) + weights(0.55+0.45)\n- option 7 -> LB=? work (5,6)\n- option 8 -> LB=? work (5,6) + weights(0.90+0.10)\n- option 10-> LB=0.54 work (6,7) + weights(0.005+0.995)\n- option 11-> LB=? work (5,6,7) + weights(0.0005+0.0025+0.997)\n- option 13-> LB=**0.52** work (7,8) + weights(0.95+0.05) - It came from an ensemble of two works (7 & \n8) <=> (0.54 & 0.55)\n- option 14-> LB=0.51 work (7,8) + weights(0.85+0.15) - It came from an ensemble of two works (7 & \n8) <=> (0.54 & 0.55)\n- option 12-> LB=0.48 work (7,9) + weights(0.20+0.80) - It came from an ensemble of two works (7 & \n9) <=> (0.54 & 0.51) \n- option 17-> LB=0.48 work (7,9) + weights (0.257+0.743) \n- option 18-> LB=0.48 work ( 7,9 ) + weights ( 0.33+0.67 )\n- option 19-> LB=0.48 work ( 7,9 ) + weights ( 0.450+0.550 ) - previus.1 best option\n- option 20-> LB=0.48 work ( 7,9 ) + weights ( 0.50+0.50 )\n- option 23-> LB=0.48 work ( 7,9 ) + weights ( 0.473+0.527 )\n- option 24-> LB=0.48 work ( 7,9 ) + weights ( 0.427+0.573 ) - previus.2 best option\n- option 25-> LB=0.48 work ( 7,9 ) + weights ( 0.402+0.598 ) - previus.3 best option\n- option 26-> LB=0.49 work ( 7,9 ) + weights ( 0.377+0.623 )\n- **option 27**-> LB=**0.48** work ( 7,9 ) + weights ( 0.407+0.593 ) - **best option**\n- option 28-> LB=0.48 work ( 7,9 ) + weights ( 0.395+0.605 )\n- option 29-> LB=0.48 work ( 7,9 ) + weights ( 0.402+0.598 )\n- option 30-> LB=0.48 work ( 7,9 ) + weights ( 0.405+0.595 )\n- option 31-> LB=0.48 work ( 7,9 ) + weights ( 0.415+0.585 )\n- option 32-> LB=0.48 work ( 7,9 ) + weights ( 0.410+0.590 )\n- option 33-> LB=0.51 work ( 8,9 ) + weights ( 0.407+0.593 )\n- option 34-> LB=0.51 work ( 8,9 ) + weights ( 0.410+0.590 )\n- option 16-> LB=0.48 work ( 7,8,9 ) + weights ( 0.250+0.200+0.550 )\n- option 35-> LB=0.48 work ( 7,8,9 ) + weights ( 0.320+0.080+0.600 )\n- option 36-> LB=0.51 work ( 7,8,9 ) + weights ( 0.335+0.070+0.595 )\n- option 37-> LB=0.51 work ( 7,8,9 ) + weights ( 0.330+0.065+0.605 )\n\nsome rezult:\n* option.18 < option.20 < option.19 < option.24 == (version.48 < version.51 < version.49 < version.55-56)\n* option.24 < option.25 == (version.55-56 < version.58)\n* option.25 < option.28 < **option.27** == (version.58 < version.63 < **version.62**)\n* option.29 < option.30 < **option.27** == (version.58 < version.63 < **version.62**)\n* option.31 < option.32 < **option.27** == (version.58 < version.63 < **version.62**)\n\nbest option:\n- option.32 < **option.27**\n\ncurrent option: \n- **option 27**-> LB=**0.48** work ( 7,9 ) + weights ( 0.407+0.593 ) - **best option**\n\nnext option:\n- ?","metadata":{}},{"cell_type":"code","source":"LAUNCH_VARIANT = 'option 27'","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:57:16.474381Z","iopub.execute_input":"2024-10-07T18:57:16.475335Z","iopub.status.idle":"2024-10-07T18:57:16.479599Z","shell.execute_reply.started":"2024-10-07T18:57:16.475291Z","shell.execute_reply":"2024-10-07T18:57:16.478543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [[Inference] RSNA2024](https://www.kaggle.com/code/takashimanaoya/inference-rsna2024)\n\n### [Naoya](https://www.kaggle.com/takashimanaoya)","metadata":{}},{"cell_type":"markdown","source":"### My Training Code\nhttps://www.kaggle.com/code/takashimanaoya/train-lightning-hydra-baseline","metadata":{}},{"cell_type":"markdown","source":"[References]\nhttps://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-submission-baseline","metadata":{}},{"cell_type":"markdown","source":"### Import Libralies","metadata":{}},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt\n# from sklearn.model_selection import KFold\n\n# from collections import OrderedDict\n\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n\n# import albumentations as A\n\n# from sklearn.model_selection import KFold\n\n# import re\n# import pydicom\n# from typing import Optional\n# import glob","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:16.481383Z","iopub.execute_input":"2024-10-07T18:57:16.481917Z","iopub.status.idle":"2024-10-07T18:57:20.364244Z","shell.execute_reply.started":"2024-10-07T18:57:16.481876Z","shell.execute_reply":"2024-10-07T18:57:20.363258Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.365681Z","iopub.execute_input":"2024-10-07T18:57:20.366276Z","iopub.status.idle":"2024-10-07T18:57:20.375582Z","shell.execute_reply.started":"2024-10-07T18:57:20.366229Z","shell.execute_reply":"2024-10-07T18:57:20.374676Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Config","metadata":{}},{"cell_type":"code","source":"# EXP_NO = \"006\"\n# MODEL_DIR = f\"/kaggle/input/rsna24-a-{EXP_NO}\"\n# MODEL_NAME = \"tf_efficientnet_b5.ns_jft_in1k\"\n\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 42\n# NUM_FOLDS = 5\n\n\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 30\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n\n# BATCH_SIZE = 1","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.376826Z","iopub.execute_input":"2024-10-07T18:57:20.377185Z","iopub.status.idle":"2024-10-07T18:57:20.386532Z","shell.execute_reply.started":"2024-10-07T18:57:20.377152Z","shell.execute_reply":"2024-10-07T18:57:20.385734Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.389107Z","iopub.execute_input":"2024-10-07T18:57:20.389403Z","iopub.status.idle":"2024-10-07T18:57:20.400884Z","shell.execute_reply.started":"2024-10-07T18:57:20.389372Z","shell.execute_reply":"2024-10-07T18:57:20.399975Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')\n# device","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.402050Z","iopub.execute_input":"2024-10-07T18:57:20.402472Z","iopub.status.idle":"2024-10-07T18:57:20.410531Z","shell.execute_reply.started":"2024-10-07T18:57:20.402440Z","shell.execute_reply":"2024-10-07T18:57:20.409602Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df = pd.read_csv(f'{rd}/test_series_descriptions.csv')\n# df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.411813Z","iopub.execute_input":"2024-10-07T18:57:20.412248Z","iopub.status.idle":"2024-10-07T18:57:20.420920Z","shell.execute_reply.started":"2024-10-07T18:57:20.412205Z","shell.execute_reply":"2024-10-07T18:57:20.420006Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# study_ids = list(df['study_id'].unique())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.422085Z","iopub.execute_input":"2024-10-07T18:57:20.422398Z","iopub.status.idle":"2024-10-07T18:57:20.430092Z","shell.execute_reply.started":"2024-10-07T18:57:20.422355Z","shell.execute_reply":"2024-10-07T18:57:20.429269Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sample_sub = pd.read_csv(f'{rd}/sample_submission.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.431338Z","iopub.execute_input":"2024-10-07T18:57:20.432107Z","iopub.status.idle":"2024-10-07T18:57:20.439879Z","shell.execute_reply.started":"2024-10-07T18:57:20.432062Z","shell.execute_reply":"2024-10-07T18:57:20.438881Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# LABELS = list(sample_sub.columns[1:])\n# LABELS","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.440943Z","iopub.execute_input":"2024-10-07T18:57:20.441300Z","iopub.status.idle":"2024-10-07T18:57:20.449822Z","shell.execute_reply.started":"2024-10-07T18:57:20.441262Z","shell.execute_reply":"2024-10-07T18:57:20.448807Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.451130Z","iopub.execute_input":"2024-10-07T18:57:20.451536Z","iopub.status.idle":"2024-10-07T18:57:20.461068Z","shell.execute_reply.started":"2024-10-07T18:57:20.451468Z","shell.execute_reply":"2024-10-07T18:57:20.460143Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [ atoi(c) for c in re.split(r'(\\d+)', text) ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.462164Z","iopub.execute_input":"2024-10-07T18:57:20.462450Z","iopub.status.idle":"2024-10-07T18:57:20.470525Z","shell.execute_reply.started":"2024-10-07T18:57:20.462419Z","shell.execute_reply":"2024-10-07T18:57:20.469688Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Dataset","metadata":{}},{"cell_type":"code","source":"# class RSNA24TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n \n# def __len__(self):\n# return len(self.study_ids)\n \n# def get_img_paths(self, study_id, series_desc):\n# pdf = self.df[self.df['study_id']==study_id]\n# pdf_ = pdf[pdf['series_description']==series_desc]\n# allimgs = []\n# for i, row in pdf_.iterrows():\n# pimgs = glob.glob(f'{rd}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm')\n# pimgs = sorted(pimgs, key=natural_keys)\n# allimgs.extend(pimgs)\n \n# return allimgs\n \n# def read_dcm_ret_arr(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# image = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# img = cv2.resize(image, (IMG_SIZE[0], IMG_SIZE[1]),interpolation=cv2.INTER_CUBIC)\n# assert img.shape==(IMG_SIZE[0], IMG_SIZE[1])\n# return img\n\n# def __getitem__(self, idx):\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], IN_CHANS), dtype=np.uint8)\n# st_id = self.study_ids[idx] \n \n# # Sagittal T1\n# allimgs_st1 = self.get_img_paths(st_id, 'Sagittal T1')\n# if len(allimgs_st1)==0:\n# print(st_id, ': Sagittal T1, has no images')\n \n# else:\n# step = len(allimgs_st1) / 10.0\n# st = len(allimgs_st1)/2.0 - 4.0*step\n# end = len(allimgs_st1)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st1[ind2])\n# x[..., j] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T1')\n# pass\n \n# # Sagittal T2/STIR\n# allimgs_st2 = self.get_img_paths(st_id, 'Sagittal T2/STIR')\n# if len(allimgs_st2)==0:\n# print(st_id, ': Sagittal T2/STIR, has no images')\n \n# else:\n# step = len(allimgs_st2) / 10.0\n# st = len(allimgs_st2)/2.0 - 4.0*step\n# end = len(allimgs_st2)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st2[ind2])\n# x[..., j+10] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T2/STIR')\n# pass\n \n# # Axial T2\n# allimgs_at2 = self.get_img_paths(st_id, 'Axial T2')\n# if len(allimgs_at2)==0:\n# print(st_id, ': Axial T2, has no images')\n \n# else:\n# step = len(allimgs_at2) / 10.0\n# st = len(allimgs_at2)/2.0 - 4.0*step\n# end = len(allimgs_at2)+0.0001\n\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_at2[ind2])\n# x[..., j+20] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Axial T2')\n# pass \n \n \n# if self.transform is not None:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1)\n \n# return x, str(st_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.471907Z","iopub.execute_input":"2024-10-07T18:57:20.472212Z","iopub.status.idle":"2024-10-07T18:57:20.482297Z","shell.execute_reply.started":"2024-10-07T18:57:20.472181Z","shell.execute_reply":"2024-10-07T18:57:20.481472Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.5, std=0.5)\n# ])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.486828Z","iopub.execute_input":"2024-10-07T18:57:20.487194Z","iopub.status.idle":"2024-10-07T18:57:20.495388Z","shell.execute_reply.started":"2024-10-07T18:57:20.487161Z","shell.execute_reply":"2024-10-07T18:57:20.494519Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_ds = RSNA24TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(\n# test_ds, \n# batch_size=1, \n# shuffle=False,\n# num_workers=N_WORKERS,\n# pin_memory=True,\n# drop_last=False\n# )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.496523Z","iopub.execute_input":"2024-10-07T18:57:20.496894Z","iopub.status.idle":"2024-10-07T18:57:20.505474Z","shell.execute_reply.started":"2024-10-07T18:57:20.496850Z","shell.execute_reply":"2024-10-07T18:57:20.504458Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Model","metadata":{}},{"cell_type":"code","source":"# class RSNA24Model(nn.Module):\n# def __init__(\n# self,\n# model_name: str,\n# pretrained: bool,\n# features_only: bool,\n# in_chans: int,\n# n_classes: int,\n# n_labels: int,\n# loss_name: str,\n# ):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name=model_name,\n# pretrained=pretrained, \n# features_only=features_only,\n# in_chans=in_chans,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n# self.loss_fn = loss_name\n# self.n_labels = n_labels\n \n# def forward(\n# self,\n# x: torch.Tensor,\n# y: Optional[torch.Tensor],\n# ) -> dict[str, torch.Tensor]:\n \n# logits = self.model(x)\n \n# output = {\"logits\": logits}\n# if y is not None:\n# loss = 0\n# for col in range(self.n_labels):\n# pred = logits[:,col*3:col*3+3]\n# gt = y[:,col]\n# loss = loss + self.loss_fn(pred, gt) / self.n_labels\n# output[\"loss\"] = loss\n \n# return output","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.506754Z","iopub.execute_input":"2024-10-07T18:57:20.507068Z","iopub.status.idle":"2024-10-07T18:57:20.518249Z","shell.execute_reply.started":"2024-10-07T18:57:20.507037Z","shell.execute_reply":"2024-10-07T18:57:20.517533Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load Models","metadata":{}},{"cell_type":"code","source":"# models = []","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.519315Z","iopub.execute_input":"2024-10-07T18:57:20.519695Z","iopub.status.idle":"2024-10-07T18:57:20.527659Z","shell.execute_reply.started":"2024-10-07T18:57:20.519657Z","shell.execute_reply":"2024-10-07T18:57:20.526718Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for i in range(NUM_FOLDS):\n# cp = f\"{MODEL_DIR}/{EXP_NO}-{i}/best_model.pth\"\n# # cp = f\"{MODEL_DIR}/best_model.pth\"\n# print(f'loading {cp}...')\n# model = RSNA24Model(MODEL_NAME, False, False, IN_CHANS, N_CLASSES, 25, 'dummy')\n# state_dict = torch.load(cp)\n\n# # 予期しないキーを削除\n# if 'loss_fn.weight' in state_dict:\n# del state_dict['loss_fn.weight']\n# model.load_state_dict(state_dict)\n# model.eval()\n# model.to(device)\n# models.append(model)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.528644Z","iopub.execute_input":"2024-10-07T18:57:20.528942Z","iopub.status.idle":"2024-10-07T18:57:20.537463Z","shell.execute_reply.started":"2024-10-07T18:57:20.528902Z","shell.execute_reply":"2024-10-07T18:57:20.536542Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Inference loop","metadata":{}},{"cell_type":"code","source":"# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# y_preds = []\n# row_names = []\n\n# with tqdm(test_dl, leave=True) as pbar:\n# with torch.no_grad():\n# for idx, (x, si) in enumerate(pbar):\n# x = x.to(device)\n# pred_per_study = np.zeros((25, 3))\n \n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(si[0] + '_' + cond + '_' + level)\n \n# with autocast:\n# for m in models:\n# y = m(x, None)[\"logits\"][0]\n# for col in range(N_LABELS):\n# pred = y[col*3:col*3+3]\n# y_pred = pred.float().softmax(0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n# y_preds.append(pred_per_study)\n\n# y_preds = np.concatenate(y_preds, axis=0)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.538639Z","iopub.execute_input":"2024-10-07T18:57:20.538997Z","iopub.status.idle":"2024-10-07T18:57:20.547672Z","shell.execute_reply.started":"2024-10-07T18:57:20.538956Z","shell.execute_reply":"2024-10-07T18:57:20.546828Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Make Submission","metadata":{}},{"cell_type":"code","source":"# sub = pd.DataFrame()\n# sub['row_id'] = row_names\n# sub[LABELS] = y_preds\n# sub.head(25)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.548752Z","iopub.execute_input":"2024-10-07T18:57:20.549019Z","iopub.status.idle":"2024-10-07T18:57:20.560812Z","shell.execute_reply.started":"2024-10-07T18:57:20.548990Z","shell.execute_reply":"2024-10-07T18:57:20.559983Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sub.to_csv('submission_1.csv', index=False)\n# pd.read_csv('submission_1.csv').head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.561923Z","iopub.execute_input":"2024-10-07T18:57:20.562252Z","iopub.status.idle":"2024-10-07T18:57:20.569362Z","shell.execute_reply.started":"2024-10-07T18:57:20.562213Z","shell.execute_reply":"2024-10-07T18:57:20.568432Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Conclusion\nWe created the dataset, performed training, and inference in this notebook. \n\nThis competition is a bit complicated to handle the dataset, so there may be a better way.\n\nI think there are many other areas to improve in my notebook. I hope you can learn from my notebook and get a better score.","metadata":{}},{"cell_type":"markdown","source":"## [[Inference] RSNA2024-Normalize-Change](https://www.kaggle.com/code/drqingyang/inference-rsna2024-normalize-change) \n### [Yang](https://www.kaggle.com/drqingyang)","metadata":{}},{"cell_type":"markdown","source":"### My Training Code\nhttps://www.kaggle.com/code/takashimanaoya/train-lightning-hydra-baseline","metadata":{}},{"cell_type":"markdown","source":"[References]\nhttps://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-submission-baseline","metadata":{}},{"cell_type":"markdown","source":"### Import Libralies","metadata":{}},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt\n# from sklearn.model_selection import KFold\n\n# from collections import OrderedDict\n\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n\n# import albumentations as A\n\n# from sklearn.model_selection import KFold\n\n# import re\n# import pydicom\n# from typing import Optional\n# import glob","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.570412Z","iopub.execute_input":"2024-10-07T18:57:20.570725Z","iopub.status.idle":"2024-10-07T18:57:20.580340Z","shell.execute_reply.started":"2024-10-07T18:57:20.570690Z","shell.execute_reply":"2024-10-07T18:57:20.579404Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.581461Z","iopub.execute_input":"2024-10-07T18:57:20.583381Z","iopub.status.idle":"2024-10-07T18:57:20.589768Z","shell.execute_reply.started":"2024-10-07T18:57:20.583346Z","shell.execute_reply":"2024-10-07T18:57:20.588837Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Config","metadata":{}},{"cell_type":"code","source":"# EXP_NO = \"006\"\n# MODEL_DIR = f\"/kaggle/input/rsna24-a-{EXP_NO}\"\n# MODEL_NAME = \"tf_efficientnet_b5.ns_jft_in1k\"\n\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 42\n# NUM_FOLDS = 5\n\n\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 30\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n\n# BATCH_SIZE = 1","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.590927Z","iopub.execute_input":"2024-10-07T18:57:20.591748Z","iopub.status.idle":"2024-10-07T18:57:20.600587Z","shell.execute_reply.started":"2024-10-07T18:57:20.591705Z","shell.execute_reply":"2024-10-07T18:57:20.599628Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.601695Z","iopub.execute_input":"2024-10-07T18:57:20.601945Z","iopub.status.idle":"2024-10-07T18:57:20.614161Z","shell.execute_reply.started":"2024-10-07T18:57:20.601917Z","shell.execute_reply":"2024-10-07T18:57:20.613209Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')\n# device","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.615287Z","iopub.execute_input":"2024-10-07T18:57:20.615626Z","iopub.status.idle":"2024-10-07T18:57:20.623898Z","shell.execute_reply.started":"2024-10-07T18:57:20.615575Z","shell.execute_reply":"2024-10-07T18:57:20.622986Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df = pd.read_csv(f'{rd}/test_series_descriptions.csv')\n# df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.624917Z","iopub.execute_input":"2024-10-07T18:57:20.625200Z","iopub.status.idle":"2024-10-07T18:57:20.633794Z","shell.execute_reply.started":"2024-10-07T18:57:20.625169Z","shell.execute_reply":"2024-10-07T18:57:20.632804Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# study_ids = list(df['study_id'].unique())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.634898Z","iopub.execute_input":"2024-10-07T18:57:20.635170Z","iopub.status.idle":"2024-10-07T18:57:20.643836Z","shell.execute_reply.started":"2024-10-07T18:57:20.635127Z","shell.execute_reply":"2024-10-07T18:57:20.642884Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sample_sub = pd.read_csv(f'{rd}/sample_submission.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.644894Z","iopub.execute_input":"2024-10-07T18:57:20.645251Z","iopub.status.idle":"2024-10-07T18:57:20.653565Z","shell.execute_reply.started":"2024-10-07T18:57:20.645209Z","shell.execute_reply":"2024-10-07T18:57:20.652748Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# LABELS = list(sample_sub.columns[1:])\n# LABELS","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.654568Z","iopub.execute_input":"2024-10-07T18:57:20.654853Z","iopub.status.idle":"2024-10-07T18:57:20.663555Z","shell.execute_reply.started":"2024-10-07T18:57:20.654823Z","shell.execute_reply":"2024-10-07T18:57:20.662652Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.664600Z","iopub.execute_input":"2024-10-07T18:57:20.664964Z","iopub.status.idle":"2024-10-07T18:57:20.674226Z","shell.execute_reply.started":"2024-10-07T18:57:20.664926Z","shell.execute_reply":"2024-10-07T18:57:20.673431Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [ atoi(c) for c in re.split(r'(\\d+)', text) ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.675454Z","iopub.execute_input":"2024-10-07T18:57:20.676074Z","iopub.status.idle":"2024-10-07T18:57:20.684287Z","shell.execute_reply.started":"2024-10-07T18:57:20.676031Z","shell.execute_reply":"2024-10-07T18:57:20.683465Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Dataset","metadata":{}},{"cell_type":"code","source":"# class RSNA24TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n \n# def __len__(self):\n# return len(self.study_ids)\n \n# def get_img_paths(self, study_id, series_desc):\n# pdf = self.df[self.df['study_id']==study_id]\n# pdf_ = pdf[pdf['series_description']==series_desc]\n# allimgs = []\n# for i, row in pdf_.iterrows():\n# pimgs = glob.glob(f'{rd}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm')\n# pimgs = sorted(pimgs, key=natural_keys)\n# allimgs.extend(pimgs)\n \n# return allimgs\n \n# def read_dcm_ret_arr(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# image = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# img = cv2.resize(image, (IMG_SIZE[0], IMG_SIZE[1]),interpolation=cv2.INTER_CUBIC)\n# assert img.shape==(IMG_SIZE[0], IMG_SIZE[1])\n# return img\n\n# def __getitem__(self, idx):\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], IN_CHANS), dtype=np.uint8)\n# st_id = self.study_ids[idx] \n \n# # Sagittal T1\n# allimgs_st1 = self.get_img_paths(st_id, 'Sagittal T1')\n# if len(allimgs_st1)==0:\n# print(st_id, ': Sagittal T1, has no images')\n \n# else:\n# step = len(allimgs_st1) / 10.0\n# st = len(allimgs_st1)/2.0 - 4.0*step\n# end = len(allimgs_st1)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st1[ind2])\n# x[..., j] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T1')\n# pass\n \n# # Sagittal T2/STIR\n# allimgs_st2 = self.get_img_paths(st_id, 'Sagittal T2/STIR')\n# if len(allimgs_st2)==0:\n# print(st_id, ': Sagittal T2/STIR, has no images')\n \n# else:\n# step = len(allimgs_st2) / 10.0\n# st = len(allimgs_st2)/2.0 - 4.0*step\n# end = len(allimgs_st2)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st2[ind2])\n# x[..., j+10] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T2/STIR')\n# pass\n \n# # Axial T2\n# allimgs_at2 = self.get_img_paths(st_id, 'Axial T2')\n# if len(allimgs_at2)==0:\n# print(st_id, ': Axial T2, has no images')\n \n# else:\n# step = len(allimgs_at2) / 10.0\n# st = len(allimgs_at2)/2.0 - 4.0*step\n# end = len(allimgs_at2)+0.0001\n\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_at2[ind2])\n# x[..., j+20] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Axial T2')\n# pass \n \n \n# if self.transform is not None:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1)\n \n# return x, str(st_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.685700Z","iopub.execute_input":"2024-10-07T18:57:20.686187Z","iopub.status.idle":"2024-10-07T18:57:20.695741Z","shell.execute_reply.started":"2024-10-07T18:57:20.686116Z","shell.execute_reply":"2024-10-07T18:57:20.694893Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.4, std=0.2)\n# ])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.696854Z","iopub.execute_input":"2024-10-07T18:57:20.697152Z","iopub.status.idle":"2024-10-07T18:57:20.707752Z","shell.execute_reply.started":"2024-10-07T18:57:20.697121Z","shell.execute_reply":"2024-10-07T18:57:20.706976Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_ds = RSNA24TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(\n# test_ds, \n# batch_size=1, \n# shuffle=False,\n# num_workers=N_WORKERS,\n# pin_memory=True,\n# drop_last=False\n# )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.708738Z","iopub.execute_input":"2024-10-07T18:57:20.709016Z","iopub.status.idle":"2024-10-07T18:57:20.718357Z","shell.execute_reply.started":"2024-10-07T18:57:20.708987Z","shell.execute_reply":"2024-10-07T18:57:20.717618Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Model","metadata":{}},{"cell_type":"code","source":"# class RSNA24Model(nn.Module):\n# def __init__(\n# self,\n# model_name: str,\n# pretrained: bool,\n# features_only: bool,\n# in_chans: int,\n# n_classes: int,\n# n_labels: int,\n# loss_name: str,\n# ):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name=model_name,\n# pretrained=pretrained, \n# features_only=features_only,\n# in_chans=in_chans,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n# self.loss_fn = loss_name\n# self.n_labels = n_labels\n \n# def forward(\n# self,\n# x: torch.Tensor,\n# y: Optional[torch.Tensor],\n# ) -> dict[str, torch.Tensor]:\n \n# logits = self.model(x)\n \n# output = {\"logits\": logits}\n# if y is not None:\n# loss = 0\n# for col in range(self.n_labels):\n# pred = logits[:,col*3:col*3+3]\n# gt = y[:,col]\n# loss = loss + self.loss_fn(pred, gt) / self.n_labels\n# output[\"loss\"] = loss\n \n# return output","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.719394Z","iopub.execute_input":"2024-10-07T18:57:20.719711Z","iopub.status.idle":"2024-10-07T18:57:20.731409Z","shell.execute_reply.started":"2024-10-07T18:57:20.719661Z","shell.execute_reply":"2024-10-07T18:57:20.730558Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load Models","metadata":{}},{"cell_type":"code","source":"# models = []","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.732515Z","iopub.execute_input":"2024-10-07T18:57:20.732882Z","iopub.status.idle":"2024-10-07T18:57:20.740008Z","shell.execute_reply.started":"2024-10-07T18:57:20.732839Z","shell.execute_reply":"2024-10-07T18:57:20.739091Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for i in range(NUM_FOLDS):\n# cp = f\"{MODEL_DIR}/{EXP_NO}-{i}/best_model.pth\"\n# # cp = f\"{MODEL_DIR}/best_model.pth\"\n# print(f'loading {cp}...')\n# model = RSNA24Model(MODEL_NAME, False, False, IN_CHANS, N_CLASSES, 25, 'dummy')\n# state_dict = torch.load(cp)\n\n# # 予期しないキーを削除\n# if 'loss_fn.weight' in state_dict:\n# del state_dict['loss_fn.weight']\n# model.load_state_dict(state_dict)\n# model.eval()\n# model.to(device)\n# models.append(model)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.741253Z","iopub.execute_input":"2024-10-07T18:57:20.741568Z","iopub.status.idle":"2024-10-07T18:57:20.749804Z","shell.execute_reply.started":"2024-10-07T18:57:20.741525Z","shell.execute_reply":"2024-10-07T18:57:20.748917Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# y_preds = []\n# row_names = []\n\n# with tqdm(test_dl, leave=True) as pbar:\n# with torch.no_grad():\n# for idx, (x, si) in enumerate(pbar):\n# x = x.to(device)\n# pred_per_study = np.zeros((25, 3))\n \n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(si[0] + '_' + cond + '_' + level)\n \n# with autocast:\n# for m in models:\n# y = m(x, None)[\"logits\"][0]\n# for col in range(N_LABELS):\n# pred = y[col*3:col*3+3]\n# y_pred = pred.float().softmax(0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n# y_preds.append(pred_per_study)\n\n# y_preds = np.concatenate(y_preds, axis=0)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.750834Z","iopub.execute_input":"2024-10-07T18:57:20.751155Z","iopub.status.idle":"2024-10-07T18:57:20.758969Z","shell.execute_reply.started":"2024-10-07T18:57:20.751123Z","shell.execute_reply":"2024-10-07T18:57:20.758226Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Make Submission","metadata":{}},{"cell_type":"code","source":"# sub = pd.DataFrame()\n# sub['row_id'] = row_names\n# sub[LABELS] = y_preds\n# sub.head(25)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.760110Z","iopub.execute_input":"2024-10-07T18:57:20.760471Z","iopub.status.idle":"2024-10-07T18:57:20.771777Z","shell.execute_reply.started":"2024-10-07T18:57:20.760426Z","shell.execute_reply":"2024-10-07T18:57:20.770787Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sub.to_csv('submission_2.csv', index=False)\n# pd.read_csv('submission_2.csv').head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.784965Z","iopub.execute_input":"2024-10-07T18:57:20.785344Z","iopub.status.idle":"2024-10-07T18:57:20.789248Z","shell.execute_reply.started":"2024-10-07T18:57:20.785310Z","shell.execute_reply":"2024-10-07T18:57:20.788299Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Conclusion\nWe created the dataset, performed training, and inference in this notebook. \n\nThis competition is a bit complicated to handle the dataset, so there may be a better way.\n\nI think there are many other areas to improve in my notebook. I hope you can learn from my notebook and get a better score.","metadata":{}},{"cell_type":"markdown","source":"## [84% Accuracy in Spine Classification Using CNN](https://www.kaggle.com/code/regisvargas/fork-of-neurips-ariel-2024-starter-5be123)\n### [AIMED](https://www.kaggle.com/ceng49)","metadata":{}},{"cell_type":"markdown","source":"**AIMED Research Group Members:** İnci Kızıldağ Yırgın, Assoc Prof. Dr.(M.D); Murat Emeç, Ph.D.; Mustafa Durmaz, M.D\n\n**Problem Statement**\n\nOverview\nThis competition aims to develop models that can assist in detecting and classifying degenerative spine conditions using MR images of the lumbar spine. Competitors will create models that simulate radiologists' performance in diagnosing spine conditions.\n\n**Background**\nAccording to the World Health Organization, low back pain is the most common cause of disability worldwide, affecting 619 million people in 2020. Most people experience lower back pain at some point in their lives, and the frequency increases with age. Pain and limitation of movement are often symptoms of spondylosis, characterized by degeneration of the intervertebral discs and resulting compression or irritation of nerves associated with narrowing of the spinal canal (spinal stenosis), subarticular spaces, or neural foramen.\n\nMagnetic resonance imaging (MRI) shows the lumbar spine vertebrae, discs, and nerves in detail, allowing radiologists to assess the presence and severity of these conditions. Accurate diagnosis and grading of these conditions is critical in guiding treatment and possible surgical interventions. It is essential to relieve patients' lower back pain and improve their overall health and quality of life.\n\n**Challenge**\nRSNA (Radiological Society of North America), in collaboration with ASNR (American Society of Neuroradiology), is organizing this competition to investigate the use of artificial intelligence in detecting and classifying degenerative spine conditions using lumbar spine MR images.\n\nThis competition focuses on the classification of five lumbar spine degenerative conditions:\n1.\tLeft Neural Foramen Narrowing\n2.\tRight Neural Foramen Narrowing\n3.\tLeft Subarticular Stenosis\n4.\tRight Subarticular Stenosis\n5.\tSpinal Canal Stenosis\n\nFor each imaging study in the dataset, severity scores (Normal/Mild, Moderate or Severe) were provided for each of these five conditions at L1/L2, L2/L3, L3/L4, L4/L5, and L5/S1 intervertebral disc levels.\n\nObjective\nThis competition aims to develop machine learning models that can accurately classify the severity of degenerative spine conditions based on lumbar spine MRI images. Models:\n-\tDetect the presence of degenerative conditions,\n-\tClassify the severity of these conditions as Normal/Mild, Moderate or Severe,\n-\tIt must be able to perform these tasks at more than one intervertebral disc level.\n\n**Dataset**\nThe dataset created for this competition includes imaging data collected by the RSNA competition planning team from eight sites on five continents. This multi-institutional, expertly curated dataset aims to improve the standardized classification of degenerative lumbar spine conditions and enable the development of tools to automate accurate and rapid disease classification.\n\nImpact\nThe successful development of these models:\n- Can improve the diagnostic capabilities of radiologists,\n- Provide rapid and accurate classification of degenerative spine conditions,\n- Improve patient outcomes by guiding appropriate treatment and surgical decisions,\n- It can alleviate the global burden of low back pain and related disabilities.\n\n\n**Limitations**\n1.\tClass Imbalance: The dataset used in this study exhibits significant class imbalance, with certain degenerative spine conditions being underrepresented. This imbalance can lead to biased model performance, where the model may perform well on the majority classes but poorly on the minority classes. Techniques such as class weighting, oversampling, or undersampling may be necessary to address this issue.\n2.\tSingle-Patient Test Data: The test data in this study is derived from a single patient. This limitation can affect the model's generalizability, as its performance on this specific patient may not accurately reflect its performance on a broader population. It is crucial to validate the model on a more diverse test set to ensure its robustness and applicability to different patients.\n3.\tLimited Data Augmentation: The current implementation may not include extensive data augmentation techniques for improving model generalization, especially in medical imaging tasks. Incorporating various augmentation strategies such as rotation, scaling, and flipping could enhance the model's ability to handle data variations.\n4.\tPotential Overfitting: Given the complexity of the convolutional neural network (CNN) model and the relatively small size of the dataset, there is a risk of overfitting. Overfitting occurs when the model learns to perform well on the training data but fails to generalize to new, unseen data. Techniques such as dropout, early stopping, and cross-validation should be employed to mitigate this risk.\n5.\tLack of External Validation: The study does not include external validation using independent datasets from different sources. External validation is crucial for assessing the model's performance in real-world scenarios and ensuring its reliability across different imaging centers and patient populations.\n6.\tLimited Feature Engineering: The current approach primarily relies on raw pixel data from MR images. Additional features such as patient demographics, clinical history, and other relevant metadata could improve the model's predictive performance.\n7.\tComputational Resources: Training deep learning models, especially CNNs, requires substantial computational resources. Limited access to high-performance computing infrastructure can constrain the ability to experiment with different model architectures and hyperparameters, potentially impacting the model's performance.\n\n\n**The study continues in two phases:**\n1.\tData preprocessing\n2.\tPresenting the results of the development of the AI model\n","metadata":{}},{"cell_type":"markdown","source":"### Phase 1: Data preprocessing","metadata":{}},{"cell_type":"markdown","source":"**Train and Test Data Preprocessing**","metadata":{}},{"cell_type":"markdown","source":"**Importing Libraries:**","metadata":{}},{"cell_type":"markdown","source":"**Data Path and Uploading Files:**","metadata":{}},{"cell_type":"markdown","source":"**Creating New Columns and Adding Target Values:**","metadata":{}},{"cell_type":"markdown","source":"**Adding File Path Column and Creating Target Index:**","metadata":{}},{"cell_type":"markdown","source":"**Function that Extracts Last Values from File Path:**","metadata":{}},{"cell_type":"markdown","source":"**Processing and Copying of Files:**","metadata":{}},{"cell_type":"markdown","source":"**Creating a New Train DataFrame and Clearing Memory:**","metadata":{}},{"cell_type":"markdown","source":"**Reorganizing Columns and Saving Data:**","metadata":{}},{"cell_type":"markdown","source":"### Phase 2: Development of the AI model and presentation of results","metadata":{}},{"cell_type":"markdown","source":"****Importing Libraries****","metadata":{}},{"cell_type":"markdown","source":"**DICOM Image Upload Function**","metadata":{}},{"cell_type":"code","source":"# '''\n# import os\n# import pandas as pd\n# import numpy as np\n# import pydicom\n# import cv2\n# from sklearn.preprocessing import LabelEncoder\n# from tensorflow.keras.utils import to_categorical\n# from tensorflow.keras.models import Sequential\n# from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout\n# from skimage.transform import resize\n# import matplotlib.pyplot as plt\n# from matplotlib.colors import LinearSegmentedColormap\n# from matplotlib.patches import Rectangle\n# import warnings\n# warnings.filterwarnings('ignore')\n# # Paths\n# train_path = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train.csv'\n# train_label_coordinates_path = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_label_coordinates.csv'\n# train_series_descriptions_path = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_series_descriptions.csv'\n# test_series_descriptions_path = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/test_series_descriptions.csv'\n# sample_submission_path = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/sample_submission.csv'\n\n# # Load data\n# df_train = pd.read_csv(train_path)\n# df_train_label_coordinates = pd.read_csv(train_label_coordinates_path)\n# df_train_series_descriptions = pd.read_csv(train_series_descriptions_path)\n# df_test_series_descriptions = pd.read_csv(test_series_descriptions_path)\n# df_sample_submission = pd.read_csv(sample_submission_path)\n# # Preprocess data\n# df_train = df_train.dropna()\n# df_train_label_coordinates = df_train_label_coordinates.dropna()\n# df_train_series_descriptions = df_train_series_descriptions.dropna()\n# df_test_series_descriptions = df_test_series_descriptions.dropna()\n\n# # Drop unnecessary columns\n# df_train = df_train.drop('Unnamed: 0', axis=1, errors='ignore')\n# df_train_label_coordinates = df_train_label_coordinates.drop('Unnamed: 0', axis=1, errors='ignore')\n# df_train_series_descriptions = df_train_series_descriptions.drop('Unnamed: 0', axis=1, errors='ignore')\n# df_test_series_descriptions = df_test_series_descriptions.drop('Unnamed: 0', axis=1, errors='ignore')\n# # Encode labels\n# le_condition = LabelEncoder()\n# df_train_label_coordinates['condition'] = le_condition.fit_transform(df_train_label_coordinates['condition'])\n\n# le_level = LabelEncoder()\n# df_train_label_coordinates['level'] = le_level.fit_transform(df_train_label_coordinates['level'])\n\n# le_target = LabelEncoder()\n# df_train_label_coordinates['target'] = le_target.fit_transform(df_train_label_coordinates['condition'])\n# # Function to load DICOM images\n# def load_dicom_image(path):\n# dicom = pydicom.read_file(path)\n# image = dicom.pixel_array\n# if image.dtype != np.uint8:\n# image = cv2.normalize(image, None, 0, 255, cv2.NORM_MINMAX).astype(np.uint8)\n# return image\n# # Function to extract region\n# def extract_region(image, x, y, width=128, height=128):\n# start_x = int(x - width / 2)\n# end_x = int(x + width / 2)\n# start_y = int(y - height / 2)\n# end_y = int(y + height / 2)\n\n# start_x = max(0, start_x)\n# end_x = min(image.shape[1], end_x)\n# start_y = max(0, start_y)\n# end_y = min(image.shape[0], end_y)\n\n# region = image[start_y:end_y, start_x:end_x]\n \n# if region.size == 0:\n# raise ValueError(\"Extracted region is empty. Please check the coordinates and image dimensions.\")\n \n# region = cv2.resize(region, (128, 128))\n# return region\n# # Function to draw rectangle\n# def draw_rectangle(image, x_coord, y_coord, size, color, label):\n# fig, ax = plt.subplots(1, 2, figsize=(10, 5)) \n\n# ax[0].imshow(image, cmap='gray')\n# ax[0].set_title('Original Image')\n\n# window_size = int(0.2 * min(image.shape)) # Adaptive window size (20%)\n# selected_area = image[max(0, int(y_coord) - window_size // 2):min(image.shape[0], int(y_coord) + window_size // 2),\n# max(0, int(x_coord) - window_size // 2):min(image.shape[1], int(x_coord) + window_size // 2)]\n\n# ax[1].imshow(selected_area, cmap='gray')\n# rect = Rectangle((window_size // 2 - size // 2, window_size // 2 - size // 2), size, size, linewidth=2, edgecolor=color, facecolor='none')\n# ax[1].add_patch(rect)\n# ax[1].set_title(f'{label} Area at ({x_coord:.2f}, {y_coord:.2f})')\n\n# plt.show()\n# # Function to draw severity\n# def draw_severe(image, x_coord, y_coord, severity):\n# colors = [(1, 1, 0), (1, 0.5, 0), (1, 0, 0)] # Yellow to Red\n# cmap = LinearSegmentedColormap.from_list(\"severity_cmap\", colors, N=3)\n \n# severity_level = le_target.inverse_transform([severity])[0]\n# severity_levels = le_target.classes_\n \n# if severity_level not in severity_levels:\n# raise ValueError(f\"Unexpected severity level: {severity_level}\")\n \n# severity_index = list(severity_levels).index(severity_level)\n# color = cmap(severity_index)\n\n# clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))\n# image = clahe.apply(image)\n\n# draw_rectangle(image, x_coord, y_coord, 24, color, severity)\n\n# extracted_region = extract_region(image, x_coord, y_coord, width=16, height=16)\n# return extracted_region\n# # Function to load images from study\n# def load_images_from_study(df, folder):\n# base_folder = folder\n# images = []\n# conditions = []\n# levels = []\n# targets = []\n# for _, row in df.iterrows():\n# study_id = row['study_id']\n# series_id = row['series_id']\n# x = row['x']\n# y = row['y']\n# condition = row['condition']\n# level = row['level']\n# target = row['target']\n \n# dicom_folder = f\"{base_folder}/{study_id}/{series_id}\"\n \n# if os.path.isdir(dicom_folder):\n# for dicom_file in os.listdir(dicom_folder):\n# dicom_path = os.path.join(dicom_folder, dicom_file)\n# img = load_dicom_image(dicom_path)\n# if img is not None:\n# extracted_region = draw_severe(img, x, y, int(target))\n# region = extract_region(img, x, y)\n# images.append(region)\n# conditions.append(condition)\n# levels.append(level)\n# targets.append(target)\n# else:\n# print(f\"Failed to load image for {dicom_path}\")\n# else:\n# print(f\"Folder not found: {dicom_folder}\")\n# return np.array(images), np.array(conditions), np.array(levels), np.array(targets)\n# # Prepare test data\n# df_test = df_test_series_descriptions.merge(df_train_label_coordinates[['study_id', 'series_id', 'x', 'y', 'condition', 'level', 'target']], on=['study_id', 'series_id'], how='left')\n# df_test = df_test.dropna(subset=['x', 'y', 'condition', 'level', 'target'])\n\n# if df_test.empty:\n# df_test = pd.DataFrame({\n# 'study_id': [44036939],\n# 'series_id': [2828203845],\n# 'x': [240],\n# 'y': [120],\n# 'condition': [0],\n# 'level': [0],\n# 'target': [0]\n# })\n\n# X_test_images, X_test_conditions, X_test_levels, _ = load_images_from_study(df_test, '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/test_images')\n\n# if X_test_images.size == 0 or X_test_conditions.size == 0 or X_test_levels.size == 0:\n# raise ValueError(\"Error\")\n\n# min_samples = min(X_test_images.shape[0], X_test_conditions.shape[0], X_test_levels.shape[0])\n\n# X_test_images = X_test_images[:min_samples]\n# X_test_conditions = X_test_conditions[:min_samples]\n# X_test_levels = X_test_levels[:min_samples]\n\n# X_test_images_resized = np.array([resize(image, (64, 64)) for image in X_test_images])\n# X_test_images_resized = X_test_images_resized.reshape(-1, 64, 64, 1)\n\n# X_test_conditions_encoded = to_categorical(X_test_conditions)\n# X_test_levels_encoded = to_categorical(X_test_levels)\n# # Create CNN model\n# def create_cnn_model(input_shape):\n# model = Sequential()\n# model.add(Conv2D(32, (3, 3), activation='relu', input_shape=input_shape))\n# model.add(MaxPooling2D((2, 2)))\n# model.add(Conv2D(64, (3, 3), activation='relu'))\n# model.add(MaxPooling2D((2, 2)))\n# model.add(Conv2D(128, (3, 3), activation='relu'))\n# model.add(MaxPooling2D((2, 2)))\n# model.add(Flatten())\n# model.add(Dense(128, activation='relu'))\n# model.add(Dropout(0.5))\n# model.add(Dense(3, activation='softmax'))\n# model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n# return model\n\n# input_shape = (64, 64, 1)\n# model = create_cnn_model(input_shape)\n\n# # Dummy training data\n# X_train_images = np.random.rand(10, 64, 64, 1)\n# X_train_conditions = np.random.randint(0, 3, 10)\n# X_train_levels = np.random.randint(0, 3, 10)\n# y_train = to_categorical(np.random.randint(0, 3, 10))\n\n# # Train model\n# model.fit([X_train_images, to_categorical(X_train_conditions), to_categorical(X_train_levels)], y_train, epochs=1)\n# # Load data\n# df_train_data = pd.read_csv(train_path)\n# df_train_labels = pd.read_csv(train_label_coordinates_path)\n# df_train_series_desc = pd.read_csv(train_series_descriptions_path)\n# df_test_series_desc = pd.read_csv(test_series_descriptions_path)\n# df_submission_template = pd.read_csv(sample_submission_path)\n# # Merge data\n# df_merged_train_labels = pd.merge(left=df_train_labels, right=df_train_data, how='left', on='study_id').reset_index(drop=True)\n# df_complete_train_data = pd.merge(left=df_merged_train_labels, right=df_train_series_desc, how='left', on=['study_id', 'series_id']).reset_index(drop=True)\n# # Convert to category\n# df_complete_train_data.study_id = df_complete_train_data.study_id.astype('category')\n# df_complete_train_data.series_id = df_complete_train_data.series_id.astype('category')\n# # Calculate frequencies\n# label_columns = df_train_data.columns.drop('study_id').tolist()\n# df_label_frequencies = pd.DataFrame(label_columns, columns=['label'])\n# df_label_frequencies['p1'] = 1.0\n# df_label_frequencies['p2'] = 0.0\n# df_label_frequencies['p3'] = 0.0\n# for label in label_columns:\n# relative_counts = df_train_data[label].value_counts(normalize=True)\n# df_label_frequencies.loc[df_label_frequencies.label == label, 'p1'] = relative_counts.get('Normal/Mild', 0)\n# df_label_frequencies.loc[df_label_frequencies.label == label, 'p2'] = relative_counts.get('Moderate', 0)\n# df_label_frequencies.loc[df_label_frequencies.label == label, 'p3'] = relative_counts.get('Severe', 0)\n# # Frequency adjustment\n# labels = df_train.columns.drop('study_id').tolist()\n# freqs = pd.DataFrame(labels, columns=['label'])\n# freqs['p1'] = 1.0\n# freqs['p2'] = 0.0\n# freqs['p3'] = 0.0\n# for l in labels:\n# rel_counts = df_train[l].value_counts(normalize=True)\n# freqs.loc[freqs.label==l, 'p1'] = rel_counts.get('Normal/Mild', 0)\n# freqs.loc[freqs.label==l, 'p2'] = rel_counts.get('Moderate', 0)\n# freqs.loc[freqs.label==l, 'p3'] = rel_counts.get('Severe', 0)\n# # Save train data\n# df_complete_train_data.to_csv('complete_train_data.csv', index=False)\n# # Explore specific study and series\n# study_id = 100206310\n# df_study_example = df_complete_train_data[df_complete_train_data.study_id == study_id]\n# series_id = 1012284084\n# df_series_example = df_study_example[df_study_example.series_id == series_id]\n# # Explore DICOM files\n# dicom_path = f'/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_images/{study_id}/{series_id}/'\n# for dirname, _, filenames in os.walk(dicom_path):\n# for filename in filenames:\n# print(os.path.join(dirname, filename))\n\n# for i in range(1, 10 + 1):\n# dicom_file = dicom_path + str(i) + '.dcm'\n# print(dicom_file)\n# ds = pydicom.dcmread(dicom_file)\n# # Apply frequency adjustment to submission\n# num_rows = df_submission_template.shape[0]\n# for i in range(num_rows):\n# current_label = df_submission_template.loc[i, 'row_id'].split('_', 1)[1]\n# p1 = df_label_frequencies.loc[df_label_frequencies.label == current_label, 'p1'].min()\n# p2 = df_label_frequencies.loc[df_label_frequencies.label == current_label, 'p2'].min()\n# p3 = df_label_frequencies.loc[df_label_frequencies.label == current_label, 'p3'].min()\n# df_submission_template.loc[i, 'normal_mild'] = p1\n# df_submission_template.loc[i, 'moderate'] = p2\n# df_submission_template.loc[i, 'severe'] = p3\n# # Save frequency-adjusted submission\n# df_submission_template.to_csv('submission_with_frequencies.csv', index=False)\n\n# # Apply uniform distribution to submission\n# for i in range(num_rows):\n# df_submission_template.loc[i, 'normal_mild'] = 0.34\n# df_submission_template.loc[i, 'moderate'] = 0.33\n# df_submission_template.loc[i, 'severe'] = 0.33\n\n# # Save uniform distribution submission\n# df_submission_template.to_csv('submission.csv', index=False)\n# '''","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.790881Z","iopub.execute_input":"2024-10-07T18:57:20.791265Z","iopub.status.idle":"2024-10-07T18:57:20.808862Z","shell.execute_reply.started":"2024-10-07T18:57:20.791224Z","shell.execute_reply":"2024-10-07T18:57:20.807968Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt\n# from sklearn.model_selection import KFold\n# from collections import OrderedDict\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n# import albumentations as A\n# from sklearn.model_selection import KFold\n# import re\n# import pydicom\n# import glob","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.809881Z","iopub.execute_input":"2024-10-07T18:57:20.810147Z","iopub.status.idle":"2024-10-07T18:57:20.820707Z","shell.execute_reply.started":"2024-10-07T18:57:20.810118Z","shell.execute_reply":"2024-10-07T18:57:20.819732Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'\n# OUTPUT_DIR = f'/kaggle/input/rsna2024-lsdc-training-baseline/rsna2024-results'\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 8620\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 42\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n# N_FOLDS = 5\n# MODEL_NAME = \"edgenext_base.in21k_ft_in1k\"\n\n# BATCH_SIZE = 1\n# device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')\n# device\n# df = pd.read_csv(f'{rd}/test_series_descriptions.csv')\n# df.head()\n# study_ids = list(df['study_id'].unique())\n# sample_sub = pd.read_csv(f'{rd}/sample_submission.csv')\n# LABELS = list(sample_sub.columns[1:])\n# LABELS","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.821902Z","iopub.execute_input":"2024-10-07T18:57:20.822265Z","iopub.status.idle":"2024-10-07T18:57:20.834902Z","shell.execute_reply.started":"2024-10-07T18:57:20.822222Z","shell.execute_reply":"2024-10-07T18:57:20.833970Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.836097Z","iopub.execute_input":"2024-10-07T18:57:20.836822Z","iopub.status.idle":"2024-10-07T18:57:20.844066Z","shell.execute_reply.started":"2024-10-07T18:57:20.836778Z","shell.execute_reply":"2024-10-07T18:57:20.843145Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [ atoi(c) for c in re.split(r'(\\d+)', text) ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.845375Z","iopub.execute_input":"2024-10-07T18:57:20.846299Z","iopub.status.idle":"2024-10-07T18:57:20.853878Z","shell.execute_reply.started":"2024-10-07T18:57:20.846255Z","shell.execute_reply":"2024-10-07T18:57:20.852891Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# class RSNA2024TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n \n# def __len__(self):\n# return len(self.study_ids)\n \n# def get_img_paths(self, study_id, series_desc):\n# pdf = self.df[self.df['study_id']==study_id]\n# pdf_ = pdf[pdf['series_description']==series_desc]\n# allimgs = []\n# for i, row in pdf_.iterrows():\n# pimgs = glob.glob(f'{rd}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm')\n# pimgs = sorted(pimgs, key=natural_keys)\n# allimgs.extend(pimgs)\n \n# return allimgs\n \n# def read_dcm_ret_arr(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# image = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# img = cv2.resize(image, (IMG_SIZE[0], IMG_SIZE[1]),interpolation=cv2.INTER_CUBIC)\n# assert img.shape==(IMG_SIZE[0], IMG_SIZE[1])\n# return img\n\n# def __getitem__(self, idx):\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], IN_CHANS), dtype=np.uint8)\n# st_id = self.study_ids[idx] \n \n# # Sagittal T1\n# allimgs_st1 = self.get_img_paths(st_id, 'Sagittal T1')\n# if len(allimgs_st1)==0:\n# print(st_id, ': Sagittal T1, has no images')\n \n# else:\n# step = len(allimgs_st1) / 14.0\n# st = len(allimgs_st1)/2.0 - 6.0*step\n# end = len(allimgs_st1)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st1[ind2])\n# x[..., j] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T1')\n# pass\n \n# # Sagittal T2/STIR\n# allimgs_st2 = self.get_img_paths(st_id, 'Sagittal T2/STIR')\n# if len(allimgs_st2)==0:\n# print(st_id, ': Sagittal T2/STIR, has no images')\n \n# else:\n# step = len(allimgs_st2) / 14.0\n# st = len(allimgs_st2)/2.0 - 6.0*step\n# end = len(allimgs_st2)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st2[ind2])\n# x[..., j+14] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T2/STIR')\n# pass\n \n# # Axial T2\n# allimgs_at2 = self.get_img_paths(st_id, 'Axial T2')\n# if len(allimgs_at2)==0:\n# print(st_id, ': Axial T2, has no images')\n \n# else:\n# step = len(allimgs_at2) / 14.0\n# st = len(allimgs_at2)/2.0 - 6.0*step\n# end = len(allimgs_at2)+0.0001\n\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_at2[ind2])\n# x[..., j+28] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Axial T2')\n# pass \n \n \n# if self.transform is not None:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1)\n \n# return x, str(st_id)\n# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.5, std=0.5)\n# ])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.855142Z","iopub.execute_input":"2024-10-07T18:57:20.855436Z","iopub.status.idle":"2024-10-07T18:57:20.865399Z","shell.execute_reply.started":"2024-10-07T18:57:20.855405Z","shell.execute_reply":"2024-10-07T18:57:20.864541Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_ds = RSNA2024TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(\n# test_ds, \n# batch_size=1, \n# shuffle=False,\n# num_workers=N_WORKERS,\n# pin_memory=True,\n# drop_last=False\n# )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.866646Z","iopub.execute_input":"2024-10-07T18:57:20.866936Z","iopub.status.idle":"2024-10-07T18:57:20.878527Z","shell.execute_reply.started":"2024-10-07T18:57:20.866906Z","shell.execute_reply":"2024-10-07T18:57:20.877662Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# class RSNA2024Model(nn.Module):\n# def __init__(self, model_name, in_c=42, n_classes=75, pretrained=True, features_only=False):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name,\n# pretrained=pretrained, \n# features_only=features_only,\n# in_chans=in_c,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n \n# def forward(self, x):\n# y = self.model(x)\n# return y","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.879551Z","iopub.execute_input":"2024-10-07T18:57:20.879857Z","iopub.status.idle":"2024-10-07T18:57:20.891311Z","shell.execute_reply.started":"2024-10-07T18:57:20.879825Z","shell.execute_reply":"2024-10-07T18:57:20.890546Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import torch\n\n# models = []\n\n# CKPT_PATHS = [\n# \"/kaggle/input/base-models/model0.pt\",\n# \"/kaggle/input/base-models/model1.pt\",\n# \"/kaggle/input/base-models/model2.pt\",\n# ]\n\n# CKPT_PATHS = sorted(CKPT_PATHS)\n\n# # Check if CUDA is available\n# use_cuda = torch.cuda.is_available()\n# device = torch.device('cuda' if use_cuda else 'cpu')\n\n# for i, cp in enumerate(CKPT_PATHS):\n# print(f'loading {cp}...')\n# model = RSNA2024Model(MODEL_NAME, IN_CHANS, N_CLASSES, pretrained=False)\n \n# # Load model with appropriate map_location\n# model.load_state_dict(torch.load(cp, map_location=device))\n# model.to(device)\n# model.eval()\n# model.half() if use_cuda else model.float()\n# models.append(model)\n\n# # Use autocast with appropriate dtype\n# if use_cuda:\n# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# else:\n# autocast = torch.cpu.amp.autocast(enabled=USE_AMP, dtype=torch.bfloat16)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.892358Z","iopub.execute_input":"2024-10-07T18:57:20.892744Z","iopub.status.idle":"2024-10-07T18:57:20.902436Z","shell.execute_reply.started":"2024-10-07T18:57:20.892712Z","shell.execute_reply":"2024-10-07T18:57:20.901609Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# y_preds = []\n# row_names = []\n\n# # Check if CUDA is available\n# use_cuda = torch.cuda.is_available()\n\n# # Use autocast with appropriate dtype\n# if use_cuda:\n# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# else:\n# autocast = torch.cpu.amp.autocast(enabled=USE_AMP, dtype=torch.bfloat16)\n\n# with tqdm(test_dl, leave=True) as pbar:\n# with torch.no_grad():\n# for idx, (x, si) in enumerate(pbar):\n# # Ensure input tensor is of type float\n# x = x.to(device).float()\n# pred_per_study = np.zeros((25, 3))\n \n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(si[0] + '_' + cond + '_' + level)\n \n# with autocast:\n# for m in models:\n# # Ensure model parameters are of type float\n# m = m.float()\n# y = m(x)[0]\n# for col in range(N_LABELS):\n# pred = y[col*3:col*3+3]\n# y_pred = pred.float().softmax(0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n# y_preds.append(pred_per_study)\n\n# y_preds = np.concatenate(y_preds, axis=0)\n# sub = pd.DataFrame()\n# sub['row_id'] = row_names\n# sub[LABELS] = y_preds\n# sub.head(25)\n# sub.to_csv('submission_3.csv', index=False)\n# #pd.read_csv('submission_3.csv').head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.903570Z","iopub.execute_input":"2024-10-07T18:57:20.903878Z","iopub.status.idle":"2024-10-07T18:57:20.913318Z","shell.execute_reply.started":"2024-10-07T18:57:20.903847Z","shell.execute_reply":"2024-10-07T18:57:20.912272Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Conclusion**\n\nThe AIMED research group successfully developed a convolutional neural network (CNN) model to classify degenerative spine conditions using lumbar spine MR images in this study. The model achieved a commendable accuracy of 75%, demonstrating the potential of deep learning techniques in medical imaging and diagnostics.\n\n**Key Findings**\n\n•\tModel Performance: The CNN model achieved a 75% accuracy in classifying various degenerative spine conditions, including Left Neural Foraminal Narrowing, Right Neural Foraminal Narrowing, Left Subarticular Stenosis, Right Subarticular Stenosis, and Spinal Canal Stenosis. This performance indicates that the model can effectively simulate a radiologist's diagnostic capabilities to a significant extent.\n•\tClass Imbalance: One of the primary challenges encountered was the class imbalance within the dataset. Certain conditions were underrepresented, which could potentially bias the model's performance. Addressing this imbalance through class weighting or data augmentation techniques could further enhance the model's accuracy and generalizability.\n\n•\tSingle Patient Test Data: The test data was derived from a single patient, which limits the model's generalizability. Future studies should include a more diverse test set to ensure the model's robustness across different patient populations.\n\n•\tData Augmentation and Feature Engineering: The study primarily relied on raw pixel data from MR images. Incorporating advanced data augmentation techniques and additional features such as patient demographics and clinical history could improve the model's predictive performance.\n\n•\tComputational Resources: Training deep learning models requires substantial computational resources. Limited access to high-performance computing infrastructure can constrain the ability to experiment with different model architectures and hyperparameters.\nImplications and Future Work\n\nThe results of this study highlight the potential of CNNs in automating the detection and classification of degenerative spine conditions, which can significantly aid radiologists in clinical settings. However, to fully realize this potential, several steps need to be taken:\n\n•\tAddressing Class Imbalance: Implementing techniques to handle class imbalance will improve the model's performance in underrepresented conditions.\n\n•\tDiverse Test Sets: Validating the model on a more diverse and independent test set will be essential to ensure its applicability in real-world scenarios.\n\n•\tAdvanced Data Augmentation: Employing more sophisticated data augmentation strategies can help the model generalize better to variations in the data.\n\n•\tIncorporating Additional Features: Integrating additional patient-specific features and metadata can enhance the model's diagnostic accuracy.\n\n•\tExternal Validation: External validation using datasets from different sources will be essential to assess the model's reliability across various imaging centers and patient populations.\n\nIn conclusion, the AIMED research group's work demonstrates a significant step forward in applying deep learning to medical imaging. By addressing the identified limitations and building on the current findings, future research can further enhance the accuracy and utility of AI-driven diagnostic tools in healthcare.\n","metadata":{}},{"cell_type":"markdown","source":"### About AIMED Research Group","metadata":{}},{"cell_type":"markdown","source":"The AIMED research group is a collaborative team composed of two radiology research specialists and one computer engineering research specialist. This interdisciplinary team combines expertise in medical imaging and advanced computational techniques to address complex challenges in the field of radiology. The radiology research specialists bring extensive clinical knowledge and experience, ensuring that the research is deeply rooted in practical medical applications and diagnostic accuracy. The computer engineering research specialist contributes advanced skills in machine learning, data processing, and algorithm development, enabling the creation of sophisticated models and tools.\n\nTogether, the AIMED research group leverages their combined expertise to develop innovative solutions that enhance diagnostic accuracy and efficiency in medical imaging. Their collaborative efforts aim to improve patient outcomes, advance the field of radiology, and push the boundaries of what is possible with artificial intelligence in healthcare.","metadata":{}},{"cell_type":"markdown","source":"## [RSNA2024 LSDC DenseNet Submission](https://www.kaggle.com/code/hugowjd/rsna2024-lsdc-densenet-submission)\n### [Jiadi Wang](https://www.kaggle.com/hugowjd)","metadata":{}},{"cell_type":"markdown","source":"### RSNA2024 LSDC Submission Baseline\nThis notebook is forked [here](https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-submission-baseline). In the [previous notebook](https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-making-dataset), the author selected the images we wanted to use and exported them to png.The original notebook training. And the original other trained EfficientNet_B4 model with these images. \n\nI desided to change the model to see if there is any improvement. The reason why I choose DenseNet201 for training is that DenseNet201 has generally same number of parameters and size with EfficientNet_B4 so that I believe that Kaggle GPU could handle it and we don't need extra machine.\n\n### My other Notebooks\n- [RSNA2024 LSDC Making Dataset](https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-making-dataset) \n- [RSNA2024 LSDC Training DenseNet](https://www.kaggle.com/code/hugowjd/rsna2024-lsdc-training-densenet) \n- [RSNA2024 LSDC Submission DenseNet](https://www.kaggle.com/code/hugowjd/rsna2024-lsdc-densenet-submission) <- you're reading now\n\n### Reference:\n* [Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q. Densely connected convolutional networks. CVPR, 2017.](https://arxiv.org/abs/1608.06993)\n* [Mingxing Tan and Quoc V. Le. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. ICML 2019.](https://arxiv.org/abs/1905.11946)\n\n### Future Improvement\n* I'm certain that we can run other models to train these images. We can do it by changing ***MODEL_NAME*** parameters in **Config** session. I would list some CNN models which are suitable for image classification and these numbers of parameters.\n * ResNet:\n * ResNet-18: ~11.7 million parameters\n * **ResNet-34: ~21.8 million parameters**\n * **ResNet-50: ~25.6 million parameters**\n * ResNet-101: ~44.5 million parameters\n * ResNet-152: ~60 million parameters\n * VGG:\n * VGG-16: ~138 million parameters\n * VGG-19: ~143 million parameters\n * Inception Networks:\n * Inception v1 (GoogleNet): ~6.8 million parameters\n * Inception v3: ~23.8 million parameters\n * DenseNet:\n * DenseNet-121: ~8 million parameters\n * **DenseNet-169: ~14 million parameters**\n * Waiting to do\n * **DenseNet-201: ~20 million parameters** \n * My Submission LB: 0.61\n * [10 folds submission LB](https://www.kaggle.com/code/sadidul012/densenet201-submission): 0.6\n * **DenseNet-161: ~28.7 million parameters**\n * 10 folds(This notebook V9): 0.59\n * DenseNet-264: ~33 million parameters\n * MobileNets (parameters can vary significantly with changes in alpha and resolution multipliers):\n * MobileNetV1 (1.0 224): ~4.2 million parameters\n * MobileNetV2 (1.0 224): ~3.5 million parameters\n * MobileNetV3 Large: ~5.4 million parameters\n * Vision Transformers (ViT):\n * ViT-B/16 (base model with patch size 16x16): ~86 million parameters\n * Xception:\n * **Xception: ~22.9 million parameters**\n * my test LB : 0.66\n * EfficientNet\n * EfficientNet-B0: ~5.3 million parameters\n * EfficientNet-B1: ~7.8 million parameters\n * EfficientNet-B2: ~9.2 million parameters\n * [EFNetV2 LB](https://www.kaggle.com/code/shubhamcodez/rsna-efficientnet-starter-notebook): 1.01 \n * EfficientNet-B3: ~12 million parameters\n * [Original Notebook](https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-submission-baseline) LB: 0.69\n * **EfficientNet-B4: ~19 million parameters**\n * [Original Notebook](https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-submission-baseline) LB: 0.70\n * EfficientNet-B5: ~30 million parameters\n * EfficientNet-B6: ~43 million parameters\n * EfficientNet-B7: ~66 million parameters\n* The original author said that we can improve the dataset making process\n* I only trained 5 **folds** and 10 **epochs**, you can modify these parameters. But take care of overfitting.","metadata":{}},{"cell_type":"markdown","source":"### Import Libralies","metadata":{}},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt\n# from sklearn.model_selection import KFold\n\n# from collections import OrderedDict\n\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n\n# import albumentations as A\n\n# from sklearn.model_selection import KFold\n\n# import re\n# import pydicom","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.914588Z","iopub.execute_input":"2024-10-07T18:57:20.914953Z","iopub.status.idle":"2024-10-07T18:57:20.927059Z","shell.execute_reply.started":"2024-10-07T18:57:20.914907Z","shell.execute_reply":"2024-10-07T18:57:20.926164Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# DENSE201_DIR = f'/kaggle/input/densenet-weights-for-rsna-2024/'\n# DENSE161_DIR = f'/kaggle/input/densenet161-weights-rsna2024/'\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 8620\n\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 30\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n\n# N_FOLDS = 5\n\n# # MODEL_NAME = \"tf_efficientnet_b4.ns_jft_in1k\"\n# # DENSE_MODEL_NAME = \"densenet201\"\n# DENSE_MODEL_NAME = 'densenet161.tv_in1k'\n# BATCH_SIZE = 1","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.928365Z","iopub.execute_input":"2024-10-07T18:57:20.928759Z","iopub.status.idle":"2024-10-07T18:57:20.939173Z","shell.execute_reply.started":"2024-10-07T18:57:20.928717Z","shell.execute_reply":"2024-10-07T18:57:20.938243Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.940332Z","iopub.execute_input":"2024-10-07T18:57:20.940671Z","iopub.status.idle":"2024-10-07T18:57:20.949101Z","shell.execute_reply.started":"2024-10-07T18:57:20.940638Z","shell.execute_reply":"2024-10-07T18:57:20.948220Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Config","metadata":{}},{"cell_type":"code","source":"# DENSE201_DIR = f'/kaggle/input/densenet-weights-for-rsna-2024/'\n# DENSE161_DIR = f'/kaggle/input/densenet161-weights-rsna2024/'\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 8620\n\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 30\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n\n# N_FOLDS = 5\n\n# # MODEL_NAME = \"tf_efficientnet_b4.ns_jft_in1k\"\n# # DENSE_MODEL_NAME = \"densenet201\"\n# DENSE_MODEL_NAME = 'densenet161.tv_in1k'\n# BATCH_SIZE = 1","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.950202Z","iopub.execute_input":"2024-10-07T18:57:20.950510Z","iopub.status.idle":"2024-10-07T18:57:20.959045Z","shell.execute_reply.started":"2024-10-07T18:57:20.950459Z","shell.execute_reply":"2024-10-07T18:57:20.958209Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.960277Z","iopub.execute_input":"2024-10-07T18:57:20.961152Z","iopub.status.idle":"2024-10-07T18:57:20.969729Z","shell.execute_reply.started":"2024-10-07T18:57:20.961106Z","shell.execute_reply":"2024-10-07T18:57:20.968993Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')\n# device","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.970905Z","iopub.execute_input":"2024-10-07T18:57:20.971242Z","iopub.status.idle":"2024-10-07T18:57:20.979710Z","shell.execute_reply.started":"2024-10-07T18:57:20.971210Z","shell.execute_reply":"2024-10-07T18:57:20.978934Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df = pd.read_csv(f'{rd}/test_series_descriptions.csv')\n# df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.980844Z","iopub.execute_input":"2024-10-07T18:57:20.981210Z","iopub.status.idle":"2024-10-07T18:57:20.990916Z","shell.execute_reply.started":"2024-10-07T18:57:20.981165Z","shell.execute_reply":"2024-10-07T18:57:20.989998Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# study_ids = list(df['study_id'].unique())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:20.991988Z","iopub.execute_input":"2024-10-07T18:57:20.992279Z","iopub.status.idle":"2024-10-07T18:57:21.000506Z","shell.execute_reply.started":"2024-10-07T18:57:20.992247Z","shell.execute_reply":"2024-10-07T18:57:20.999681Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sample_sub = pd.read_csv(f'{rd}/sample_submission.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.001757Z","iopub.execute_input":"2024-10-07T18:57:21.002232Z","iopub.status.idle":"2024-10-07T18:57:21.014270Z","shell.execute_reply.started":"2024-10-07T18:57:21.002185Z","shell.execute_reply":"2024-10-07T18:57:21.013347Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# LABELS = list(sample_sub.columns[1:])\n# LABELS","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.015481Z","iopub.execute_input":"2024-10-07T18:57:21.015953Z","iopub.status.idle":"2024-10-07T18:57:21.024133Z","shell.execute_reply.started":"2024-10-07T18:57:21.015908Z","shell.execute_reply":"2024-10-07T18:57:21.023249Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.025354Z","iopub.execute_input":"2024-10-07T18:57:21.025731Z","iopub.status.idle":"2024-10-07T18:57:21.032523Z","shell.execute_reply.started":"2024-10-07T18:57:21.025698Z","shell.execute_reply":"2024-10-07T18:57:21.031741Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [ atoi(c) for c in re.split(r'(\\d+)', text) ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.033816Z","iopub.execute_input":"2024-10-07T18:57:21.034467Z","iopub.status.idle":"2024-10-07T18:57:21.043028Z","shell.execute_reply.started":"2024-10-07T18:57:21.034422Z","shell.execute_reply":"2024-10-07T18:57:21.042209Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Dataset","metadata":{}},{"cell_type":"code","source":"# class RSNA24TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n \n# def __len__(self):\n# return len(self.study_ids)\n \n# def get_img_paths(self, study_id, series_desc):\n# pdf = self.df[self.df['study_id']==study_id]\n# pdf_ = pdf[pdf['series_description']==series_desc]\n# allimgs = []\n# for i, row in pdf_.iterrows():\n# pimgs = glob.glob(f'{rd}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm')\n# pimgs = sorted(pimgs, key=natural_keys)\n# allimgs.extend(pimgs)\n \n# return allimgs\n \n# def read_dcm_ret_arr(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# image = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# img = cv2.resize(image, (IMG_SIZE[0], IMG_SIZE[1]),interpolation=cv2.INTER_CUBIC)\n# assert img.shape==(IMG_SIZE[0], IMG_SIZE[1])\n# return img\n\n# def __getitem__(self, idx):\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], IN_CHANS), dtype=np.uint8)\n# st_id = self.study_ids[idx] \n \n# # Sagittal T1\n# allimgs_st1 = self.get_img_paths(st_id, 'Sagittal T1')\n# if len(allimgs_st1)==0:\n# print(st_id, ': Sagittal T1, has no images')\n \n# else:\n# step = len(allimgs_st1) / 10.0\n# st = len(allimgs_st1)/2.0 - 4.0*step\n# end = len(allimgs_st1)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st1[ind2])\n# x[..., j] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T1')\n# pass\n \n# # Sagittal T2/STIR\n# allimgs_st2 = self.get_img_paths(st_id, 'Sagittal T2/STIR')\n# if len(allimgs_st2)==0:\n# print(st_id, ': Sagittal T2/STIR, has no images')\n \n# else:\n# step = len(allimgs_st2) / 10.0\n# st = len(allimgs_st2)/2.0 - 4.0*step\n# end = len(allimgs_st2)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st2[ind2])\n# x[..., j+10] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T2/STIR')\n# pass\n \n# # Axial T2\n# allimgs_at2 = self.get_img_paths(st_id, 'Axial T2')\n# if len(allimgs_at2)==0:\n# print(st_id, ': Axial T2, has no images')\n \n# else:\n# step = len(allimgs_at2) / 10.0\n# st = len(allimgs_at2)/2.0 - 4.0*step\n# end = len(allimgs_at2)+0.0001\n\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_at2[ind2])\n# x[..., j+20] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Axial T2')\n# pass \n \n \n# if self.transform is not None:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1)\n \n# return x, str(st_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.044476Z","iopub.execute_input":"2024-10-07T18:57:21.045064Z","iopub.status.idle":"2024-10-07T18:57:21.054284Z","shell.execute_reply.started":"2024-10-07T18:57:21.045020Z","shell.execute_reply":"2024-10-07T18:57:21.053371Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.5, std=0.5)\n# ])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.055392Z","iopub.execute_input":"2024-10-07T18:57:21.055748Z","iopub.status.idle":"2024-10-07T18:57:21.067347Z","shell.execute_reply.started":"2024-10-07T18:57:21.055715Z","shell.execute_reply":"2024-10-07T18:57:21.066412Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_ds = RSNA24TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(\n# test_ds, \n# batch_size=1, \n# shuffle=False,\n# num_workers=N_WORKERS,\n# pin_memory=True,\n# drop_last=False\n# )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.068454Z","iopub.execute_input":"2024-10-07T18:57:21.068764Z","iopub.status.idle":"2024-10-07T18:57:21.076949Z","shell.execute_reply.started":"2024-10-07T18:57:21.068732Z","shell.execute_reply":"2024-10-07T18:57:21.076147Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Model","metadata":{}},{"cell_type":"code","source":"# class RSNA24Model(nn.Module):\n# def __init__(self, model_name, in_c=30, n_classes=75, pretrained=True, features_only=False):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name,\n# pretrained=pretrained, \n# features_only=features_only,\n# in_chans=in_c,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n \n# def forward(self, x):\n# y = self.model(x)\n# return y","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.078171Z","iopub.execute_input":"2024-10-07T18:57:21.078461Z","iopub.status.idle":"2024-10-07T18:57:21.090544Z","shell.execute_reply.started":"2024-10-07T18:57:21.078430Z","shell.execute_reply":"2024-10-07T18:57:21.089631Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load Models","metadata":{}},{"cell_type":"code","source":"# models = []","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.091738Z","iopub.execute_input":"2024-10-07T18:57:21.092654Z","iopub.status.idle":"2024-10-07T18:57:21.099431Z","shell.execute_reply.started":"2024-10-07T18:57:21.092617Z","shell.execute_reply":"2024-10-07T18:57:21.098651Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import glob\n# DENSE_CKPT_PATHS = glob.glob(f'{DENSE161_DIR}best_wll_model_fold-*.pt')\n# DENSE_CKPT_PATHS = sorted(DENSE_CKPT_PATHS)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.100572Z","iopub.execute_input":"2024-10-07T18:57:21.100890Z","iopub.status.idle":"2024-10-07T18:57:21.109648Z","shell.execute_reply.started":"2024-10-07T18:57:21.100859Z","shell.execute_reply":"2024-10-07T18:57:21.108794Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for i, cp in enumerate(DENSE_CKPT_PATHS):\n# print(f'loading {cp}...')\n# model = RSNA24Model(DENSE_MODEL_NAME, IN_CHANS, N_CLASSES, pretrained=False)\n# model.load_state_dict(torch.load(cp))\n# model.eval()\n# model.half()\n# model.to(device)\n# models.append(model)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.110810Z","iopub.execute_input":"2024-10-07T18:57:21.111175Z","iopub.status.idle":"2024-10-07T18:57:21.118780Z","shell.execute_reply.started":"2024-10-07T18:57:21.111142Z","shell.execute_reply":"2024-10-07T18:57:21.117910Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Inference loop","metadata":{}},{"cell_type":"code","source":"# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# y_preds = []\n# row_names = []\n\n# with tqdm(test_dl, leave=True) as pbar:\n# with torch.no_grad():\n# for idx, (x, si) in enumerate(pbar):\n# x = x.to(device)\n# pred_per_study = np.zeros((25, 3))\n \n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(si[0] + '_' + cond + '_' + level)\n \n# with autocast:\n# for m in models:\n# y = m(x)[0]\n# for col in range(N_LABELS):\n# pred = y[col*3:col*3+3]\n# y_pred = pred.float().softmax(0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n# y_preds.append(pred_per_study)\n\n# y_preds = np.concatenate(y_preds, axis=0)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.120218Z","iopub.execute_input":"2024-10-07T18:57:21.120605Z","iopub.status.idle":"2024-10-07T18:57:21.128805Z","shell.execute_reply.started":"2024-10-07T18:57:21.120562Z","shell.execute_reply":"2024-10-07T18:57:21.127972Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Make Submission","metadata":{}},{"cell_type":"code","source":"# sub = pd.DataFrame()\n# sub['row_id'] = row_names\n# sub[LABELS] = y_preds\n# sub.head(25)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.129967Z","iopub.execute_input":"2024-10-07T18:57:21.130292Z","iopub.status.idle":"2024-10-07T18:57:21.141849Z","shell.execute_reply.started":"2024-10-07T18:57:21.130260Z","shell.execute_reply":"2024-10-07T18:57:21.140875Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sub.to_csv('submission_4.csv', index=False)\n# pd.read_csv('submission_4.csv').head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.143015Z","iopub.execute_input":"2024-10-07T18:57:21.143765Z","iopub.status.idle":"2024-10-07T18:57:21.151146Z","shell.execute_reply.started":"2024-10-07T18:57:21.143731Z","shell.execute_reply":"2024-10-07T18:57:21.150311Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Conclusion\nWe created the dataset, performed training, and inference in this notebook. \n\nThis competition is a bit complicated to handle the dataset, so there may be a better way.\n\nI think there are many other areas to improve in my notebook. I hope you can learn from my notebook and get a better score.","metadata":{}},{"cell_type":"markdown","source":"## [RSNA Lumbar Spine Prediction](https://www.kaggle.com/code/marwanashref/rsna-lumbar-spine-prediction) \n### [Marwan Ashref](https://www.kaggle.com/marwanashref)","metadata":{}},{"cell_type":"markdown","source":"### Import the Libraries","metadata":{}},{"cell_type":"code","source":"# from sklearn.model_selection import KFold\n# from collections import OrderedDict\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n# import albumentations as A\n# from sklearn.model_selection import KFold\n# import re\n# import pydicom","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.152182Z","iopub.execute_input":"2024-10-07T18:57:21.152517Z","iopub.status.idle":"2024-10-07T18:57:21.161344Z","shell.execute_reply.started":"2024-10-07T18:57:21.152464Z","shell.execute_reply":"2024-10-07T18:57:21.160395Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.162610Z","iopub.execute_input":"2024-10-07T18:57:21.163239Z","iopub.status.idle":"2024-10-07T18:57:21.171125Z","shell.execute_reply.started":"2024-10-07T18:57:21.163193Z","shell.execute_reply":"2024-10-07T18:57:21.170283Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load & Edit The Data","metadata":{}},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'\n# OUTPUT_DIR = f'/kaggle/input/rsna2024-lsdc-training-baseline/rsna24-results'\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.172215Z","iopub.execute_input":"2024-10-07T18:57:21.172575Z","iopub.status.idle":"2024-10-07T18:57:21.182123Z","shell.execute_reply.started":"2024-10-07T18:57:21.172542Z","shell.execute_reply":"2024-10-07T18:57:21.181185Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Configuration\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 8620\n# IMG_SIZE = (512, 512)\n# IN_CHANS = 42\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n# N_FOLDS = 5\n# MODEL_NAME = \"edgenext_base.in21k_ft_in1k\"\n# BATCH_SIZE = 1\n# data_dir = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.183327Z","iopub.execute_input":"2024-10-07T18:57:21.183718Z","iopub.status.idle":"2024-10-07T18:57:21.192428Z","shell.execute_reply.started":"2024-10-07T18:57:21.183661Z","shell.execute_reply":"2024-10-07T18:57:21.191542Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Device setup\n# device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n\n# # Load DataFrames\n# df = pd.read_csv(f'{data_dir}/test_series_descriptions.csv')\n# study_ids = df['study_id'].unique().tolist()\n\n# sample_sub = pd.read_csv(f'{data_dir}/sample_submission.csv')\n# LABELS = sample_sub.columns[1:].tolist()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.193659Z","iopub.execute_input":"2024-10-07T18:57:21.194504Z","iopub.status.idle":"2024-10-07T18:57:21.205335Z","shell.execute_reply.started":"2024-10-07T18:57:21.194445Z","shell.execute_reply":"2024-10-07T18:57:21.204551Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Conditions and Levels\n# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.206323Z","iopub.execute_input":"2024-10-07T18:57:21.206636Z","iopub.status.idle":"2024-10-07T18:57:21.216091Z","shell.execute_reply.started":"2024-10-07T18:57:21.206605Z","shell.execute_reply":"2024-10-07T18:57:21.215225Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Helper functions\n# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [atoi(c) for c in re.split(r'(\\d+)', text)]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.217184Z","iopub.execute_input":"2024-10-07T18:57:21.217682Z","iopub.status.idle":"2024-10-07T18:57:21.225279Z","shell.execute_reply.started":"2024-10-07T18:57:21.217646Z","shell.execute_reply":"2024-10-07T18:57:21.224550Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Feature Engineering","metadata":{}},{"cell_type":"code","source":"# import glob\n# import cv2\n# import pydicom\n# import numpy as np\n# from torch.utils.data import Dataset, DataLoader\n# import albumentations as A\n\n# class RSNA24TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n\n# def __len__(self):\n# return len(self.study_ids)\n\n# def get_img_paths(self, study_id, series_desc):\n# series_df = self.df[(self.df['study_id'] == study_id) & \n# (self.df['series_description'] == series_desc)]\n# img_paths = []\n# for _, row in series_df.iterrows():\n# paths = sorted(glob.glob(f'{data_dir}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm'), \n# key=natural_keys)\n# img_paths.extend(paths)\n# return img_paths\n\n# def read_dcm_image(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# norm_img = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# resized_img = cv2.resize(norm_img, IMG_SIZE, interpolation=cv2.INTER_CUBIC)\n# return resized_img.astype(np.uint8)\n\n# def load_series_images(self, study_id, series_desc, start_idx):\n# images = np.zeros((IMG_SIZE[0], IMG_SIZE[1], 14), dtype=np.uint8)\n# img_paths = self.get_img_paths(study_id, series_desc)\n \n# if not img_paths:\n# print(f'{study_id}: {series_desc} has no images')\n# return images\n \n# step = len(img_paths) / 14.0\n# mid_point = len(img_paths) / 2.0 - 6.0 * step\n \n# for j, i in enumerate(np.arange(mid_point, len(img_paths), step)):\n# try:\n# idx = max(0, int(round(i - 0.5)))\n# images[..., j] = self.read_dcm_image(img_paths[idx])\n# except Exception as e:\n# print(f'Failed to load {series_desc} for {study_id}: {e}')\n \n# return images\n\n# def __getitem__(self, idx):\n# study_id = self.study_ids[idx]\n# channels = 42\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], channels), dtype=np.uint8)\n\n# # Load images for each series\n# x[..., :14] = self.load_series_images(study_id, 'Sagittal T1', 0)\n# x[..., 14:28] = self.load_series_images(study_id, 'Sagittal T2/STIR', 14)\n# x[..., 28:] = self.load_series_images(study_id, 'Axial T2', 28)\n\n# # Apply transformations\n# if self.transform:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1) # Channels-first for PyTorch\n# return x, str(study_id)\n\n# # Transformations\n# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.5, std=0.5)\n# ])\n\n# # Dataset and DataLoader\n# test_ds = RSNA24TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(test_ds, batch_size=1, shuffle=False, num_workers=N_WORKERS,\n# pin_memory=True, drop_last=False)\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.226769Z","iopub.execute_input":"2024-10-07T18:57:21.227055Z","iopub.status.idle":"2024-10-07T18:57:21.236999Z","shell.execute_reply.started":"2024-10-07T18:57:21.227024Z","shell.execute_reply":"2024-10-07T18:57:21.236110Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import torch.nn as nn\n# import timm\n\n# class RSNA24Model(nn.Module):\n# \"\"\"\n# Custom model class for RSNA 2024 Lumbar Spine Degenerative Classification.\n \n# Args:\n# model_name (str): Name of the model architecture from the TIMM library.\n# in_c (int): Number of input channels. Default is 42.\n# n_classes (int): Number of output classes. Default is 75.\n# pretrained (bool): If True, use pre-trained weights. Default is True.\n# features_only (bool): If True, return features only instead of final output. Default is False.\n# \"\"\"\n \n# def __init__(self, model_name, in_c=42, n_classes=75, pretrained=True, features_only=False):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name,\n# pretrained=pretrained,\n# features_only=features_only,\n# in_chans=in_c,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n \n# def forward(self, x):\n# \"\"\"\n# Forward pass for the model.\n \n# Args:\n# x (torch.Tensor): Input tensor with shape (batch_size, in_c, height, width).\n \n# Returns:\n# torch.Tensor: Model output with shape (batch_size, n_classes).\n# \"\"\"\n# return self.model(x)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.238076Z","iopub.execute_input":"2024-10-07T18:57:21.238436Z","iopub.status.idle":"2024-10-07T18:57:21.251205Z","shell.execute_reply.started":"2024-10-07T18:57:21.238404Z","shell.execute_reply":"2024-10-07T18:57:21.250351Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.252475Z","iopub.execute_input":"2024-10-07T18:57:21.252915Z","iopub.status.idle":"2024-10-07T18:57:21.264446Z","shell.execute_reply.started":"2024-10-07T18:57:21.252865Z","shell.execute_reply":"2024-10-07T18:57:21.263544Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import torch\n# import numpy as np\n# import pandas as pd\n# from tqdm import tqdm\n\n# # Constants\n# CKPT_PATHS = sorted([\n# \"/kaggle/input/rsna-2024-edgenext-base/model_fold-0.pt\",\n# \"/kaggle/input/rsna-2024-edgenext-base/model_fold-1.pt\",\n# \"/kaggle/input/rsna-2024-edgenext-base/model_fold-2.pt\",\n# ])\n\n# # Load Models\n# models = []\n# for cp in CKPT_PATHS:\n# print(f'Loading checkpoint: {cp}...')\n# model = RSNA24Model(MODEL_NAME, IN_CHANS, N_CLASSES, pretrained=False)\n# try:\n# model.load_state_dict(torch.load(cp))\n# except Exception as e:\n# print(f'Error loading {cp}: {e}')\n# continue\n\n# model.eval()\n# model.half()\n# model.to(device)\n# models.append(model)\n\n# # Autocast for mixed precision\n# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n\n# # Predictions and Submission\n# y_preds = []\n# row_names = []\n\n# with torch.no_grad():\n# for x, study_id in tqdm(test_dl, leave=True):\n# x = x.to(device)\n# pred_per_study = np.zeros((N_LABELS, 3))\n\n# # Generate row names for submission\n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(f'{study_id[0]}_{cond}_{level}')\n\n# with autocast:\n# for model in models:\n# y = model(x)[0]\n# for col in range(N_LABELS):\n# pred = y[col * 3 : (col + 1) * 3]\n# y_pred = pred.float().softmax(dim=0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n \n# y_preds.append(pred_per_study)\n\n# # Combine and save predictions\n# y_preds = np.concatenate(y_preds, axis=0)\n\n# submission_df = pd.DataFrame({\n# 'row_id': row_names,\n# **{label: y_preds[:, i] for i, label in enumerate(LABELS)}\n# })\n\n# submission_df.to_csv('submission_5.csv', index=False)\n\n# # Verify Submission\n# print(pd.read_csv('submission_5.csv').head(3))\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.265966Z","iopub.execute_input":"2024-10-07T18:57:21.266269Z","iopub.status.idle":"2024-10-07T18:57:21.274341Z","shell.execute_reply.started":"2024-10-07T18:57:21.266237Z","shell.execute_reply":"2024-10-07T18:57:21.273467Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Japan [RSNA[INF]edgenext_base_x512[LB.57]](https://www.kaggle.com/code/hideyukizushi/rsna-inf-edgenext-base-x512-lb-57)\n### [yukiZ](https://www.kaggle.com/hideyukizushi)","metadata":{}},{"cell_type":"markdown","source":"### ℹ️ **Info**\n* **[2024/8/23]forked original great work kernels**\n * https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-submission-baseline\n * https://www.kaggle.com/code/wanyanwendi/rsna2024\n* **[2024/8/23]add my dataset**\n * https://www.kaggle.com/datasets/hideyukizushi/rsna-2024-edgenext-base","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt\n# from sklearn.model_selection import KFold\n# from collections import OrderedDict\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n# import albumentations as A\n# from sklearn.model_selection import KFold\n# import re\n# import pydicom","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.275330Z","iopub.execute_input":"2024-10-07T18:57:21.275653Z","iopub.status.idle":"2024-10-07T18:57:21.287330Z","shell.execute_reply.started":"2024-10-07T18:57:21.275605Z","shell.execute_reply":"2024-10-07T18:57:21.286364Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'\n# OUTPUT_DIR = f'/kaggle/input/rsna2024-lsdc-training-baseline/rsna24-results'\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 8620\n\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 42\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n\n# N_FOLDS = 5\n\n# MODEL_NAME = \"edgenext_base.in21k_ft_in1k\"\n\n# BATCH_SIZE = 1\n# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'\n# device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')\n# device\n# df = pd.read_csv(f'{rd}/test_series_descriptions.csv')\n# df.head()\n# study_ids = list(df['study_id'].unique())\n# sample_sub = pd.read_csv(f'{rd}/sample_submission.csv')\n# LABELS = list(sample_sub.columns[1:])\n# LABELS\n# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.288458Z","iopub.execute_input":"2024-10-07T18:57:21.288790Z","iopub.status.idle":"2024-10-07T18:57:21.301257Z","shell.execute_reply.started":"2024-10-07T18:57:21.288746Z","shell.execute_reply":"2024-10-07T18:57:21.300436Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [ atoi(c) for c in re.split(r'(\\d+)', text) ]","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.302272Z","iopub.execute_input":"2024-10-07T18:57:21.302596Z","iopub.status.idle":"2024-10-07T18:57:21.310664Z","shell.execute_reply.started":"2024-10-07T18:57:21.302530Z","shell.execute_reply":"2024-10-07T18:57:21.309872Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# class RSNA24TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n \n# def __len__(self):\n# return len(self.study_ids)\n \n# def get_img_paths(self, study_id, series_desc):\n# pdf = self.df[self.df['study_id']==study_id]\n# pdf_ = pdf[pdf['series_description']==series_desc]\n# allimgs = []\n# for i, row in pdf_.iterrows():\n# pimgs = glob.glob(f'{rd}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm')\n# pimgs = sorted(pimgs, key=natural_keys)\n# allimgs.extend(pimgs)\n \n# return allimgs\n \n# def read_dcm_ret_arr(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# image = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# img = cv2.resize(image, (IMG_SIZE[0], IMG_SIZE[1]),interpolation=cv2.INTER_CUBIC)\n# assert img.shape==(IMG_SIZE[0], IMG_SIZE[1])\n# return img\n\n# def __getitem__(self, idx):\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], IN_CHANS), dtype=np.uint8)\n# st_id = self.study_ids[idx] \n \n# # Sagittal T1\n# allimgs_st1 = self.get_img_paths(st_id, 'Sagittal T1')\n# if len(allimgs_st1)==0:\n# print(st_id, ': Sagittal T1, has no images')\n \n# else:\n# step = len(allimgs_st1) / 14.0\n# st = len(allimgs_st1)/2.0 - 6.0*step\n# end = len(allimgs_st1)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st1[ind2])\n# x[..., j] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T1')\n# pass\n \n# # Sagittal T2/STIR\n# allimgs_st2 = self.get_img_paths(st_id, 'Sagittal T2/STIR')\n# if len(allimgs_st2)==0:\n# print(st_id, ': Sagittal T2/STIR, has no images')\n \n# else:\n# step = len(allimgs_st2) / 14.0\n# st = len(allimgs_st2)/2.0 - 6.0*step\n# end = len(allimgs_st2)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st2[ind2])\n# x[..., j+14] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T2/STIR')\n# pass\n \n# # Axial T2\n# allimgs_at2 = self.get_img_paths(st_id, 'Axial T2')\n# if len(allimgs_at2)==0:\n# print(st_id, ': Axial T2, has no images')\n \n# else:\n# step = len(allimgs_at2) / 14.0\n# st = len(allimgs_at2)/2.0 - 6.0*step\n# end = len(allimgs_at2)+0.0001\n\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_at2[ind2])\n# x[..., j+28] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Axial T2')\n# pass \n \n \n# if self.transform is not None:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1)\n \n# return x, str(st_id)\n# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.5, std=0.5)\n# ])\n# test_ds = RSNA24TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(\n# test_ds, \n# batch_size=1, \n# shuffle=False,\n# num_workers=N_WORKERS,\n# pin_memory=True,\n# drop_last=False\n# )","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.311880Z","iopub.execute_input":"2024-10-07T18:57:21.312168Z","iopub.status.idle":"2024-10-07T18:57:21.322597Z","shell.execute_reply.started":"2024-10-07T18:57:21.312137Z","shell.execute_reply":"2024-10-07T18:57:21.321649Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# class RSNA24Model(nn.Module):\n# def __init__(self, model_name, in_c=42, n_classes=75, pretrained=True, features_only=False):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name,\n# pretrained=pretrained, \n# features_only=features_only,\n# in_chans=in_c,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n \n# def forward(self, x):\n# y = self.model(x)\n# return y","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.323920Z","iopub.execute_input":"2024-10-07T18:57:21.324582Z","iopub.status.idle":"2024-10-07T18:57:21.336686Z","shell.execute_reply.started":"2024-10-07T18:57:21.324541Z","shell.execute_reply":"2024-10-07T18:57:21.335761Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# models = []\n# import glob\n\n# CKPT_PATHS = [\n# \"/kaggle/input/rsna-2024-edgenext-base/model_fold-0.pt\",\n# \"/kaggle/input/rsna-2024-edgenext-base/model_fold-1.pt\",\n# \"/kaggle/input/rsna-2024-edgenext-base/model_fold-2.pt\",\n# ]\n\n# CKPT_PATHS = sorted(CKPT_PATHS)\n# for i, cp in enumerate(CKPT_PATHS):\n# print(f'loading {cp}...')\n# model = RSNA24Model(MODEL_NAME, IN_CHANS, N_CLASSES, pretrained=False)\n# model.load_state_dict(torch.load(cp))\n# model.eval()\n# model.half()\n# model.to(device)\n# models.append(model)\n# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# y_preds = []\n# row_names = []\n\n# with tqdm(test_dl, leave=True) as pbar:\n# with torch.no_grad():\n# for idx, (x, si) in enumerate(pbar):\n# x = x.to(device)\n# pred_per_study = np.zeros((25, 3))\n \n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(si[0] + '_' + cond + '_' + level)\n \n# with autocast:\n# for m in models:\n# y = m(x)[0]\n# for col in range(N_LABELS):\n# pred = y[col*3:col*3+3]\n# y_pred = pred.float().softmax(0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n# y_preds.append(pred_per_study)\n\n# y_preds = np.concatenate(y_preds, axis=0)\n# sub = pd.DataFrame()\n# sub['row_id'] = row_names\n# sub[LABELS] = y_preds\n# sub.head(25)\n# sub.to_csv('submission_6.csv', index=False)\n# pd.read_csv('submission_6.csv').head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.337728Z","iopub.execute_input":"2024-10-07T18:57:21.338077Z","iopub.status.idle":"2024-10-07T18:57:21.347046Z","shell.execute_reply.started":"2024-10-07T18:57:21.338045Z","shell.execute_reply":"2024-10-07T18:57:21.346117Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Germany [LSDC Yolo Approach](https://www.kaggle.com/code/namgalielei/lsdc-yolo-approach)\n### [Liam Nguyen](https://www.kaggle.com/namgalielei)","metadata":{}},{"cell_type":"code","source":"!pip install -q --no-index --find-links /kaggle/input/ultralytics ultralytics","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:21.348148Z","iopub.execute_input":"2024-10-07T18:57:21.348475Z","iopub.status.idle":"2024-10-07T18:57:33.231317Z","shell.execute_reply.started":"2024-10-07T18:57:21.348443Z","shell.execute_reply":"2024-10-07T18:57:33.230032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport pydicom\nfrom PIL import Image\nimport numpy as np\nfrom multiprocessing import Pool, cpu_count\n\nimport sklearn.metrics\nimport torch\nimport cv2\nimport numpy as np \nimport pandas as pd \nfrom tqdm.auto import tqdm","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.232906Z","iopub.execute_input":"2024-10-07T18:57:33.233270Z","iopub.status.idle":"2024-10-07T18:57:33.239038Z","shell.execute_reply.started":"2024-10-07T18:57:33.233233Z","shell.execute_reply":"2024-10-07T18:57:33.238126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EVAL = False # Change to True to compute the validation score\nIMG_DIR = '/images'\nFOLD = 0\nSAMPLE = False # True for quick debugging\nSEVERITIES = ['Normal/Mild', 'Moderate', 'Severe']\nLEVELS = ['l1_l2', 'l2_l3', 'l3_l4', 'l4_l5', 'l5_s1']\n\nSCS_WEIGHTS = ['/kaggle/input/lsdc-train-yolo-scs/lsdc_yolov8/train/weights/best.pt']\n\nSS_WEIGHTS = ['/kaggle/input/lsdc-train-yolo-ss/lsdc_yolov8/train/weights/best.pt',\n '/kaggle/input/lsdc-yolo-ssv3/best.pt']\n\nNFN_WEIGHTS = ['/kaggle/input/lsdc-train-yolo-nfn/lsdc_yolov8/train/weights/best.pt',\n '/kaggle/input/lsdc-yolo-nfnv3/best.pt']","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.240457Z","iopub.execute_input":"2024-10-07T18:57:33.241079Z","iopub.status.idle":"2024-10-07T18:57:33.251638Z","shell.execute_reply.started":"2024-10-07T18:57:33.241037Z","shell.execute_reply":"2024-10-07T18:57:33.250833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if EVAL:\n import sys\n sys.path.append('/kaggle/input/lsdc-utils')\n from metrics import score as lsdc_scoring","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.252755Z","iopub.execute_input":"2024-10-07T18:57:33.253515Z","iopub.status.idle":"2024-10-07T18:57:33.267701Z","shell.execute_reply.started":"2024-10-07T18:57:33.253451Z","shell.execute_reply":"2024-10-07T18:57:33.266921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_val_df = pd.read_csv('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.268848Z","iopub.execute_input":"2024-10-07T18:57:33.269143Z","iopub.status.idle":"2024-10-07T18:57:33.296405Z","shell.execute_reply.started":"2024-10-07T18:57:33.269111Z","shell.execute_reply":"2024-10-07T18:57:33.295407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if EVAL:\n train_xy = pd.read_csv('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_label_coordinates.csv')\n des = pd.read_csv('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_series_descriptions.csv')\nelse: \n des = pd.read_csv('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/test_series_descriptions.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.297862Z","iopub.execute_input":"2024-10-07T18:57:33.298306Z","iopub.status.idle":"2024-10-07T18:57:33.308925Z","shell.execute_reply.started":"2024-10-07T18:57:33.298263Z","shell.execute_reply":"2024-10-07T18:57:33.307997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def read_dcm(src_path):\n dicom_data = pydicom.dcmread(src_path)\n image = dicom_data.pixel_array\n image = (image - image.min()) / (image.max() - image.min() +1e-6) * 255\n return image\n\ndef convert_dcm_to_jpg(file_path):\n try:\n # Read the DICOM file\n image_array = read_dcm(file_path)\n \n # Define the output path\n relative_path = os.path.relpath(file_path, start=input_directory)\n output_path = os.path.join(output_directory, relative_path)\n output_path = output_path.replace('.dcm', '.jpg')\n \n # Create the output directory if it doesn't exist\n os.makedirs(os.path.dirname(output_path), exist_ok=True)\n \n # Save the image as a JPEG file\n cv2.imwrite(output_path, image_array)\n \n return output_path\n except Exception as e:\n print(f\"Error processing file {file_path}: {e}\")\n return None\n\ndef process_files(dcm_files):\n with Pool(cpu_count()) as pool:\n # Wrap pool.map with tqdm to show the progress bar\n list(tqdm(pool.imap(convert_dcm_to_jpg, dcm_files), total=len(dcm_files)))\n\ndef get_dcm_files(directory):\n dcm_files = []\n for root, dirs, files in os.walk(directory):\n for file in files:\n if file.endswith('.dcm'):\n dcm_files.append(os.path.join(root, file))\n return dcm_files ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.310071Z","iopub.execute_input":"2024-10-07T18:57:33.310599Z","iopub.status.idle":"2024-10-07T18:57:33.320791Z","shell.execute_reply.started":"2024-10-07T18:57:33.310561Z","shell.execute_reply":"2024-10-07T18:57:33.319888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Replace these with your input and output directories\nif not EVAL:\n input_directory = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/test_images'\n\n output_directory = IMG_DIR\n\n # Get all .dcm files in the input directory\n dcm_files = get_dcm_files(input_directory)\n\n # Process the files using multiprocessing\n process_files(dcm_files)\n\n print(f\"Conversion completed. Images saved to {output_directory}\")\nelse:\n if not os.path.exists(IMG_DIR):\n print('Unziping data..')\n !unzip -q -d / /kaggle/input/lsdc-get-all-images/images.zip\n print('Done unziping data')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:33.322134Z","iopub.execute_input":"2024-10-07T18:57:33.322534Z","iopub.status.idle":"2024-10-07T18:57:34.299575Z","shell.execute_reply.started":"2024-10-07T18:57:33.322472Z","shell.execute_reply":"2024-10-07T18:57:34.298439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if EVAL:\n fold_df = pd.read_csv('/kaggle/input/lsdc-fold-split/5folds.csv')\n test_df = fold_df[fold_df.fold == FOLD]\n \nelse:\n test_df = os.listdir('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/test_images')\n test_df = pd.DataFrame(test_df, columns=['study_id'])\n test_df['study_id'] = test_df['study_id'].astype(int)\n \ntest_df = test_df.merge(des, on=['study_id'])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:34.301023Z","iopub.execute_input":"2024-10-07T18:57:34.301356Z","iopub.status.idle":"2024-10-07T18:57:34.312875Z","shell.execute_reply.started":"2024-10-07T18:57:34.301319Z","shell.execute_reply":"2024-10-07T18:57:34.311951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def gen_label_map(CONDITIONS):\n label2id = {}\n id2label = {}\n i = 0\n for cond in CONDITIONS:\n for level in LEVELS:\n for severity in SEVERITIES:\n cls_ = f\"{cond.lower().replace(' ', '_')}_{level}_{severity.lower()}\"\n label2id[cls_] = i\n id2label[i] = cls_\n i+=1\n return label2id, id2label\n \nscs_label2id, scs_id2label = gen_label_map(['Spinal Canal Stenosis'])\nss_label2id, ss_id2label = gen_label_map(['Left Subarticular Stenosis', 'Right Subarticular Stenosis'])\nnfn_label2id, nfn_id2label = gen_label_map(['Left Neural Foraminal Narrowing', 'Right Neural Foraminal Narrowing'])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:34.314758Z","iopub.execute_input":"2024-10-07T18:57:34.315060Z","iopub.status.idle":"2024-10-07T18:57:34.343892Z","shell.execute_reply.started":"2024-10-07T18:57:34.315030Z","shell.execute_reply":"2024-10-07T18:57:34.342908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from ultralytics import YOLO\n\n# Load YOLO Model\nscs_models = []\nfor weight in SCS_WEIGHTS:\n scs_models.append(YOLO(weight))\n \nss_models = []\nfor weight in SS_WEIGHTS:\n ss_models.append(YOLO(weight))\n \nnfn_models = []\nfor weight in NFN_WEIGHTS:\n nfn_models.append(YOLO(weight))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:34.345163Z","iopub.execute_input":"2024-10-07T18:57:34.345564Z","iopub.status.idle":"2024-10-07T18:57:35.227681Z","shell.execute_reply.started":"2024-10-07T18:57:34.345513Z","shell.execute_reply":"2024-10-07T18:57:35.226636Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"all_label_set = train_val_df.iloc[0, 1:].index.tolist()\nscs_label_set = all_label_set[:5]\nnfn_label_set = all_label_set[5:15]\nss_label_set = all_label_set[15:]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:35.228963Z","iopub.execute_input":"2024-10-07T18:57:35.229287Z","iopub.status.idle":"2024-10-07T18:57:35.234621Z","shell.execute_reply.started":"2024-10-07T18:57:35.229252Z","shell.execute_reply":"2024-10-07T18:57:35.233730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"settings = [\n ( 'Sagittal T2/STIR', scs_models, scs_id2label, scs_label_set, 0.01),\n ( 'Axial T2', ss_models, ss_id2label, ss_label_set, 0.01),\n ( 'Sagittal T1', nfn_models, nfn_id2label, nfn_label_set, 0.1)\n]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:35.235931Z","iopub.execute_input":"2024-10-07T18:57:35.236220Z","iopub.status.idle":"2024-10-07T18:57:35.249450Z","shell.execute_reply.started":"2024-10-07T18:57:35.236189Z","shell.execute_reply":"2024-10-07T18:57:35.248541Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from collections import defaultdict","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:35.252547Z","iopub.execute_input":"2024-10-07T18:57:35.252886Z","iopub.status.idle":"2024-10-07T18:57:35.258948Z","shell.execute_reply.started":"2024-10-07T18:57:35.252851Z","shell.execute_reply":"2024-10-07T18:57:35.258054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_rows = []\n\nfor modality, models, id2label, label_set, thresh in settings:\n mod_df = test_df[test_df.series_description == modality]\n \n if SAMPLE:\n mod_df = mod_df.sample(20, random_state=610)\n \n # for each study, at each level and condition, get the maximum probability score\n for study_id, group in tqdm(mod_df.groupby('study_id')):\n predictions = defaultdict(list)\n for i, row in group.iterrows():\n # predict on all images from all the series\n series_dir = os.path.join(IMG_DIR, str(row['study_id']), str(row['series_id']))\n for model in models:\n results = model(series_dir, conf=thresh, verbose=False)\n for res in results:\n for pred_class, conf in zip(res.boxes.cls, res.boxes.conf):\n pred_class = pred_class.item()\n conf = conf.item()\n _class = id2label[pred_class]\n predictions[_class].append(conf)\n \n # aggregate the result on images to obtain study-level prediction\n for condition in label_set:\n res_dict = {'row_id': f'{study_id}_{condition}' }\n\n score_vec = []\n for severity in SEVERITIES:\n severity = severity.lower()\n key = f'{condition}_{severity}'\n if len(predictions[key]) > 0:\n score = np.max(predictions[key])\n else:\n score = thresh\n score_vec.append(score)\n \n # normalize score to sum to 1\n score_vec = torch.tensor(score_vec)\n score_vec = score_vec / score_vec.sum()\n\n for idx, severity in enumerate(SEVERITIES):\n res_dict[severity.replace('/', '_').lower()] = score_vec[idx].item()\n\n pred_rows.append(res_dict)\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:35.260133Z","iopub.execute_input":"2024-10-07T18:57:35.262280Z","iopub.status.idle":"2024-10-07T18:57:39.069517Z","shell.execute_reply.started":"2024-10-07T18:57:35.262236Z","shell.execute_reply":"2024-10-07T18:57:39.068535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df = pd.DataFrame(pred_rows)\npred_df","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:39.070679Z","iopub.execute_input":"2024-10-07T18:57:39.070965Z","iopub.status.idle":"2024-10-07T18:57:39.086257Z","shell.execute_reply.started":"2024-10-07T18:57:39.070934Z","shell.execute_reply":"2024-10-07T18:57:39.085261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df.to_csv('submission_7.csv', index=False)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:39.087442Z","iopub.execute_input":"2024-10-07T18:57:39.087788Z","iopub.status.idle":"2024-10-07T18:57:39.096080Z","shell.execute_reply.started":"2024-10-07T18:57:39.087754Z","shell.execute_reply":"2024-10-07T18:57:39.095308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def sample_weight(row):\n if row['normal_mild'] == 1:\n return 1\n if row['moderate'] == 1:\n return 2\n if row['severe'] == 1:\n return 4\n raise ValueError('No such value')\n \ndef get_class(row):\n return np.argmax([row['normal_mild'], row['moderate'], row['severe']])","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:39.097272Z","iopub.execute_input":"2024-10-07T18:57:39.097620Z","iopub.status.idle":"2024-10-07T18:57:39.106548Z","shell.execute_reply.started":"2024-10-07T18:57:39.097586Z","shell.execute_reply":"2024-10-07T18:57:39.105650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if EVAL:\n gt_df = train_val_df.dropna().melt(id_vars=['study_id'], value_vars=all_label_set)\n gt_df['row_id'] = gt_df['study_id'].astype(str) + '_' + gt_df['variable']\n gt_df= gt_df[['row_id', 'value']]\n gt_df = pd.get_dummies(gt_df, columns=['value'], dtype=int)\n gt_df.columns = ['row_id', 'moderate', 'normal_mild', 'severe']\n gt_df = gt_df[['row_id', 'normal_mild', 'moderate', 'severe']]\n gt_df['sample_weight'] = gt_df.apply(sample_weight, axis=1)\n\n gt_df1 = gt_df.merge(pred_df['row_id'], how='inner', on='row_id').sort_values('row_id').reset_index(drop=True)\n pred_df1 = pred_df.merge(gt_df1['row_id'], how='inner', on='row_id').sort_values('row_id').reset_index(drop=True)\n gt_df1['pred_cls'] = gt_df1.apply(get_class, axis=1)\n pred_df1['pred_cls'] = pred_df1.apply(get_class, axis=1)\n\n gt_df1[(gt_df1['pred_cls'] != pred_df1['pred_cls'])]\n pred_df1[(gt_df1['pred_cls'] != pred_df1['pred_cls'])]\n print('Label count:\\n', gt_df1['pred_cls'].value_counts(normalize=True))\n print('Prediction accuracy:', (gt_df1['pred_cls'] == pred_df1['pred_cls']).mean())\n print()\n\n target_levels = ['normal_mild', 'moderate', 'severe']\n loss = lsdc_scoring(gt_df1.drop(['pred_cls'], axis=1), pred_df1.drop(['pred_cls'], axis=1), row_id_column_name='row_id', any_severe_scalar=1)\n print('Total weighted log loss:', loss)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:39.107723Z","iopub.execute_input":"2024-10-07T18:57:39.108044Z","iopub.status.idle":"2024-10-07T18:57:39.120881Z","shell.execute_reply.started":"2024-10-07T18:57:39.108010Z","shell.execute_reply":"2024-10-07T18:57:39.119968Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:39.122006Z","iopub.execute_input":"2024-10-07T18:57:39.122299Z","iopub.status.idle":"2024-10-07T18:57:40.146727Z","shell.execute_reply.started":"2024-10-07T18:57:39.122265Z","shell.execute_reply":"2024-10-07T18:57:40.145640Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Japan [LB0.55 | Inference by tempature0.9 | RSNA2024](https://www.kaggle.com/code/taikimori/lb0-55-inference-by-tempature0-9-rsna2024)\n### [Taimo](https://www.kaggle.com/taikimori)","metadata":{}},{"cell_type":"markdown","source":"### Training Setting\n\n* model: timm/efficientnet_b5.sw_in12k_ft_in1k\n* tempature: 0.9\n* training code: I borrowed [the notebook](https://www.kaggle.com/code/itsuki9180/rsna2024-lsdc-training-baseline) for training ","metadata":{}},{"cell_type":"markdown","source":"### Import Libralies","metadata":{}},{"cell_type":"code","source":"# import os\n# import gc\n# import sys\n# from PIL import Image\n# import cv2\n# import math, random\n# import numpy as np\n# import pandas as pd\n# from glob import glob\n# from tqdm import tqdm\n# import matplotlib.pyplot as plt\n# from sklearn.model_selection import KFold\n\n# from collections import OrderedDict\n\n# import torch\n# import torch.nn.functional as F\n# from torch import nn\n# from torch.utils.data import DataLoader, Dataset\n# from torch.optim import AdamW\n\n# import timm\n# from timm.utils import ModelEmaV2\n# from transformers import get_cosine_schedule_with_warmup\n\n# import albumentations as A\n\n# from sklearn.model_selection import KFold\n\n# import re\n# import pydicom","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.148285Z","iopub.execute_input":"2024-10-07T18:57:40.148658Z","iopub.status.idle":"2024-10-07T18:57:40.154161Z","shell.execute_reply.started":"2024-10-07T18:57:40.148621Z","shell.execute_reply":"2024-10-07T18:57:40.153312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.155196Z","iopub.execute_input":"2024-10-07T18:57:40.155479Z","iopub.status.idle":"2024-10-07T18:57:40.179452Z","shell.execute_reply.started":"2024-10-07T18:57:40.155449Z","shell.execute_reply":"2024-10-07T18:57:40.178527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# OUTPUT_DIR = f'/kaggle/input/rsna-2024-externals/cv_0.618_rsna24-results'\n# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n# N_WORKERS = os.cpu_count()\n# USE_AMP = True\n# SEED = 8620\n\n# IMG_SIZE = [512, 512]\n# IN_CHANS = 30\n# N_LABELS = 25\n# N_CLASSES = 3 * N_LABELS\n\n# N_FOLDS = 5\n\n# MODEL_NAME = \"tf_efficientnet_b5.ns_jft_in1k\"\n\n# BATCH_SIZE = 1","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.180519Z","iopub.execute_input":"2024-10-07T18:57:40.180805Z","iopub.status.idle":"2024-10-07T18:57:40.189628Z","shell.execute_reply.started":"2024-10-07T18:57:40.180775Z","shell.execute_reply":"2024-10-07T18:57:40.188674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# rd = '/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification'","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.190749Z","iopub.execute_input":"2024-10-07T18:57:40.191071Z","iopub.status.idle":"2024-10-07T18:57:40.198721Z","shell.execute_reply.started":"2024-10-07T18:57:40.191040Z","shell.execute_reply":"2024-10-07T18:57:40.197890Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')\n# device","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.200001Z","iopub.execute_input":"2024-10-07T18:57:40.200281Z","iopub.status.idle":"2024-10-07T18:57:40.209251Z","shell.execute_reply.started":"2024-10-07T18:57:40.200250Z","shell.execute_reply":"2024-10-07T18:57:40.208366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# df = pd.read_csv(f'{rd}/test_series_descriptions.csv')\n# df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.210285Z","iopub.execute_input":"2024-10-07T18:57:40.210635Z","iopub.status.idle":"2024-10-07T18:57:40.218096Z","shell.execute_reply.started":"2024-10-07T18:57:40.210603Z","shell.execute_reply":"2024-10-07T18:57:40.217197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# study_ids = list(df['study_id'].unique())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.220977Z","iopub.execute_input":"2024-10-07T18:57:40.221423Z","iopub.status.idle":"2024-10-07T18:57:40.228718Z","shell.execute_reply.started":"2024-10-07T18:57:40.221390Z","shell.execute_reply":"2024-10-07T18:57:40.227807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sample_sub = pd.read_csv(f'{rd}/sample_submission.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.229877Z","iopub.execute_input":"2024-10-07T18:57:40.230135Z","iopub.status.idle":"2024-10-07T18:57:40.237440Z","shell.execute_reply.started":"2024-10-07T18:57:40.230107Z","shell.execute_reply":"2024-10-07T18:57:40.236697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# LABELS = list(sample_sub.columns[1:])\n# LABELS","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.238527Z","iopub.execute_input":"2024-10-07T18:57:40.238930Z","iopub.status.idle":"2024-10-07T18:57:40.247334Z","shell.execute_reply.started":"2024-10-07T18:57:40.238898Z","shell.execute_reply":"2024-10-07T18:57:40.246542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CONDITIONS = [\n# 'spinal_canal_stenosis', \n# 'left_neural_foraminal_narrowing', \n# 'right_neural_foraminal_narrowing',\n# 'left_subarticular_stenosis',\n# 'right_subarticular_stenosis'\n# ]\n\n# LEVELS = [\n# 'l1_l2',\n# 'l2_l3',\n# 'l3_l4',\n# 'l4_l5',\n# 'l5_s1',\n# ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.248482Z","iopub.execute_input":"2024-10-07T18:57:40.248782Z","iopub.status.idle":"2024-10-07T18:57:40.257559Z","shell.execute_reply.started":"2024-10-07T18:57:40.248751Z","shell.execute_reply":"2024-10-07T18:57:40.256702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def atoi(text):\n# return int(text) if text.isdigit() else text\n\n# def natural_keys(text):\n# return [ atoi(c) for c in re.split(r'(\\d+)', text) ]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.258764Z","iopub.execute_input":"2024-10-07T18:57:40.259077Z","iopub.status.idle":"2024-10-07T18:57:40.269167Z","shell.execute_reply.started":"2024-10-07T18:57:40.259045Z","shell.execute_reply":"2024-10-07T18:57:40.268310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Dataset","metadata":{}},{"cell_type":"code","source":"# class RSNA24TestDataset(Dataset):\n# def __init__(self, df, study_ids, phase='test', transform=None):\n# self.df = df\n# self.study_ids = study_ids\n# self.transform = transform\n# self.phase = phase\n \n# def __len__(self):\n# return len(self.study_ids)\n \n# def get_img_paths(self, study_id, series_desc):\n# pdf = self.df[self.df['study_id']==study_id]\n# pdf_ = pdf[pdf['series_description']==series_desc]\n# allimgs = []\n# for i, row in pdf_.iterrows():\n# pimgs = glob.glob(f'{rd}/test_images/{study_id}/{row[\"series_id\"]}/*.dcm')\n# pimgs = sorted(pimgs, key=natural_keys)\n# allimgs.extend(pimgs)\n \n# return allimgs\n \n# def read_dcm_ret_arr(self, src_path):\n# dicom_data = pydicom.dcmread(src_path)\n# image = dicom_data.pixel_array\n# image = (image - image.min()) / (image.max() - image.min() + 1e-6) * 255\n# img = cv2.resize(image, (IMG_SIZE[0], IMG_SIZE[1]),interpolation=cv2.INTER_CUBIC)\n# assert img.shape==(IMG_SIZE[0], IMG_SIZE[1])\n# return img\n\n# def __getitem__(self, idx):\n# x = np.zeros((IMG_SIZE[0], IMG_SIZE[1], IN_CHANS), dtype=np.uint8)\n# st_id = self.study_ids[idx] \n \n# # Sagittal T1\n# allimgs_st1 = self.get_img_paths(st_id, 'Sagittal T1')\n# if len(allimgs_st1)==0:\n# print(st_id, ': Sagittal T1, has no images')\n \n# else:\n# step = len(allimgs_st1) / 10.0\n# st = len(allimgs_st1)/2.0 - 4.0*step\n# end = len(allimgs_st1)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st1[ind2])\n# x[..., j] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T1')\n# pass\n \n# # Sagittal T2/STIR\n# allimgs_st2 = self.get_img_paths(st_id, 'Sagittal T2/STIR')\n# if len(allimgs_st2)==0:\n# print(st_id, ': Sagittal T2/STIR, has no images')\n \n# else:\n# step = len(allimgs_st2) / 10.0\n# st = len(allimgs_st2)/2.0 - 4.0*step\n# end = len(allimgs_st2)+0.0001\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_st2[ind2])\n# x[..., j+10] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Sagittal T2/STIR')\n# pass\n \n# # Axial T2\n# allimgs_at2 = self.get_img_paths(st_id, 'Axial T2')\n# if len(allimgs_at2)==0:\n# print(st_id, ': Axial T2, has no images')\n \n# else:\n# step = len(allimgs_at2) / 10.0\n# st = len(allimgs_at2)/2.0 - 4.0*step\n# end = len(allimgs_at2)+0.0001\n\n# for j, i in enumerate(np.arange(st, end, step)):\n# try:\n# ind2 = max(0, int((i-0.5001).round()))\n# img = self.read_dcm_ret_arr(allimgs_at2[ind2])\n# x[..., j+20] = img.astype(np.uint8)\n# except:\n# print(f'failed to load on {st_id}, Axial T2')\n# pass \n \n \n# if self.transform is not None:\n# x = self.transform(image=x)['image']\n\n# x = x.transpose(2, 0, 1)\n \n# return x, str(st_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.289326Z","iopub.execute_input":"2024-10-07T18:57:40.289644Z","iopub.status.idle":"2024-10-07T18:57:40.297249Z","shell.execute_reply.started":"2024-10-07T18:57:40.289611Z","shell.execute_reply":"2024-10-07T18:57:40.296336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# transforms_test = A.Compose([\n# A.Resize(IMG_SIZE[0], IMG_SIZE[1]),\n# A.Normalize(mean=0.5, std=0.5)\n# ])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.298518Z","iopub.execute_input":"2024-10-07T18:57:40.298873Z","iopub.status.idle":"2024-10-07T18:57:40.309245Z","shell.execute_reply.started":"2024-10-07T18:57:40.298831Z","shell.execute_reply":"2024-10-07T18:57:40.308354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_ds = RSNA24TestDataset(df, study_ids, transform=transforms_test)\n# test_dl = DataLoader(\n# test_ds, \n# batch_size=1, \n# shuffle=False,\n# num_workers=N_WORKERS,\n# pin_memory=True,\n# drop_last=False\n# )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.310540Z","iopub.execute_input":"2024-10-07T18:57:40.311080Z","iopub.status.idle":"2024-10-07T18:57:40.319019Z","shell.execute_reply.started":"2024-10-07T18:57:40.311036Z","shell.execute_reply":"2024-10-07T18:57:40.318178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Define Model","metadata":{}},{"cell_type":"code","source":"# class RSNA24Model(nn.Module):\n# def __init__(self, model_name, in_c=30, n_classes=75, pretrained=True, features_only=False):\n# super().__init__()\n# self.model = timm.create_model(\n# model_name,\n# pretrained=pretrained, \n# features_only=features_only,\n# in_chans=in_c,\n# num_classes=n_classes,\n# global_pool='avg'\n# )\n \n# def forward(self, x):\n# y = self.model(x)\n# return y","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.321230Z","iopub.execute_input":"2024-10-07T18:57:40.321780Z","iopub.status.idle":"2024-10-07T18:57:40.330855Z","shell.execute_reply.started":"2024-10-07T18:57:40.321748Z","shell.execute_reply":"2024-10-07T18:57:40.330081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load Models","metadata":{}},{"cell_type":"code","source":"# models = []","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.331811Z","iopub.execute_input":"2024-10-07T18:57:40.332076Z","iopub.status.idle":"2024-10-07T18:57:40.340643Z","shell.execute_reply.started":"2024-10-07T18:57:40.332047Z","shell.execute_reply":"2024-10-07T18:57:40.339822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import glob\n# CKPT_PATHS = glob.glob('/kaggle/input/rsna-2024-externals/cv_0.618_rsna24-results/best_wll_model_fold-*.pt')\n# CKPT_PATHS = sorted(CKPT_PATHS)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.341638Z","iopub.execute_input":"2024-10-07T18:57:40.341906Z","iopub.status.idle":"2024-10-07T18:57:40.349545Z","shell.execute_reply.started":"2024-10-07T18:57:40.341876Z","shell.execute_reply":"2024-10-07T18:57:40.348729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for i, cp in enumerate(CKPT_PATHS):\n# print(f'loading {cp}...')\n# model = RSNA24Model(MODEL_NAME, IN_CHANS, N_CLASSES, pretrained=False)\n# model.load_state_dict(torch.load(cp))\n# model.eval()\n# model.half()\n# model.to(device)\n# models.append(model)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.350526Z","iopub.execute_input":"2024-10-07T18:57:40.350805Z","iopub.status.idle":"2024-10-07T18:57:40.359237Z","shell.execute_reply.started":"2024-10-07T18:57:40.350775Z","shell.execute_reply":"2024-10-07T18:57:40.358408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Inference loop","metadata":{}},{"cell_type":"code","source":"# autocast = torch.cuda.amp.autocast(enabled=USE_AMP, dtype=torch.half)\n# y_preds = []\n# row_names = []\n\n# with tqdm(test_dl, leave=True) as pbar:\n# with torch.no_grad():\n# for idx, (x, si) in enumerate(pbar):\n# x = x.to(device)\n# pred_per_study = np.zeros((25, 3))\n \n# for cond in CONDITIONS:\n# for level in LEVELS:\n# row_names.append(si[0] + '_' + cond + '_' + level)\n \n# with autocast:\n# for m in models:\n# y = m(x)[0]\n# for col in range(N_LABELS):\n# pred = y[col*3:col*3+3]\n# y_pred = (pred.float()/0.9).softmax(0).cpu().numpy()\n# pred_per_study[col] += y_pred / len(models)\n# y_preds.append(pred_per_study)\n\n# y_preds = np.concatenate(y_preds, axis=0)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.360342Z","iopub.execute_input":"2024-10-07T18:57:40.360650Z","iopub.status.idle":"2024-10-07T18:57:40.368628Z","shell.execute_reply.started":"2024-10-07T18:57:40.360619Z","shell.execute_reply":"2024-10-07T18:57:40.367859Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Make Submission","metadata":{}},{"cell_type":"code","source":"# sub = pd.DataFrame()\n# sub['row_id'] = row_names\n# sub[LABELS] = y_preds\n# sub.head(25)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.369822Z","iopub.execute_input":"2024-10-07T18:57:40.370116Z","iopub.status.idle":"2024-10-07T18:57:40.382082Z","shell.execute_reply.started":"2024-10-07T18:57:40.370084Z","shell.execute_reply":"2024-10-07T18:57:40.381099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sub.to_csv('submission_8.csv', index=False)\n# pd.read_csv('submission_8.csv').head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.383308Z","iopub.execute_input":"2024-10-07T18:57:40.383922Z","iopub.status.idle":"2024-10-07T18:57:40.392450Z","shell.execute_reply.started":"2024-10-07T18:57:40.383879Z","shell.execute_reply":"2024-10-07T18:57:40.391617Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## USA [3D Vision Transformer Single-Stage (Inference)](https://www.kaggle.com/code/vsahin/3d-vision-transformer-single-stage-inference)\n### [Victor S](https://www.kaggle.com/vsahin)","metadata":{}},{"cell_type":"markdown","source":"### Dependencies\n#### timm_3d\nTimm models adapted for 3D data\n#### torchio\nMedical imaging focused library. Used for 3D augmentations here\n#### spacecutter\nOrdinal regression layer and related loss and callback functions i.e. `LogisticCumulativeLink`\n#### open3d\nUsed for point cloud initialization and transformation. At first I used this for voxelization too, but dropped it in favor of a simpler sampling approach due to slow runtime.","metadata":{}},{"cell_type":"code","source":"%pip install --quiet /kaggle/input/timm_3d_deps/other/initial/9/pydicom/pydicom/pydicom-2.4.4-py3-none-any.whl\n%pip install timm_3d --no-index --quiet --find-links=/kaggle/input/timm_3d_deps/other/initial/9/timm_3d/\n%pip install torchio --no-index --quiet --find-links=/kaggle/input/timm_3d_deps/other/initial/9/torchio/\n%pip install itk --no-index --quiet --find-links=/kaggle/input/timm_3d_deps/other/initial/9/itk/itk\n%pip install skorch --no-index --quiet --find-links=/kaggle/input/timm_3d_deps/other/initial/9/skorch/skorch\n%pip install spacecutter --no-index --quiet --find-links=/kaggle/input/timm_3d_deps/other/initial/9/spacecutter/\n%pip install open3d --no-index --quiet --find-links=/kaggle/input/timm_3d_deps/other/initial/9/open3d","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-10-07T18:57:40.393554Z","iopub.execute_input":"2024-10-07T18:57:40.394295Z","iopub.status.idle":"2024-10-07T18:59:02.814675Z","shell.execute_reply.started":"2024-10-07T18:57:40.394261Z","shell.execute_reply":"2024-10-07T18:59:02.813381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_path = \"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/\"","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.816332Z","iopub.execute_input":"2024-10-07T18:59:02.816700Z","iopub.status.idle":"2024-10-07T18:59:02.821817Z","shell.execute_reply.started":"2024-10-07T18:59:02.816662Z","shell.execute_reply":"2024-10-07T18:59:02.820836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\n\ndef retrieve_test_data(data_path):\n test_df = pd.read_csv(data_path + 'test_series_descriptions.csv')\n\n return test_df\n\nretrieve_test_data(data_path)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.823254Z","iopub.execute_input":"2024-10-07T18:59:02.823760Z","iopub.status.idle":"2024-10-07T18:59:02.848930Z","shell.execute_reply.started":"2024-10-07T18:59:02.823716Z","shell.execute_reply":"2024-10-07T18:59:02.848022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\n\ndef retrieve_image_paths(base_path, study_id, series_id):\n series_dir = os.path.join(base_path, str(study_id), str(series_id))\n images = os.listdir(series_dir)\n image_paths = [os.path.join(series_dir, img) for img in images]\n return image_paths","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.850142Z","iopub.execute_input":"2024-10-07T18:59:02.850716Z","iopub.status.idle":"2024-10-07T18:59:02.856257Z","shell.execute_reply.started":"2024-10-07T18:59:02.850673Z","shell.execute_reply":"2024-10-07T18:59:02.855414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Data Loading\n\nQuite a few things are happening here. Refer here for more details on where the affine transforms come from: https://nipy.org/nibabel/dicom/dicom_orientation.html\n\nThe rough idea is -- converting the slices into point cloud format, transforming them into the same patient coordinate space, then converting into a voxel grid.\n\n1. Resizing each slice -- doing this first reduces the runtime from having to transform each point just to resize/downsample later anyway.\n2. Duplicating the slices and applying the affine transforms -- I found duplicating each slice by slice thickness makes it easier for the model to learn. The spatial information between slices might get somewhat warped, but the relative ordering is still the same but with much less empty space.\n3. Sampling into a voxel grid -- I am simply scaling the coordinates into the desired grid size, then sampling each channel by max.","metadata":{}},{"cell_type":"code","source":"import open3d as o3d\nfrom pydicom import dcmread\nimport math\nimport numpy as np\nimport cv2\nimport copy\n\ndef read_study_as_pcd(dir_path, series_types_dict=None, downsampling_factor=1, img_size=(256, 256)):\n pcd_overall = o3d.geometry.PointCloud()\n\n for path in glob.glob(os.path.join(dir_path, \"**/*.dcm\"), recursive=True):\n dicom_slice = dcmread(path)\n\n series_id = os.path.basename(os.path.dirname(path))\n study_id = os.path.basename(os.path.dirname(os.path.dirname(path)))\n if series_types_dict is None or int(series_id) not in series_types_dict:\n series_desc = dicom_slice.SeriesDescription\n else:\n series_desc = series_types_dict[int(series_id)]\n series_desc = series_desc.split(\" \")[-1]\n\n x_orig, y_orig = dicom_slice.pixel_array.shape\n img = np.expand_dims(cv2.resize(dicom_slice.pixel_array, img_size, interpolation=cv2.INTER_AREA), -1)\n x, y, z = np.where(img)\n\n downsampling_factor_iter = max(downsampling_factor, int(math.ceil(len(x) / 6e6)))\n\n index_voxel = np.vstack((x, y, z))[:, ::downsampling_factor_iter]\n grid_index_array = index_voxel.T\n pcd = o3d.geometry.PointCloud(o3d.utility.Vector3dVector(grid_index_array.astype(np.float64)))\n\n vals = np.expand_dims(img[x, y, z][::downsampling_factor_iter], -1)\n if series_desc == \"T1\":\n vals = np.pad(vals, ((0, 0), (0, 2)))\n elif series_desc == \"T2\":\n vals = np.pad(vals, ((0, 0), (1, 1)))\n elif series_desc == \"T2/STIR\":\n vals = np.pad(vals, ((0, 0), (2, 0)))\n else:\n raise ValueError(f\"Unknown series desc: {series_desc}\")\n\n pcd.colors = o3d.utility.Vector3dVector(vals.astype(np.float64))\n\n dX, dY = dicom_slice.PixelSpacing\n dZ = dicom_slice.SliceThickness\n\n X = np.array(list(dicom_slice.ImageOrientationPatient[:3]) + [0]) * dX\n Y = np.array(list(dicom_slice.ImageOrientationPatient[3:]) + [0]) * dY\n\n for z in range(int(dZ)):\n pos = list(dicom_slice.ImagePositionPatient)\n if series_desc == \"T2\":\n pos[-1] += z\n else:\n pos[0] += z\n S = np.array(pos + [1])\n\n transform_matrix = np.array([X, Y, np.zeros(len(X)), S]).T\n transform_matrix = transform_matrix @ np.matrix(\n [[0, y_orig / img_size[1], 0, 0],\n [x_orig / img_size[0], 0, 0, 0],\n [0, 0, 1, 0],\n [0, 0, 0, 1]]\n )\n\n pcd_overall += copy.deepcopy(pcd).transform(transform_matrix)\n\n return pcd_overall\n\n\n\ndef read_study_as_voxel_grid(dir_path, series_type_dict=None, downsampling_factor=1, img_size=(256, 256)):\n pcd_overall = read_study_as_pcd(dir_path,\n series_types_dict=series_type_dict,\n downsampling_factor=downsampling_factor,\n img_size=img_size)\n box = pcd_overall.get_axis_aligned_bounding_box()\n\n max_b = np.array(box.get_max_bound())\n min_b = np.array(box.get_min_bound())\n\n pts = (np.array(pcd_overall.points) - (min_b)) * (\n (img_size[0] - 1, img_size[0] - 1, img_size[0] - 1) / (max_b - min_b))\n coords = np.round(pts).astype(np.int32)\n vals = np.array(pcd_overall.colors, dtype=np.float16)\n\n grid = np.zeros((3, img_size[0], img_size[0], img_size[0]), dtype=np.float16)\n indices = coords[:, 0], coords[:, 1], coords[:, 2]\n\n np.maximum.at(grid[0], indices, vals[:, 0])\n np.maximum.at(grid[1], indices, vals[:, 1])\n np.maximum.at(grid[2], indices, vals[:, 2])\n\n\n return grid","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.858011Z","iopub.execute_input":"2024-10-07T18:59:02.858416Z","iopub.status.idle":"2024-10-07T18:59:02.888986Z","shell.execute_reply.started":"2024-10-07T18:59:02.858374Z","shell.execute_reply":"2024-10-07T18:59:02.888119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nfrom torch.utils.data import Dataset, DataLoader\nimport torchio as tio\nimport torch.nn as nn\nimport pydicom\n\nCONDITIONS = {\n \"Sagittal T2/STIR\": [\"Spinal Canal Stenosis\"],\n \"Axial T2\": [\"Left Subarticular Stenosis\", \"Right Subarticular Stenosis\"],\n \"Sagittal T1\": [\"Left Neural Foraminal Narrowing\", \"Right Neural Foraminal Narrowing\"],\n}\n\n\nclass PatientLevelTestset(Dataset):\n def __init__(self,\n base_path: str,\n dataframe: pd.DataFrame,\n transform_3d=None):\n self.base_path = base_path\n\n self.dataframe = (dataframe[['study_id', \"series_id\", \"series_description\"]]\n .drop_duplicates())\n\n self.subjects = self.dataframe[['study_id']].drop_duplicates().reset_index(drop=True)\n self.series_descs = {e[0]: e[1] for e in self.dataframe[[\"series_id\", \"series_description\"]].drop_duplicates().values}\n\n self.transform_3d = transform_3d\n\n def __len__(self):\n return len(self.subjects)\n\n def __getitem__(self, index):\n curr = self.subjects.iloc[index]\n study_path = os.path.join(self.base_path, str(curr[\"study_id\"]))\n\n study_images = read_study_as_voxel_grid(study_path, self.series_descs)\n\n if self.transform_3d is not None:\n study_images = self.transform_3d(torch.FloatTensor(study_images)) # .data\n return study_images.to(torch.half), str(curr[\"study_id\"])\n\n return torch.HalfTensor(study_images), str(curr[\"study_id\"])\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.890077Z","iopub.execute_input":"2024-10-07T18:59:02.890375Z","iopub.status.idle":"2024-10-07T18:59:02.905934Z","shell.execute_reply.started":"2024-10-07T18:59:02.890339Z","shell.execute_reply":"2024-10-07T18:59:02.905193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transform_3d = tio.Compose([\n tio.RescaleIntensity([0, 1]),\n])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.906997Z","iopub.execute_input":"2024-10-07T18:59:02.907304Z","iopub.status.idle":"2024-10-07T18:59:02.918878Z","shell.execute_reply.started":"2024-10-07T18:59:02.907255Z","shell.execute_reply":"2024-10-07T18:59:02.918132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_subject_level_testset_and_loader(df: pd.DataFrame,\n transform_3d,\n base_path: str,\n batch_size=1,\n num_workers=0):\n testset = PatientLevelTestset(base_path, df, transform_3d=transform_3d)\n test_loader = DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=num_workers)\n\n return testset, test_loader","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.919986Z","iopub.execute_input":"2024-10-07T18:59:02.920290Z","iopub.status.idle":"2024-10-07T18:59:02.928498Z","shell.execute_reply.started":"2024-10-07T18:59:02.920257Z","shell.execute_reply":"2024-10-07T18:59:02.927625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\n\ndata = retrieve_test_data(data_path)\ndataset, dataloader = create_subject_level_testset_and_loader(data, transform_3d, os.path.join(data_path, \"test_images\"))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.929543Z","iopub.execute_input":"2024-10-07T18:59:02.929829Z","iopub.status.idle":"2024-10-07T18:59:02.947901Z","shell.execute_reply.started":"2024-10-07T18:59:02.929799Z","shell.execute_reply":"2024-10-07T18:59:02.946878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport glob \nimport torch\n\ngrid = dataset[0][0]\n\nfig, axs = plt.subplots(3, 3)\n\naxs[0, 0].imshow(grid[0, 128])\naxs[1, 0].imshow(grid[1, 128])\naxs[2, 0].imshow(grid[2, 128])\n\naxs[0, 1].imshow(grid[0, :, 128])\naxs[1, 1].imshow(grid[1, :, 128])\naxs[2, 1].imshow(grid[2, :, 128])\n\naxs[0, 2].imshow(grid[0, :, :, 128])\naxs[1, 2].imshow(grid[1, :, :, 128])\naxs[2, 2].imshow(grid[2, :, :, 128])\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:02.949003Z","iopub.execute_input":"2024-10-07T18:59:02.949284Z","iopub.status.idle":"2024-10-07T18:59:22.736940Z","shell.execute_reply.started":"2024-10-07T18:59:02.949253Z","shell.execute_reply":"2024-10-07T18:59:22.736102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\ndevice = torch.device(\"cuda\") if torch.cuda.is_available() else \"cpu\"","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:22.738319Z","iopub.execute_input":"2024-10-07T18:59:22.738672Z","iopub.status.idle":"2024-10-07T18:59:22.743523Z","shell.execute_reply.started":"2024-10-07T18:59:22.738639Z","shell.execute_reply":"2024-10-07T18:59:22.742526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model Architecture\n\nI have somewhat better performance from a ViT. Might be due to better handling the gaps between slices. \n\nAlso note the LogisticCumulativeLink as the final head layer. This allows learning a continuous severity feature from ordinal labels and vice versa for inference.\n\n","metadata":{}},{"cell_type":"code","source":"import torch.nn as nn\nimport timm_3d\nfrom spacecutter import *\nfrom spacecutter.losses import *\nfrom spacecutter.models import *\nfrom spacecutter.callbacks import *\n\n\nclass CNN_Model_3D_Multihead(nn.Module):\n def __init__(self,\n backbone=\"efficientnet_lite0\",\n in_chans=1,\n out_classes=5,\n cutpoint_margin=0.15,\n pretrained=False):\n super(CNN_Model_3D_Multihead, self).__init__()\n self.out_classes = out_classes\n\n self.encoder = timm_3d.create_model(\n backbone,\n features_only=False,\n drop_rate=0,\n drop_path_rate=0,\n pretrained=pretrained,\n in_chans=in_chans,\n global_pool=\"max\"\n )\n if \"efficientnet\" in backbone:\n head_in_dim = self.encoder.classifier.in_features\n self.encoder.classifier = nn.Sequential(\n nn.LayerNorm(head_in_dim),\n nn.Dropout(0),\n )\n\n elif \"vit\" in backbone:\n self.encoder.head.drop = nn.Dropout(0)\n head_in_dim = self.encoder.head.fc.in_features\n self.encoder.head.fc = nn.Identity()\n\n self.heads = nn.ModuleList(\n [nn.Sequential(\n nn.Linear(head_in_dim, 1),\n LogisticCumulativeLink(3)\n ) for i in range(out_classes)]\n )\n\n self.ascension_callback = AscensionCallback(margin=cutpoint_margin)\n\n def forward(self, x):\n feat = self.encoder(x)\n return torch.swapaxes(torch.stack([head(feat) for head in self.heads]), 0, 1)\n\n def _ascension_callback(self):\n for head in self.heads:\n self.ascension_callback.clip(head[-1])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:22.744784Z","iopub.execute_input":"2024-10-07T18:59:22.745072Z","iopub.status.idle":"2024-10-07T18:59:24.359964Z","shell.execute_reply.started":"2024-10-07T18:59:22.745040Z","shell.execute_reply":"2024-10-07T18:59:24.359153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = CNN_Model_3D_Multihead(backbone=\"maxvit_rmlp_tiny_rw_256\", in_chans=3, out_classes=25).to(device)\nmodel.load_state_dict(torch.load(\"/kaggle/input/rsna-2024/pytorch/vit_voxel_v2/6/maxvit_rmlp_tiny_rw_256_256_v2_fold_3_32.pt\"))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:24.361392Z","iopub.execute_input":"2024-10-07T18:59:24.361683Z","iopub.status.idle":"2024-10-07T18:59:26.354117Z","shell.execute_reply.started":"2024-10-07T18:59:24.361652Z","shell.execute_reply":"2024-10-07T18:59:26.352980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"CONDITIONS = {\n \"Sagittal T2/STIR\": [\"spinal_canal_stenosis\"],\n \"Axial T2\": [\"left_subarticular_stenosis\", \"right_subarticular_stenosis\"],\n \"Sagittal T1\": [\"left_neural_foraminal_narrowing\", \"right_neural_foraminal_narrowing\"],\n}\n\nALL_CONDITIONS = sorted([\"spinal_canal_stenosis\", \"left_subarticular_stenosis\", \"right_subarticular_stenosis\", \"left_neural_foraminal_narrowing\", \"right_neural_foraminal_narrowing\"])\nLEVELS = [\"l1_l2\", \"l2_l3\", \"l3_l4\", \"l4_l5\", \"l5_s1\"]\n\nresults_df = pd.DataFrame({\"row_id\":[], \"normal_mild\": [], \"moderate\": [], \"severe\": []})\n\nALL_CONDITIONS","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:26.355446Z","iopub.execute_input":"2024-10-07T18:59:26.355868Z","iopub.status.idle":"2024-10-07T18:59:26.365439Z","shell.execute_reply.started":"2024-10-07T18:59:26.355824Z","shell.execute_reply":"2024-10-07T18:59:26.364654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Pre-populate results df\nimport glob\nimport os\n\nstudy_ids = glob.glob(\"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/test_images/*\")\nstudy_ids = [os.path.basename(e) for e in study_ids]\n\nresults_df = pd.DataFrame({\"row_id\":[], \"normal_mild\": [], \"moderate\": [], \"severe\": []})\nfor study_id in study_ids:\n for condition in ALL_CONDITIONS:\n for level in LEVELS:\n row_id = f\"{study_id}_{condition}_{level}\"\n results_df = results_df._append({\"row_id\": row_id, \"normal_mild\": 1/3, \"moderate\": 1/3, \"severe\": 1/3}, ignore_index=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:26.366608Z","iopub.execute_input":"2024-10-07T18:59:26.366900Z","iopub.status.idle":"2024-10-07T18:59:26.406509Z","shell.execute_reply.started":"2024-10-07T18:59:26.366868Z","shell.execute_reply":"2024-10-07T18:59:26.405713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset.dataframe","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:26.407613Z","iopub.execute_input":"2024-10-07T18:59:26.407912Z","iopub.status.idle":"2024-10-07T18:59:26.416902Z","shell.execute_reply.started":"2024-10-07T18:59:26.407880Z","shell.execute_reply":"2024-10-07T18:59:26.415948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nfrom torch.cuda.amp import autocast\nimport time\n\nstart_time = time.time()\n\nwith torch.no_grad():\n with autocast(dtype=torch.float16):\n model.eval()\n\n for images, study_id in dataloader:\n output = model(images.to(device))\n for i, batch_out in enumerate(output):\n batch_out = output.cpu().numpy()[i]\n for index, level in enumerate(batch_out):\n row_id = f\"{study_id[i]}_{ALL_CONDITIONS[index // 5]}_{LEVELS[index % 5]}\"\n results_df.loc[results_df.row_id == row_id,'normal_mild'] = level[0]\n results_df.loc[results_df.row_id == row_id,'moderate'] = level[1]\n results_df.loc[results_df.row_id == row_id,'severe'] = level[2]\n \nprint(\"--- %s seconds ---\" % (time.time() - start_time))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:26.418057Z","iopub.execute_input":"2024-10-07T18:59:26.418341Z","iopub.status.idle":"2024-10-07T18:59:45.797174Z","shell.execute_reply.started":"2024-10-07T18:59:26.418309Z","shell.execute_reply":"2024-10-07T18:59:45.796203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results_df","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:45.798683Z","iopub.execute_input":"2024-10-07T18:59:45.799064Z","iopub.status.idle":"2024-10-07T18:59:45.814270Z","shell.execute_reply.started":"2024-10-07T18:59:45.799019Z","shell.execute_reply":"2024-10-07T18:59:45.813348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results_df.to_csv('submission_9.csv', index=False)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:45.815346Z","iopub.execute_input":"2024-10-07T18:59:45.815658Z","iopub.status.idle":"2024-10-07T18:59:45.824642Z","shell.execute_reply.started":"2024-10-07T18:59:45.815625Z","shell.execute_reply":"2024-10-07T18:59:45.823705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ensemble submissions","metadata":{}},{"cell_type":"code","source":"# - option 17-> LB=0.48 work (7,9) + weights (0.257+0.743) \n# - option 18-> LB=0.48 work ( 7,9 ) + weights ( 0.33+0.67 )\n# - option 19-> LB=0.48 work ( 7,9 ) + weights ( 0.450+0.550 ) - previus.1 best option\n# - option 20-> LB=0.48 work ( 7,9 ) + weights ( 0.50+0.50 )\n# - option 23-> LB=0.48 work ( 7,9 ) + weights ( 0.473+0.527 )\n# - option 24-> LB=0.48 work ( 7,9 ) + weights ( 0.427+0.573 ) - previus.2 best option\n# - option 25-> LB=0.48 work ( 7,9 ) + weights ( 0.402+0.598 ) - previus.3 best option\n# - option 26-> LB=0.49 work ( 7,9 ) + weights ( 0.377+0.623 )\n# - **option 27**-> LB=**0.48** work ( 7,9 ) + weights ( 0.407+0.593 ) - **best option**\n# - option 28-> LB=0.48 work ( 7,9 ) + weights ( 0.395+0.605 )\n# - option 29-> LB=0.48 work ( 7,9 ) + weights ( 0.402+0.598 )\n# - option 30-> LB=0.48 work ( 7,9 ) + weights ( 0.405+0.595 )\n# - option 31-> LB=0.48 work ( 7,9 ) + weights ( 0.415+0.585 )\n# - option 32-> LB=0.48 work ( 7,9 ) + weights ( 0.410+0.590 )\n\n# some rezult:\n# * option.18 < option.20 < option.19 < option.24 == (version.48 < version.51 < version.49 < version.55-56)\n# * option.24 < option.25 == (version.55-56 < version.58)\n# * option.25 < option.28 < **option.27** == (version.58 < version.63 < **version.62**)\n# * option.29 < option.30 < **option.27** == (version.58 < version.63 < **version.62**)\n# * option.31 < option.32 < **option.27** == (version.58 < version.63 < **version.62**)\n\n# best option:\n# - option.32 < **option.27**\n\n# current option: \n# - option 33-> LB=0.4? work ( 8,9 ) + weights ( 0.407+0.593 )\n# - option 34-> LB=0.4? work ( 8,9 ) + weights ( 0.410+0.590 )\n\n# next option:\n# - option 16-> LB=0.4? work ( 7,8,9 ) + weights ( 0.25+0.20+0.55 )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:45.826029Z","iopub.execute_input":"2024-10-07T18:59:45.826767Z","iopub.status.idle":"2024-10-07T18:59:45.834983Z","shell.execute_reply.started":"2024-10-07T18:59:45.826722Z","shell.execute_reply":"2024-10-07T18:59:45.834103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if LAUNCH_VARIANT == 'option 27':\n sm7 = pd.read_csv('submission_7.csv')\n sm9 = pd.read_csv('submission_9.csv')\n display(sm7,sm9)\n sms = pd.merge(sm7,sm9, on=['row_id'])\n sms['normal_mild'] = sms['normal_mild_x'] *0.407 + 0.593* sms['normal_mild_y']\n sms['moderate'] = sms['moderate_x'] *0.407 + 0.593* sms['moderate_y']\n sms['severe'] = sms['severe_x'] *0.407 + 0.593* sms['severe_y']","metadata":{"_kg_hide-input":false,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2024-10-07T18:59:45.836023Z","iopub.execute_input":"2024-10-07T18:59:45.836355Z","iopub.status.idle":"2024-10-07T18:59:45.878129Z","shell.execute_reply.started":"2024-10-07T18:59:45.836323Z","shell.execute_reply":"2024-10-07T18:59:45.877058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Applying the aggressive modification to 'normal_mild' in the 'sms' DataFrame\nsms['normal_mild'] = sms['normal_mild'].apply(lambda x: max(x - 0.1, 0.01) if 0.1 <= x <= 0.2 else (0.01 if x < 0.1 else x))","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.879428Z","iopub.execute_input":"2024-10-07T18:59:45.879854Z","iopub.status.idle":"2024-10-07T18:59:45.885696Z","shell.execute_reply.started":"2024-10-07T18:59:45.879808Z","shell.execute_reply":"2024-10-07T18:59:45.884686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set 'severe' values under 0.1 to 0.01\nsms.loc[sms['severe'] < 0.1, 'severe'] = 0.01","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.887031Z","iopub.execute_input":"2024-10-07T18:59:45.887400Z","iopub.status.idle":"2024-10-07T18:59:45.898620Z","shell.execute_reply.started":"2024-10-07T18:59:45.887364Z","shell.execute_reply":"2024-10-07T18:59:45.897670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalization to ensure the probabilities sum to 1\ntotal = sms['normal_mild'] + sms['moderate'] + sms['severe']\nsms['normal_mild'] /= total\nsms['moderate'] /= total\nsms['severe'] /= total","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.899716Z","iopub.execute_input":"2024-10-07T18:59:45.900096Z","iopub.status.idle":"2024-10-07T18:59:45.909729Z","shell.execute_reply.started":"2024-10-07T18:59:45.900048Z","shell.execute_reply":"2024-10-07T18:59:45.908880Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Enhance 'severe' probabilities greater than 0.5\nmask_severe = sms['severe'] > 0.5\nincrease_amount_severe = 0.3 * sms['severe'][mask_severe]\nsms.loc[mask_severe, 'severe'] += increase_amount_severe\nsms.loc[mask_severe, 'normal_mild'] -= increase_amount_severe * 0.66 # 66% from 'normal_mild'\nsms.loc[mask_severe, 'moderate'] -= increase_amount_severe * 0.34 # 34% from 'moderate'","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.910850Z","iopub.execute_input":"2024-10-07T18:59:45.911234Z","iopub.status.idle":"2024-10-07T18:59:45.925285Z","shell.execute_reply.started":"2024-10-07T18:59:45.911187Z","shell.execute_reply":"2024-10-07T18:59:45.924428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalization to ensure the probabilities sum to 1\ntotal = sms['normal_mild'] + sms['moderate'] + sms['severe']\nsms['normal_mild'] /= total\nsms['moderate'] /= total\nsms['severe'] /= total","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.926821Z","iopub.execute_input":"2024-10-07T18:59:45.927159Z","iopub.status.idle":"2024-10-07T18:59:45.935796Z","shell.execute_reply.started":"2024-10-07T18:59:45.927126Z","shell.execute_reply":"2024-10-07T18:59:45.934952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Enhance 'moderate' probabilities when >0.5 and both others <0.3\nmask_moderate = (sms['moderate'] > 0.5) & (sms['normal_mild'] < 0.3) & (sms['severe'] < 0.3)\nincrease_amount_moderate = 0.3 * sms['moderate'][mask_moderate]\nsms.loc[mask_moderate, 'moderate'] += increase_amount_moderate\nsms.loc[mask_moderate, 'normal_mild'] -= increase_amount_moderate * 0.66 # 66% from 'normal_mild'\nsms.loc[mask_moderate, 'severe'] -= increase_amount_moderate * 0.34 # 34% from 'severe'","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.937146Z","iopub.execute_input":"2024-10-07T18:59:45.938208Z","iopub.status.idle":"2024-10-07T18:59:45.952658Z","shell.execute_reply.started":"2024-10-07T18:59:45.938154Z","shell.execute_reply":"2024-10-07T18:59:45.951772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Replace any negative values that might have occurred due to adjustments with 0.01\nsms.loc[sms['normal_mild'] < 0.01, 'normal_mild'] = 0.01\nsms.loc[sms['moderate'] < 0.01, 'moderate'] = 0.01\nsms.loc[sms['severe'] < 0.01, 'severe'] = 0.01","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.953856Z","iopub.execute_input":"2024-10-07T18:59:45.954267Z","iopub.status.idle":"2024-10-07T18:59:45.964206Z","shell.execute_reply.started":"2024-10-07T18:59:45.954223Z","shell.execute_reply":"2024-10-07T18:59:45.963402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalization to ensure the probabilities sum to 1\ntotal = sms['normal_mild'] + sms['moderate'] + sms['severe']\nsms['normal_mild'] /= total\nsms['moderate'] /= total\nsms['severe'] /= total","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.965200Z","iopub.execute_input":"2024-10-07T18:59:45.965536Z","iopub.status.idle":"2024-10-07T18:59:45.978830Z","shell.execute_reply.started":"2024-10-07T18:59:45.965503Z","shell.execute_reply":"2024-10-07T18:59:45.977758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Find the minimum value greater than 0.7 across the specific columns as basedline\nthreshold = sms[['normal_mild', 'moderate', 'severe']].apply(lambda x: x[x > 0.7]).min().min()","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.979918Z","iopub.execute_input":"2024-10-07T18:59:45.980243Z","iopub.status.idle":"2024-10-07T18:59:45.993332Z","shell.execute_reply.started":"2024-10-07T18:59:45.980211Z","shell.execute_reply":"2024-10-07T18:59:45.992536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(type(threshold))","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:45.994684Z","iopub.execute_input":"2024-10-07T18:59:45.995038Z","iopub.status.idle":"2024-10-07T18:59:46.003071Z","shell.execute_reply.started":"2024-10-07T18:59:45.994996Z","shell.execute_reply":"2024-10-07T18:59:46.002138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sms.dtypes","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:46.004215Z","iopub.execute_input":"2024-10-07T18:59:46.004612Z","iopub.status.idle":"2024-10-07T18:59:46.016009Z","shell.execute_reply.started":"2024-10-07T18:59:46.004569Z","shell.execute_reply":"2024-10-07T18:59:46.015154Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\n\n# Calculate the mask for the top probability being under 0.75\nmask_top_under_07 = sms[['normal_mild', 'moderate', 'severe']].max(axis=1) < 0.75\n\n# Calculate the boost amounts and convert to Series\nboost_amount = (threshold - sms[['normal_mild', 'moderate', 'severe']].max(axis=1)) * 0.6\nboost_amount = pd.Series(np.where(mask_top_under_07, boost_amount, 0), index=sms.index)\n\n# Apply boosts to the top probabilities\nfor col in ['normal_mild', 'moderate', 'severe']:\n max_col = sms[['normal_mild', 'moderate', 'severe']].idxmax(axis=1)\n sms.loc[max_col == col, col] += boost_amount\n\n# Calculate secondary boosts when the second highest probability is within 90% of the top\nfor col in ['normal_mild', 'moderate', 'severe']:\n # Calculate second highest by sorting the columns and picking the second last item\n second_highest = sms[['normal_mild', 'moderate', 'severe']].apply(lambda x: x.sort_values().iloc[-2], axis=1)\n max_value = sms[['normal_mild', 'moderate', 'severe']].max(axis=1)\n mask_second_close_to_top = (second_highest / max_value > 0.9) & mask_top_under_07\n sms.loc[mask_second_close_to_top & (sms[col] == second_highest), col] += boost_amount * 0.66","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:46.017411Z","iopub.execute_input":"2024-10-07T18:59:46.017819Z","iopub.status.idle":"2024-10-07T18:59:46.062328Z","shell.execute_reply.started":"2024-10-07T18:59:46.017767Z","shell.execute_reply":"2024-10-07T18:59:46.061601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalization to ensure the probabilities sum to 1\ntotal = sms['normal_mild'] + sms['moderate'] + sms['severe']\nsms['normal_mild'] /= total\nsms['moderate'] /= total\nsms['severe'] /= total","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:46.063407Z","iopub.execute_input":"2024-10-07T18:59:46.063704Z","iopub.status.idle":"2024-10-07T18:59:46.069791Z","shell.execute_reply.started":"2024-10-07T18:59:46.063672Z","shell.execute_reply":"2024-10-07T18:59:46.068860Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # One more iteration (extreme agreesive)\n# # Calculate the mask for the top probability being under 0.75\n# mask_top_under_07 = sms[['normal_mild', 'moderate', 'severe']].max(axis=1) < 1\n\n# # Calculate the boost amounts and convert to Series\n# boost_amount = (threshold - sms[['normal_mild', 'moderate', 'severe']].max(axis=1)) * 2\n# boost_amount = pd.Series(np.where(mask_top_under_07, boost_amount, 0), index=sms.index)\n\n# # Apply boosts to the top probabilities\n# for col in ['normal_mild', 'moderate', 'severe']:\n# max_col = sms[['normal_mild', 'moderate', 'severe']].idxmax(axis=1)\n# sms.loc[max_col == col, col] += boost_amount\n\n# # Calculate secondary boosts when the second highest probability is within 90% of the top\n# for col in ['normal_mild', 'moderate', 'severe']:\n# # Calculate second highest by sorting the columns and picking the second last item\n# second_highest = sms[['normal_mild', 'moderate', 'severe']].apply(lambda x: x.sort_values().iloc[-2], axis=1)\n# max_value = sms[['normal_mild', 'moderate', 'severe']].max(axis=1)\n# mask_second_close_to_top = (second_highest / max_value > 0.9) & mask_top_under_07\n# sms.loc[mask_second_close_to_top & (sms[col] == second_highest), col] += boost_amount * 0.66","metadata":{"execution":{"iopub.status.busy":"2024-10-07T18:59:46.070840Z","iopub.execute_input":"2024-10-07T18:59:46.071109Z","iopub.status.idle":"2024-10-07T18:59:46.084391Z","shell.execute_reply.started":"2024-10-07T18:59:46.071079Z","shell.execute_reply":"2024-10-07T18:59:46.083555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# One more iteration (adjusted to boost only when difference > 0.2)\n# Calculate the top two probabilities and their difference\nprobabilities = sms[['normal_mild', 'moderate', 'severe']]\n\n# Find the top two probabilities for each row\ntop_two_probs = probabilities.apply(lambda row: row.nlargest(2).values, axis=1)\n\n# Extract top and second highest probabilities\ntop_probs = top_two_probs.apply(lambda x: x[0])\nsecond_probs = top_two_probs.apply(lambda x: x[1])\n\n# Calculate the difference between the top and second highest probabilities\ndiff_top_second = top_probs - second_probs\n\n# Create a mask where the difference is greater than 0.25\nmask_diff_gt_02 = diff_top_second > 0.20\n\n# Apply boosts only to rows where the difference is greater than 0.25\nboost_amount = 0.8\n\n# Create a Series of boost amounts, where boost_amount is applied only to rows satisfying the mask\nboost_series = pd.Series(np.where(mask_diff_gt_02, boost_amount, 0), index=sms.index)\n\n# Identify the column with the maximum probability for each row\nmax_col = probabilities.idxmax(axis=1)\n\n# Apply boosts to the top probabilities where the mask is True\nfor col in ['normal_mild', 'moderate', 'severe']:\n mask = (max_col == col) & mask_diff_gt_02\n sms.loc[mask, col] += boost_series[mask]\n","metadata":{"execution":{"iopub.status.busy":"2024-10-07T19:18:22.199457Z","iopub.execute_input":"2024-10-07T19:18:22.199886Z","iopub.status.idle":"2024-10-07T19:18:22.241879Z","shell.execute_reply.started":"2024-10-07T19:18:22.199848Z","shell.execute_reply":"2024-10-07T19:18:22.240751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalization to ensure the probabilities sum to 1\ntotal = sms['normal_mild'] + sms['moderate'] + sms['severe']\nsms['normal_mild'] /= total\nsms['moderate'] /= total\nsms['severe'] /= total","metadata":{"execution":{"iopub.status.busy":"2024-10-07T19:12:57.709412Z","iopub.execute_input":"2024-10-07T19:12:57.710176Z","iopub.status.idle":"2024-10-07T19:12:57.717734Z","shell.execute_reply.started":"2024-10-07T19:12:57.710134Z","shell.execute_reply":"2024-10-07T19:12:57.716659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Calculate the top two probabilities and the minimum probability for each row\ntop_two = sms[['normal_mild', 'moderate', 'severe']].apply(lambda x: x.nlargest(2).values, axis=1)\nmin_prob = sms[['normal_mild', 'moderate', 'severe']].idxmin(axis=1)\n\n# Calculate the difference between the top and the second highest probabilities\ndiff_top_second = top_two.apply(lambda x: x[0] - x[1])\n\n# Create a mask where the difference is less than 0.1\nmask_diff_lt_01 = diff_top_second < 0.1\n\n# Set the lowest probability to 0.1 where the mask condition is met\nsms.loc[mask_diff_lt_01, min_prob[mask_diff_lt_01]] = 0.01","metadata":{"execution":{"iopub.status.busy":"2024-10-07T19:19:03.585349Z","iopub.execute_input":"2024-10-07T19:19:03.585758Z","iopub.status.idle":"2024-10-07T19:19:03.617149Z","shell.execute_reply.started":"2024-10-07T19:19:03.585718Z","shell.execute_reply":"2024-10-07T19:19:03.616118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalization to ensure the probabilities sum to 1\ntotal = sms['normal_mild'] + sms['moderate'] + sms['severe']\nsms['normal_mild'] /= total\nsms['moderate'] /= total\nsms['severe'] /= total","metadata":{"execution":{"iopub.status.busy":"2024-10-07T19:19:06.677912Z","iopub.execute_input":"2024-10-07T19:19:06.678293Z","iopub.status.idle":"2024-10-07T19:19:06.686796Z","shell.execute_reply.started":"2024-10-07T19:19:06.678257Z","shell.execute_reply":"2024-10-07T19:19:06.685696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = sms[['row_id','normal_mild','moderate','severe']]\nsub.to_csv('submission.csv', index=False, float_format='%.7f')\ndisplay(sub)","metadata":{"execution":{"iopub.status.busy":"2024-10-07T19:19:08.477830Z","iopub.execute_input":"2024-10-07T19:19:08.478727Z","iopub.status.idle":"2024-10-07T19:19:08.496410Z","shell.execute_reply.started":"2024-10-07T19:19:08.478685Z","shell.execute_reply":"2024-10-07T19:19:08.495390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# arhiv\n\n# elif LAUNCH_VARIANT == 'option 31':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.415 + 0.585* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.415 + 0.585* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.415 + 0.585* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 32':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.410 + 0.590* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.410 + 0.590* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.410 + 0.590* sms['severe_y']\n \n# elif LAUNCH_VARIANT == 'option 33':\n# sm8 = pd.read_csv('submission_8.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm8,sm9)\n# sms = pd.merge(sm8,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.407 + 0.593* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.407 + 0.593* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.407 + 0.593* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 34':\n# sm8 = pd.read_csv('submission_8.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm8,sm9)\n# sms = pd.merge(sm8,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.410 + 0.590* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.410 + 0.590* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.410 + 0.590* sms['severe_y']\n\n# if LAUNCH_VARIANT == 'option 29':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.402 + 0.598* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.402 + 0.598* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.402 + 0.598* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 30':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.405 + 0.595* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.405 + 0.595* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.405 + 0.595* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 28':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.395 + 0.605* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.395 + 0.605* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.395 + 0.605* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 27':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.407 + 0.593* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.407 + 0.593* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.407 + 0.593* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 26':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.377 + 0.523* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.377 + 0.523* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.377 + 0.523* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 25':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.402 + 0.598* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.402 + 0.598* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.402 + 0.598* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 24':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.427 + 0.573* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.427 + 0.573* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.427 + 0.573* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 23':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.473 + 0.527* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.473 + 0.527* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.473 + 0.527* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 22':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.58 + 0.42* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.58 + 0.42* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.58 + 0.42* sms['severe_y']\n \n# elif LAUNCH_VARIANT == 'option 21':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.55 + 0.45* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.55 + 0.45* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.55 + 0.45* sms['severe_y']\n\n# if LAUNCH_VARIANT == 'option 20':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.50 + 0.50* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.50 + 0.50* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.50 + 0.50* sms['severe_y']\n\n# if LAUNCH_VARIANT == 'option 18':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.33 + 0.67* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.33 + 0.67* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.33 + 0.67* sms['severe_y'] \n\n# if LAUNCH_VARIANT == 'option 12':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.20 + 0.80* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.20 + 0.80* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.20 + 0.80* sms['severe_y']\n\n# elif LAUNCH_VARIANT == 'option 17':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm9)\n# sms = pd.merge(sm7,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.257 + 0.743* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.257 + 0.743* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.257 + 0.743* sms['severe_y'] \n \n# elif LAUNCH_VARIANT == 'option 15':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm8 = pd.read_csv('submission_8.csv')\n# display(sm7,sm8)\n# sms = pd.merge(sm7,sm8, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.70 + 0.30* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.70 + 0.30* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.70 + 0.30* sms['severe_y']\n \n# elif LAUNCH_VARIANT == 'option 16':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm8 = pd.read_csv('submission_8.csv')\n# sm9 = pd.read_csv('submission_9.csv')\n# display(sm7,sm8,sm9)\n# sms = pd.merge(sm7,sm8, on=['row_id'])\n# sms = pd.merge(sms,sm9, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.35 + 0.20* sms['normal_mild_y'] + 0.45* sms['normal_mild']\n# sms['moderate'] = sms['moderate_x'] *0.35 + 0.20* sms['moderate_y'] + 0.45* sms['moderate']\n# sms['severe'] = sms['severe_x'] *0.35 + 0.20* sms['severe_y'] + 0.45* sms['severe']\n\n# elif LAUNCH_VARIANT == 'option 13':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm8 = pd.read_csv('submission_8.csv')\n# display(sm7,sm8)\n# sms = pd.merge(sm7,sm8, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.95 + 0.05* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.95 + 0.05* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.95 + 0.05* sms['severe_y'] \n \n# elif LAUNCH_VARIANT == 'option 14':\n# sm7 = pd.read_csv('submission_7.csv')\n# sm8 = pd.read_csv('submission_8.csv')\n# display(sm7,sm8)\n# sms = pd.merge(sm7,sm8, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.85 + 0.15* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.85 + 0.15* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.85 + 0.15* sms['severe_y']\n \n# elif LAUNCH_VARIANT == 'option 10':\n# sm6 = pd.read_csv('submission_6.csv')\n# sm7 = pd.read_csv('submission_7.csv')\n# display(sm6,sm7)\n# sms = pd.merge(sm6,sm7, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.005 + 0.995* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.005 + 0.995* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.005 + 0.995* sms['severe_y']\n \n# elif LAUNCH_VARIANT == 'option 11':\n# sm5 = pd.read_csv('submission_5.csv')\n# sm6 = pd.read_csv('submission_6.csv')\n# sm7 = pd.read_csv('submission_7.csv')\n# display(sm5,sm6,sm7)\n# sms = pd.merge(sm5,sm6, on=['row_id'])\n# sms = pd.merge(sm6,sm7, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.0005 + 0.0025* sms['normal_mild_y'] + 0.997* sms['normal_mild']\n# sms['moderate'] = sms['moderate_x'] *0.0005 + 0.0025* sms['moderate_y'] + 0.997* sms['moderate']\n# sms['severe'] = sms['severe_x'] *0.0005 + 0.0025* sms['severe_y'] + 0.997* sms['severe']\n \n# elif LAUNCH_VARIANT == 'option 8':\n# sm5 = pd.read_csv('submission_5.csv')\n# sm6 = pd.read_csv('submission_6.csv')\n# display(sm5,sm6)\n# sms = pd.merge(sm5,sm6, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.90 + 0.10* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.90 + 0.10* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.90 + 0.10* sms['severe_y']\n \n# elif LAUNCH_VARIANT == 'option 7':\n# sm5 = pd.read_csv('submission_5.csv')\n# sm6 = pd.read_csv('submission_6.csv')\n# display(sm5,sm6)\n# sms = pd.merge(sm5,sm6, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.50 + 0.50* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.50 + 0.50* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.50 + 0.50* sms['severe_y']\n \n# if LAUNCH_VARIANT == 'option 1':\n# sm1 = pd.read_csv('submission_1.csv')\n# sm2 = pd.read_csv('submission_2.csv')\n# display(sm1,sm2)\n# sms = pd.merge(sm1,sm2, on=['row_id'])\n# sms['normal_mild'] = (sms['normal_mild_x'] + sms['normal_mild_y']) / 2\n# sms['moderate'] = (sms['moderate_x'] + sms['moderate_y']) / 2\n# sms['severe'] = (sms['severe_x'] + sms['severe_y']) / 2\n\n# elif LAUNCH_VARIANT == 'option 2':\n# sm2 = pd.read_csv('submission_2.csv')\n# sm3 = pd.read_csv('submission_3.csv')\n# display(sm2,sm3)\n# sms = pd.merge(sm2,sm3, on=['row_id'])\n# sms['normal_mild'] = (sms['normal_mild_x'] + sms['normal_mild_y']) / 2\n# sms['moderate'] = (sms['moderate_x'] + sms['moderate_y']) / 2\n# sms['severe'] = (sms['severe_x'] + sms['severe_y']) / 2\n# # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\n\n# elif LAUNCH_VARIANT == 'option 3':\n# sm3 = pd.read_csv('submission_3.csv')\n# sm4 = pd.read_csv('submission_4.csv')\n# display(sm3,sm4)\n# sms = pd.merge(sm3,sm4, on=['row_id'])\n# sms['normal_mild'] = (sms['normal_mild_x'] + sms['normal_mild_y']) / 2\n# sms['moderate'] = (sms['moderate_x'] + sms['moderate_y']) / 2\n# sms['severe'] = (sms['severe_x'] + sms['severe_y']) / 2\n# # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\n\n# elif LAUNCH_VARIANT == 'option 4':\n# sm1 = pd.read_csv('submission_1.csv')\n# sm2 = pd.read_csv('submission_2.csv')\n# sm4 = pd.read_csv('submission_4.csv')\n# display(sm1,sm2,sm4)\n# sms = pd.merge(sm1,sm2, on=['row_id'])\n# sms = pd.merge(sms,sm4, on=['row_id'])\n# sms['normal_mild'] = (sms['normal_mild_x'] + sms['normal_mild_y'] + sms['normal_mild']) / 3\n# sms['moderate'] = (sms['moderate_x'] + sms['moderate_y']) + sms['moderate'] / 3\n# sms['severe'] = (sms['severe_x'] + sms['severe_y']) + sms['severe'] / 3\n# # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\n \n# elif LAUNCH_VARIANT == 'option 5':\n# sm2 = pd.read_csv('submission_2.csv')\n# sm3 = pd.read_csv('submission_3.csv')\n# display(sm2,sm3)\n# sms = pd.merge(sm2,sm3, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.45 + 0.55* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.45 + 0.55* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.45 + 0.55* sms['severe_y']\n# # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\n \n# elif LAUNCH_VARIANT == 'option 6':\n# sm2 = pd.read_csv('submission_2.csv')\n# sm3 = pd.read_csv('submission_3.csv')\n# display(sm2,sm3)\n# sms = pd.merge(sm2,sm3, on=['row_id'])\n# sms['normal_mild'] = sms['normal_mild_x'] *0.55 + 0.45* sms['normal_mild_y']\n# sms['moderate'] = sms['moderate_x'] *0.55 + 0.45* sms['moderate_y']\n# sms['severe'] = sms['severe_x'] *0.55 + 0.45* sms['severe_y']\n# # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-07T18:59:46.131917Z","iopub.execute_input":"2024-10-07T18:59:46.132260Z","iopub.status.idle":"2024-10-07T18:59:46.149269Z","shell.execute_reply.started":"2024-10-07T18:59:46.132228Z","shell.execute_reply":"2024-10-07T18:59:46.148467Z"},"trusted":true},"execution_count":null,"outputs":[]}]}