{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Hi everybody!!! This notebook is the work of a couple of weeks with nnUnet software, which was the BRATS'20 winner. I have been working with the BRATS'21 task1 data, and also competition data for brain tumour segmentation. I really expect what could be helpfull for you!**","metadata":{}},{"cell_type":"markdown","source":"- My first approximation was training with nnUnet data my own model, but due to kaggle GPU limits, when I was trying to train 2D, and 3D-FullRes models, I couldn't finish because it took more than 9 hours. Neverless, I used the pretrained models, and here you can see the results:","metadata":{}},{"cell_type":"markdown","source":"# 1.- Installing nnUNET and import libs. ","metadata":{}},{"cell_type":"code","source":"!git clone https://github.com/MIC-DKFZ/nnUNet.git\n!git clone https://github.com/NVIDIA/apex\n!pip install -e ./nnUNet\n!pip install --upgrade git+https://github.com/nanohanno/hiddenlayer.git@bugfix/get_trace_graph#egg=hiddenlayer","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-09-10T12:30:38.562073Z","iopub.execute_input":"2021-09-10T12:30:38.562438Z","iopub.status.idle":"2021-09-10T12:31:16.755091Z","shell.execute_reply.started":"2021-09-10T12:30:38.562356Z","shell.execute_reply":"2021-09-10T12:31:16.753781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"###Libraries and imports\nimport numpy as np\nimport math\nimport random\nimport os\nimport shutil\nimport gzip\nimport shutil\nimport glob\nimport gc\nimport cv2 as cv\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nimport numpy as np\nimport pandas as pd\nimport tarfile\nimport PIL\nimport scipy.misc\nimport skimage\nimport nibabel as nib\nimport matplotlib.pyplot as plt\nimport sys\n\nfrom PIL import Image\nfrom tqdm import tqdm\nfrom tensorflow import keras\nfrom keras.models import Model, load_model\nfrom keras.layers import Input ,BatchNormalization , Activation \nfrom keras.layers.convolutional import Conv2D, UpSampling2D\nfrom keras.layers.pooling import MaxPooling2D\nfrom keras.layers.merge import concatenate\nfrom keras.callbacks import EarlyStopping, ModelCheckpoint\nfrom keras import optimizers \nfrom keras import backend as K\nfrom sklearn.model_selection import train_test_split\nfrom skimage.morphology import ball, disk, dilation, binary_erosion, remove_small_objects, erosion, closing, reconstruction, binary_closing\nfrom skimage.measure import label,regionprops, perimeter\nfrom skimage.morphology import binary_dilation, binary_opening\nfrom skimage.filters import roberts, sobel\nfrom skimage import measure, feature\nfrom skimage.segmentation import clear_border\nfrom skimage import data\nfrom skimage.io import imread\nfrom scipy import ndimage as ndi\nfrom mpl_toolkits.mplot3d.art3d import Poly3DCollection\nfrom glob import glob\nimport numpy as np\nimport nibabel as nib\nimport matplotlib.pyplot as plt\nimport SimpleITK as sitk\nfrom skimage.transform import resize\nimport os\nimport sys \nfrom tqdm import tqdm\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom PIL import Image\nimport pydicom\nfrom os.path import join\nfrom fastai.vision.all import *\nDEVICE = \"GPU\"\n\n","metadata":{"execution":{"iopub.status.busy":"2021-09-10T12:36:41.802418Z","iopub.execute_input":"2021-09-10T12:36:41.802827Z","iopub.status.idle":"2021-09-10T12:36:41.817390Z","shell.execute_reply.started":"2021-09-10T12:36:41.802778Z","shell.execute_reply":"2021-09-10T12:36:41.816095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2.- Download Brats'20 pretrained model and prediction with BRATS'21 dataset","metadata":{}},{"cell_type":"markdown","source":"- Download pretrained Task001_BrainTumour, and setting up the nnUnet environment.","metadata":{}},{"cell_type":"code","source":"\nos.mkdir('./nnUNet_raw_data_base')\nos.mkdir('./nnUNet_raw_data_base/nnUNet_raw_data')\nos.mkdir('./RESULTS_FOLDER')\nos.environ['nnUNet_raw_data_base'] = './nnUNet_raw_data_base/nnUNet_raw_data'\nos.environ['RESULTS_FOLDER'] = './RESULTS_FOLDER'\nos.environ['nnUNet_preprocessed'] = './nnUNet_preprocessed'\n!nnUNet_download_pretrained_model Task001_BrainTumour","metadata":{"execution":{"iopub.status.busy":"2021-09-10T12:31:25.285626Z","iopub.execute_input":"2021-09-10T12:31:25.286089Z","iopub.status.idle":"2021-09-10T12:33:11.198159Z","shell.execute_reply.started":"2021-09-10T12:31:25.286043Z","shell.execute_reply":"2021-09-10T12:33:11.196796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- In order to show how it work for BRATS'21, I will execute one example case.\n\n-- Important: The data as i told you should be structurated as follow, and as you can see in brats2021dataset:\nin the same study, you have to rename the files with 00000, 00001, 00002 and 00003 depending if it is an MRI type FLAIR, T1w, T1wCE of T2 respectively, and it should be in .gz format.","metadata":{}},{"cell_type":"code","source":"!mkdir -p ./RESULTS_FOLDER/nnUNet/2d/image_raw\n!cp ../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/imagesTs/BRATS_188_0000.nii ./RESULTS_FOLDER/nnUNet/2d/image_raw/BRATS_188_0000.nii.gz\n!cp ../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/imagesTs/BRATS_188_0001.nii ./RESULTS_FOLDER/nnUNet/2d/image_raw/BRATS_188_0001.nii.gz\n!cp ../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/imagesTs/BRATS_188_0002.nii ./RESULTS_FOLDER/nnUNet/2d/image_raw/BRATS_188_0002.nii.gz\n!cp ../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/imagesTs/BRATS_188_0003.nii ./RESULTS_FOLDER/nnUNet/2d/image_raw/BRATS_188_0003.nii.gz\n!nnUNet_predict -i ./RESULTS_FOLDER/nnUNet/2d/image_raw -o ./RESULTS_FOLDER/nnUNet/2d/ -t \"001\" -tr nnUNetTrainerV2 -m 2d\n","metadata":{"execution":{"iopub.status.busy":"2021-09-10T11:58:51.341519Z","iopub.execute_input":"2021-09-10T11:58:51.341845Z","iopub.status.idle":"2021-09-10T11:59:31.028558Z","shell.execute_reply.started":"2021-09-10T11:58:51.341808Z","shell.execute_reply":"2021-09-10T11:59:31.027605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls ./RESULTS_FOLDER/nnUNet/2d/image_raw/","metadata":{"execution":{"iopub.status.busy":"2021-09-10T11:59:31.030191Z","iopub.execute_input":"2021-09-10T11:59:31.030555Z","iopub.status.idle":"2021-09-10T11:59:31.735257Z","shell.execute_reply.started":"2021-09-10T11:59:31.030495Z","shell.execute_reply":"2021-09-10T11:59:31.734175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path_raw=\"../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/imagesTs/BRATS_188_0000.nii\"\npath=\"./RESULTS_FOLDER/nnUNet/2d/\"\n\nplt.figure(figsize=(12,6))\nplt.subplot(121)\nflair_nib = nib.load(path_raw)\nflair_nib_array = flair_nib.get_fdata()\nplt.imshow(flair_nib_array[:,:,flair_nib_array.shape[2]//2])\nplt.subplot(122)\nflair_nib = nib.load(path+\"BRATS_188.nii.gz\")\nflair_nib_array = flair_nib.get_fdata()\nplt.imshow(flair_nib_array[:,:,flair_nib_array.shape[2]//2])","metadata":{"execution":{"iopub.status.busy":"2021-09-10T11:59:31.739682Z","iopub.execute_input":"2021-09-10T11:59:31.740359Z","iopub.status.idle":"2021-09-10T11:59:32.180887Z","shell.execute_reply.started":"2021-09-10T11:59:31.740314Z","shell.execute_reply":"2021-09-10T11:59:32.179479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.- Prediction of RSNA-MICCAI challenge code, transform size and format previously, and show the predictions","metadata":{}},{"cell_type":"markdown","source":"- Now that you can see that it works for BRAT'S 21, lets see how to manage with competition data:\n-- You have to move all the data to nii.gz format, and also move to this shape configuration (240,240,155). So, the way that I have used this data, first resize to 240x240, and secondly, if there are enought .dcm images, it is no problem, I can cut in 155, and get the 240x240x155, but it is not the case, you can create empty slices.","metadata":{}},{"cell_type":"code","source":"!mkdir -p ./test/post/\nreader = sitk.ImageSeriesReader()\nreader.LoadPrivateTagsOn()\ntrain_path=\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/\"\nstudi_id=[\"00000\",\"00002\",\"00003\"]\nmri_types = [\"FLAIR\",\"T1w\",\"T1wCE\",\"T2w\"]\npixel_size_h=240\npixel_size_w=240\ninner_count=0\n\ndef dicom2nifti(image_dir, out_dir, save=True):\n    \"given a dicom directory, loads them into single file and can save it as .nii file\"\n    reader = sitk.ImageSeriesReader()\n    reader.LoadPrivateTagsOn()\n    filenamesDICOM = reader.GetGDCMSeriesFileNames(str(image_dir))\n    reader.SetFileNames(filenamesDICOM)\n    img = reader.Execute()\n    img = sitk.Cast(img, sitk.sitkFloat32)\n    \n    if save:\n        sitk.WriteImage(img, f'{out_dir}/{image_dir.parent.name}.nii')\n    else:\n        return img\n\ndef resample_nifti(image_dir, ref_image, fn, save=True):\n    \"resample using a reference image\"\n\n    image = sitk.ReadImage(str(image_dir), sitk.sitkFloat32)\n    \n    initial_transform = sitk.CenteredTransformInitializer(ref_image, \n                                                          image, \n                                                          sitk.Euler3DTransform(), \n                                                          sitk.CenteredTransformInitializerFilter.GEOMETRY)\n\n    resampler = sitk.ResampleImageFilter()\n    resampler.SetReferenceImage(ref_image)\n    resampler.SetInterpolator(sitk.sitkLinear)\n    resampler.SetTransform(initial_transform)\n    resampler.SetOutputSpacing(ref_image.GetSpacing())\n    resampler.SetSize((ref_image.GetSize()))\n    resampler.SetOutputDirection(ref_image.GetDirection())\n    resampler.SetOutputOrigin(ref_image.GetOrigin())\n    resampler.SetDefaultPixelValue(image.GetPixelIDValue())\n    resamped_image = resampler.Execute(image)\n    \n    if save:\n        sitk.WriteImage(resamped_image, fn)\n\n    return resamped_image\n\nref_image = sitk.ReadImage('../input/sri24-dataset/sri24/spgr.nii', sitk.sitkFloat32)\n!mkdir -p ./tmp/T1w\n!mkdir -p ./tmp/T1wCE\n!mkdir -p ./tmp/T2w\n!mkdir -p ./tmp/FLAIR\n\nfor c in tqdm(studi_id):\n    path=join(train_path,c)\n    samples = [Path(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/\"+c)]\n    path_train_t2w, path_train_t1wce,path_train_t1w,path_train_flair = [],[],[],[]\n    for each in samples:\n        path_train_t2w.append(each.ls()[0])\n        path_train_t1wce.append(each.ls()[1])\n        path_train_t1w.append(each.ls()[2])\n        path_train_flair.append(each.ls()[3])\n    for fn in path_train_t1w: dicom2nifti(fn, \"./tmp/T1w/\")\n    for fn in path_train_t1wce: dicom2nifti(fn, \"./tmp/T1wCE/\")\n    for fn in path_train_t1w: dicom2nifti(fn, \"./tmp/T2w/\")\n    for fn in path_train_flair: dicom2nifti(fn, \"./tmp/FLAIR/\")      \n    for b in tqdm(mri_types):\n        path=join(train_path,c)\n        file=[]\n        for each in  [Path(join('./tmp/',b))]:\n            file.append(each.ls()[0])        \n        for fn2 in file:\n            pat_id = str(fn2).split('/')[-1].split('.')[0]\n            if b==\"FLAIR\":\n                final_fn = f\"./test/post/\"+pat_id+\"_0000.nii.gz\"\n            if b==\"T1w\":\n                final_fn = f\"./test/post/\"+pat_id+\"_0001.nii.gz\"\n            if b==\"T1wCE\":\n                final_fn = f\"./test/post/\"+pat_id+\"_0002.nii.gz\"    \n            if b==\"T2w\":\n                final_fn = f\"./test/post/\"+pat_id+\"_0003.nii.gz\"\n            resample_nifti(fn2, ref_image, final_fn, True)\n            os.remove(str(fn2))","metadata":{"execution":{"iopub.status.busy":"2021-09-10T13:12:41.444294Z","iopub.execute_input":"2021-09-10T13:12:41.444780Z","iopub.status.idle":"2021-09-10T13:13:14.360165Z","shell.execute_reply.started":"2021-09-10T13:12:41.444715Z","shell.execute_reply":"2021-09-10T13:13:14.358810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls ./test/post/","metadata":{"execution":{"iopub.status.busy":"2021-09-10T13:13:17.289114Z","iopub.execute_input":"2021-09-10T13:13:17.289513Z","iopub.status.idle":"2021-09-10T13:13:18.133309Z","shell.execute_reply.started":"2021-09-10T13:13:17.289477Z","shell.execute_reply":"2021-09-10T13:13:18.131975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- I do the predictions for 2d and 3d_fullres, and show the result, that it is not so good, and this is the main problem, I don't know if this is registration problem, or other kind of problem.","metadata":{}},{"cell_type":"code","source":"!nnUNet_predict -i ./test/post/ -o ./RESULTS_FOLDER/nnUNet/2d/ -t 001 -tr nnUNetTrainerV2 -m 3d_fullres --disable_tta ","metadata":{"execution":{"iopub.status.busy":"2021-09-10T13:13:21.328778Z","iopub.execute_input":"2021-09-10T13:13:21.329336Z","iopub.status.idle":"2021-09-10T13:15:19.760614Z","shell.execute_reply.started":"2021-09-10T13:13:21.329301Z","shell.execute_reply":"2021-09-10T13:15:19.759107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls ./RESULTS_FOLDER/nnUNet/2d/","metadata":{"execution":{"iopub.status.busy":"2021-09-10T13:07:30.822289Z","iopub.execute_input":"2021-09-10T13:07:30.822896Z","iopub.status.idle":"2021-09-10T13:07:31.773409Z","shell.execute_reply.started":"2021-09-10T13:07:30.822815Z","shell.execute_reply":"2021-09-10T13:07:31.772018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_array(fn):\n    \"opens .nii file and return the array\"\n    img = sitk.ReadImage(str(fn))\n    imgd = sitk.GetArrayFromImage(img)\n    return imgd\n\ndef plot_slice(imgd, sli):\n    \"given an image of shape slices x height x width, plots a slice\"\n    plt.imshow(imgd[sli], cmap='gray')\n    plt.axis('off')\n    \ndef get_array_plot(fn, sli):\n    imgd = get_array(fn)\n    plot_slice(imgd, sli)\n\nget_array_plot(f'./RESULTS_FOLDER/nnUNet/2d/00003.nii.gz', 105)","metadata":{"execution":{"iopub.status.busy":"2021-09-10T13:19:48.192796Z","iopub.execute_input":"2021-09-10T13:19:48.193274Z","iopub.status.idle":"2021-09-10T13:19:48.313531Z","shell.execute_reply.started":"2021-09-10T13:19:48.193240Z","shell.execute_reply":"2021-09-10T13:19:48.312233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_array_plot(f'./test/post/00003_0003.nii.gz', 125)","metadata":{"execution":{"iopub.status.busy":"2021-09-10T13:20:14.490719Z","iopub.execute_input":"2021-09-10T13:20:14.491213Z","iopub.status.idle":"2021-09-10T13:20:14.751558Z","shell.execute_reply.started":"2021-09-10T13:20:14.491148Z","shell.execute_reply":"2021-09-10T13:20:14.750180Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!nnUNet_predict -i ./test/post/ -o ./RESULTS_FOLDER/nnUNet/2d/ -t \"001\" -tr nnUNetTrainerV2 -m 2d","metadata":{"execution":{"iopub.status.busy":"2021-09-09T09:00:00.630308Z","iopub.execute_input":"2021-09-09T09:00:00.630689Z","iopub.status.idle":"2021-09-09T09:00:07.888145Z","shell.execute_reply.started":"2021-09-09T09:00:00.630648Z","shell.execute_reply":"2021-09-09T09:00:07.887014Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4.- How to train your own model with BRATs'21 dataset","metadata":{}},{"cell_type":"markdown","source":"- **If you have interest and home machines in order to train the Brats'21 model, you can use the next code in order to create the specific configurations with nnUnet (https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/dataset_conversion.md):**\n\nI did for you and I share my dataset with you, please if it is hepfull upvote: https://www.kaggle.com/victorfernandezalbor/brats2021dataset","metadata":{}},{"cell_type":"code","source":"path_raw_brain=\"../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/\"\npath_tr=\"imagesTr/\"\npath_ts=\"imagesTs/\"\npath_lbr=\"labelsTr/\"\npath_lbs=\"labelsTs/\"\npath_raw_brain=\"nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/\"\ntask=\"Task101_BrainTumour\"\n\n#Uncomment and try the next code by yourself......\n\n#!ln -rs ../input/brats2021dataset/nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/* ./nnUNet_raw_data_base/nnUNet_raw_data/Task101_BrainTumour/\n\n##The next code generate one json which the nnUnet software use for know the test and train data, in my own dataset \n##I've added the propper one, so maybe it is not usefull for you, but it is necessary.\n## Please feel free if you miss something to ask me.\n\n#sys.path.insert(1,'../input/generatejsonbrats/')\n#sys.path.insert(1,'../input/nnunet-batchgenerators/batchgenerators/')\n#from batchgenerators.utilities.file_and_folder_operations import *\n#from batchgenerators import *\n#import file_and_folder_operations\n#from utils import generate_dataset_json\n#generate_dataset_json(join(path_raw_brain,'dataset.json'), path_raw_brain+path_tr, path_raw_brain+path_ts, (\"flair\", \"t1\", \"t1ce\", \"t2\"),\n#                    labels={\"0\": \"background\", \"1\": \"necrotic tumor core\",\"2\": \"peritumoral edematous/invaded tissue\",\"3\":\"\", \"4\": \"GD-enhancing tumor\"} ,dataset_name=task, license='hands off!')\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Uncoment the next lines for running, first the preprocess, which make some augmentation and proper preprocessing in the images, and then for training the 2d and 3d_fullres configuration, with 4 folds.\n\nMore info, in the web page of the project: https://github.com/MIC-DKFZ/nnUNet#model-training","metadata":{}},{"cell_type":"code","source":"#!nnUNet_plan_and_preprocess -t 101 -tf 1 --verify_dataset_integrity","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#!nnUNet_train 2d nnUNetTrainerV2 101 4 --npz","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#nnUNet_train 3d_fullres nnUNetTrainerV2 TaskXXX_MYTASK FOLD --npz\n#!nnUNet_train 3d_fullres nnUNetTrainerV2 101 4 --npz","metadata":{},"execution_count":null,"outputs":[]}]}