{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

RSNA-MICCAI Brain Tumor Radiogenomic Classification

\n

Predict the status of a genetic biomarker important for brain cancer treatment

\n\n \n\n

About RSNA

\n\n\n

The Radiological Society of North America ( RSNA ) is an international society of radiologists, medical physicists and other medical professionals with more than 54,000 members across the globe. Based in Oak Brook, Illinois, RSNA was established in 1915.

\n \n

RSNA hosts one of the world's largest annual medical meetings, a week-long event which starts the last Sunday of November in Chicago. The society also publishes two peer-reviewed journals: Radiology, offering radiology research and reviews, and RadioGraphics, dedicated to continuing education in radiology.

\n \n

The Society also develops and offers informatics-based software solutions in support of a universal electronic health record, sponsors research to advance quantitative imaging biomarkers, and conducts outreach to enhance education in developing nations. Through its Research and Education Foundation, RSNA provides millions of dollars in funding to investigators.

","metadata":{}},{"cell_type":"code","source":"#------------------------------------------------\n##install gdcm for DICOM image data preprocessing\n#------------------------------------------------\n\n!conda install -c conda-forge gdcm -y","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-14T17:50:51.129104Z","iopub.execute_input":"2021-07-14T17:50:51.129588Z","iopub.status.idle":"2021-07-14T17:52:01.022170Z","shell.execute_reply.started":"2021-07-14T17:50:51.129485Z","shell.execute_reply":"2021-07-14T17:52:01.020991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#---------------------------\n##import necessary libraries\n#---------------------------\n\nimport pandas as pd\nfrom pandas_profiling import ProfileReport\nimport numpy as np\n\nimport cv2\nimport matplotlib.pyplot as plt\nfrom matplotlib import rc\nfrom matplotlib.animation import FuncAnimation\nimport matplotlib.animation as animation\nimport PIL\nfrom PIL import Image\n\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n\nimport warnings\nimport os\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:52:01.028357Z","iopub.execute_input":"2021-07-14T17:52:01.028730Z","iopub.status.idle":"2021-07-14T17:52:03.848653Z","shell.execute_reply.started":"2021-07-14T17:52:01.028695Z","shell.execute_reply":"2021-07-14T17:52:03.847485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TRAIN_PATH = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train\"\nTEST_PATH = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/test\"\n\nHEIGHT,WIDTH = 512,512\n\n#----------------------------\n##Function to obtain metadata\n#----------------------------\ndef get_metadata(mainpath):\n cases = os.listdir(mainpath)\n flair,t1w,t1wce,t2w = [],[],[],[]\n for case in cases:\n casepath = os.path.join(mainpath,case)\n flairpath = os.path.join(casepath,\"FLAIR\")\n t1wpath = os.path.join(casepath,\"T1w\")\n t1wcepath = os.path.join(casepath,\"T1wCE\")\n t2wpath = os.path.join(casepath,\"T2w\")\n \n flair.append(len(os.listdir(flairpath)))\n t1w.append(len(os.listdir(t1wpath)))\n t1wce.append(len(os.listdir(t1wcepath)))\n t2w.append(len(os.listdir(t2wpath)))\n df = pd.DataFrame.from_dict({\n \"CASE ID\" : cases,\n \"FLAIR\" : flair,\n \"T1w\" : t1w,\n \"T1wCE\" : t1wce,\n \"T2w\" : t2w\n })\n return df","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:52:03.859365Z","iopub.execute_input":"2021-07-14T17:52:03.859785Z","iopub.status.idle":"2021-07-14T17:52:03.871713Z","shell.execute_reply.started":"2021-07-14T17:52:03.859738Z","shell.execute_reply":"2021-07-14T17:52:03.870512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Information about Training Images

\n\n## Dataset Structure\n\n- train / test\n - case [5 digited number]\n - FLAIR\n - Image-1.dicom\n - ....\n - T1w\n - Image-1.dicom\n - ....\n - T1wCE\n - Image-1.dicom\n - ....\n - T2w\n - Image-1.dicom\n - ....\n - .... \n- train_labels.csv\n- sample_submission.csv","metadata":{}},{"cell_type":"markdown","source":"

Pandas Profiling

\n\n\n\n

Pandas profiling is an open source Python module with which we can quickly do an exploratory data analysis with just a few lines of code. Besides, if this is not enough to convince us to use this tool, it also generates interactive reports in web format that can be presented to any person, even if they don’t know programming.

\n Know more about Pandas Profiling ","metadata":{}},{"cell_type":"code","source":"#----------------------\n##Get Training Metadata\n#----------------------\n\ntrain_df = get_metadata(TRAIN_PATH)\ntrain_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:52:03.873328Z","iopub.execute_input":"2021-07-14T17:52:03.873730Z","iopub.status.idle":"2021-07-14T17:53:03.604941Z","shell.execute_reply.started":"2021-07-14T17:52:03.873681Z","shell.execute_reply":"2021-07-14T17:53:03.604072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#-------------\n##Train report\n#-------------\n\ntrain_report = ProfileReport(train_df,title=\"Metadata of Training images\")\ntrain_report.to_file(\"./train_metadata.html\")\ntrain_report","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:53:03.606348Z","iopub.execute_input":"2021-07-14T17:53:03.606666Z","iopub.status.idle":"2021-07-14T17:54:14.674545Z","shell.execute_reply.started":"2021-07-14T17:53:03.606636Z","shell.execute_reply":"2021-07-14T17:54:14.673480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Information about Training Labels

","metadata":{}},{"cell_type":"code","source":"label_df = pd.read_csv(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv\")\nlabel_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:14.676836Z","iopub.execute_input":"2021-07-14T17:54:14.677154Z","iopub.status.idle":"2021-07-14T17:54:14.700536Z","shell.execute_reply.started":"2021-07-14T17:54:14.677123Z","shell.execute_reply":"2021-07-14T17:54:14.699510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#--------------------\n##Train Labels report\n#--------------------\n\nlabels_report = ProfileReport(label_df,title=\"Training labels\")\nlabels_report.to_file(\"./train_labels.html\")\nlabels_report","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:14.702351Z","iopub.execute_input":"2021-07-14T17:54:14.702724Z","iopub.status.idle":"2021-07-14T17:54:17.823986Z","shell.execute_reply.started":"2021-07-14T17:54:14.702689Z","shell.execute_reply":"2021-07-14T17:54:17.822924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Information about Test Images

","metadata":{}},{"cell_type":"code","source":"#------------------\n##Get Test Metadata\n#------------------\n\ntest_df = get_metadata(TEST_PATH)\ntest_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:17.825566Z","iopub.execute_input":"2021-07-14T17:54:17.825967Z","iopub.status.idle":"2021-07-14T17:54:26.494625Z","shell.execute_reply.started":"2021-07-14T17:54:17.825914Z","shell.execute_reply":"2021-07-14T17:54:26.493585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#------------\n##Test report\n#------------\n\ntest_report = ProfileReport(test_df,title=\"Metadata of Test images\")\ntest_report.to_file(\"./test_metadata.html\")\ntest_report","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:26.496141Z","iopub.execute_input":"2021-07-14T17:54:26.496740Z","iopub.status.idle":"2021-07-14T17:54:41.838753Z","shell.execute_reply.started":"2021-07-14T17:54:26.496692Z","shell.execute_reply":"2021-07-14T17:54:41.837739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

DICOM to Array

","metadata":{}},{"cell_type":"markdown","source":"

What is DICOM format

\n

Digital Imaging and Communications in Medicine (DICOM) is the standard for the communication and management of medical imaging information and related data. DICOM is most commonly used for storing and transmitting medical images enabling the integration of medical imaging devices such as scanners, servers, workstations, printers, network hardware.

\n\n Read more about DICOM here\n\n

Why is DICOM used

\n\n

Following are some of applications of using DICOM format in medical imaging

\n\n- Collaboration With Existing IT Systems\n- High-Performance Review\n- Complete Scanning and Image Reviewing\n\n Read more about DICOM applications here","metadata":{}},{"cell_type":"code","source":"##---------------------------------------\n#DICOM to pixel array converting function\n##---------------------------------------\n\n# Ref : https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\ndef dicom2arr(path, voi_lut = True, fix_monochrome = True):\n dicom = pydicom.read_file(path)\n \n if voi_lut:\n arr = apply_voi_lut(dicom.pixel_array, dicom)\n else:\n arr = dicom.pixel_array\n \n if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n arr = np.amax(arr) - arr\n \n arr = arr - np.min(arr)\n arr = arr / np.max(arr)\n arr = (arr * 255).astype(np.uint8)\n \n arr = cv2.resize(arr,(HEIGHT,WIDTH))\n return arr","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:41.840263Z","iopub.execute_input":"2021-07-14T17:54:41.840651Z","iopub.status.idle":"2021-07-14T17:54:41.849970Z","shell.execute_reply.started":"2021-07-14T17:54:41.840612Z","shell.execute_reply":"2021-07-14T17:54:41.848891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

MRIs Visualization

","metadata":{}},{"cell_type":"markdown","source":"

What is MRI

\n\n

Medical Resonance Imaging (MRI) is a medical imaging technique used in radiology to form pictures of the anatomy and the physiological processes of the body. MRI scanners use strong magnetic fields, magnetic field gradients, and radio waves to generate images of the organs in the body. MRI does not involve X-rays or the use of ionizing radiation, which distinguishes it from CT and PET scans. MRI is a medical application of Nuclear Magnetic Resonance (NMR) which can also be used for imaging in other NMR applications, such as NMR spectroscopy.

\n\n

Types of MRI Scans

\n\n- **Fluid Attenuated Inversion Recovery (FLAIR)**\n\n Fluid attenuated inversion recovery (FLAIR) is a special inversion recovery sequence with a long inversion time. This removes signal from the cerebrospinal fluid in the resulting images 1. Brain tissue on FLAIR images appears similar to T2 weighted images with grey matter brighter than white matter but CSF is dark instead of bright.\n \n \n- **T1-weighted (T1)**\n T1-weighted images are produced by using short TE and TR times. The contrast and brightness of the image are predominately determined by T1 properties of tissue.\n - **T1-weighted pre-contrast (T1w)**\n - **T1-weighted post-contrast (T1Gd)**\n \n \n- **T2-weighted (T2)**\n \n T2 weighted image (T2WI) is one of the basic pulse sequences on MRI. The sequence weighting highlights differences on the T2 relaxation time of tissues.\n \n \n \n Read more about MRI Scanning here\n\n For quick learning check this.","metadata":{}},{"cell_type":"code","source":"#----------------------\n##sample images display\n#----------------------\n\nflair = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/FLAIR/Image-109.dcm\"\nt1w = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/T1w/Image-19.dcm\"\nt1wce = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/T1wCE/Image-50.dcm\"\nt2w = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/T2w/Image-180.dcm\"\n\nflair_arr = dicom2arr(flair)\nt1w_arr = dicom2arr(t1w)\nt1wce_arr = dicom2arr(t1wce)\nt2w_arr = dicom2arr(t2w)\n\nimg_ls = [flair_arr,t1w_arr,t1wce_arr,t2w_arr]\ntitle_ls = [\"FLAIR\",\"T1-weighted pre contrast\", \"T1-weighted post contrast\", \"T2-weighted\"]\n\nfig,ax = plt.subplots(nrows=1,ncols=4,figsize=(20,10))\nfor i in range(4):\n ax[i].imshow(img_ls[i],cmap=\"viridis\")\n ax[i].axis(\"off\")\n ax[i].set_title(title_ls[i])\n \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:41.851367Z","iopub.execute_input":"2021-07-14T17:54:41.851719Z","iopub.status.idle":"2021-07-14T17:54:42.352427Z","shell.execute_reply.started":"2021-07-14T17:54:41.851686Z","shell.execute_reply":"2021-07-14T17:54:42.351466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Animation of MRIs

","metadata":{}},{"cell_type":"code","source":"#--------------------------\n##Aniamating FLAIR type MRI\n#--------------------------\nflair = \"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/00000/FLAIR\"\nname_ls = os.listdir(flair)\npath_ls = [os.path.join(flair,i) for i in name_ls]\n\nimg_ls = [dicom2arr(path) for path in path_ls]","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:54:42.353634Z","iopub.execute_input":"2021-07-14T17:54:42.354138Z","iopub.status.idle":"2021-07-14T17:54:49.001796Z","shell.execute_reply.started":"2021-07-14T17:54:42.354098Z","shell.execute_reply":"2021-07-14T17:54:49.000895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rc('animation', html='jshtml') \n\ndef get_animation(img_ls):\n fig = plt.figure(figsize=(8,8))\n im_arr = plt.imshow(img_ls[0],cmap=\"viridis\")\n plt.axis(\"off\")\n plt.title(\"FLAIR\")\n def update(i):\n im_arr.set_array(img_ls[i])\n return [im_arr]\n anim = animation.FuncAnimation(fig,update,frames=len(img_ls),interval=50)\n anim.save(\"./FLAIR.mp4\")\n return anim\n\nget_animation(img_ls)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T17:57:48.407944Z","iopub.execute_input":"2021-07-14T17:57:48.408621Z","iopub.status.idle":"2021-07-14T17:58:47.206344Z","shell.execute_reply.started":"2021-07-14T17:57:48.408577Z","shell.execute_reply":"2021-07-14T17:58:47.204984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n

Do upvote if you found this notebook useful🙂

","metadata":{}}]}