{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<div class='alert alert-info' style='text-align: center'><h1>Brain Tumor Object Detection</h1>\n- yet another MR processing notebook -</div>\n\n#### In this notebook, we'll use YOLOv5 to try and detect tumors in brain MRs.\n#### There are three models (pytorch weights), one for each plane, that work on the T1wCE series only.\n#### They were trained on a small sample (~400 images each) of the RSNA dataset I hand labeled using makesense.ai.\n#### The image plane is checked and the appropriate weights are used in detection/classification.\n#### They perform object detection and classification (positive/negative) simulataneously.\n\n##### Model Dataset -> https://www.kaggle.com/davidbroberts/brain-tumor-yolo-od\n##### Training Notebook -> https://www.kaggle.com/davidbroberts/brain-tumor-yolo-od-train\n##### Training Dataset -> https://www.kaggle.com/davidbroberts/brain-tumor-object-detection-datasets\n\n- You can cross reference the classification accuracy by comparing to the study's MGMT value.\n- I included classification for demonstration purposes only (it's not very accurate .. of course none of the models in this comp are).\n- The OD models find a lot of false positives on the inferior axial images especially. Excluding non-diagnostic (all black, peripheral images etc) slices is important here.\n- I created OD models for the other series types, but I did not include them in this demo. Let me know if you're interested in them.","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-09-30T17:32:47.581383Z","iopub.execute_input":"2021-09-30T17:32:47.582388Z","iopub.status.idle":"2021-09-30T17:32:47.58664Z","shell.execute_reply.started":"2021-09-30T17:32:47.582341Z","shell.execute_reply":"2021-09-30T17:32:47.586051Z"}}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport matplotlib.pyplot as plt\nimport cv2\nimport shutil\nfrom shutil import copyfile\nimport path","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.447769Z","iopub.execute_input":"2021-09-30T18:33:34.448074Z","iopub.status.idle":"2021-09-30T18:33:34.45326Z","shell.execute_reply.started":"2021-09-30T18:33:34.448041Z","shell.execute_reply":"2021-09-30T18:33:34.452475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict_dir = '/kaggle/working/predict'\nlabel_dir = '/kaggle/working/runs/detect/exp/labels'\nimage_dir = '/kaggle/working/runs/detect/exp'","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.609048Z","iopub.execute_input":"2021-09-30T18:33:34.609609Z","iopub.status.idle":"2021-09-30T18:33:34.614141Z","shell.execute_reply.started":"2021-09-30T18:33:34.609566Z","shell.execute_reply":"2021-09-30T18:33:34.61312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Make a dir to export jpgs to\nif not os.path.exists(f'{predict_dir}'):\n    os.mkdir(f'{predict_dir}')","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.620689Z","iopub.execute_input":"2021-09-30T18:33:34.621023Z","iopub.status.idle":"2021-09-30T18:33:34.625938Z","shell.execute_reply.started":"2021-09-30T18:33:34.620982Z","shell.execute_reply":"2021-09-30T18:33:34.624861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Copy YOLO\nif not os.path.exists('/kaggle/working/yolov5'):\n    shutil.copytree('/kaggle/input/yolov5-official-v31-dataset/yolov5', '/kaggle/working/yolov5')","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.627823Z","iopub.execute_input":"2021-09-30T18:33:34.628121Z","iopub.status.idle":"2021-09-30T18:33:34.636646Z","shell.execute_reply.started":"2021-09-30T18:33:34.628089Z","shell.execute_reply":"2021-09-30T18:33:34.635955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This function gives a basic plane from the ImageOrientationPatient tag. It doesn't account for obliqueness. But we don't need to care about it.\n# Will return 'unknown' if the image isn't exactly orthogonal.\ndef get_image_plane(loc):\n    row_x = round(loc[0])\n    row_y = round(loc[1])\n    row_z = round(loc[2])\n    col_x = round(loc[3])\n    col_y = round(loc[4])\n    col_z = round(loc[5])\n    if (row_x, row_y, col_x, col_y) == (1,0,0,0):\n        return \"coronal\"\n    if (row_x, row_y, col_x, col_y) == (0,1,0,0):\n        return \"sagittal\"\n    if (row_x, row_y, col_x, col_y) == (1,0,0,1):\n        return \"axial\"\n    return \"Unknown\"","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.637821Z","iopub.execute_input":"2021-09-30T18:33:34.63806Z","iopub.status.idle":"2021-09-30T18:33:34.647722Z","shell.execute_reply.started":"2021-09-30T18:33:34.638034Z","shell.execute_reply":"2021-09-30T18:33:34.646869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Call yolo detect.py on an image\ndef detect(plane):\n    !python yolov5/detect.py --source {predict_dir} --weights ../input/brain-tumor-yolo-od/{plane}_t1wce_2_class.pt --img 512 --exist-ok --save-txt","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.651056Z","iopub.execute_input":"2021-09-30T18:33:34.651281Z","iopub.status.idle":"2021-09-30T18:33:34.657882Z","shell.execute_reply.started":"2021-09-30T18:33:34.651255Z","shell.execute_reply":"2021-09-30T18:33:34.657352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Delete all images and labels\ndef cleanup():\n    if os.path.exists(label_dir):\n        filelist = [ f for f in os.listdir(label_dir) if f.endswith(\".txt\") ]\n        for f in filelist:\n            os.remove(os.path.join(label_dir, f))\n\n        filelist = [ f for f in os.listdir(image_dir) if f.endswith(\".jpg\") ]\n        for f in filelist:\n            os.remove(os.path.join(image_dir, f))\n            \n    filelist = [ f for f in os.listdir(predict_dir) if f.endswith(\".jpg\") ]\n    for f in filelist:\n        os.remove(os.path.join(predict_dir, f))  ","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.658746Z","iopub.execute_input":"2021-09-30T18:33:34.659327Z","iopub.status.idle":"2021-09-30T18:33:34.666827Z","shell.execute_reply.started":"2021-09-30T18:33:34.659298Z","shell.execute_reply":"2021-09-30T18:33:34.666233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def detect_tumor(study, image_number):\n\n    # Make sure there aren't files hanging around from the last run\n    cleanup()\n\n    # Load an image\n    image = pydicom.dcmread(f'../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/{study}/T1wCE/Image-{image_number}.dcm')\n    pixels = image.pixel_array\n\n    # Crunch pixels down to 8 bit\n    pixels = pixels - np.min(pixels)\n    pixels = pixels / np.max(pixels)\n    pixels = (pixels * 255).astype(np.uint8)\n\n    # Get the plane\n    plane = get_image_plane(image.ImageOrientationPatient)\n    print(\"Plane:\", plane)\n\n    # Expor the image as a JPG\n    filename = f'{predict_dir}/{study}_t1wce_{image_number}.jpg'\n    cv2.imwrite(filename, pixels)\n    \n    # Run YOLO detect on the exported image\n    detect(plane)\n    \n    # Get the YOLO image and label/BB coords .. if they exist\n    image_name = f'{image_dir}/{study}_t1wce_{image_number}.jpg'\n    if os.path.isfile(image_name):\n        img = cv2.imread(image_name)\n\n        label_name = f'{label_dir}/{study}_t1wce_{image_number}.txt'\n        if os.path.isfile(label_name):\n            label_file = open(label_name, \"r\")\n            label_text = label_file.read()\n\n            fig, axes = plt.subplots(nrows=1, ncols=2,sharex=False, sharey=False, figsize=(10, 10))\n            ax = axes.ravel()\n            ax[0].set_title('Original')\n            ax[0].imshow(pixels, cmap='gray')\n            ax[1].set_title('OD Detect')\n            ax[1].imshow(img, cmap='gray')\n\n            plt.tight_layout()\n            plt.show()\n\n            print(\"Label/BB coords:\", label_text)\n        else:\n            plt.figure(figsize= (6,6))\n            plt.title('No Tumor Detected')\n            plt.imshow(pixels, cmap='gray');","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.66774Z","iopub.execute_input":"2021-09-30T18:33:34.668299Z","iopub.status.idle":"2021-09-30T18:33:34.683827Z","shell.execute_reply.started":"2021-09-30T18:33:34.668269Z","shell.execute_reply":"2021-09-30T18:33:34.683078Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Specify a study,series and image number and call the main function\nstudy = '00216'\nimage_number = '98'\n\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:34.685104Z","iopub.execute_input":"2021-09-30T18:33:34.685921Z","iopub.status.idle":"2021-09-30T18:33:39.385607Z","shell.execute_reply.started":"2021-09-30T18:33:34.685881Z","shell.execute_reply":"2021-09-30T18:33:39.384753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### The bounding box coords can be used to extract 'patches' or to aid with segmentation.\n- Let's try some other images","metadata":{}},{"cell_type":"code","source":"study = '00006'\nimage_number = '90'\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:39.38687Z","iopub.execute_input":"2021-09-30T18:33:39.387116Z","iopub.status.idle":"2021-09-30T18:33:45.03456Z","shell.execute_reply.started":"2021-09-30T18:33:39.387086Z","shell.execute_reply":"2021-09-30T18:33:45.033911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Sometimes the OD will predict two tumors in one. Like above.","metadata":{}},{"cell_type":"code","source":"study = '00019'\nimage_number = '50'\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:45.036017Z","iopub.execute_input":"2021-09-30T18:33:45.036451Z","iopub.status.idle":"2021-09-30T18:33:49.40132Z","shell.execute_reply.started":"2021-09-30T18:33:45.036415Z","shell.execute_reply":"2021-09-30T18:33:49.400488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"study = '00008'\nimage_number = '80'\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2021-09-30T18:33:49.40418Z","iopub.execute_input":"2021-09-30T18:33:49.405045Z","iopub.status.idle":"2021-09-30T18:33:53.155579Z","shell.execute_reply.started":"2021-09-30T18:33:49.404996Z","shell.execute_reply":"2021-09-30T18:33:53.154732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - This tool can be useful to find tumors if the images are sorted well and poor images are removed prior to prediction.\n\nSome of my other MR processing notebooks:\n\n- Determining MR image planes -> https://www.kaggle.com/davidbroberts/determining-mr-image-planes\n- Determining MR Slice Orientation -> https://www.kaggle.com/davidbroberts/determining-mr-slice-orientation\n- Determining DICOM image order -> https://www.kaggle.com/davidbroberts/determining-dicom-image-order\n- Reference Lines on MR images -> https://www.kaggle.com/davidbroberts/mr-reference-lines\n- Manual VOI LUT on MR images -> https://www.kaggle.com/davidbroberts/manual-voi-lut-on-mr-images\n- Standardizing MR Images -> https://www.kaggle.com/davidbroberts/standardizing-mr-images\n- Export DICOM Images by Plane -> https://www.kaggle.com/davidbroberts/export-dicom-series-by-plane/","metadata":{}}]}