{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<div class='alert alert-info' style='text-align: center'><h1>Brain Tumor Object Detection</h1>\n- yet another MR processing notebook -</div>\n\n#### In this notebook, we'll use YOLOv5 to try and detect tumors in brain MRs.\n#### There are three models (pytorch weights), one for each plane, that work on the T1wCE series only.\n#### They were trained on a small sample (~400 images each) of the RSNA dataset I hand labeled using makesense.ai.\n#### The image plane is checked and the appropriate weights are used in detection/classification.\n#### They perform object detection and classification (positive/negative) simulataneously.\n\n##### Model Dataset -> https://www.kaggle.com/davidbroberts/brain-tumor-yolo-od\n##### Training Notebook -> https://www.kaggle.com/davidbroberts/brain-tumor-yolo-od-train\n##### Training Dataset -> https://www.kaggle.com/davidbroberts/brain-tumor-object-detection-datasets\n\n- You can cross reference the classification accuracy by comparing to the study's MGMT value.\n- I included classification for demonstration purposes only (it's not very accurate .. of course none of the models in this comp are).\n- The OD models find a lot of false positives on the inferior axial images especially. Excluding non-diagnostic (all black, peripheral images etc) slices is important here.\n- I created OD models for the other series types, but I did not include them in this demo. Let me know if you're interested in them.","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-09-30T17:32:47.581383Z","iopub.execute_input":"2021-09-30T17:32:47.582388Z","iopub.status.idle":"2021-09-30T17:32:47.58664Z","shell.execute_reply.started":"2021-09-30T17:32:47.582341Z","shell.execute_reply":"2021-09-30T17:32:47.586051Z"}}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport matplotlib.pyplot as plt\nimport cv2\nimport shutil\nfrom shutil import copyfile\nimport path","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:55.381650Z","iopub.execute_input":"2023-06-06T00:55:55.382008Z","iopub.status.idle":"2023-06-06T00:55:55.719705Z","shell.execute_reply.started":"2023-06-06T00:55:55.381925Z","shell.execute_reply":"2023-06-06T00:55:55.718402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict_dir = '/kaggle/working/predict'\nlabel_dir = '/kaggle/working/runs/detect/exp/labels'\nimage_dir = '/kaggle/working/runs/detect/exp'","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:55.721303Z","iopub.execute_input":"2023-06-06T00:55:55.721566Z","iopub.status.idle":"2023-06-06T00:55:55.725346Z","shell.execute_reply.started":"2023-06-06T00:55:55.721536Z","shell.execute_reply":"2023-06-06T00:55:55.724597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Make a dir to export jpgs to\nif not os.path.exists(f'{predict_dir}'):\n    os.mkdir(f'{predict_dir}')","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:55.726401Z","iopub.execute_input":"2023-06-06T00:55:55.727087Z","iopub.status.idle":"2023-06-06T00:55:55.739971Z","shell.execute_reply.started":"2023-06-06T00:55:55.727052Z","shell.execute_reply":"2023-06-06T00:55:55.738728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Copy YOLO\nif not os.path.exists('/kaggle/working/yolov5'):\n    shutil.copytree('/kaggle/input/yolov5-official-v31-dataset/yolov5', '/kaggle/working/yolov5')","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:55.741604Z","iopub.execute_input":"2023-06-06T00:55:55.741846Z","iopub.status.idle":"2023-06-06T00:55:56.658126Z","shell.execute_reply.started":"2023-06-06T00:55:55.741813Z","shell.execute_reply":"2023-06-06T00:55:56.657366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This function gives a basic plane from the ImageOrientationPatient tag. It doesn't account for obliqueness. But we don't need to care about it.\n# Will return 'unknown' if the image isn't exactly orthogonal.\ndef get_image_plane(loc):\n    row_x = round(loc[0])\n    row_y = round(loc[1])\n    row_z = round(loc[2])\n    col_x = round(loc[3])\n    col_y = round(loc[4])\n    col_z = round(loc[5])\n    if (row_x, row_y, col_x, col_y) == (1,0,0,0):\n        return \"coronal\"\n    if (row_x, row_y, col_x, col_y) == (0,1,0,0):\n        return \"sagittal\"\n    if (row_x, row_y, col_x, col_y) == (1,0,0,1):\n        return \"axial\"\n    return \"Unknown\"","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:56.659179Z","iopub.execute_input":"2023-06-06T00:55:56.659391Z","iopub.status.idle":"2023-06-06T00:55:56.666931Z","shell.execute_reply.started":"2023-06-06T00:55:56.659366Z","shell.execute_reply":"2023-06-06T00:55:56.665864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Call yolo detect.py on an image\ndef detect(plane):\n    !python yolov5/detect.py --source {predict_dir} --weights ../input/brain-tumor-yolo-od/{plane}_t1wce_2_class.pt --img 512 --exist-ok --save-txt","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:56.668207Z","iopub.execute_input":"2023-06-06T00:55:56.668652Z","iopub.status.idle":"2023-06-06T00:55:56.679191Z","shell.execute_reply.started":"2023-06-06T00:55:56.668623Z","shell.execute_reply":"2023-06-06T00:55:56.678606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Delete all images and labels\ndef cleanup():\n    if os.path.exists(label_dir):\n        filelist = [ f for f in os.listdir(label_dir) if f.endswith(\".txt\") ]\n        for f in filelist:\n            os.remove(os.path.join(label_dir, f))\n\n        filelist = [ f for f in os.listdir(image_dir) if f.endswith(\".jpg\") ]\n        for f in filelist:\n            os.remove(os.path.join(image_dir, f))\n            \n    filelist = [ f for f in os.listdir(predict_dir) if f.endswith(\".jpg\") ]\n    for f in filelist:\n        os.remove(os.path.join(predict_dir, f))  ","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:56.680351Z","iopub.execute_input":"2023-06-06T00:55:56.680691Z","iopub.status.idle":"2023-06-06T00:55:56.693752Z","shell.execute_reply.started":"2023-06-06T00:55:56.680663Z","shell.execute_reply":"2023-06-06T00:55:56.692982Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def detect_tumor(study, image_number):\n\n    # Make sure there aren't files hanging around from the last run\n    cleanup()\n\n    # Load an image\n    image = pydicom.dcmread(f'../input/rsna-miccai-brain-tumor-radiogenomic-classification/train/{study}/T1wCE/Image-{image_number}.dcm')\n    pixels = image.pixel_array\n\n    # Crunch pixels down to 8 bit\n    pixels = pixels - np.min(pixels)\n    pixels = pixels / np.max(pixels)\n    pixels = (pixels * 255).astype(np.uint8)\n\n    # Get the plane\n    plane = get_image_plane(image.ImageOrientationPatient)\n    print(\"Plane:\", plane)\n\n    # Expor the image as a JPG\n    filename = f'{predict_dir}/{study}_t1wce_{image_number}.jpg'\n    cv2.imwrite(filename, pixels)\n    \n    # Run YOLO detect on the exported image\n    detect(plane)\n    \n    # Get the YOLO image and label/BB coords .. if they exist\n    image_name = f'{image_dir}/{study}_t1wce_{image_number}.jpg'\n    if os.path.isfile(image_name):\n        img = cv2.imread(image_name)\n\n        label_name = f'{label_dir}/{study}_t1wce_{image_number}.txt'\n        if os.path.isfile(label_name):\n            label_file = open(label_name, \"r\")\n            label_text = label_file.read()\n\n            fig, axes = plt.subplots(nrows=1, ncols=2,sharex=False, sharey=False, figsize=(10, 10))\n            ax = axes.ravel()\n            ax[0].set_title('Original')\n            ax[0].imshow(pixels, cmap='gray')\n            ax[1].set_title('OD Detect')\n            ax[1].imshow(img, cmap='gray')\n\n            plt.tight_layout()\n            plt.show()\n\n            print(\"Label/BB coords:\", label_text)\n        else:\n            plt.figure(figsize= (6,6))\n            plt.title('No Tumor Detected')\n            plt.imshow(pixels, cmap='gray');","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:56.695236Z","iopub.execute_input":"2023-06-06T00:55:56.695629Z","iopub.status.idle":"2023-06-06T00:55:56.708137Z","shell.execute_reply.started":"2023-06-06T00:55:56.695593Z","shell.execute_reply":"2023-06-06T00:55:56.707305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Specify a study,series and image number and call the main function\nstudy = '00216'\nimage_number = '98'\n\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:55:56.709150Z","iopub.execute_input":"2023-06-06T00:55:56.709349Z","iopub.status.idle":"2023-06-06T00:56:02.192521Z","shell.execute_reply.started":"2023-06-06T00:55:56.709325Z","shell.execute_reply":"2023-06-06T00:56:02.191895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### The bounding box coords can be used to extract 'patches' or to aid with segmentation.\n- Let's try some other images","metadata":{}},{"cell_type":"code","source":"study = '00006'\nimage_number = '90'\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:59:55.411354Z","iopub.execute_input":"2023-06-06T00:59:55.412570Z","iopub.status.idle":"2023-06-06T00:59:58.473069Z","shell.execute_reply.started":"2023-06-06T00:59:55.412520Z","shell.execute_reply":"2023-06-06T00:59:58.471552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Sometimes the OD will predict two tumors in one. Like above.","metadata":{}},{"cell_type":"code","source":"study = '00019'\nimage_number = '50'\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:56:06.176471Z","iopub.execute_input":"2023-06-06T00:56:06.176688Z","iopub.status.idle":"2023-06-06T00:56:09.384335Z","shell.execute_reply.started":"2023-06-06T00:56:06.176661Z","shell.execute_reply":"2023-06-06T00:56:09.383287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"study = '00008'\nimage_number = '80'\ndetect_tumor(study, image_number)","metadata":{"execution":{"iopub.status.busy":"2023-06-06T00:56:09.386025Z","iopub.execute_input":"2023-06-06T00:56:09.386333Z","iopub.status.idle":"2023-06-06T00:56:12.269630Z","shell.execute_reply.started":"2023-06-06T00:56:09.386277Z","shell.execute_reply":"2023-06-06T00:56:12.268778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### - This tool can be useful to find tumors if the images are sorted well and poor images are removed prior to prediction.\n\nSome of my other MR processing notebooks:\n\n- Determining MR image planes -> https://www.kaggle.com/davidbroberts/determining-mr-image-planes\n- Determining MR Slice Orientation -> https://www.kaggle.com/davidbroberts/determining-mr-slice-orientation\n- Determining DICOM image order -> https://www.kaggle.com/davidbroberts/determining-dicom-image-order\n- Reference Lines on MR images -> https://www.kaggle.com/davidbroberts/mr-reference-lines\n- Manual VOI LUT on MR images -> https://www.kaggle.com/davidbroberts/manual-voi-lut-on-mr-images\n- Standardizing MR Images -> https://www.kaggle.com/davidbroberts/standardizing-mr-images\n- Export DICOM Images by Plane -> https://www.kaggle.com/davidbroberts/export-dicom-series-by-plane/","metadata":{}}]}