{"cells":[{"metadata":{"_uuid":"09f14d9199ffa8cd84c499f2e9497bc8e27b15fd"},"cell_type":"markdown","source":"# Overview\n\nWelcome to the 2018 RSNA Challenge co-hosted by Kaggle. In this competition, the primary endpoint will be the detection of bounding boxes corresponding to the diagnosis of pneumonia (e.g. lung infection) on chest radiographs, a special 2D high resolution grayscale medical image. Note that pnuemonia is just one of many possible disease processes that can occur on a chest radiograph, and that any given single image may contain 0, 1 or many boxes corresponding to possible pneumonia locations.\n\nMy name is Peter Chang, MD. I am both a radiologist physician and a data scientist / software engineer with machine learning experience. Today, in this Jupyter notebook, we will explore the 2018 RSNA Challenge dataset including underlying data structures, imaging file formats and label types."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import glob, pandas as pd\nimport matplotlib.pyplot as plt\nimport pydicom, numpy as np","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0f368672adb58cc7bb57ecc95a04487251bb9c2c"},"cell_type":"markdown","source":"# Challenge Data\n\nThe challenge data is organized in several files and folders. If you are following along in the Kaggle kernel, this data will be preloaded in the `../input` directory:"},{"metadata":{"trusted":true,"_uuid":"1b6c13edf09c7d80145173edad847c34db1ba5f2"},"cell_type":"code","source":"!ls ../input","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"88fb25d6864f224a0995a28140de532f80cb3e6b"},"cell_type":"markdown","source":"The several key items in this folder:\n* `stage_1_train_labels.csv`: CSV file containing training set patientIds and  labels (including bounding boxes)\n* `stage_1_detailed_class_info.csv`: CSV file containing detailed labels (explored further below)\n* `stage_1_train_images/`:  directory containing training set raw image (DICOM) files\n\nLet's go ahead and take a look at the first labels CSV file first:"},{"metadata":{"trusted":true,"_uuid":"b4c54974f191b477e46370c79254f75657ed2a85","scrolled":true},"cell_type":"code","source":"df = pd.read_csv('../input/stage_2_train_labels.csv')\nprint(df.iloc[0])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5435a871306c834a2282ae867a86861f9b80a610"},"cell_type":"markdown","source":"As you can see, each row in the CSV file contains a `patientId` (one unique value per patient), a target (either 0 or 1 for absence or presence of pneumonia, respectively) and the corresponding abnormality bounding box defined by the upper-left hand corner (x, y) coordinate and its corresponding width and height. In this particular case, the patient does *not* have pneumonia and so the corresponding bounding box information is set to `NaN`. See an example case with pnuemonia here:"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"98c60a2ddf5bba070c0908d9bb705c0c5976ac7a"},"cell_type":"code","source":"print(df.iloc[4])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7ad152504ec74be4ac1992ca669da7ad3ce3aac2"},"cell_type":"markdown","source":"One important thing to keep in mind is that a given `patientId` may have **multiple** boxes if more than one area of pneumonia is detected (see below for example images)."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"# Overview of DICOM files and medical images\n\nMedical images are stored in a special format known as DICOM files (`*.dcm`). They contain a combination of header metadata as well as underlying raw image arrays for pixel data. In Python, one popular library to access and manipulate DICOM files is the `pydicom` module. To use the `pydicom` library, first find the DICOM file for a given `patientId` by simply looking for the matching file in the `stage_1_train_images/` folder, and the use the `pydicom.read_file()` method to load the data:"},{"metadata":{"trusted":true,"_uuid":"5f2c15162a0d1390624b42ef94d4f9e260be56ac"},"cell_type":"code","source":"patientId = df['patientId'][0]\ndcm_file = f'../input/stage_2_train_images/{patientId}.dcm'\ndcm_data = pydicom.read_file(dcm_file)\n\nprint(dcm_data)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2cfbe9eb43f4e4922c42481739046943767f765c"},"cell_type":"markdown","source":"Most of the standard headers containing patient identifable information have been anonymized (removed) so we are left with a relatively sparse set of metadata. The primary field we will be accessing is the underlying pixel data as follows:"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"953ccb9e176398bcad8dad0241dbc1e194fa8a8e"},"cell_type":"code","source":"im = dcm_data.pixel_array\nprint(type(im))\nprint(im.dtype)\nprint(im.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ed51c8db0a041023f7fb19ecd66b08bb630a1485"},"cell_type":"markdown","source":"## Considerations\n\nAs we can see here, the pixel array data is stored as a Numpy array, a powerful numeric Python library for handling and manipulating matrix data (among other things). In addition, it is apparent here that the original radiographs have been preprocessed for us as follows:\n\n* The relatively high dynamic range, high bit-depth original images have been rescaled to 8-bit encoding (256 grayscales). For the radiologists out there, this means that the images have been windowed and leveled already. In clinical practice, manipulating the image bit-depth is typically done manually by a radiologist to highlight certain disease processes. To visually assess the quality of the automated bit-depth downscaling and for considerations on potentially improving this baseline, consider consultation with a radiologist physician.\n\n* The relativley large original image matrices (typically acquired at >2000 x 2000) have been resized to the data-science friendly shape of 1024 x 1024. For the purposes of this challenge, the diagnosis of most pneumonia cases can typically be made at this resolution. To visually assess the feasibility of diagnosis at this resolution, and to determine the optimal resolution for pneumonia detection (oftentimes can be done at a resolution *even smaller* than 1024 x 1024), consider consultation with a radiogist physician.\n\n## Visualizing An Example\n\nTo take a look at this first DICOM image, let's use the `pylab.imshow()` method:"},{"metadata":{"trusted":true,"_uuid":"59ac176a83e8fc1f11531a216723791577948fbe"},"cell_type":"code","source":"plt.imshow(im)#, cmap=pylab.cm.gist_gray)\nplt.gray()\nplt.axis('off')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"78c0e24742ce4f61c153f363e66ac5d7e5efbb0e"},"cell_type":"markdown","source":"# Exploring the Data and Labels\n\nAs alluded to above, any given patient may potentially have many boxes if there are several different suspicious areas of pneumonia. To collapse the current CSV file dataframe into a dictionary with unique entries, consider the following method:"},{"metadata":{"trusted":true,"_uuid":"626ee0aa4038a988bac8e8c800c4bf4270af1b76"},"cell_type":"code","source":"def parse_data(df):\n    \"\"\"\n    Method to read a CSV file (Pandas dataframe) and parse the \n    data into the following nested dictionary:\n\n      parsed = {\n        \n        'patientId-00': {\n            'dicom': path/to/dicom/file,\n            'label': either 0 or 1 for normal or pnuemonia, \n            'boxes': list of box(es)\n        },\n        'patientId-01': {\n            'dicom': path/to/dicom/file,\n            'label': either 0 or 1 for normal or pnuemonia, \n            'boxes': list of box(es)\n        }, ...\n\n      }\n\n    \"\"\"\n    # --- Define lambda to extract coords in list [y, x, height, width]\n    extract_box = lambda row: [row['y'], row['x'], row['height'], row['width']]\n\n    parsed = {}\n    for n, row in df.iterrows():\n        # --- Initialize patient entry into parsed \n        pid = row['patientId']\n        if pid not in parsed:\n            parsed[pid] = {\n                'dicom': '../input/stage_2_train_images/%s.dcm' % pid,\n                'label': row['Target'],\n                'boxes': []}\n\n        # --- Add box if opacity is present\n        if parsed[pid]['label'] == 1:\n            parsed[pid]['boxes'].append(extract_box(row))\n\n    return parsed","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cbb1a9cf9eb93df25896f47f80257aba561c0d00"},"cell_type":"markdown","source":"Let's use the method here:"},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"fbf2c2f60dcc661c249989eb063382ad7304aa10"},"cell_type":"code","source":"parsed = parse_data(df)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7b851e847ce442d690a8dc55937070235c7a16f0"},"cell_type":"markdown","source":"As we saw above, patient `00436515-870c-4b36-a041-de91049b9ab4` has pnuemonia so lets check our new `parsed` dict here to see the patients corresponding bounding boxes:"},{"metadata":{"trusted":true,"_uuid":"459c07f8fadd3f30aff6ff26bdd4b4c8fe4f79f5"},"cell_type":"code","source":"print(parsed['00436515-870c-4b36-a041-de91049b9ab4'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d54c891b2477a2c022b8b4e3ffd73422eaf862e1"},"cell_type":"markdown","source":"# Visualizing Boxes\n\nIn order to overlay color boxes on the original grayscale DICOM files, consider using the following  methods (below, the main method `draw()` requires the method `overlay_box()`):"},{"metadata":{"trusted":true,"_uuid":"6be7b744e42ac7204ea661c0c7175881b043c5c6"},"cell_type":"code","source":"def draw(data):\n    \"\"\"\n    Method to draw single patient with bounding box(es) if present \n\n    \"\"\"\n    # --- Open DICOM file\n    d = pydicom.read_file(data['dicom'])\n    im = d.pixel_array\n\n    # --- Convert from single-channel grayscale to 3-channel RGB\n    im = np.stack([im] * 3, axis=2)\n\n    # --- Add boxes with random color if present\n    for box in data['boxes']:\n        rgb = np.floor(np.random.rand(3) * 256).astype('int')\n        im = overlay_box(im=im, box=box, rgb=rgb, stroke=6)\n\n    plt.imshow(im)\n    plt.gray()\n    plt.axis('off')\n\ndef overlay_box(im, box, rgb, stroke=1):\n    \"\"\"\n    Method to overlay single box on image\n\n    \"\"\"\n    # --- Convert coordinates to integers\n    box = [int(b) for b in box]\n    \n    # --- Extract coordinates\n    y1, x1, height, width = box\n    y2 = y1 + height\n    x2 = x1 + width\n\n    im[y1:y1 + stroke, x1:x2] = rgb\n    im[y2:y2 + stroke, x1:x2] = rgb\n    im[y1:y2, x1:x1 + stroke] = rgb\n    im[y1:y2, x2:x2 + stroke] = rgb\n\n    return im","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d10aef5768f6147aaa4c5c8dec7444fb90f0842d"},"cell_type":"markdown","source":"As we saw above, patient `00436515-870c-4b36-a041-de91049b9ab4` has pnuemonia so let's take a look at the overlaid bounding boxes:"},{"metadata":{"trusted":true,"_uuid":"ee8f69746904ae88da12f631b6c9e0b1e214c4fa","scrolled":true},"cell_type":"code","source":"draw(parsed['00436515-870c-4b36-a041-de91049b9ab4'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"eb00fb542999b5ffe6b32ee38321830f9b090d26"},"cell_type":"markdown","source":"## Exploring Detailed Labels\n\nIn this challenge, the primary endpoint will be the detection of bounding boxes consisting of a binary classification---e.g. the presence or absence of pneumonia. However, in addition to the binary classification, each bounding box *without* pneumonia is further categorized into *normal* or *no lung opacity / not normal*. This extra third class indicates that while pneumonia was determined not to be present, there was nonetheless some type of abnormality on the image---and oftentimes this finding may mimic the appearance of true pneumonia. Keep in mind that this extra class is provided as supplemental information to help improve algorithm accuracy if needed; generation of this separate class **will not** be a formal metric used to evaluate performance in this competition.\n\nAs above, we saw that the first patient in the CSV file did not have pneumonia. Let's look at the detailed label information for this patient:"},{"metadata":{"trusted":true,"_uuid":"a639baad366af644bdd8efe63712dd47cc0fe9ed"},"cell_type":"code","source":"df_detailed = pd.read_csv('../input/stage_2_detailed_class_info.csv')\nprint(df_detailed.iloc[0])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8bb1f27ca89d8f05142e7ac462ccbde61d4228bd"},"cell_type":"markdown","source":"As we see here, the patient does not have pneumonia however *does* have another imaging abnormality present. Let's take a closer look:"},{"metadata":{"trusted":true,"_uuid":"ce3843dbf0f47cda1299d05fa8791148c1c2e806"},"cell_type":"code","source":"patientId = df_detailed['patientId'][0]\ndraw(parsed[patientId])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b54a1136f966df74c15872a073fc333a8dd37294"},"cell_type":"markdown","source":"While the image displayed inline within the notebook is small, as a radiologist it is evident that the patient has several well circumscribed nodular densities in the left lung (right side of image). In addition there is a large chest tube in the right lung (left side of the image) which has been placed to drain fluid accumulation (e.g. pleural effusion) at the right lung base that also demonstrates overlying patchy densities (e.g. possibly atelectasis or partial lung collapse).\n\nAs you can see, there are a number of abnormalities on the image, and the determination that none of these findings correlate to pneumonia is somewhat subjective even among expert physicians. Therefore, as is almost always the case in medical imaging datasets, the provided ground-truth labels are far from 100% objective. Keep this in mind as you develop your algorithm, and consider consultation with a radiologist physician to help determine an optimal strategy for mitigating these discrepencies."},{"metadata":{"_uuid":"f52f87aafb11d94916d984e19089c28e25b95eed"},"cell_type":"markdown","source":"## Label Summary\n\nFinally, let us take a closer look at the distribution of labels in the dataset. To do so we will first parse the detailed label information:"},{"metadata":{"trusted":true,"_uuid":"44cdd1bff2692927dabd6d08f231be9aae4d31c9"},"cell_type":"code","source":"summary = {}\nfor n, row in df_detailed.iterrows():\n    if row['class'] not in summary:\n        summary[row['class']] = 0\n    summary[row['class']] += 1\n    \nprint(summary)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7e1575f979fe05c0853ba19bf28b56099fd38d7a"},"cell_type":"markdown","source":"As we can see, there is a relatively even split between the three classes, with nearly 2/3rd of the data comprising of no pneumonia (either completely *normal* or *no lung opacity / not normal*). Compared to most medical imaging datasets, where the prevalence of disease is quite low, this dataset has been significantly enriched with pathology."},{"metadata":{"_uuid":"2f4da05fb65ba9b4ce1b14b7a9c85140fcd037d5"},"cell_type":"markdown","source":"# Next Steps\n\nNow that you understand the data structures, imaging file formats and label types, it's time to make an algorithm! Keep in mind that the primary endpoint is the detection of bounding boxes, thus you will likely be considering various **object localization** algorithms. An alternative strategy is to consider the related family of **segmentation** algorithms with the acknowledgement that bounding boxes will only be a coarse approximation to true pixel-by-pixel image segmentation masks.\n\nFinally, as alluded to several times in this notebook, a radiologist physican may often times provide useful ancillary information, strategy for algorithm development and/or additional label reconciliation. In addition to physicians you may have access to locally, the RSNA will reach out to radiologists and facilitate engagement remotely through the Kaggle online forums. As a medical professional, I know that many of my colleagues are very interested in getting started so please feel free to reach out and start a conversation! \n\nGood luck!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}