{"cells":[{"metadata":{"_uuid":"e7d66579ecc1cdf2b0002ddd0ce8d77c340c8b8c"},"cell_type":"markdown","source":"Dear Kagglers,\n\nThis kernel is intended to help beginners who are overwhelmed by this project get \"up and running.\" We hope to provide a basic understanding of the data and how to work with it to make predictions that can be submitted to the competition. If you have any questions, comments, or especially criticisms please let us know in the comments so we can address them! Have a nice day, everyone!\n\n---"},{"metadata":{"_uuid":"7d3fa2412af2cc98d0d01451a42af204bd3cb8d8"},"cell_type":"markdown","source":"# <a href='#contents'>Table of Contents</a> <a id=\"contents\"></a> \n1. <a href='#intro'>Introduction</a>\n   *   <a href='#overview'>Project Overview</a>\n   *   <a href='#goals'>Goals</a>\n2. <a href='#setup'>Set-up\n   *   <a href='#dependencies'>Install Dependencies</a>\n   *   <a href='#files'>Examine Files</a>\n3. <a href='#data'>Data Exploration and Visualization</a>\n   *   <a href='#classes'>Classes</a>\n   *   <a href='#locations'>Pneumonia Locations</a>\n4. <a href='#conclusion'>Conclusion</a>\n    *   <a href='#kernels'>Other Helpful Kernels</a>\n    *   <a href='#refs'>References</a>\n\n ---\n ---"},{"metadata":{"_uuid":"f468d3614a785b543639285d471b628f5dd435ae"},"cell_type":"markdown","source":"## <span style=\"color:darkgreen\">Introduction</span> <a id=\"intro\"></a>\nThis kernel will provide a simplified approach to loading and examining data for the RSNA Pneumonia Detection Challenge. First we will briefly discuss the challenge. Then we will look at the `csv` files and examine the [DICOM](https://en.wikipedia.org/wiki/DICOM) images. Next we will manipulate the images to maximize their usefulness for model training. Lastly, we will point you to models to explore, including our own version of YOLO.\n\n---"},{"metadata":{"_uuid":"b34855ee1226e2a820744d4faf681b167041df8f"},"cell_type":"markdown","source":"### Project Overview <a id='overview'></a>"},{"metadata":{"_uuid":"c02f3e655375ca1d258db91c7c738f0514fc81a2"},"cell_type":"markdown","source":"Numerous advances in medicine have been accomplished through the use of machine learning on medical imagery. The Radiological Society of North America ([RSNA](http://www.rsna.org)) has sponsored this competition on Kaggle to incentivize the creation of new algorithms that can detect pneumonia in radiographic images. A more detailed definition of the of the competition is provided on the [Kaggle RSNA Pneumonia Detection Challenge website](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge). \n\n---"},{"metadata":{"_uuid":"6e0e405f1e47d492579652382da01099901c96ca"},"cell_type":"markdown","source":"### Goals <a id='goals'></a>"},{"metadata":{"_uuid":"64c35b0bd397708198584797f788aab0184374cb"},"cell_type":"markdown","source":"To classify pneumonia in medical images we will build an algorithm that detects lung opacities. We will need to differentiate between pneumonia opacities and other lung conditions that can create signals such as fluid overload, bleeding, volume loss, lung cancer, post-radiation or surgical changes, and fluid in the pleural space.\n\nThis will be done by training machine learning models on a set of images that have been labeled by experts to show a box drawn around the presumed pneumonia lung opacity or opacities, when present. Many control images with no pneumonia opacity or with other lung conditions are also present with which to train. A successful algorithm will be able to take unlabeled images and label them accurately by drawing a box around pneumonia lung opacities.\n\nScoring in the competition will be by finding the mean average precision of the predicted boxes at different intersection over union thresholds. A more detailed explanation can be found on the [challenge's evaluation page](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge#evaluation).\n\n---"},{"metadata":{"_uuid":"53330655f0591eac9836bdc2125f00ff8a501bb4"},"cell_type":"markdown","source":"(return to <a href='#contents'>Table of Contents</a>)\n\n---\n---"},{"metadata":{"_uuid":"04f3a246bfe28b5d70239d5313672fc1fb6baa95"},"cell_type":"markdown","source":"## <span style=\"color:darkgreen\">Set-up</span> <a id='setup'></a>"},{"metadata":{"_uuid":"3e1d0014351062ffb35e02835d8d1cbb428e53ee"},"cell_type":"markdown","source":"### Install Dependencies <a id='dependencies'></a>\n\nWe need to import the libraries and packages we'll be using. These are:\n*   [os](https://docs.python.org/3/library/os.html): Python module for operating system functionality (<a href='#os'>here</a>)\n*   [pandas](https://pandas.pydata.org/index.html): Python data analysis and processing library, used for reading`csv`files (<a href='#pd'>here</a>)\n*   [seaborn](https://seaborn.pydata.org/): Python visualization library (<a href='#seaborn'>here</a>)\n*   [matplotlib.pyplot](https://matplotlib.org/users/pyplot_tutorial.html): graphical plotting output library (<a href='#plt'>here</a>)\n*   [pydicom](https://github.com/pydicom/pydicom): inspect and modify [DICOM](https://www.dicomstandard.org/) data (<a href='#pydicom'>here</a>)\n*   [numpy](https://docs.scipy.org/doc/numpy/reference/): Python package for scientific computations, including linear algebra (<a href='#np'>here</a>)\n*   [multiprocessing](https://docs.python.org/3/library/multiprocessing.html): supports spawning processes (<a href='#multiprocessing'>here</a>)\n*   [warnings](https://docs.python.org/3/library/warnings.html): provides options for warning control (<a href='#warnings'>here</a>)\n*   [matplotlib.patches](https://matplotlib.org/api/_as_gen/matplotlib.patches.Patch.html#matplotlib.patches.Patch): 2D shapes with a face color and an edge color (<a href='#patches'>here</a>)\n*   [imgaug](https://imgaug.readthedocs.io/en/latest/): library for image augmentation in machine learning experiments (<a href='#imgaug'>here</a>)\n*   [tqdm](https://pypi.org/project/tqdm/): fast, extensible progress meter (<a href='#tqdm'>here</a>)\n*   [scikit-learn GaussianMixture](http://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html): allows estimation of the parameters of a Gaussian mixture distribution (<a href='#gaussianmixture'>here</a>)\n*   [scikit-image morphology, feature, measure, util, & transform](https://scikit-image.org/): a collection of algorithms for image processing (<a href='#skimage'>here</a>)\n\nClick on `here` to the right of the library to find the code block where it is first used. <a id='warnings'></a>"},{"metadata":{"trusted":true,"_uuid":"034a4b05b49c982137a7bd597a64a9c9091d6fb5"},"cell_type":"code","source":"import os, pandas as pd, seaborn as sns, matplotlib.pyplot as plt, pydicom, numpy as np\nimport multiprocessing, warnings\nimport matplotlib.patches as patches\n\nfrom imgaug import augmenters as iaa\nfrom tqdm import tqdm\nfrom sklearn.mixture import GaussianMixture\nfrom skimage import feature\nfrom skimage import morphology\nfrom skimage import measure\nfrom skimage import util\nfrom skimage import transform\n\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2cc4f0658deebfed8aefa841bb67f4660e829695"},"cell_type":"markdown","source":"### Examine Files <a id='files'></a>\n\nWe will start by listing the files included in the project. <a id='os'></a>"},{"metadata":{"trusted":true,"_uuid":"097a513931eb33ebcca7643681ca55de33517ffd"},"cell_type":"code","source":"os.listdir(\"../input\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c9008612ee003eb734206950150b9b70e8dce192"},"cell_type":"markdown","source":"Let's first look at the `csv` files in order. We'll use [`pandas.read_csv`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html) to read the data and then look at a few examples. <a id='pd'></a>"},{"metadata":{"trusted":true,"_uuid":"608c9105367c18910c72412d3fcc33fae9343fac"},"cell_type":"code","source":"stage_1_detailed_class_info = pd.read_csv('../input/stage_1_detailed_class_info.csv')\nprint(stage_1_detailed_class_info.iloc[2:5])\nprint(\"Rows:\", stage_1_detailed_class_info.shape[0])\nprint(\"Columns:\", stage_1_detailed_class_info.shape[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4b6fe4daa74c6cd475a310d142760c47bcdc4a6b"},"cell_type":"markdown","source":"There are 28,989 rows with two columns. The first column is the `patientId` and the second is the `class`. How many of those patient IDs are unique?"},{"metadata":{"trusted":true,"_uuid":"4f159d23c39e98784cab20bc3b208608d82b10c1"},"cell_type":"code","source":"print(\"# of unique patient IDs: \", stage_1_detailed_class_info['patientId'].nunique())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6b9daec4ce5a4bea5209d1ab5e149f7271997dae"},"cell_type":"markdown","source":"Some patients are represented multiple times in the `patientId` column.\n\nLet's break that down.\n\nThis line below counts up each `patientId`'s number of occurrences and then counts up how many have each number of occurrences. Got it?"},{"metadata":{"trusted":true,"_uuid":"72c94145a5a91664111ffd10818ef8253571d315"},"cell_type":"code","source":"stage_1_detailed_class_info['patientId'].value_counts().value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2c5a2ea80a58d5ffee01a91d4f6bb45c6324b164"},"cell_type":"markdown","source":"So 3062 patients are represented twice, 105 thrice, and 11 four times.\n\n---"},{"metadata":{"_uuid":"5ae8df9362d32e1dfa910eafff15a28b710c0435"},"cell_type":"markdown","source":"The sample submission file will show us the format that we should be using for our competition submission after we have created our predictions."},{"metadata":{"trusted":true,"_uuid":"5ac64c293a7f5664ddd35e10ffa0348502488e83"},"cell_type":"code","source":"stage_1_sample_submission = pd.read_csv('../input/stage_1_sample_submission.csv')\nprint(stage_1_sample_submission.iloc[:3])\nprint(\"Rows:\", stage_1_sample_submission.shape[0])\nprint(\"Columns:\", stage_1_sample_submission.shape[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"47f0e2347f7eb6ad0b57727dc6aff839c4da71e6"},"cell_type":"markdown","source":"We see that it has two columns, titled `patientId` and `PredictionString`, and 1000 rows. This tells us we do NOT want to include the row index counters on our submission. We also know, from the [competition evaluation page](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge#evaluation), that if a patient has multiple predicted bounding boxes that the submission should include them all by listing them one after another in the prediction column like this: \n\n`00322d4d-1c29-4943-afc9-b6754be640eb,0.8 10 10 50 50 0.75 100 100 5 5`"},{"metadata":{"_uuid":"4c7900106ea770e1d34df70f74980ab3c7e414ac"},"cell_type":"markdown","source":"---\n\nNext up we have the training labels file."},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"b7c96df54409f862e7164f93d7ca49ea88a1a2ea"},"cell_type":"code","source":"stage_1_train_labels = pd.read_csv('../input/stage_1_train_labels.csv')\npd.set_option('display.max_columns', 1000)\npd.set_option('display.width', 1000)\nprint(stage_1_train_labels.iloc[2:5])\nprint(\"Rows:\", stage_1_train_labels.shape[0])\nprint(\"Columns:\", stage_1_train_labels.shape[1])\nprint(\"# of unique patient IDs: \", len(list(stage_1_train_labels.patientId.unique())))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"24125a2f7c663f44765798d3372f09cfd9b126a7"},"cell_type":"markdown","source":"Each row consists of the `patientId`, as well as either `NaN` (not a number) if there are no pneumonia bounding boxes indicated or values representing the `x` and `y` coordinates of the upper-left corner of the bounding box followed by the `width` and `height` of the bounding box. `Target` is 0 for no boxes and 1 when a box is present. And again, some patients are represented multiple times. Spoiler: that is because some patients have multiple bounding boxes labeled!"},{"metadata":{"_uuid":"2dc1f991ee83b2ee574ff04a27e882bc4cc500ba"},"cell_type":"markdown","source":"---\n\nLet's now look at the metadata available with each image. We'll use pydicom to read the images. <a id='pydicom'></a>"},{"metadata":{"trusted":true,"_uuid":"3c90ea90ee659d9eaae08f7901e63131b3a14bb6"},"cell_type":"code","source":"patientId = stage_1_train_labels['patientId'][0]\ndicom_file = '../input/stage_1_train_images/%s.dcm' % patientId\ndicom_data = pydicom.read_file(dicom_file)\n\ndicom_data","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"31122665651b62022cba953424bcfc19b1b247c9"},"cell_type":"markdown","source":"Wow. There is a lot here. The information we are most interested in at this point are the patient's sex and age, the view position of the image (PA for posterior -> anterior or AP for anterior -> posterior), the image pixel spacing, and of course the image itself, which is contained in the `Pixel Data` array."},{"metadata":{"_uuid":"75ada715a204a083d6a0dbcf028280346cc12a49"},"cell_type":"markdown","source":"(return to <a href='#contents'>Table of Contents</a>)\n\n---\n---"},{"metadata":{"_uuid":"5e31ac356dbed80c4ed12e0c9c7751f59795af5f"},"cell_type":"markdown","source":"## <span style=\"color:darkgreen\">Data Exploration and Visualization</span> <a id='data'></a>"},{"metadata":{"_uuid":"7ed3a514412222dd4be79b436e89e08dbb933718"},"cell_type":"markdown","source":"First of all, a giant shout-out to [thomasjpfan's kernel](https://www.kaggle.com/thomasjpfan/q-a-with-only-pictures) for a lot of these visualizations. We've adapted some and used others as is and explained the code behind them for this tutorial."},{"metadata":{"_uuid":"950503416a026deb0d608874cfde5f75f2a0876e"},"cell_type":"markdown","source":"---\n\nNext we will define functions that parse the DICOM metadata for the attributes we are interested in. We get the age, gender, view position, patient ID, pixel spacing, and a metric to determine the number of black pixels in each image (we'll see why later).\n\nWe use Python's multiprocessing package to accelerate the speed of these tasks. <a id='multiprocessing'></a> <a id='np'></a>"},{"metadata":{"trusted":true,"_uuid":"3a99d17fc6545b30cb1ba61325ee4f292bccb5fa"},"cell_type":"code","source":"stage_1_train_labels['aspect_ratio'] = (stage_1_train_labels['width'] / \n                                        stage_1_train_labels['height'])\nstage_1_train_labels['area'] = stage_1_train_labels['width'] * stage_1_train_labels['height']\n\ndef get_info(patientId, root_dir='../input/stage_1_train_images/'):\n    file_name = os.path.join(root_dir, f'{patientId}.dcm')\n    dicom_data = pydicom.read_file(file_name)\n    return {'age': dicom_data.PatientAge, \n            'gender': dicom_data.PatientSex,\n            'view_position': dicom_data.ViewPosition,\n            'id': os.path.basename(file_name).split('.')[0],\n            'pixel_spacing': float(dicom_data.PixelSpacing[0]),\n            'mean_black_pixels': np.mean(dicom_data.pixel_array == 0)}\n\npatient_ids = list(stage_1_train_labels.patientId.unique())\nwith multiprocessing.Pool(4) as pool:\n    result = pool.map(get_info, patient_ids)\n    \ndemo = pd.DataFrame(result)\ndemo['age'] = demo['age'].astype(int)\ndemo['gender'] = demo['gender'].astype('category')\ndemo['view_position'] = demo['view_position'].astype('category')\n\nstage_1_train_labels = (stage_1_train_labels.merge(demo, left_on='patientId', \n                                                   right_on='id', how='left')\n                        .drop(columns='id'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9df8de18dbffdf3e9da9d5f860ee1ca45446d495"},"cell_type":"code","source":"stage_1_train_labels[2:5]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6bcb928e019b2f8c8c6459d9ccb979855f60c6ff"},"cell_type":"markdown","source":"---\n### Classes <a id='classes'></a>"},{"metadata":{"_uuid":"e8396af97b092025c195e3c0c764fb6012855ec9"},"cell_type":"markdown","source":"How many of the patients have pneumonia compared to those that don't? <a id='seaborn'></a> <a id='plt'></a>"},{"metadata":{"trusted":true,"_uuid":"fb18fc01119fc7e0688753596390919a331961f1"},"cell_type":"code","source":"sns.set_style('darkgrid')\nsns.set_context('notebook', font_scale=1.4)\n\nplt.rcParams['figure.figsize'] = [12, 3]\nplt.rcParams['lines.linewidth'] = 1\n\nboxes_per_patient = stage_1_train_labels.groupby('patientId')['Target'].sum()\n\nax = (boxes_per_patient > 0).value_counts().plot.barh(color=['teal','orange'])\n_ = ax.set_title('Pneumonia opacity present')\n_ = ax.set_xlabel('Number of patients')\n_ = ax.xaxis.set_tick_params(rotation=0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4f2c4c0f9a32092f7242e056be4ff32c752239b0"},"cell_type":"markdown","source":"---\n\nHow many bounding boxes designating a pneumonia opacity are present in each image?"},{"metadata":{"trusted":true,"_uuid":"01b89745b918312009ac4cde03f96d2939cbdc69"},"cell_type":"code","source":"ax = boxes_per_patient.value_counts().sort_index().plot.barh()\n_ = ax.set_title('Pneumonia opacity bounding boxes per image')\n_ = ax.set_xlabel('Number of patients')\n_ = ax.set_ylabel('Boxes')\n_ = ax.xaxis.set_tick_params(rotation=0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"617d4c881f931354a10b54dad51488fd9dd82e84"},"cell_type":"markdown","source":"---\n\nWhat is the age distribution by gender and target?"},{"metadata":{"trusted":true,"_uuid":"e7028a0a2d64dcad85d36b4d4b46a07f709513cd"},"cell_type":"code","source":"g = sns.FacetGrid(col='Target', hue='gender', \n                  data=stage_1_train_labels.drop_duplicates(subset=['patientId']), \n                  height=9, palette=dict(F=\"red\", M=\"blue\"))\n_ = g.map(sns.distplot, 'age', hist_kws={'alpha': 0.5}).add_legend()\n_ = g.fig.suptitle(\"What is the age distribution by gender and target?\", y=1.02, fontsize=20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f6313e2ccaf6f35726ac117fa7f27f9bf623ec44"},"cell_type":"markdown","source":"---\n\nWhat are the areas of the bounding boxes by gender?"},{"metadata":{"trusted":true,"_uuid":"a6eafb1f6945554c6d9044b7ac659a1f8362aa39"},"cell_type":"code","source":"areas = stage_1_train_labels.dropna(subset=['area'])\ng = sns.FacetGrid(hue='gender', data=areas, height=9, palette=dict(F=\"red\", M=\"blue\"), aspect=1.4)\n_ = g.map(sns.distplot, 'area', hist_kws={'alpha': 0.5}).add_legend()\n_ = g.fig.suptitle('What are the areas of the bounding boxes by gender?', y=1.01)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"809b6fe0eccc08fa4eb38e05a17caeffa33d686b"},"cell_type":"markdown","source":"Here we see that females have smaller sized pneumonia bounding boxes. This is possible because [females generally have lung volumes that are 10%-12% less than that of males](https://www.atsjournals.org/doi/pdf/10.1164/rccm.200208-876OC)."},{"metadata":{"_uuid":"3549dad355404b7d97e713f58203e1990b3b4a9b"},"cell_type":"markdown","source":"### Pneumonia Locations <a id='locations'></a>"},{"metadata":{"_uuid":"b0567a0b824faf994d25aa81b1c040816130b4f7"},"cell_type":"markdown","source":"Where is the centroid of the bounding boxes of pneumonia located?"},{"metadata":{"trusted":true,"_uuid":"69fb608dfcc15f94593c22f285bdb79a4a651484","scrolled":true},"cell_type":"code","source":"centers = (stage_1_train_labels.dropna(subset=['x'])\n           .assign(center_x=stage_1_train_labels.x + stage_1_train_labels.width / 2, \n                   center_y=stage_1_train_labels.y + stage_1_train_labels.height / 2))\nax = sns.jointplot(\"center_x\", \"center_y\", data=centers, height=6, alpha=0.03, color=\"red\")\n_ = ax.fig.suptitle(\"Where is Pneumonia located?\", y=1.01)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4bf359267e9dbcd272c8db7607610e19ebfe0ca3"},"cell_type":"markdown","source":"Without overlaying this over the mean lung outline it is difficult to make any strong conclusions from this, as the lungs may not be centered on all images. It would be helpful to recalculate the centroid in relation to each lung after segmentation."},{"metadata":{"_uuid":"54c29ca7ddf78baf1c77aeb4fd12efbdb215c447"},"cell_type":"markdown","source":"---\n\nHow is the pixel spacing distributed?"},{"metadata":{"trusted":true,"_uuid":"b8cf293ab636cdedd3d33e400548eedcfc11c865"},"cell_type":"code","source":"pixel_vc = stage_1_train_labels.drop_duplicates('patientId')['pixel_spacing'].value_counts()\npixel_vc.iloc[4] += pixel_vc.iloc[5] # combine into one as near identical values\nax = pixel_vc.iloc[0:5].plot.barh()\n_ = ax.set_yticklabels([f'{ps:.3f}' for ps in pixel_vc.index[:6]])\n_ = ax.set_xlabel('Count')\n_ = ax.set_ylabel('Pixel Spacing')\n_ = ax.set_title('How is the pixel spacing distributed?')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d69a7eefad38300ec0bec822f73bbd1350349079"},"cell_type":"markdown","source":"---\n\nHow are the bounding box areas distributed by the number of boxes?"},{"metadata":{"trusted":true,"_uuid":"bb7215cc8dd9caa0044a701219f7858d5c51f165"},"cell_type":"code","source":"areas_with_count = areas.merge(pd.DataFrame(boxes_per_patient).rename(columns={'Target': 'bbox_count'}), \n                               on='patientId')\ng = sns.FacetGrid(hue='bbox_count', data=areas_with_count, height=8, aspect=1.4)\n_ = g.map(sns.distplot, 'area').add_legend()\n_ = g.fig.suptitle(\"How are the bounding box areas distributed by the number of boxes?\", y=1.01)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c31ba6cdfffd08bd76f905c7f8df6a5acf40b337"},"cell_type":"markdown","source":"---\n\nWhere are the outliers? <a id='gaussianmixture'></a>"},{"metadata":{"trusted":true,"_uuid":"35c2b79c41c088d83f34ed2a39ab8d5262a13742"},"cell_type":"code","source":"plt.rcParams['figure.figsize'] = [10, 6]\nclf = GaussianMixture(n_components=2)\nclf.fit(centers[['center_x', 'center_y']])\ncenter_probs = clf.predict_proba(centers[['center_x', 'center_y']])\nZ = -clf.score_samples(centers[['center_x', 'center_y']])\noutliers = centers.iloc[Z > 17]\nfig, ax = plt.subplots()\ncenters.plot.scatter('center_x', 'center_y', c=Z, alpha=0.07, cmap='viridis', ax=ax)\noutliers.plot.scatter('center_x', 'center_y', c='red', marker='x', s=100, ax=ax)\n_ = ax.set_title('Where are the outliers?', fontsize=18)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5f3b8ae968d3f30feecc5ecaef0c2a4626b0519b"},"cell_type":"markdown","source":"Again, this is difficult to interpret without normalizing the location of the lungs in each image."},{"metadata":{"_uuid":"cdcba24e13bb134a11164a3b99a1f2801d85f8af"},"cell_type":"markdown","source":"---\n\nWhat do the outliers look like? <a id='patches'></a>"},{"metadata":{"trusted":true,"_uuid":"6caf604a850248556d0eaf83a047293483ffa42a"},"cell_type":"code","source":"def get_image(patientId, root_dir='../input/stage_1_train_images/'):\n    fn = os.path.join(root_dir, f'{patientId}.dcm')\n    dcm_data = pydicom.read_file(fn)\n    return dcm_data.pixel_array\n\ndef draw_bbs(bbs, ax):\n    for bb in bbs.itertuples():\n        rect = patches.Rectangle(\n            (bb.x, bb.y), bb.width, bb.height,\n            linewidth=2, edgecolor='red', facecolor='none')\n        ax.add_patch(rect)\n\ndef draw_image(img, bbs, ax):\n    ax.imshow(img, cmap='gray')\n    ax.grid(False)\n    ax.set_xticks([])\n    ax.set_yticks([])\n    if bbs is not None:\n        draw_bbs(bbs, ax)\n\noutliers_15 = outliers.drop_duplicates(subset=['patientId']).iloc[:15]\nfig, axes = plt.subplots(3, 5)\nfor row, ax in zip(outliers_15.itertuples(), axes.flatten()):\n    img = get_image(row.patientId)\n    bbs = stage_1_train_labels.loc[stage_1_train_labels.patientId == row.patientId, ['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-0.5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7b07ac1a1756b3ae9bdb8664722b85b3a99d9cde"},"cell_type":"markdown","source":"Aha! As feared, there are images that are cropped and off-center. One thing we notice about those is that the have a large quantity of black pixels resulting from the cropping. Let's look into that."},{"metadata":{"_uuid":"de1523bd2b5b977d52f84351af6ad488f0051ef7"},"cell_type":"markdown","source":"---\n\nWhat is the distribution of black pixels saturation in the images? How many have more than 10% of images comprised of black pixels?"},{"metadata":{"trusted":true,"_uuid":"f873befa7792219ee46b1ac926b0ac30c50a30c3"},"cell_type":"code","source":"plt.rcParams['figure.figsize'] = [12, 4]\nax = sns.distplot(stage_1_train_labels.mean_black_pixels)\n_ = ax.set_xlabel('Percentage of black pixels')\n_ = ax.set_title('Are there images with mostly black pixels?')\nprint(\"Images with more than 10% black pixels: \", len(stage_1_train_labels[stage_1_train_labels.mean_black_pixels > 0.1]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2795df694f42bc25f06fceac461c56936071e1b2"},"cell_type":"markdown","source":"---\n\nWhat do the images with mostly black pixels look like?"},{"metadata":{"trusted":true,"_uuid":"f7090e8912b0720a496fc2ee7b5a9599af3fc68a"},"cell_type":"code","source":"high_black_pixel_patientIds = stage_1_train_labels.loc[stage_1_train_labels.mean_black_pixels > 0.1, \n                                                       'patientId'].drop_duplicates()\nfig, axes = plt.subplots(4, 5)\nfor i, (patient_id, ax) in enumerate(zip(high_black_pixel_patientIds, axes.flatten())):\n    row = stage_1_train_labels.loc[stage_1_train_labels.patientId == patient_id]\n    img = get_image(row.patientId.iloc[0])\n    bbs = row[['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9d61ee2f50c9bdd3268d3620553bbc6644145351"},"cell_type":"markdown","source":"---\n\nWhile we're at it, what do the images with mostly white pixels look like?"},{"metadata":{"trusted":true,"_uuid":"9e581a0f6fc0e94977d22cf67fa45df41edfa15b"},"cell_type":"code","source":"high_white_pixel_patientIds = stage_1_train_labels.loc[stage_1_train_labels.mean_black_pixels < 0.000001, 'patientId'].drop_duplicates()\nfig, axes = plt.subplots(4, 5)\nfor patient_id, ax in zip(high_white_pixel_patientIds, axes.flatten()):\n    row = stage_1_train_labels.loc[stage_1_train_labels.patientId == patient_id]\n    img = get_image(row.patientId.iloc[0])\n    bbs = row[['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"58e18ffaabd0d3b2457dd8e903688f37b6063ce8"},"cell_type":"markdown","source":"These ones are generally full frame. Some contain pneumonia opacities and others contain other conditions."},{"metadata":{"_uuid":"40f74acdba739c2afd7e9ead9294a119f8c3ea59"},"cell_type":"markdown","source":"---\n\nCan traditional image processing find a bounding box in the cropped images? (<a href='#skimage'>here</a>)"},{"metadata":{"trusted":true,"_uuid":"bd13133da403e18d4a87341a57014b68a0b346b5"},"cell_type":"code","source":"high_black_pixel_images = np.empty(shape=(high_black_pixel_patientIds.shape[0], 1024, 1024))\n\nfor i, patient_id in enumerate(high_black_pixel_patientIds):\n    row = stage_1_train_labels.loc[stage_1_train_labels.patientId == patient_id]\n    img = get_image(row.patientId.iloc[0])\n    high_black_pixel_images[i] = img \n    \nhigh_black_pixel_contours = []\nfor img in high_black_pixel_images:\n    img2 = feature.canny(img != 0)\n    img2 = morphology.convex_hull_image(img2)\n    c = measure.find_contours(img2, 0)[0]\n    c = measure.approximate_polygon(c, 20)\n    high_black_pixel_contours.append(c)\n\nfig, axes = plt.subplots(4, 5)\ncontours = []\nfor c, img, ax in zip(high_black_pixel_contours, high_black_pixel_images, axes.flatten()):\n    draw_image(img, None, ax)\n    _ = ax.plot(c[:, 1], c[:, 0], '-b', linewidth=4)\nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"38a2459c75620bf4572ed7c8df930ea6249c9edd"},"cell_type":"markdown","source":"---\n\nCan the bounding boxes be resized when cropping and resizing the cropped images?"},{"metadata":{"trusted":true,"_uuid":"c2fe1653ccfd0a4caf65b89760098a6379839be7"},"cell_type":"code","source":"def order_coordinates(coords):\n    \"\"\"Returns coordinates with order:\n    (top left, top right, bottom right, bottom left)\n    \"\"\"\n    coords = coords[:-1]\n    output = np.empty((4, 2), dtype=np.float32)\n    dists = coords[:, 1]**2 + coords[:, 0]**2\n    ratios = coords[:, 1]/np.sqrt(dists)\n    \n    tl = coords[np.argmin(dists)]\n    br = coords[np.argmax(dists)]\n    \n    tr = coords[np.argmax(ratios)]\n    bl = coords[np.argmin(ratios)]\n    \n    output[0] = tl\n    output[1] = tr\n    output[2] = br\n    output[3] = bl\n    \n    return output[:,::-1]\n\ndef _convert_bb(bb, tfm):\n    x, y, w, h = bb.x, bb.y, bb.width, bb.height\n    pts = np.array([\n        [x, y],\n        [x + w, y],\n        [x + w, y + h],\n        [x, y + h]\n    ])\n    new_pts = tfm.inverse(pts)\n    pts_min = np.min(new_pts, axis=0)\n    pts_max = np.max(new_pts, axis=0)\n    \n    x, y = pts_min\n    w, h = pts_max - pts_min\n    \n    return np.array([x, y, w, h])\n\ndef convert_bbs(bboxs, tfm):\n    output = np.empty_like(bboxs, dtype=np.float32)\n    \n    for i, bb in enumerate(bboxs.itertuples()):\n        output[i] = _convert_bb(bb, tfm)\n    \n    return pd.DataFrame(output, columns=['x', 'y', 'width', 'height'])\n\nfig, axes = plt.subplots(4, 2, figsize=(8, 10))\n\norig_coords = np.array([[0, 0], [1024, 0], [1024, 1024], [0, 1024]])\ninteresting_idices = [0, 2, 3, 17]\n\nfor i, (ax1, ax2) in zip(interesting_idices, axes):\n    patient_id = high_black_pixel_patientIds.iloc[i]\n    img = high_black_pixel_images[i]\n    contour = high_black_pixel_contours[i]\n    \n    row = stage_1_train_labels.loc[stage_1_train_labels.patientId == patient_id]\n    bbs = row[['x', 'y', 'width', 'height']]\n    ordered_coors = order_coordinates(contour)\n    tform = transform.estimate_transform('projective', orig_coords, ordered_coors)\n    img_t = transform.warp(img, tform, output_shape=(1024, 1024))\n    \n    new_bbs = convert_bbs(bbs, tform)\n    _ = draw_image(img, bbs, ax1)\n    _ = draw_image(img_t, new_bbs, ax2)\n    \nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b5a91ca7f9f9ab2205916b5572501198660e3df9"},"cell_type":"markdown","source":"---\n\nHow are the bounding box aspect ratios distributed?"},{"metadata":{"trusted":true,"_uuid":"6311a45dfbcd96ed4cb4859c2d39b37b2ccc77d2"},"cell_type":"code","source":"ax = sns.distplot(stage_1_train_labels['aspect_ratio'].dropna(), norm_hist=True)\n_ = ax.set_title(\"What does the distribution of bounding aspect ratios look like?\")\n_ = ax.set_xlabel(\"Aspect Ratio\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fb991d4bc135b4d9f4a48fe24c175f599f6d6d43"},"cell_type":"markdown","source":"---\n\nWhat does the images with a high aspect ratio look like?"},{"metadata":{"trusted":true,"_uuid":"758f3214057380f5b7168e9c8277f9a2550c05f8"},"cell_type":"code","source":"aspect_ratios = stage_1_train_labels['aspect_ratio'].dropna()\nhigh_aspect_ratio_tr = (stage_1_train_labels.iloc[aspect_ratios[aspect_ratios > aspect_ratios.quantile(q=0.99)].index]\n                          .drop_duplicates(['patientId']))\nfig, axes = plt.subplots(3, 5)\nfor row, ax in zip(high_aspect_ratio_tr.itertuples(), axes.flatten()):\n    img = get_image(row.patientId)\n    bbs = stage_1_train_labels.loc[stage_1_train_labels.patientId == row.patientId, ['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-0.5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a775d6b5274b8129c45dac051fbfa42aad510329"},"cell_type":"markdown","source":"---\n\nIs there a relationship between the bounding box's aspect ratio and area?"},{"metadata":{"trusted":true,"_uuid":"a2fdf1f84ddcca623f27a8445491ff1ae9688938"},"cell_type":"code","source":"g = sns.relplot(x='area', y='aspect_ratio', \n            data=stage_1_train_labels.dropna(subset=['area', 'aspect_ratio']), \n            height=8, alpha=0.8, aspect=1.4,)\n_ = g.fig.suptitle(\"Is there a relationship between the bounding box's aspect ratio and area?\", y=1.005)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a67beacc02d64a1c06a92b6d67ce769eb1a95336"},"cell_type":"markdown","source":"(return to <a href='#contents'>Table of Contents</a>)\n\n---\n---"},{"metadata":{"_uuid":"0c48059c23b878bdda691aa1f1944fd5e1b6371e"},"cell_type":"markdown","source":"## <span style=\"color:darkgreen\">Conclusion</span> <a id=\"conclusion\"></a>"},{"metadata":{"_uuid":"665dce57993a054125e473507b153ee1e1e8b6ad"},"cell_type":"markdown","source":"So there you have it. We've loaded the data, explored the files included, examined images to help us understand our goals and to brainstorm approaches, and run the provided model on the data to create our first submission file. Where do we go next? Well, there are several other models that have been shared in the kernels."},{"metadata":{"_uuid":"2a6c6b8e30eab0c0f4f4e151546fffbf606585c0"},"cell_type":"markdown","source":"### Other Kernels  <a id=\"kernels\"></a>\n\n*   [CNN with segmentation](https://www.kaggle.com/jonnedtc/cnn-segmentation-connected-components)\n*   [CheXNet](https://www.kaggle.com/ashishpatel26/chexnet-radiologist-level-pneumonia-detection)\n*   [NASNet](https://www.kaggle.com/ashishpatel26/beginner-tutorial-nasnet-pneumonia-detection)"},{"metadata":{"_uuid":"2a76b61cc39c6dd4339935b35e62a3ef013c08df"},"cell_type":"markdown","source":"You can also check out these excellent Medium articles by NoMonia team members:\n* [YOLO Object Detection Walkthrough for the RSNA Pneumonia Detection Challenge](https://medium.com/@hjhuney/yolo-object-detection-walkthrough-for-the-rsna-pneumonia-detection-challenge-123ec9a9adf2) by Jake Huneycutt: a walk-through to get a YOLOv3 model working locally\n* [Kaggle RSNA Pneumonia Detection Challenge Explained](https://medium.com/@sebastiannorena/c140b19bf903) by Sebastian Norena: ideas for improving upon the competition-provided starter kernel\n* [No-more-moaning…ia: A Journey in Medical Imaging](https://medium.com/@t7jackso/26951f901707) by Robert Jackson: An overview of the competition challenge with discussion on pneumonia, neural networks, and more"},{"metadata":{"_uuid":"1a8ad4c48a2d9f4d0fe1411c353106686d05b7d8"},"cell_type":"markdown","source":"### References <a id=\"refs\"></a>"},{"metadata":{"_uuid":"3eb40f4c4274ab39e5b00eb9bd17f4d12bf6c03b"},"cell_type":"markdown","source":"https://www.atsjournals.org/doi/pdf/10.1164/rccm.200208-876OC"},{"metadata":{"_uuid":"0b045b4f2a3b7d76b55c9a2c9bc51108685114ff"},"cell_type":"markdown","source":"(return to <a href='#contents'>Table of Contents</a>)\n\n---\n---"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}