{"cells":[{"metadata":{"_uuid":"de737f8a0a2e9fa172005cfa372f8a2d40b0cfef"},"cell_type":"markdown","source":"# Q&A with Only Pictuers! (RSNA Pneumonia Detection Challenge)\n\nThis kernel poses and answers questions about the dataset using images!\n\n## Questions\n\n- [Are the classes imbalanced?](#Are-the-classes-imbalanced?)\n- [How many cases are there per image?](#How-many-cases-are-there-per-image?)\n- [Where is Pneumonia located?](#Where-is-Pneumonia-located?)\n- [What is the age distribution by gender and target?](#What-is-the-age-distribution-by-gender-and-target?)\n- [What are the areas of the bounding boxes by gender?](#What-are-the-areas-of-the-bounding-boxes-by-gender?)\n- [How is the pixel spacing distributed?](#How-is-the-pixel-spacing-distributed?)\n- [How are the bounding box areas distributed by the number of boxes?](#How-are-the-bounding-box-areas-distributed-by-the-number-of-boxes?)\n- [Where are the outliers?](#Where-are-the-outliers?)\n- [What does the outliers look like?](#What-does-the-outliers-look-like?)\n- [Are there images with mostly black pixels?](#Are-there-images-with-mostly-black-pixels?)\n- [What does the mostly black pixels look like?](#What-does-the-mostly-black-pixels-look-like?)\n- [What does the mostly white pixel images look like?](#What-does-the-mostly-white-pixel-images-look-like?)\n- [Can tradiational image processing find a bounding box around the cropped images?](#Can-tradiational-image-processing-find-a-bounding-box-around-the-cropped-images?)\n- [Can the bounding boxes be resized when cropping and resizing the cropped images?](#Can-the-bounding-boxes-be-resized-when-cropping-and-resizing-the-cropped-images?)\n- [How are the bounding box aspect ratios distributed?](#How-are-the-bounding-box-aspect-ratios-distributed?)\n- [What does the images with a high aspect ratio look like?](#What-does-the-images-with-a-high-aspect-ratio-look-like?)\n- [Is there a relationship between the bounding box's aspect ratio and area?](#Is-there-a-relationship-between-the-bounding-box's-aspect-ratio-and-area?)\n\nMore questions to come! 🧐"},{"metadata":{"_uuid":"fc797afc430bf8d639b322e144cda532fdef371f"},"cell_type":"markdown","source":"## Load and Prepare data\n\nIn this section, the data is loaded and prepared for analysis."},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":false,"_kg_hide-output":false,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import glob\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport pandas as pd\nimport pydicom\nimport numpy as np\nimport warnings\nimport multiprocessing\nimport os\nfrom skimage import morphology\nfrom skimage import feature\nfrom skimage import measure\nfrom skimage import util\nfrom skimage import transform\n\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cdcf33271f5c8f5f10c71ad26dc5de5db0b09bda"},"cell_type":"markdown","source":"Most of objects used in the figures are defined in the next cell:"},{"metadata":{"_uuid":"df679db2020d6e1965bf76c13a87bc663e5b94fa","trusted":true},"cell_type":"code","source":"sns.set_style('darkgrid')\nsns.set_context('notebook', font_scale=1.2)\nplt.rcParams['figure.figsize'] = [14, 8]\nplt.rcParams['lines.linewidth'] = 2.5\n\n# Get all data\ntr = pd.read_csv('../input/stage_1_train_labels.csv')\ntr['aspect_ratio'] = (tr['width']/tr['height'])\ntr['area'] = tr['width'] * tr['height']\n\ndef get_info(patientId, root_dir='../input/stage_1_train_images/'):\n    fn = os.path.join(root_dir, f'{patientId}.dcm')\n    dcm_data = pydicom.read_file(fn)\n    return {'age': dcm_data.PatientAge, \n            'gender': dcm_data.PatientSex, \n            'id': os.path.basename(fn).split('.')[0],\n            'pixel_spacing': float(dcm_data.PixelSpacing[0]),\n            'mean_black_pixels': np.mean(dcm_data.pixel_array == 0)}\n\npatient_ids = list(tr.patientId.unique())\nwith multiprocessing.Pool(4) as pool:\n    result = pool.map(get_info, patient_ids)\n    \ndemo = pd.DataFrame(result)\ndemo['gender'] = demo['gender'].astype('category')\ndemo['age'] = demo['age'].astype(int)\n\ntr = (tr.merge(demo, left_on='patientId', right_on='id', how='left')\n        .drop(columns='id'))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4f753593f2c335a87a3b45079480561d827e6d51"},"cell_type":"markdown","source":"## Are the classes imbalanced?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"2908724f67853bde6faac9f7f70471d5616e9a98","trusted":true},"cell_type":"code","source":"boxes_per_patient = tr.groupby('patientId')['Target'].sum()\n\nax = (boxes_per_patient > 0).value_counts().plot.bar()\n_ = ax.set_title('Are the classes imbalanced?')\n_ = ax.set_xlabel('Has Pneumonia')\n_ = ax.set_ylabel('Count')\n_ = ax.xaxis.set_tick_params(rotation=0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e42931ebc5e5a7a258014fbb95cd8796dca1a690"},"cell_type":"markdown","source":"## How many cases are there per image?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"74444dabb9b2259876f75adc044238637fd0aca5","trusted":true},"cell_type":"code","source":"ax = boxes_per_patient.value_counts().plot.bar()\n_ = ax.set_title('How many cases are there per image?')\n_ = ax.set_xlabel('Number of cases')\n_ = ax.xaxis.set_tick_params(rotation=0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2c7c44e67bd69f5cb5bf1248d2ee6cdc7c5228ad"},"cell_type":"markdown","source":"## Where is Pneumonia located?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"46bf5332a09c14dd44ac54dfd1b444c504b12505","scrolled":false,"trusted":true},"cell_type":"code","source":"centers = (tr.dropna(subset=['x'])\n           .assign(center_x=tr.x + tr.width / 2, center_y=tr.y + tr.height / 2))\nax = sns.jointplot(\"center_x\", \"center_y\", data=centers, height=9, alpha=0.1)\n_ = ax.fig.suptitle(\"Where is Pneumonia located?\", y=1.01)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8184755df856611afa1159d85a47f3678a45b960"},"cell_type":"markdown","source":"## What is the age distribution by gender and target?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"1fe15d77a32fcb95ad89cd645d6c3794006dadbf","scrolled":false,"trusted":true},"cell_type":"code","source":"g = sns.FacetGrid(col='Target', hue='gender', \n                  data=tr.drop_duplicates(subset=['patientId']), \n                  height=9, palette=dict(F=\"red\", M=\"blue\"))\n_ = g.map(sns.distplot, 'age', hist_kws={'alpha': 0.3}).add_legend()\n_ = g.fig.suptitle(\"What is the age distribution by gender and target?\", y=1.02, fontsize=20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"89cb53e97403fd9b9cbc74cdec703d5ea95936a5"},"cell_type":"markdown","source":"## What are the areas of the bounding boxes by gender?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"880f4929aaccbd8a8f838cd7893b531e9caca722","trusted":true},"cell_type":"code","source":"areas = tr.dropna(subset=['area'])\ng = sns.FacetGrid(hue='gender', data=areas, height=9, palette=dict(F=\"red\", M=\"blue\"), aspect=1.4)\n_ = g.map(sns.distplot, 'area', hist_kws={'alpha': 0.3}).add_legend()\n_ = g.fig.suptitle('What are the areas of the bounding boxes by gender?', y=1.01)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"58ae531ef0cd5873603afc5f4d7e0b2cf39da989"},"cell_type":"markdown","source":"## How is the pixel spacing distributed?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"959d29e1d56ba2ad3034a44cad68f6cd8d9b1bee","trusted":true},"cell_type":"code","source":"pixel_vc = tr.drop_duplicates('patientId')['pixel_spacing'].value_counts()\nax = pixel_vc.iloc[:6].plot.bar()\n_ = ax.set_xticklabels([f'{ps:.4f}' for ps in pixel_vc.index[:6]])\n_ = ax.set_xlabel('Pixel Spacing')\n_ = ax.set_ylabel('Count')\n_ = ax.set_title('How is the pixel spacing distributed?', fontsize=20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bcf407ecefe0d03c5e58bd374a27b2f0fa783e70"},"cell_type":"markdown","source":"## How are the bounding box areas distributed by the number of boxes?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"e938b2414c68ba9a0c07d35ef14909fc27262d08","scrolled":false,"trusted":true},"cell_type":"code","source":"areas_with_count = areas.merge(pd.DataFrame(boxes_per_patient).rename(columns={'Target': 'bbox_count'}), \n                               on='patientId')\ng = sns.FacetGrid(hue='bbox_count', data=areas_with_count, height=8, aspect=1.4)\n_ = g.map(sns.distplot, 'area').add_legend()\n_ = g.fig.suptitle(\"How are the bounding box areas distributed by the number of boxes?\", y=1.01)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5e00eefb6bf87b8cb37bcf801bdd82ba53b373b0"},"cell_type":"markdown","source":"## Where are the outliers?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"98675cd17709edd521eab8c8a5e66285729a0730","trusted":true},"cell_type":"code","source":"from sklearn.mixture import GaussianMixture\nclf = GaussianMixture(n_components=2)\nclf.fit(centers[['center_x', 'center_y']])\ncenter_probs = clf.predict_proba(centers[['center_x', 'center_y']])\nZ = -clf.score_samples(centers[['center_x', 'center_y']])\noutliers = centers.iloc[Z > 17]\nfig, ax = plt.subplots()\ncenters.plot.scatter('center_x', 'center_y', c=Z, alpha=0.5, cmap='viridis', ax=ax)\noutliers.plot.scatter('center_x', 'center_y', c='red', marker='x', s=100, ax=ax)\n_ = ax.set_title('Where are the outliers?', fontsize=18)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e75f0af250ba18e584c5f04ac69679208369b11b"},"cell_type":"markdown","source":"## What does the outliers look like?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"8381bf4e8aee7499eb8274f656302c8324902067","scrolled":false,"trusted":true},"cell_type":"code","source":"import matplotlib.patches as patches\n\ndef get_image(patientId, root_dir='../input/stage_1_train_images/'):\n    fn = os.path.join(root_dir, f'{patientId}.dcm')\n    dcm_data = pydicom.read_file(fn)\n    return dcm_data.pixel_array\n\ndef draw_bbs(bbs, ax):\n    for bb in bbs.itertuples():\n        rect = patches.Rectangle(\n            (bb.x, bb.y), bb.width, bb.height,\n            linewidth=2, edgecolor='red', facecolor='none')\n        ax.add_patch(rect)\n\ndef draw_image(img, bbs, ax):\n    ax.imshow(img, cmap='gray')\n    ax.grid(False)\n    ax.set_xticks([])\n    ax.set_yticks([])\n    if bbs is not None:\n        draw_bbs(bbs, ax)\n\noutliers_15 = outliers.drop_duplicates(subset=['patientId']).iloc[:15]\nfig, axes = plt.subplots(3, 5)\nfor row, ax in zip(outliers_15.itertuples(), axes.flatten()):\n    img = get_image(row.patientId)\n    bbs = tr.loc[tr.patientId == row.patientId, ['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-0.5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6bdd8ede1225f09d93ee0521cf735ebfc0d120f1"},"cell_type":"markdown","source":"## Are there images with mostly black pixels?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"4e5e334a53f0c6b302e516bffe4f115f3960db2e","scrolled":false,"trusted":true},"cell_type":"code","source":"ax = sns.boxplot(tr.mean_black_pixels)\n_ = ax.set_xlabel('Percentage of black pixels')\n_ = ax.set_title('Are there images with mostly black pixels?')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"287b560447fe91c30b85c5dd4c9e8d57cef0c50f"},"cell_type":"markdown","source":"## What does the mostly black pixels look like?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"86fc18c9e01b4386eb9fe70883cea1ec98ae6151","scrolled":false,"trusted":true},"cell_type":"code","source":"high_black_pixel_patientIds = tr.loc[tr.mean_black_pixels > 0.55, 'patientId'].drop_duplicates()\nfig, axes = plt.subplots(4, 5)\nfor i, (patient_id, ax) in enumerate(zip(high_black_pixel_patientIds, axes.flatten())):\n    row = tr.loc[tr.patientId == patient_id]\n    img = get_image(row.patientId.iloc[0])\n    bbs = row[['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"62320eae9e5ad0d66d283b4cfabe6cc579dc44da"},"cell_type":"markdown","source":"## What does the mostly white pixel images look like?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"df54c0d5695e743d2a70973ce8698b83fb498a5d","trusted":true},"cell_type":"code","source":"high_white_pixel_patientIds = tr.loc[tr.mean_black_pixels < 0.000001, 'patientId'].drop_duplicates()\nfig, axes = plt.subplots(4, 5)\nfor patient_id, ax in zip(high_white_pixel_patientIds, axes.flatten()):\n    row = tr.loc[tr.patientId == patient_id]\n    img = get_image(row.patientId.iloc[0])\n    bbs = row[['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4972448bacf9530f13a2f25d774dea8bc4b000cb"},"cell_type":"markdown","source":"## Can tradiational image processing find a bounding box around the cropped images?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"933428aa4b35004dd044809cbcf13474e94d2a5a","trusted":true},"cell_type":"code","source":"high_black_pixel_images = np.empty(shape=(high_black_pixel_patientIds.shape[0], 1024, 1024))\n\nfor i, patient_id in enumerate(high_black_pixel_patientIds):\n    row = tr.loc[tr.patientId == patient_id]\n    img = get_image(row.patientId.iloc[0])\n    high_black_pixel_images[i] = img \n    \nhigh_black_pixel_contours = []\nfor img in high_black_pixel_images:\n    img2 = feature.canny(img != 0)\n    img2 = morphology.convex_hull_image(img2)\n    c = measure.find_contours(img2, 0)[0]\n    c = measure.approximate_polygon(c, 20)\n    high_black_pixel_contours.append(c)\n\nfig, axes = plt.subplots(4, 5)\ncontours = []\nfor c, img, ax in zip(high_black_pixel_contours, high_black_pixel_images, axes.flatten()):\n    draw_image(img, None, ax)\n    _ = ax.plot(c[:, 1], c[:, 0], '-b', linewidth=4)\nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1df3a97fa16f5016391e2acfc7cbd15aa463b2dc"},"cell_type":"markdown","source":"## Can the bounding boxes be resized when cropping and resizing the cropped images?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"_uuid":"c1d69c15918f0ed47cd12a83d6beb2e81159dfef","trusted":true},"cell_type":"code","source":"def order_coordinates(coords):\n    \"\"\"Returns coordinates with order:\n    (top left, top right, bottom right, bottom left)\n    \"\"\"\n    coords = coords[:-1]\n    output = np.empty((4, 2), dtype=np.float32)\n    dists = coords[:, 1]**2 + coords[:, 0]**2\n    ratios = coords[:, 1]/np.sqrt(dists)\n    \n    tl = coords[np.argmin(dists)]\n    br = coords[np.argmax(dists)]\n    \n    tr = coords[np.argmax(ratios)]\n    bl = coords[np.argmin(ratios)]\n    \n    output[0] = tl\n    output[1] = tr\n    output[2] = br\n    output[3] = bl\n    \n    return output[:,::-1]\n\ndef _convert_bb(bb, tfm):\n    x, y, w, h = bb.x, bb.y, bb.width, bb.height\n    pts = np.array([\n        [x, y],\n        [x + w, y],\n        [x + w, y + h],\n        [x, y + h]\n    ])\n    new_pts = tfm.inverse(pts)\n    pts_min = np.min(new_pts, axis=0)\n    pts_max = np.max(new_pts, axis=0)\n    \n    x, y = pts_min\n    w, h = pts_max - pts_min\n    \n    return np.array([x, y, w, h])\n\ndef convert_bbs(bboxs, tfm):\n    output = np.empty_like(bboxs, dtype=np.float32)\n    \n    for i, bb in enumerate(bboxs.itertuples()):\n        output[i] = _convert_bb(bb, tfm)\n    \n    return pd.DataFrame(output, columns=['x', 'y', 'width', 'height'])\n\nfig, axes = plt.subplots(4, 2, figsize=(8, 10))\n\norig_coords = np.array([[0, 0], [1024, 0], [1024, 1024], [0, 1024]])\ninteresting_idices = [0, 2, 3, 17]\n\nfor i, (ax1, ax2) in zip(interesting_idices, axes):\n    patient_id = high_black_pixel_patientIds.iloc[i]\n    img = high_black_pixel_images[i]\n    contour = high_black_pixel_contours[i]\n    \n    row = tr.loc[tr.patientId == patient_id]\n    bbs = row[['x', 'y', 'width', 'height']]\n    ordered_coors = order_coordinates(contour)\n    tform = transform.estimate_transform('projective', orig_coords, ordered_coors)\n    img_t = transform.warp(img, tform, output_shape=(1024, 1024))\n    \n    new_bbs = convert_bbs(bbs, tform)\n    _ = draw_image(img, bbs, ax1)\n    _ = draw_image(img_t, new_bbs, ax2)\n    \nfig.tight_layout(pad=-1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dbebc554a7c71437614cea642e5e0d7dc1886c70"},"cell_type":"markdown","source":"## How are the bounding box aspect ratios distributed?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"74773dc8ba23493a6f23c5b9bdeb4f2f35d0c88a"},"cell_type":"code","source":"ax = sns.distplot(tr['aspect_ratio'].dropna(), norm_hist=True)\n_ = ax.set_title(\"What does the distribution of bounding aspect ratios look like?\")\n_ = ax.set_xlabel(\"Aspect Ratio\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e46ceb9907710b119ef55640b52761094fb278ea"},"cell_type":"markdown","source":"## What does the images with a high aspect ratio look like?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"cb2f0b23323a1b2bfdb41499aca15fe2c4288b13"},"cell_type":"code","source":"aspect_ratios = tr['aspect_ratio'].dropna()\nhigh_aspect_ratio_tr = (tr.iloc[aspect_ratios[aspect_ratios > aspect_ratios.quantile(q=0.99)].index]\n                          .drop_duplicates(['patientId']))\nfig, axes = plt.subplots(3, 5)\nfor row, ax in zip(high_aspect_ratio_tr.itertuples(), axes.flatten()):\n    img = get_image(row.patientId)\n    bbs = tr.loc[tr.patientId == row.patientId, ['x', 'y', 'width', 'height']]\n    draw_image(img, bbs, ax)\nfig.tight_layout(pad=-0.5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"72a1ef44daf753b3542b6a65528a13a821b0340b"},"cell_type":"markdown","source":"## Is there a relationship between the bounding box's aspect ratio and area?\n[Back to top](#Questions)"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"29fa36f6de9295950f0591e6fa6d7a2677ebff4a"},"cell_type":"code","source":"g = sns.relplot(x='area', y='aspect_ratio', \n            data=tr.dropna(subset=['area', 'aspect_ratio']), \n            height=8, alpha=0.8, aspect=1.4,)\n_ = g.fig.suptitle(\"Is there a relationship between the bounding box's aspect ratio and area?\", y=1.005)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"661ee2caa1ca4006fcf7022bd1bed87610cc0ffc"},"cell_type":"markdown","source":"### More questions to come! 🧐"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}