{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# HuBMAP - Exploratory Data Analysis\n\nQuick Exploratory Data Analysis for [HuBMAP: Hacking the Kidney](https://www.kaggle.com/c/hubmap-kidney-segmentation) challenge\n\nThe HuBMAP data used in this hackathon includes 11 fresh frozen and 9 Formalin Fixed Paraffin Embedded (FFPE) PAS kidney images. Glomeruli FTU annotations exist for all 20 tissue samples; some of these will be shared for training, and others will be used to judge submissions.","metadata":{}},{"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-competitions/kaggle/22990/logos/header.png)","metadata":{}},{"cell_type":"markdown","source":"<a id=\"top\"></a>\n\n<div class=\"list-group\" id=\"list-tab\" role=\"tablist\">\n<h3 class=\"list-group-item list-group-item-action active\" data-toggle=\"list\" style='color:white; background:#EAA6D1; border:0' role=\"tab\" aria-controls=\"home\"><center>Quick Navigation</center></h3>\n\n* [1. Basic Data Exploration](#1)\n* [2. Image and Masks Visualizations](#2)\n* [3. Metadata Analysis](#3)","metadata":{}},{"cell_type":"code","source":"!pip install -q -U pip\n!pip install -q -U seaborn","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:42:19.212146Z","iopub.execute_input":"2023-05-23T20:42:19.212516Z","iopub.status.idle":"2023-05-23T20:42:37.758569Z","shell.execute_reply.started":"2023-05-23T20:42:19.212486Z","shell.execute_reply":"2023-05-23T20:42:37.757001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"1\"></a>\n<h2 style='background:#EAA6D1; border:0; color:white'><center>Basic Data Exploration<center><h2>","metadata":{}},{"cell_type":"code","source":"import os\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sn\nimport cv2\nimport tifffile","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-05-23T20:42:37.761245Z","iopub.execute_input":"2023-05-23T20:42:37.761653Z","iopub.status.idle":"2023-05-23T20:42:38.664467Z","shell.execute_reply.started":"2023-05-23T20:42:37.761610Z","shell.execute_reply":"2023-05-23T20:42:38.663457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BASE_PATH = \"../input/hubmap-kidney-segmentation/\"\nTRAIN_PATH = os.path.join(BASE_PATH, \"train\")\n\nprint(os.listdir(BASE_PATH))","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:42:38.665861Z","iopub.execute_input":"2023-05-23T20:42:38.666187Z","iopub.status.idle":"2023-05-23T20:42:38.672803Z","shell.execute_reply.started":"2023-05-23T20:42:38.666156Z","shell.execute_reply":"2023-05-23T20:42:38.671914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train masks","metadata":{}},{"cell_type":"markdown","source":"**train.csv** contains the unique IDs for each image, as well as an RLE-encoded representation of the mask for the objects in the image. See the evaluation tab for details of the RLE encoding scheme.","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_csv(\n    os.path.join(BASE_PATH, \"train.csv\")\n)\ndf_train","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:42:38.674046Z","iopub.execute_input":"2023-05-23T20:42:38.674345Z","iopub.status.idle":"2023-05-23T20:42:39.234628Z","shell.execute_reply.started":"2023-05-23T20:42:38.674317Z","shell.execute_reply":"2023-05-23T20:42:39.233832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Submission df","metadata":{}},{"cell_type":"code","source":"df_sub = pd.read_csv(\n    os.path.join(BASE_PATH, \"sample_submission.csv\"))\ndf_sub","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:42:39.238383Z","iopub.execute_input":"2023-05-23T20:42:39.238686Z","iopub.status.idle":"2023-05-23T20:42:39.257922Z","shell.execute_reply.started":"2023-05-23T20:42:39.238658Z","shell.execute_reply":"2023-05-23T20:42:39.257167Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Number of samples","metadata":{}},{"cell_type":"code","source":"print(f\"Number of train images: {df_train.shape[0]}\")\nprint(f\"Number of test images: {df_sub.shape[0]}\")","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:42:39.259412Z","iopub.execute_input":"2023-05-23T20:42:39.259681Z","iopub.status.idle":"2023-05-23T20:42:39.264390Z","shell.execute_reply.started":"2023-05-23T20:42:39.259655Z","shell.execute_reply":"2023-05-23T20:42:39.263469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Train and test metadata","metadata":{}},{"cell_type":"markdown","source":"**HuBMAP-20-dataset_information.csv** contains additional information (including anonymized patient data) about each image.","metadata":{}},{"cell_type":"code","source":"df_info = pd.read_csv(\n    os.path.join(BASE_PATH, \"HuBMAP-20-dataset_information.csv\")\n)\ndf_info.sample(3)","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:42:39.266100Z","iopub.execute_input":"2023-05-23T20:42:39.266496Z","iopub.status.idle":"2023-05-23T20:42:39.299852Z","shell.execute_reply.started":"2023-05-23T20:42:39.266457Z","shell.execute_reply":"2023-05-23T20:42:39.299043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Utility functions","metadata":{}},{"cell_type":"code","source":"# https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode\ndef rle2mask(mask_rle, shape):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (width,height) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n\n    '''\n    s = mask_rle.split()\n    starts, lengths = [\n        np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])\n    ]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo : hi] = 1\n    return img.reshape(shape).T\n\n\ndef read_image(image_id, scale=None, verbose=1):\n    image = tifffile.imread(\n        os.path.join(BASE_PATH, f\"train/{image_id}.tiff\")\n    )\n    print(image.shape)\n    if len(image.shape) == 5 or image.shape[0] == 3:\n        image = image.squeeze().transpose(1, 2, 0)\n    \n    mask = rle2mask(\n        df_train[df_train[\"id\"] == image_id][\"encoding\"].values[0], \n        (image.shape[1], image.shape[0])\n    )\n    \n    if verbose:\n        print(f\"[{image_id}] Image shape: {image.shape}\")\n        print(f\"[{image_id}] Mask shape: {mask.shape}\")\n    \n    if scale:\n        new_size = (image.shape[1] // scale, image.shape[0] // scale)\n        image = cv2.resize(image, new_size)\n        mask = cv2.resize(mask, new_size)\n        \n        if verbose:\n            print(f\"[{image_id}] Resized Image shape: {image.shape}\")\n            print(f\"[{image_id}] Resized Mask shape: {mask.shape}\")\n        \n    return image, mask\n\n\ndef read_test_image(image_id, scale=None, verbose=1):\n    image = tifffile.imread(\n        os.path.join(BASE_PATH, f\"test/{image_id}.tiff\")\n    )\n    if len(image.shape) == 5 or image.shape[0] == 3:\n        image = image.squeeze().transpose(1, 2, 0)\n    \n    if verbose:\n        print(f\"[{image_id}] Image shape: {image.shape}\")\n    \n    if scale:\n        new_size = (image.shape[1] // scale, image.shape[0] // scale)\n        image = cv2.resize(image, new_size)\n        \n        if verbose:\n            print(f\"[{image_id}] Resized Image shape: {image.shape}\")\n        \n    return image\n\n\ndef plot_image_and_mask(image, mask, image_id):\n    plt.figure(figsize=(16, 10))\n    \n    plt.subplot(1, 3, 1)\n    plt.imshow(image)\n    plt.title(f\"Image {image_id}\", fontsize=18)\n    \n    plt.subplot(1, 3, 2)\n    plt.imshow(image)\n    plt.imshow(mask, cmap=\"hot\", alpha=0.5)\n    plt.title(f\"Image {image_id} + mask\", fontsize=18)    \n    \n    plt.subplot(1, 3, 3)\n    plt.imshow(mask, cmap=\"hot\")\n    plt.title(f\"Mask\", fontsize=18)    \n    \n    plt.show()\n    \n    \ndef plot_grid_image_with_mask(image, mask):\n    plt.figure(figsize=(16, 16))\n    \n    w_len = image.shape[0]\n    h_len = image.shape[1]\n    \n    min_len = min(w_len, h_len)\n    w_start = (w_len - min_len) // 2\n    h_start = (h_len - min_len) // 2\n    \n    plt.imshow(image[w_start : w_start + min_len, h_start : h_start + min_len])\n    plt.imshow(\n        mask[w_start : w_start + min_len, h_start : h_start + min_len], cmap=\"hot\", alpha=0.5,\n    )\n    plt.axis(\"off\")\n            \n    plt.show()\n    \n\ndef plot_slice_image_and_mask(image, mask, start_h, end_h, start_w, end_w):\n    plt.figure(figsize=(16, 5))\n    \n    sub_image = image[start_h:end_h, start_w:end_w, :]\n    sub_mask = mask[start_h:end_h, start_w:end_w]\n    \n    plt.subplot(1, 3, 1)\n    plt.imshow(sub_image)\n    plt.axis(\"off\")\n    \n    plt.subplot(1, 3, 2)\n    plt.imshow(sub_image)\n    plt.imshow(sub_mask, cmap=\"hot\", alpha=0.5)\n    plt.axis(\"off\")\n    \n    plt.subplot(1, 3, 3)\n    plt.imshow(sub_mask, cmap=\"hot\")\n    plt.axis(\"off\")\n    \n    plt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:42:39.301191Z","iopub.execute_input":"2023-05-23T20:42:39.301458Z","iopub.status.idle":"2023-05-23T20:42:39.331463Z","shell.execute_reply.started":"2023-05-23T20:42:39.301432Z","shell.execute_reply":"2023-05-23T20:42:39.330477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"2\"></a>\n<h2 style='background:#EAA6D1; border:0; color:white'><center>Image and Masks Visualizations<center><h2>","metadata":{}},{"cell_type":"code","source":"small_ids = [\n    \"0486052bb\", \"095bf7a1f\", \"1e2425f28\", \"2f6ecfcdf\",\n    \"54f2eec69\", \"aaa6a05cc\", \"cb2d976f4\", \"e79de561c\",\n]\nsmall_images = []\nsmall_masks = []\n\nfor small_id in small_ids:\n    tmp_image, tmp_mask = read_image(small_id, scale=20, verbose=1)\n    small_images.append(tmp_image)\n    small_masks.append(tmp_mask)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:42:39.332386Z","iopub.execute_input":"2023-05-23T20:42:39.332646Z","iopub.status.idle":"2023-05-23T20:42:58.355379Z","shell.execute_reply.started":"2023-05-23T20:42:39.332611Z","shell.execute_reply":"2023-05-23T20:42:58.354141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train images","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(16, 16))\nfor ind, (tmp_id, tmp_image) in enumerate(zip(small_ids, small_images)):\n    plt.subplot(3, 3, ind + 1)\n    plt.imshow(tmp_image)\n    plt.axis(\"off\")","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:42:58.356217Z","iopub.status.idle":"2023-05-23T20:42:58.356644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train images + masks","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(16, 16))\nfor ind, (tmp_id, tmp_image, tmp_mask) in enumerate(zip(small_ids, small_images, small_masks)):\n    plt.subplot(3, 3, ind + 1)\n    plt.imshow(tmp_image)\n    plt.imshow(tmp_mask, cmap=\"hot\", alpha=0.5)\n    plt.axis(\"off\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:42:58.357708Z","iopub.status.idle":"2023-05-23T20:42:58.358131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"small_ids = [\n    \"2ec3f1bb9\", \"3589adb90\", \"57512b7f1\", \"aa05346ff\", \"d488c759a\",\n]\nsmall_images = []\n\nfor small_id in small_ids:\n    tmp_image = read_test_image(small_id, scale=20, verbose=1)\n    small_images.append(tmp_image)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:43:06.400589Z","iopub.execute_input":"2023-05-23T20:43:06.401002Z","iopub.status.idle":"2023-05-23T20:46:34.589546Z","shell.execute_reply.started":"2023-05-23T20:43:06.400942Z","shell.execute_reply":"2023-05-23T20:46:34.588153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Test images","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(16, 11))\nfor ind, (tmp_id, tmp_image) in enumerate(zip(small_ids, small_images)):\n    plt.subplot(2, 3, ind + 1)\n    plt.imshow(tmp_image)\n    plt.axis(\"off\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:46:34.592285Z","iopub.execute_input":"2023-05-23T20:46:34.592638Z","iopub.status.idle":"2023-05-23T20:46:36.481347Z","shell.execute_reply.started":"2023-05-23T20:46:34.592601Z","shell.execute_reply":"2023-05-23T20:46:36.480567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 0486052bb","metadata":{}},{"cell_type":"code","source":"image_id = \"0486052bb\"\nimage, mask = read_image(image_id, 2)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:46:36.482441Z","iopub.execute_input":"2023-05-23T20:46:36.482853Z","iopub.status.idle":"2023-05-23T20:46:56.809225Z","shell.execute_reply.started":"2023-05-23T20:46:36.482824Z","shell.execute_reply":"2023-05-23T20:46:56.808103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_image_and_mask(image, mask, image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:46:56.810663Z","iopub.execute_input":"2023-05-23T20:46:56.811275Z","iopub.status.idle":"2023-05-23T20:47:32.182741Z","shell.execute_reply.started":"2023-05-23T20:46:56.811229Z","shell.execute_reply":"2023-05-23T20:47:32.182016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_slice_image_and_mask(image, mask, 5000, 7500, 2500, 5000)\nplot_slice_image_and_mask(image, mask, 5250, 5720, 3500, 4000)\nplot_slice_image_and_mask(image, mask, 5375, 5575, 3650, 3850)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:47:32.185190Z","iopub.execute_input":"2023-05-23T20:47:32.185611Z","iopub.status.idle":"2023-05-23T20:47:34.860374Z","shell.execute_reply.started":"2023-05-23T20:47:32.185579Z","shell.execute_reply":"2023-05-23T20:47:34.859420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_grid_image_with_mask(image, mask)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:47:34.863553Z","iopub.execute_input":"2023-05-23T20:47:34.863936Z","iopub.status.idle":"2023-05-23T20:48:00.127360Z","shell.execute_reply.started":"2023-05-23T20:47:34.863900Z","shell.execute_reply":"2023-05-23T20:48:00.126303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 095bf7a1f","metadata":{}},{"cell_type":"code","source":"image_id = \"095bf7a1f\"\nimage, mask = read_image(image_id, scale=2)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:48:00.128903Z","iopub.execute_input":"2023-05-23T20:48:00.129223Z","iopub.status.idle":"2023-05-23T20:49:30.793447Z","shell.execute_reply.started":"2023-05-23T20:48:00.129191Z","shell.execute_reply":"2023-05-23T20:49:30.792360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_image_and_mask(image, mask, image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:49:30.794764Z","iopub.execute_input":"2023-05-23T20:49:30.795066Z","iopub.status.idle":"2023-05-23T20:50:28.095411Z","shell.execute_reply.started":"2023-05-23T20:49:30.795038Z","shell.execute_reply":"2023-05-23T20:50:28.094443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_slice_image_and_mask(image, mask, 7500, 10000, 10000, 12500)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:50:28.097229Z","iopub.execute_input":"2023-05-23T20:50:28.097596Z","iopub.status.idle":"2023-05-23T20:50:30.222463Z","shell.execute_reply.started":"2023-05-23T20:50:28.097561Z","shell.execute_reply":"2023-05-23T20:50:30.221703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_grid_image_with_mask(image, mask)","metadata":{"execution":{"iopub.status.busy":"2023-05-23T20:50:30.223641Z","iopub.execute_input":"2023-05-23T20:50:30.224117Z","iopub.status.idle":"2023-05-23T20:51:07.604021Z","shell.execute_reply.started":"2023-05-23T20:50:30.224083Z","shell.execute_reply":"2023-05-23T20:51:07.602980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1e2425f28","metadata":{}},{"cell_type":"code","source":"image_id = \"1e2425f28\"\nimage, mask = read_image(image_id, scale=2)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:51:07.605454Z","iopub.execute_input":"2023-05-23T20:51:07.605759Z","iopub.status.idle":"2023-05-23T20:52:04.991083Z","shell.execute_reply.started":"2023-05-23T20:51:07.605730Z","shell.execute_reply":"2023-05-23T20:52:04.990170Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_image_and_mask(image, mask, image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:52:04.992415Z","iopub.execute_input":"2023-05-23T20:52:04.992722Z","iopub.status.idle":"2023-05-23T20:52:39.613900Z","shell.execute_reply.started":"2023-05-23T20:52:04.992690Z","shell.execute_reply":"2023-05-23T20:52:39.612822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2f6ecfcdf","metadata":{}},{"cell_type":"code","source":"image_id = \"2f6ecfcdf\"\nimage, mask = read_image(image_id, scale=2)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:52:39.615408Z","iopub.execute_input":"2023-05-23T20:52:39.615812Z","iopub.status.idle":"2023-05-23T20:52:57.865194Z","shell.execute_reply.started":"2023-05-23T20:52:39.615757Z","shell.execute_reply":"2023-05-23T20:52:57.863359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_image_and_mask(image, mask, image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:52:57.866344Z","iopub.execute_input":"2023-05-23T20:52:57.866659Z","iopub.status.idle":"2023-05-23T20:53:32.166030Z","shell.execute_reply.started":"2023-05-23T20:52:57.866630Z","shell.execute_reply":"2023-05-23T20:53:32.165267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_slice_image_and_mask(image, mask, 10000, 12000, 8000, 10000)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:53:32.167254Z","iopub.execute_input":"2023-05-23T20:53:32.167718Z","iopub.status.idle":"2023-05-23T20:53:33.569169Z","shell.execute_reply.started":"2023-05-23T20:53:32.167667Z","shell.execute_reply":"2023-05-23T20:53:33.568298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## aaa6a05cc","metadata":{}},{"cell_type":"code","source":"image_id = \"aaa6a05cc\"\nimage, mask = read_image(image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:53:33.570473Z","iopub.execute_input":"2023-05-23T20:53:33.570791Z","iopub.status.idle":"2023-05-23T20:53:35.813090Z","shell.execute_reply.started":"2023-05-23T20:53:33.570759Z","shell.execute_reply":"2023-05-23T20:53:35.811954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_image_and_mask(image, mask, image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:53:35.814613Z","iopub.execute_input":"2023-05-23T20:53:35.815213Z","iopub.status.idle":"2023-05-23T20:54:20.552943Z","shell.execute_reply.started":"2023-05-23T20:53:35.815166Z","shell.execute_reply":"2023-05-23T20:54:20.552075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_slice_image_and_mask(image, mask, 6500, 8500, 7000, 9000)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:54:20.554317Z","iopub.execute_input":"2023-05-23T20:54:20.554628Z","iopub.status.idle":"2023-05-23T20:54:22.302916Z","shell.execute_reply.started":"2023-05-23T20:54:20.554596Z","shell.execute_reply":"2023-05-23T20:54:22.302100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## e79de561c","metadata":{}},{"cell_type":"code","source":"# image_id = \"e79de561c\"\n# image, mask = read_image(image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T20:54:22.304579Z","iopub.execute_input":"2023-05-23T20:54:22.304876Z","iopub.status.idle":"2023-05-23T20:54:37.989131Z","shell.execute_reply.started":"2023-05-23T20:54:22.304846Z","shell.execute_reply":"2023-05-23T20:54:37.988115Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot_image_and_mask(image, mask, image_id)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T21:00:45.699702Z","iopub.execute_input":"2023-05-23T21:00:45.700063Z","iopub.status.idle":"2023-05-23T21:01:38.066229Z","shell.execute_reply.started":"2023-05-23T21:00:45.700032Z","shell.execute_reply":"2023-05-23T21:01:38.063058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot_slice_image_and_mask(image, mask, 4000, 6000, 2000, 4000)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-23T21:01:38.067506Z","iopub.status.idle":"2023-05-23T21:01:38.068000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"3\"></a>\n<h2 style='background:#EAA6D1; border:0; color:white'><center>Metadata Analysis<center><h2>","metadata":{}},{"cell_type":"code","source":"pd.read_json(\n    os.path.join(BASE_PATH, \"train/0486052bb-anatomical-structure.json\")\n)","metadata":{"execution":{"iopub.status.busy":"2023-05-23T21:01:50.049740Z","iopub.execute_input":"2023-05-23T21:01:50.050140Z","iopub.status.idle":"2023-05-23T21:01:50.082359Z","shell.execute_reply.started":"2023-05-23T21:01:50.050100Z","shell.execute_reply":"2023-05-23T21:01:50.081326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.read_json(\n    os.path.join(BASE_PATH, \"train/0486052bb.json\")\n)","metadata":{"execution":{"iopub.status.busy":"2023-05-23T21:01:50.801646Z","iopub.execute_input":"2023-05-23T21:01:50.802373Z","iopub.status.idle":"2023-05-23T21:01:50.855159Z","shell.execute_reply.started":"2023-05-23T21:01:50.802332Z","shell.execute_reply":"2023-05-23T21:01:50.854334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_info[\"split\"] = \"test\"\ndf_info.loc[df_info[\"image_file\"].isin(os.listdir(os.path.join(BASE_PATH, \"train\"))), \"split\"] = \"train\"","metadata":{"execution":{"iopub.status.busy":"2023-05-23T21:01:51.622365Z","iopub.execute_input":"2023-05-23T21:01:51.622762Z","iopub.status.idle":"2023-05-23T21:01:51.631176Z","shell.execute_reply.started":"2023-05-23T21:01:51.622722Z","shell.execute_reply":"2023-05-23T21:01:51.629940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_info[\"area\"] = df_info[\"width_pixels\"] * df_info[\"height_pixels\"]","metadata":{"execution":{"iopub.status.busy":"2023-05-23T21:01:51.716928Z","iopub.execute_input":"2023-05-23T21:01:51.717342Z","iopub.status.idle":"2023-05-23T21:01:51.722684Z","shell.execute_reply.started":"2023-05-23T21:01:51.717300Z","shell.execute_reply":"2023-05-23T21:01:51.721814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_info.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-23T21:01:52.016257Z","iopub.execute_input":"2023-05-23T21:01:52.016654Z","iopub.status.idle":"2023-05-23T21:01:52.040993Z","shell.execute_reply.started":"2023-05-23T21:01:52.016616Z","shell.execute_reply":"2023-05-23T21:01:52.040000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(16, 35))\nplt.subplot(6, 2, 1)\nsn.countplot(x=\"race\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 2)\nsn.countplot(x=\"ethnicity\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 3)\nsn.countplot(x=\"sex\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 4)\nsn.countplot(x=\"laterality\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 5)\nsn.histplot(x=\"age\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 6)\nsn.histplot(x=\"weight_kilograms\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 7)\nsn.histplot(x=\"height_centimeters\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 8)\nsn.histplot(x=\"bmi_kg/m^2\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 9)\nsn.histplot(x=\"percent_cortex\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 10)\nsn.histplot(x=\"percent_medulla\", hue=\"split\", data=df_info)\nplt.subplot(6, 2, 11)\nsn.histplot(x=\"area\", hue=\"split\", data=df_info);","metadata":{"execution":{"iopub.status.busy":"2023-05-23T21:01:52.262837Z","iopub.execute_input":"2023-05-23T21:01:52.263231Z","iopub.status.idle":"2023-05-23T21:01:54.422839Z","shell.execute_reply.started":"2023-05-23T21:01:52.263193Z","shell.execute_reply":"2023-05-23T21:01:54.421979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WORK IN PROGRESS...","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}