{"cells":[{"metadata":{},"cell_type":"markdown","source":"\"Human"},{"metadata":{},"cell_type":"markdown","source":"

Hacking the kidney

\n

If you feel this is something new and fresh, and it added some value to you, please consider upvoting 😄

\n

NOTE due to the size of the images, trying to read and show different images may consume all the memory and the session will automatically restart

"},{"metadata":{},"cell_type":"markdown","source":"## Contents \n\n\n

\n

\n

"},{"metadata":{},"cell_type":"markdown","source":"

1. Introduction 🏕️

"},{"metadata":{},"cell_type":"markdown","source":"
\n

Our kidneys remove wastes and extra fluid from your body, they also remove the waste products generated by the chemical reactions carried out in our cells, and maintain a healthy balance of water, salts and minerals in our blood. This balance is essential for the other tissues to be in a good state and work normally.\n At a gross level, the kidney can be divide in the following parts:

\n \n \"Parts\n \n

\n

\n
"},{"metadata":{},"cell_type":"markdown","source":"
\n

\n In the following images we can see the anatomicla part that constitutes the glomerulus, a cartoon representation (Left), and an image of how it would actually look on a Periodic acid–Schiff (PAS) image (Right), where we can apreciate the glomerulus envolved by the Bowman's capsule\n

\n
\n
\n \"Snow\"\n
\n
\n \"Forest\"\n
\n
\n \n
"},{"metadata":{},"cell_type":"markdown","source":"
\n

\n At a more granular level, we can notice that the glomerulus which is the structure of interest for this challenge, is a component of the renal corpuscle, which itself is a component of a nephron, that is found within the Cortex. The glomerulus is important because it represents the component where the filtrartion of the blood begins, in it, the hydrostatic pressure provides the force for ultrafiltration.\n

\n \"Detailed\n

\n

Here is a good video I found, if you want to learn more about the glomerulus and glomerular filtration

\n
\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import HTML\nHTML('
')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

2. Importing the necessary libraries 📗

"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture\npip install seaborn --upgrade","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport seaborn as sns\nimport pandas as pd\nimport sys\nimport matplotlib.pyplot as plt\nimport os\nimport cv2\nimport json\nimport pprint\n\n#color\nfrom colorama import Fore, Back, Style","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

3. The Data 🏕️

"},{"metadata":{},"cell_type":"markdown","source":"

3.1 Analyzing images and contours

"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"def rle2mask(mask_rle, shape):\n # https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode\n '''\n mask_rle: run-length as string formated (start length)\n shape: (width,height) of array to return \n Returns numpy array, 1 - mask, 0 - background\n\n '''\n s = mask_rle.split()\n starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n starts -= 1\n ends = starts + lengths\n img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n for lo, hi in zip(starts, ends):\n img[lo:hi] = 1\n return img.reshape(shape).T\n\ndef mask_to_contours(image, mask_layer, thickness):\n \"\"\" converts a mask to contours using OpenCV and draws them on the image\"\"\"\n contours, hierarchy = cv2.findContours(mask_layer.astype('uint8'), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)\n image = cv2.drawContours(image, contours, -1, color=(255,0,0),thickness=thickness)\n return image","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#Data paths\nimg_path = \"../input/hubmap-kidney-segmentation/train/0486052bb.tiff\"\ntrain_path = \"../input/hubmap-kidney-segmentation/train.csv\"\nimg_id = img_path.split(\"/\")[-1].split(\".\")[0]\nstruct_path = \"../input/hubmap-kidney-segmentation/train/\" + img_id + \"-anatomical-structure.json\"\nglm_path = \"../input/hubmap-kidney-segmentation/train/\" + img_id + \".json\"\n\n#Read the image an generate the binary mask from the RL encoded data\norg_img = cv2.imread(img_path)\nimg = np.copy(org_img)\ndf_train = pd.read_csv(train_path)\nmask = rle2mask(\n df_train[df_train[\"id\"] == img_id][\"encoding\"].values[0], \n (img.shape[1], img.shape[0])\n)\n\n#Merge image with glomeruli mask \nimg = mask_to_contours(img, mask, thickness=-1)\n\n#Extract contours of anathomical structures from json file\nfile = open(struct_path)\nstructures_json = json.load(file)\n\n#Merge the image with the contours of the anathomical structures \nfor structure in structures_json:\n my_class = structure['properties']['classification']['name']\n if my_class == 'Medulla': #Note that Medulla will be delimited by a green contour\n A = np.asarray(structure['geometry']['coordinates'])\n A = [np.swapaxes(A,0,1)]\n img = cv2.drawContours(img, A, -1, color=(0,255,0),thickness=50)\n else: #Note that Cortex will be delimited by a blue contour\n A = np.asarray(structure['geometry']['coordinates'])\n A = [np.swapaxes(A,0,1)]\n img = cv2.drawContours(img, A, -1, color=(0,0,255),thickness=50)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.gcf()\nfig.set_size_inches(15,12)\nplt.imshow(img)\nplt.grid(False)\nprint(\"Notice that the\",Fore.GREEN, \"Medulla\",Style.RESET_ALL ,\"is delimited by the\", Fore.GREEN,\"green\",Style.RESET_ALL,\"line, and the\" ,Fore.BLUE, \"Cortex\",Style.RESET_ALL , \"is delimited by a\", Fore.BLUE,\"blue\",Style.RESET_ALL,\"line\")\nprint(\"The\",Fore.RED, \"red\",Style.RESET_ALL ,\"dots correspond to the\",Fore.RED, \"glomerulli\",Style.RESET_ALL )\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"file = open(glm_path)\nmy_json = json.load(file)\nprint(\"This JSON file has\", Fore.RED,len(my_json), Style.RESET_ALL ,\"elements.\")\nprint(\"Each element represent a\", Fore.RED + \"glomerulus\", Style.RESET_ALL)\nprint(\"The first element is printed below\")\nprint(\"\\n\")\npprint.pprint(my_json[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#We can zoom-in on multiple regions of the image where we can find glomeruli\nseeds_y = [18000, 9000, 3000, 2000]\nseeds_x = [8000, 4000, 19500, 11000]\nglm_img = np.copy(org_img)\nf, axs = plt.subplots(2,2, figsize=(18,18))\nfor i, ax in enumerate(axs.flat):\n a,b = seeds_y[i], seeds_x[i]\n A = mask_to_contours(glm_img[a:a+1000, b:b+1000], mask[a:a+1000, b:b+1000], thickness=5)\n ax.imshow(A)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#We can zoom-in on multiple regions of the image but delimited to the Medulla\nseeds_y = [10000, 10000, 11000, 11000]\nseeds_x = [15000, 16000, 15000, 16000]\nf, axs = plt.subplots(2,2, figsize=(18,18))\nfor i, ax in enumerate(axs.flat):\n a,b = seeds_y[i], seeds_x[i]\n A = mask_to_contours(glm_img[a:a+1000, b:b+1000], mask[a:a+1000, b:b+1000], thickness=5)\n ax.imshow(A)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

3.2 Analyzing patient data

"},{"metadata":{"trusted":true},"cell_type":"code","source":"my_df = pd.read_csv(\"../input/hubmap-kidney-segmentation/HuBMAP-20-dataset_information.csv\")\nmy_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.style.use('seaborn')\n\nRC = my_df['race'].value_counts()\nSX = my_df['sex'].value_counts()\nLT = my_df['laterality'].value_counts()\n\ncolors = ['#008fd5', '#fc4f30']\nexplode = (0, 0.1)\nexplode2 = (0.2, 0)\n\nf = plt.gcf()\nf.set_size_inches(16,6)\nplt.subplot(1,3,1)\n\nplt.pie(RC, startangle=90, colors=colors, \n wedgeprops={'edgecolor': 'black'}, autopct='%1.f%%', \n explode=explode, shadow=True)\nplt.title(\"Race\")\nplt.legend(loc='best', labels=RC.keys(), fontsize='medium')\n\nplt.subplot(1,3,2)\nplt.pie(SX, startangle=90, colors=colors, \n wedgeprops={'edgecolor': 'black'}, autopct='%1.f%%', \n explode=explode, shadow=True)\nplt.title(\"Sex\")\nplt.legend(loc='best', labels=SX.keys(), fontsize='medium')\n\nplt.subplot(1,3,3)\nplt.pie(LT, startangle=90, colors=colors, \n wedgeprops={'edgecolor': 'black'}, autopct='%1.f%%', \n explode=explode, shadow=True)\nplt.title(\"Laterality\")\nplt.legend(loc='best', labels=LT.keys(), fontsize='medium')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.gcf()\nfig.set_size_inches(18,13)\nplt.subplot(3,4,1)\nsns.histplot(x=\"age\",data=my_df)\nplt.subplot(3,4,2)\nsns.histplot(x=\"weight_kilograms\",data=my_df)\nplt.subplot(3,4,3)\nsns.histplot(x=\"height_centimeters\",data=my_df)\nplt.subplot(3,4,4)\nsns.histplot(x=\"bmi_kg/m^2\",data=my_df)\n\nplt.subplot(3,4,5)\nsns.histplot(x=\"age\", hue=\"sex\", data=my_df, palette=sns.color_palette()[0:4:2])\nplt.subplot(3,4,6)\nsns.histplot(x=\"weight_kilograms\", hue=\"sex\",data=my_df, palette=sns.color_palette()[0:4:2])\nplt.subplot(3,4,7)\nsns.histplot(x=\"height_centimeters\", hue=\"sex\", data=my_df, palette=sns.color_palette()[0:4:2])\nplt.subplot(3,4,8)\nsns.histplot(x=\"bmi_kg/m^2\", hue=\"sex\", data=my_df, palette=sns.color_palette()[0:4:2])\n\nplt.subplot(3,4,9)\nsns.histplot(x=\"age\", hue=\"race\", data=my_df, palette=sns.color_palette()[0:4:2])\nplt.subplot(3,4,10)\nsns.histplot(x=\"weight_kilograms\", hue=\"race\",data=my_df, palette=sns.color_palette()[0:4:2])\nplt.subplot(3,4,11)\nsns.histplot(x=\"height_centimeters\", hue=\"race\",data=my_df, palette=sns.color_palette()[0:4:2])\nplt.subplot(3,4,12)\nsns.histplot(x=\"bmi_kg/m^2\", hue=\"race\",data=my_df, palette=sns.color_palette()[0:4:2])\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
\"Work
"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}