{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<center>\n<img src=\"https://hubmapconsortium.org/wp-content/uploads/2019/01/HuBMAP-Retina-Logo-Color.png\">\n</center>","metadata":{"execution":{"iopub.status.busy":"2022-10-10T22:07:25.985177Z","iopub.execute_input":"2022-10-10T22:07:25.985774Z","iopub.status.idle":"2022-10-10T22:07:27.103543Z","shell.execute_reply.started":"2022-10-10T22:07:25.985723Z","shell.execute_reply":"2022-10-10T22:07:27.102083Z"}}},{"cell_type":"markdown","source":"\n# HUBMAP EDA + Masks vizualisation OT/\\OT\n\nHmm, Another day Another learning journey :)\n\nSo i have learned now to **read the .tiff** files, yayy!\n\nNow i wanna **learn how to see the masks** and understand what goes behind making and plotting these MASKS\n\nbut before that, I wanna see what do we have in dataset:)\n\n***If you find my notebooks helpful, you can please leave an upvote :)***","metadata":{}},{"cell_type":"markdown","source":"# Data Exploration","metadata":{}},{"cell_type":"code","source":"# lets start by importing the libraries i would be needing :)\n\nimport cv2 \nimport tifffile\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport os","metadata":{"execution":{"iopub.status.busy":"2022-10-11T16:48:16.420013Z","iopub.execute_input":"2022-10-11T16:48:16.420434Z","iopub.status.idle":"2022-10-11T16:48:16.602795Z","shell.execute_reply.started":"2022-10-11T16:48:16.420397Z","shell.execute_reply":"2022-10-11T16:48:16.601557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv('../input/hubmap-organ-segmentation/train.csv')\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-11T16:48:17.151307Z","iopub.execute_input":"2022-10-11T16:48:17.151696Z","iopub.status.idle":"2022-10-11T16:48:17.560969Z","shell.execute_reply.started":"2022-10-11T16:48:17.151665Z","shell.execute_reply":"2022-10-11T16:48:17.559895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# so i wanna see what all organs do we have and their value counts\ndf['organ'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-10-11T16:48:19.043772Z","iopub.execute_input":"2022-10-11T16:48:19.044218Z","iopub.status.idle":"2022-10-11T16:48:19.059919Z","shell.execute_reply.started":"2022-10-11T16:48:19.044180Z","shell.execute_reply":"2022-10-11T16:48:19.058434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"these numbers look very plain, **lets spice 🌶️🌶️ it up :!!** \n### **USING PLOTLY**","metadata":{}},{"cell_type":"code","source":"import plotly.express as px","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-10-11T16:48:22.473354Z","iopub.execute_input":"2022-10-11T16:48:22.474166Z","iopub.status.idle":"2022-10-11T16:48:23.795701Z","shell.execute_reply.started":"2022-10-11T16:48:22.474116Z","shell.execute_reply":"2022-10-11T16:48:23.794431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"px.bar(df,x='organ',color='organ',color_discrete_sequence=['#ff8c1a','#804000','#008000','#0039e6','#800080'])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-10-11T16:48:52.599568Z","iopub.execute_input":"2022-10-11T16:48:52.600034Z","iopub.status.idle":"2022-10-11T16:48:53.649470Z","shell.execute_reply.started":"2022-10-11T16:48:52.599994Z","shell.execute_reply":"2022-10-11T16:48:53.648393Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Kidney has its majority established here**\n\n### Now i am wondering...\n## Does **Gender** play any role in this ?","metadata":{}},{"cell_type":"code","source":"# lets find out\n\npx.bar(df,x='sex',color='sex',barmode='group',color_discrete_sequence=['#ff8c1a','#804000'])","metadata":{"execution":{"iopub.status.busy":"2022-10-11T16:48:58.012703Z","iopub.execute_input":"2022-10-11T16:48:58.013145Z","iopub.status.idle":"2022-10-11T16:48:58.085307Z","shell.execute_reply.started":"2022-10-11T16:48:58.013107Z","shell.execute_reply":"2022-10-11T16:48:58.084155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Oohhhh so **males**, get tested more than **women** :o:o:o:o\n                \n### how about we add in the **age groups**.","metadata":{"execution":{"iopub.status.busy":"2022-10-10T23:36:30.013311Z","iopub.execute_input":"2022-10-10T23:36:30.013739Z","iopub.status.idle":"2022-10-10T23:36:30.021075Z","shell.execute_reply.started":"2022-10-10T23:36:30.013707Z","shell.execute_reply":"2022-10-10T23:36:30.019470Z"}}},{"cell_type":"code","source":"px.histogram(df,x=['age'],color='sex',barmode='group',color_discrete_sequence=['#ff8c1a','#804000'])","metadata":{"execution":{"iopub.status.busy":"2022-10-11T16:49:03.657688Z","iopub.execute_input":"2022-10-11T16:49:03.658117Z","iopub.status.idle":"2022-10-11T16:49:03.753028Z","shell.execute_reply.started":"2022-10-11T16:49:03.658081Z","shell.execute_reply":"2022-10-11T16:49:03.751716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sooo, males actually do get tested a lot \n\n#### **TIME TO EAT HEALTHY FOLKS** 😆😆\n\nso i think i am okay with this much information. Lets now **get hands dirty** and learn to plot MASKS.","metadata":{}},{"cell_type":"markdown","source":"## Before we do that !:) **LISTEN UP FOLKS **\nHere is what you need to do,  next time you encounter RLE problems :)\n\n- if you have RLE \n-  make masks out of it (READ DOCSTRING OF THE fn)\n- Resize the image and the masks and store them \n- Now, plot the image & masks !!!\n\n#### sounds easyyy right !!\nHell YESSS, its easy. **I learnt it, you can do it tooo** :)","metadata":{}},{"cell_type":"markdown","source":"# Plotting Masks from RLE ","metadata":{}},{"cell_type":"code","source":"image_list = ['10044', '10392']\n#2 samples \ninput_dir = '../input/hubmap-organ-segmentation/train_images'\noutput_dir = '.'\nprint(df['id'].dtype)# converting this int to Str\ndf['id'] = df['id'].astype(str)\n# declaring CMAP'S\ncmaps = ['coolwarm_r','bone_r']","metadata":{"execution":{"iopub.status.busy":"2022-10-11T17:43:37.756901Z","iopub.execute_input":"2022-10-11T17:43:37.757302Z","iopub.status.idle":"2022-10-11T17:43:37.764034Z","shell.execute_reply.started":"2022-10-11T17:43:37.757266Z","shell.execute_reply":"2022-10-11T17:43:37.762571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(cmaps) == len(image_list)) # this should be true :-) ","metadata":{"execution":{"iopub.status.busy":"2022-10-11T17:50:25.074596Z","iopub.execute_input":"2022-10-11T17:50:25.075720Z","iopub.status.idle":"2022-10-11T17:50:25.083568Z","shell.execute_reply.started":"2022-10-11T17:50:25.075669Z","shell.execute_reply":"2022-10-11T17:50:25.082568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def resizer(im_name,scale):\n    '''\n    The resizer takes in 2 args:\n    im_name- name of the image\n    scale- percentage by which the image has to be reduced\n    '''\n    image_path = os.path.join(input_dir, im_name +'.tiff')\n    im_read = tifffile.imread(image_path)\n    width = int(im_read.shape[1] * scale / 100)\n    height = int(im_read.shape[0] * scale / 100)\n    dim = (width, height)\n    print('File name: {}, original size: {}, resized to: {}'.format(im_name , (im_read.shape[0], im_read.shape[1]), (width, height)))\n    resized = cv2.resize(im_read, dim, interpolation=cv2.INTER_AREA)\n    image_path = os.path.join(output_dir, ('r_' + im_name +'.tiff'))\n    tifffile.imwrite(image_path, resized)    ","metadata":{"execution":{"iopub.status.busy":"2022-10-11T17:50:51.563406Z","iopub.execute_input":"2022-10-11T17:50:51.564568Z","iopub.status.idle":"2022-10-11T17:50:51.574641Z","shell.execute_reply.started":"2022-10-11T17:50:51.564512Z","shell.execute_reply":"2022-10-11T17:50:51.573321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in image_list:\n    resizer(i,5)","metadata":{"execution":{"iopub.status.busy":"2022-10-11T16:51:46.425306Z","iopub.execute_input":"2022-10-11T16:51:46.425694Z","iopub.status.idle":"2022-10-11T16:51:47.513690Z","shell.execute_reply.started":"2022-10-11T16:51:46.425663Z","shell.execute_reply":"2022-10-11T16:51:47.512461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def rle2mask(rle,shape):\n    ## to see a complete code breakdown, REFER version 3.\n    s = rle.split()\n    # the \"s\" here is of dtype('<U7') hence we convert it to \"int\"\n    # very very important step \n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape).T\n\ndef resize_mask(im_name,scale):\n    '''\n    reads RLE encodings from the df\n    converts to masks and resizes it to a scaling_percentage of original size\n    '''\n    im_read = tifffile.imread(os.path.join(input_dir, im_name +'.tiff'))\n    mask_rle = df[df[\"id\"] == im_name][\"rle\"].values[0]\n    mask = rle2mask(df[df[\"id\"] == im_name][\"rle\"].values[0], (im_read.shape[1], im_read.shape[0]))*255\n    width = int(im_read.shape[1] * scale / 100)\n    height = int(im_read.shape[0] * scale / 100)\n    dim = (width, height)\n    print('File name: {}, original size: {}, resized to: {}'.format(im_name, (im_read.shape[0], im_read.shape[1]), (width, height)))\n    resized = cv2.resize(mask, dim, interpolation=cv2.INTER_AREA)\n    image_path = os.path.join(output_dir, (im_name + '.tiff'))\n    tifffile.imwrite(image_path, resized)    ","metadata":{"execution":{"iopub.status.busy":"2022-10-11T17:08:50.195502Z","iopub.execute_input":"2022-10-11T17:08:50.195933Z","iopub.status.idle":"2022-10-11T17:08:50.207634Z","shell.execute_reply.started":"2022-10-11T17:08:50.195896Z","shell.execute_reply":"2022-10-11T17:08:50.206855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for im in image_list:\n    print(im)\n    resize_mask(im, 5)","metadata":{"execution":{"iopub.status.busy":"2022-10-11T17:08:51.524938Z","iopub.execute_input":"2022-10-11T17:08:51.525651Z","iopub.status.idle":"2022-10-11T17:08:51.611843Z","shell.execute_reply.started":"2022-10-11T17:08:51.525610Z","shell.execute_reply":"2022-10-11T17:08:51.610546Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir(output_dir)","metadata":{"execution":{"iopub.status.busy":"2022-10-11T17:09:08.862082Z","iopub.execute_input":"2022-10-11T17:09:08.862487Z","iopub.status.idle":"2022-10-11T17:09:08.871270Z","shell.execute_reply.started":"2022-10-11T17:09:08.862454Z","shell.execute_reply":"2022-10-11T17:09:08.869993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Now lets plot some Masks","metadata":{}},{"cell_type":"code","source":"def show_image(image_id,cmaps):\n    fig, ax = plt.subplots(nrows=2, ncols=1, figsize=(16, 32))\n    image_path = os.path.join(output_dir, 'r_{}.tiff'.format(image_id))\n    mask_path = os.path.join(output_dir, '{}.tiff'.format(image_id))    \n    image = tifffile.imread(image_path)\n    mask = tifffile.imread(mask_path)\n    hybr = image[:, :,0]/2 + mask[:, :]\n\n    ax[0].imshow(image)\n    ax[0].axis('off')\n    ax[0].set_title('IMAGE')\n    ax[1].imshow(hybr,cmap=cmaps)\n    ax[1].axis('off')\n    ax[1].set_title('MASK ON IMAGE')\n    plt.show()    ","metadata":{"execution":{"iopub.status.busy":"2022-10-11T18:43:50.817565Z","iopub.execute_input":"2022-10-11T18:43:50.817971Z","iopub.status.idle":"2022-10-11T18:43:50.826790Z","shell.execute_reply.started":"2022-10-11T18:43:50.817938Z","shell.execute_reply":"2022-10-11T18:43:50.825544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i,j in zip(image_list,cmaps):\n    show_image(i,j)","metadata":{"execution":{"iopub.status.busy":"2022-10-11T18:43:52.505167Z","iopub.execute_input":"2022-10-11T18:43:52.505583Z","iopub.status.idle":"2022-10-11T18:43:53.262869Z","shell.execute_reply.started":"2022-10-11T18:43:52.505546Z","shell.execute_reply":"2022-10-11T18:43:53.261768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# NEXT TOPIC\n\nI know how to see masks, but i still **dont know how to make em useful** for modelling or maybe some terms i saw like '**TILING'& \"STAINING**\", etc. so next notebook i will learn how to use these masks :) ","metadata":{}},{"cell_type":"markdown","source":"# <center> learnt something cool !:o) Do leave an Upvote </center>","metadata":{}},{"cell_type":"markdown","source":"REFERENCES-\nI used previous years Yaroslav Isaienkov & The Devastator notebook for inspiration of this notebook :)","metadata":{}},{"cell_type":"markdown","source":"### see you next time folks :)","metadata":{}},{"cell_type":"markdown","source":"# ","metadata":{}}]}