{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":30201,"databundleVersionId":2750748,"sourceType":"competition"},{"sourceId":2823038,"sourceType":"datasetVersion","datasetId":1691020},{"sourceId":2825292,"sourceType":"datasetVersion","datasetId":1703908},{"sourceId":78871800,"sourceType":"kernelVersion"}],"dockerImageVersionId":30140,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<p align=\"center\">\n<img src=\"https://i.imgur.com/XaHgL55.png\">\n</p>\n\n\n\n\n# About the competition:\n<img align=\"right\" width=\"250\" src=\"https://storage.googleapis.com/kaggle-media/competitions/Sartorius/Sartorius_Competition%20Description%20Image%20350x379.png\" />\n\nHowdy! It has been 20 whole days and the competition has started picking up speed. The leaderboard is changing every hour. I always wanted to learn instance segmentation and I got really excited when I saw a Kaggle competition on it `ToT` . Apparently every year kaggle hosts one or more instance segmentation competitions, like this year's [iMaterialist (Fashion) 2021 at FGVC8](https://www.kaggle.com/c/imaterialist-fashion-2021-fgvc8) competition. Although It was not much popular. The in-hand task for this competition is to detect the boundaries of each cell body using segmentation. In this competition we are segmenting neuronal cells in images.The training annotations are provided as run length encoded masks, and the images are in PNG format. The number of images is small, but the number of annotated objects is quite high.The hidden test set is roughly 240 images. Annotations are a bit noisy in some places because of the small size of cell bodies.\n\n\n\n\n> Yeah you know that. We just need to do segmentation for this competition to detect the boundaries of a neuron cell. But how does this help the medical industry? Actually different neurological disorders, including neurodegenerative diseases such as Alzheimer's and brain tumors are a leading cause of death and disability across the globe. And (I guess) these diseases affect the brain cells, mainly the cell body. So for detecting which part of the brain has problems in the cell body we need the help of computer vision. And with the help of segmented output for the region of interest researchers and doctors will be able to analyze properly and help the patient.\n\n\n\n\n## Why we are using Unet when it is a instance segmentation Competition:\n\n<img align=\"center\" src=\"https://i.imgflip.com/5t9p8v.jpg\" title=\"made at imgflip.com\"/>\n\n> If you are getting started with the competition then, there might be Question like, **\"This is a instance segmentation competition then why there are so many Notenooks on Unet which is a sementic segmentation model?\"** So, let me explain, we are actually doing Instance segmentation but not directly, It is a post processing part now. If we apply instance segmentation in this daataset we will get the mask for each seperated cell instances with their class labels and bounding boxes, As we dont need class informatiopn and bounding box for the submission thats why we fall back to sementic segmentation and we seperate the segmented cells from a predicted mask using opencv **`connectedComponents`** is. You can see the post-processing part of this NB  [🦠 Sartorius - Starter Baseline Torch U-net](https://www.kaggle.com/julian3833/sartorius-starter-baseline-torch-u-net#Predict)  by [@julian3833](https://www.kaggle.com/julian3833) in the Predict/Utilities part. I also took that code for preprocessing for my NB.\n\n> > Check out more about Unet utilization, here\n> > - [UNet Strikes Back 🔥 | LB: 0.155+](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/286553)\n> > - [Tips in submission and baseline (in the beginning of the competition)](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790)\n> > - [Train Attention based Residual Unet with watershed algorithm for instance segmentation](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/285422) [mainly check out the comments ]\n> > - [Are we predicting cells or a mask?](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/287487)[mainly check out the comments ]\n\n## About this NB:\n\n> 1. In this notebook I have tried to do a thorough analysis of different cell types given in the dataset.\n> 2. Did some visualizations for different types of augmentations.\n> 3. And mainly I tried to Implement Attention based Residual Unet. It is a scratch implementation.\n> 4. Used tf.data for data pipeline.\n> 5. **Tried to make this end to end, from EDA to submission.**\n\n\n## Let you get started with the competition:\n> If you are relatively new to Computer Vision, then starting out with Segmentation might be a bit hectic. But I have tried to explain everything thoroughly  with images. most of the code is with comments. I have been following the notebook section as well as the discussion section and I have some personal favorites that you must check out if you are late in the competition.\n\n> > #### Notebooks:\n> > 1. [🦠 Sartorius - Classifier + Mask R-CNN [LB=0.28]](https://www.kaggle.com/julian3833/sartorius-classifier-mask-r-cnn-lb-0-28)\n> > 2. [Sartorius Cell Instance Segmentation - EDA](https://www.kaggle.com/gunesevitan/sartorius-cell-instance-segmentation-eda)\n> > 3. [Positive score with Detectron 3/3 - Inference](https://www.kaggle.com/slawekbiel/positive-score-with-detectron-3-3-inference)\n> > 4. [Sartorius Segmentation - Keras U-Net [Training]](https://www.kaggle.com/ammarnassanalhajali/sartorius-segmentation-keras-u-net-training)\n\n\n> > #### Discussions:\n> > 1. [Tips in submission and baseline (in the beginning of the competition)](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790)\n\n> > 2. [Annotations Are too Noisy for the Metric](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/281205)\n\n> > 3. [[Info] Instance Segmentation Models (quick list)](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/278883)\n\n> > 4. [tutorial detectron 2](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280137)\n\n> > 5. [Overlaps-ambiguous or not?](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250)\n\n> > 6. [No Overlap Issue: Quick Fix](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279995)\n\n\n<font size=\"4\"\n          face=\"verdana\"\n          color=\"red\">\n            If you find this kernel useful, Please Upvote. Thank you.  \n        </font>","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-02T21:53:41.582724Z","iopub.execute_input":"2021-11-02T21:53:41.58361Z","iopub.status.idle":"2021-11-02T21:53:49.404983Z","shell.execute_reply.started":"2021-11-02T21:53:41.58349Z","shell.execute_reply":"2021-11-02T21:53:49.404313Z"}}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport random\nimport matplotlib.pyplot as plt\nimport cv2\nfrom glob import glob\nfrom sklearn.model_selection import train_test_split\nfrom tqdm import tqdm_notebook as tqdm\nfrom plotly.offline import iplot\nimport plotly as py\nimport plotly.tools as tls\nimport cufflinks as cf\nfrom IPython.core.display import display, HTML\nfrom pathlib import Path\n\nimport tensorflow as tf\nfrom tensorflow.keras.utils import CustomObjectScope\nfrom tqdm import tqdm\nimport keras\nimport imgaug.augmenters as iaa\nfrom tensorflow.keras import models, layers, regularizers\nfrom tensorflow.keras import backend as K\n\n# defining the losses\n# from keras import backend as K\nfrom keras.losses import binary_crossentropy\nimport tensorflow as tf\n\nfrom tensorflow.keras.optimizers import Adam\nfrom datetime import datetime \nfrom PIL import Image\nimport albumentations as A\n\n# from keras import backend, optimizers\n#\n\nAUTOTUNE = tf.data.experimental.AUTOTUNE\nplt.style.use('fivethirtyeight')\npy.offline.init_notebook_mode(connected = True)\ncf.go_offline()\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:25:50.123951Z","iopub.execute_input":"2021-11-19T20:25:50.12438Z","iopub.status.idle":"2021-11-19T20:25:54.524855Z","shell.execute_reply.started":"2021-11-19T20:25:50.124324Z","shell.execute_reply":"2021-11-19T20:25:54.524117Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Set the random seeds\nos.environ['TF_CUDNN_DETERMINISTIC'] = '1'\nrandom.seed(hash(\"setting random seeds\") % 2**32 - 1)\nnp.random.seed(hash(\"improves reproducibility\") % 2**32 - 1)\ntf.random.set_seed(hash(\"by removing stochasticity\") % 2**32 - 1)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:25:54.526661Z","iopub.execute_input":"2021-11-19T20:25:54.527062Z","iopub.status.idle":"2021-11-19T20:25:54.53359Z","shell.execute_reply.started":"2021-11-19T20:25:54.527023Z","shell.execute_reply":"2021-11-19T20:25:54.532991Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# import wandb\n# from wandb.keras import WandbCallback\n\n# wandb.login()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:25:54.534661Z","iopub.execute_input":"2021-11-19T20:25:54.535116Z","iopub.status.idle":"2021-11-19T20:25:54.552219Z","shell.execute_reply.started":"2021-11-19T20:25:54.535077Z","shell.execute_reply":"2021-11-19T20:25:54.551241Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import wandb\nfrom wandb.keras import WandbCallback\n\ntry:\n    from kaggle_secrets import UserSecretsClient\n    user_secrets = UserSecretsClient()\n    api_key = user_secrets.get_secret(\"WANDB\")\n    wandb.login(key=api_key)\n    anonymous = None\nexcept:\n    anonymous = \"must\"\n    print('To use your W&B account,\\nGo to Add-ons -> Secrets and provide your W&B access token. Use the Label name as WANDB. \\nGet your W&B access token from here: https://wandb.ai/authorize')","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:25:54.553413Z","iopub.execute_input":"2021-11-19T20:25:54.554971Z","iopub.status.idle":"2021-11-19T20:25:57.117075Z","shell.execute_reply.started":"2021-11-19T20:25:54.554931Z","shell.execute_reply":"2021-11-19T20:25:57.116287Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# !pip install chart-studio -q\n# %%capture\n!pip install ../input/detoxify-wheel/segmentation_models-1.0.1-py3-none-any.whl","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:25:57.119923Z","iopub.execute_input":"2021-11-19T20:25:57.12017Z","iopub.status.idle":"2021-11-19T20:26:07.083224Z","shell.execute_reply.started":"2021-11-19T20:25:57.120134Z","shell.execute_reply":"2021-11-19T20:26:07.082407Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Data Analysis and Preprocessing","metadata":{}},{"cell_type":"code","source":"train_img = \"../input/sartorius-cell-instance-segmentation/train\"\ntrain_csv = \"../input/sartorius-cell-instance-segmentation/train.csv\"\n\ndf = pd.read_csv(train_csv)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:07.085174Z","iopub.execute_input":"2021-11-19T20:26:07.085474Z","iopub.status.idle":"2021-11-19T20:26:07.662736Z","shell.execute_reply.started":"2021-11-19T20:26:07.085436Z","shell.execute_reply":"2021-11-19T20:26:07.662046Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"len(np.unique(df[\"id\"])) #number of unique ids","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:07.664344Z","iopub.execute_input":"2021-11-19T20:26:07.664838Z","iopub.status.idle":"2021-11-19T20:26:07.730606Z","shell.execute_reply.started":"2021-11-19T20:26:07.6648Z","shell.execute_reply":"2021-11-19T20:26:07.729615Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"len(df[\"id\"]) #total no of records present","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:07.732373Z","iopub.execute_input":"2021-11-19T20:26:07.732673Z","iopub.status.idle":"2021-11-19T20:26:07.740763Z","shell.execute_reply.started":"2021-11-19T20:26:07.732635Z","shell.execute_reply":"2021-11-19T20:26:07.73983Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df[\"cell_type\"].value_counts().iplot(kind='bar',color='blue') #different types of cells in the dataset","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:07.742369Z","iopub.execute_input":"2021-11-19T20:26:07.742849Z","iopub.status.idle":"2021-11-19T20:26:08.378958Z","shell.execute_reply.started":"2021-11-19T20:26:07.742806Z","shell.execute_reply":"2021-11-19T20:26:08.378276Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"> This shows the number of annotated cells are present in the `shsy5y` type cell. And the count ofr `cort` and `astro` looks similar.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\nplt.bar([0,1,2],[len(np.unique(df[df['cell_type']=='shsy5y']['id'])),len(np.unique(df[df['cell_type']=='cort']['id'])),len(np.unique(df[df['cell_type']=='astro']['id']))], label=\"Data 1\")\nplt.legend()\n\nplt.xlabel('shsy5y, cort and astro respectively')\nplt.ylabel('unique count')\nplt.title('cell type bar chart')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:08.380422Z","iopub.execute_input":"2021-11-19T20:26:08.38066Z","iopub.status.idle":"2021-11-19T20:26:08.720657Z","shell.execute_reply.started":"2021-11-19T20:26:08.380625Z","shell.execute_reply":"2021-11-19T20:26:08.719976Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"> It shows the count of the cell types are `cort>shsy5y>astro`","metadata":{}},{"cell_type":"code","source":"def rle_decode(mask_rle, shape, color=1):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height,width) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros((shape[0] * shape[1], shape[2]), dtype=np.float32)\n    for lo, hi in zip(starts, ends):\n        img[lo : hi] = color\n    return img.reshape(shape)\n    \n\n\n\ndef build_masks(image_id,input_shape, colors=True):\n    height, width = input_shape\n    labels = df[df[\"id\"] == image_id][\"annotation\"].tolist()\n    if colors:\n        mask = np.zeros((height, width, 3))\n        for label in labels:\n            mask += rle_decode(label, shape=(height,width , 3), color=np.random.rand(3))\n    else:\n        mask = np.zeros((height, width, 1))\n        for label in labels:\n            mask += rle_decode(label, shape=(height, width, 1))\n    mask = mask.clip(0, 1)\n    return mask\n    ","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:08.721888Z","iopub.execute_input":"2021-11-19T20:26:08.722208Z","iopub.status.idle":"2021-11-19T20:26:08.732462Z","shell.execute_reply.started":"2021-11-19T20:26:08.722169Z","shell.execute_reply":"2021-11-19T20:26:08.731605Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ide = 'ffdb3cc02eef'\nsample_image_df = df[df['id'] == ide]\nsample_path = f\"../input/sartorius-cell-instance-segmentation/train/{sample_image_df['id'].iloc[0]}.png\"\nsample_img = cv2.imread(sample_path)\nprint(sample_img.shape)\nsample_masks1=build_masks(ide,input_shape=(520, 704), colors=False)\nsample_masks1.shape","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:08.734193Z","iopub.execute_input":"2021-11-19T20:26:08.734818Z","iopub.status.idle":"2021-11-19T20:26:08.832326Z","shell.execute_reply.started":"2021-11-19T20:26:08.734718Z","shell.execute_reply":"2021-11-19T20:26:08.831617Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ids = ['fe3e30f849f4','ffdb3cc02eef','0140b3c8f445']\nCOLOR_MAP = 'seismic'\n\n\ndef load_data_viz(ids):\n    loaded_data = []\n    for ide in ids:\n        sample_image_df = df[df['id'] == ide]\n        sample_path = f\"../input/sartorius-cell-instance-segmentation/train/{sample_image_df['id'].iloc[0]}.png\"\n        sample_img = plt.imread(sample_path)\n        sample_masks1=build_masks(ide,input_shape=(520, 704), colors=False)\n        sample_masks2=build_masks(ide,input_shape=(520, 704), colors=True)\n        loaded_data.append([sample_img,sample_masks1,sample_masks2])\n    return loaded_data\n\ndata = load_data_viz(ids)\n\n    \nplt.figure(figsize=(14,14))\nplt.suptitle(\"subplots of shsy5y,cort and astro\",fontweight=\"bold\", size=20)\n\nplt.subplot(3, 3, 1)\nplt.imshow(data[0][0],cmap = 'seismic')\nplt.title(\"shsy5y iamge\")\n\n\nplt.subplot(3, 3, 2)\nplt.imshow(data[0][1],cmap='gray')\nplt.title(\"shsy5y mask iamge\")\n\nplt.subplot(3, 3, 3)\nplt.imshow(data[0][2])\nplt.title(\"shsy5y color mask iamge\")\n\n\n\nplt.subplot(3, 3, 4)\nplt.imshow(data[1][0],cmap = 'seismic')\nplt.title(\"cort iamge\")\n\nplt.subplot(3, 3, 5)\nplt.imshow(data[1][1],cmap='gray')\nplt.title(\"cort mask iamge\")\n\nplt.subplot(3, 3, 6)\nplt.imshow(data[1][2])\nplt.title(\"cort color mask iamge\")\n\n\n\nplt.subplot(3, 3, 7)\nplt.imshow(data[2][0],cmap = 'seismic')\nplt.title(\"astro iamge\")\n\nplt.subplot(3, 3, 8)\nplt.imshow(data[2][1],cmap='gray')\nplt.title(\"astro mask iamge\")\n\nplt.subplot(3, 3, 9)\nplt.imshow(data[2][2])\nplt.title(\"astro color mask iamge\")\n\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-19T20:26:08.833868Z","iopub.execute_input":"2021-11-19T20:26:08.834358Z","iopub.status.idle":"2021-11-19T20:26:11.770825Z","shell.execute_reply.started":"2021-11-19T20:26:08.834321Z","shell.execute_reply":"2021-11-19T20:26:11.770138Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"> Structure wise `shsy5y` cells look bigger than the `cort` types but smaller than the `astro` types.","metadata":{}},{"cell_type":"markdown","source":"# Analysis on different cell images provided in the dataset:\n## shsy5y\nLets talk about the shsy5y type cells, \nSH-SY5Y is a human derived cell line used in scientific research. And it has been used to study Parkinson's disease, neurogenesis, and other characteristics of brain cells. The main structure that is mainly annotated is the SOMA(cell body). neuclear is situated over here. Axon and dendron parts are not included in the annotations. Its not only for the sh-sy5y type cells, its for other two types also(not sure about the `astro` cell type though). `shsy5y` comes under the `neuroblastoma` cell line.\n<!-- ![]() -->\n<p align=\"center\">\n<img width = \"300\" src=\"https://www.researchgate.net/profile/Dianne-Langford/publication/256102174/figure/fig2/AS:601664889319463@1520459496437/Differentiated-SH-SY5Y-cells-Cells-do-not-cluster-and-have-a-more-pyramidal-shaped-cell.png\">\n</p>\n\n<!-- I used to bunl mu biology classes I did not know it will take revenge like this. -->","metadata":{}},{"cell_type":"code","source":"ids = ['fe3e30f849f4','ffdb3cc02eef','0140b3c8f445']\nCOLOR_MAP = 'seismic'\n\n\ndef load_data_viz(ids):\n    loaded_data = []\n    for ide in ids:\n        sample_image_df = df[df['id'] == ide]\n        sample_path = f\"../input/sartorius-cell-instance-segmentation/train/{sample_image_df['id'].iloc[0]}.png\"\n        sample_img = cv2.imread(sample_path,cv2.IMREAD_COLOR)\n        loaded_data.append(sample_img)\n    return loaded_data\n\n\n\naug_data = []\ndef get_aug(sample_img):\n    aug1 = iaa.AllChannelsCLAHE(clip_limit=(1, 10), per_channel=True)(image=sample_img)\n    aug2 = iaa.LogContrast(gain=(0.6, 1.4), per_channel=True)(image=sample_img)\n    aug3 = iaa.BlendAlpha((0.0, 1.0), iaa.HistogramEqualization())(image=sample_img)\n    aug4 = iaa.Canny(alpha=(0.1, 0.8))(image=sample_img)\n    aug5 = iaa.Canny(alpha=(0.0, 0.5),colorizer=iaa.RandomColorsBinaryImageColorizer(color_true=255,color_false=0))(image=sample_img)\n    aug6 = iaa.Canny(alpha=(0.4, 0.5), sobel_kernel_size=[3, 7])(image=sample_img)\n    aug_data.append([aug1,aug2,aug3,aug4,aug5,aug6])\n    return aug_data\n\n\ncell_img = load_data_viz(ids)\naug_img1 =  get_aug(cell_img[0])\naug_img2 =  get_aug(cell_img[1])\naug_img3 =  get_aug(cell_img[2])","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:11.774796Z","iopub.execute_input":"2021-11-19T20:26:11.775265Z","iopub.status.idle":"2021-11-19T20:26:12.259211Z","shell.execute_reply.started":"2021-11-19T20:26:11.775215Z","shell.execute_reply":"2021-11-19T20:26:12.25849Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def get_img_paths(cell_type, i,j): \n    ids = np.unique(df[df['cell_type']==cell_type]['id'])[i:j]\n    paths = []\n    for ide in ids:\n        sample_image_df = df[df['id'] == ide]\n        sample_path = f\"../input/sartorius-cell-instance-segmentation/train/{sample_image_df['id'].iloc[0]}.png\"\n        paths.append(sample_path)\n    return paths\n\n\ndef show_img_subplot(i,j,title_text,cell_type = 'shsy5y'):\n    images_paths = get_img_paths(cell_type,i,j)\n    plt.style.use('fivethirtyeight')\n    figure, ax = plt.subplots(nrows=4,ncols=4,figsize=(16,16) )\n    plt.suptitle(title_text,fontweight=\"bold\", size=20)\n    for ind,image_path in enumerate(images_paths):\n        image=plt.imread(image_path)\n        try:\n            ax.ravel()[ind].imshow(image,cmap = 'seismic')\n            ax.ravel()[ind].set_axis_off()\n        except:\n            continue;\n    plt.tight_layout()\n    plt.show()\n    \nshow_img_subplot(23,39,\"All shsy5y type cells\",'shsy5y')","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:12.260344Z","iopub.execute_input":"2021-11-19T20:26:12.260584Z","iopub.status.idle":"2021-11-19T20:26:14.732568Z","shell.execute_reply.started":"2021-11-19T20:26:12.260552Z","shell.execute_reply":"2021-11-19T20:26:14.731862Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"method_names = [\"AllChannelsCLAHE\",\"LogContrast\",\"BlendAlpha\",\"Canny\",\"RandomColorsBinaryImageColorizer\",\"canny with soble kernel\"]\n\ndef plot(data,data_idx = 0,img_idx = 0,text=\"shsy5y images\"):\n    display(HTML('<h1 style=\"background-color: #2dfafa;align=\"center\">Original Image Vs Different Augmentations</h1>'))\n    for i,j in enumerate(range(1,13,2)):\n        plt.figure(figsize=(30,30))\n        plt.subplot(6, 2, j)\n        plt.title(text)\n        plt.imshow(cell_img[img_idx])\n        plt.subplot(6, 2, j+1)\n        plt.imshow(data[data_idx][i])\n        plt.title(method_names[i])\n        plt.tight_layout()\n        plt.show()\n\n\nplot(aug_img1,0,0,text = \"shsy5y images\")","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:14.73377Z","iopub.execute_input":"2021-11-19T20:26:14.734103Z","iopub.status.idle":"2021-11-19T20:26:18.118818Z","shell.execute_reply.started":"2021-11-19T20:26:14.734071Z","shell.execute_reply":"2021-11-19T20:26:18.118085Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Lets talk about `cort` cells:\nI was reading the competition overview, but there is nothing about the cell type named `cort`, and apperantly there is nothing much about this in the internet too. But by the picture in the overview section I assume that the `neurons` are the `cort`. And I have reasons to belive that, see the 2nd bar chart from the top of the NB. actually thats the bar plot for the count of brain cell for the data provided, and you can see that the `cort` type clearly winning the race and it is because in general the number of `neurons` or `cort` are in majority in the brain of a normal human being. So, we can say that collecting data for `cort` is more easier that other ones.\n\n> According to the visualizations we can say that the smallest cell in the dataset is `cort` and according to the google search, the cell body of a motor neuron is approximately 100 microns (0.1 millimeter) in diameter and I believe its a smaller size compared to the others. And this might lead to missing of some of the cells  from the segmentation output. I have seen 1 or 2 discussion threads regarding this topic that the [`Annotations Are too Noisy`](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/281205), and the main reason behind that is the cells are too small and because of that annotations are not much clear. [@theoviel](https://www.kaggle.com/theoviel) explained that very nicely, go check that out.","metadata":{}},{"cell_type":"code","source":"show_img_subplot(23,39,\"All cort type cells\",'cort')","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:18.120024Z","iopub.execute_input":"2021-11-19T20:26:18.120793Z","iopub.status.idle":"2021-11-19T20:26:20.281764Z","shell.execute_reply.started":"2021-11-19T20:26:18.120745Z","shell.execute_reply":"2021-11-19T20:26:20.281109Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Data Augmentation for `cort`:\nI have not done much analysis on augmentations though. But watching these visualizations made me think that, as the cells are not much big so sticking with the augmentations that highlights the cells very brightly/ boldly would be a good idea, and of course with all the other rotation and flipping and random cropping type augs. I have tried mainly six augmentations, which are mainly `AllChannelsCLAHE,LogContrast,BlendAlpha,Canny, canny with RandomColorBinaryImageColorizer and canny with soble kernel`. Using blur, [meta](https://imgaug.readthedocs.io/en/latest/source/overview/meta.html), [segmentation](https://imgaug.readthedocs.io/en/latest/source/overview/segmentation.html) or gaussian blur/ noise type of augmentation might not be that helpful.\n\n","metadata":{}},{"cell_type":"code","source":"plot(aug_img2,1,1)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:20.282947Z","iopub.execute_input":"2021-11-19T20:26:20.283301Z","iopub.status.idle":"2021-11-19T20:26:23.944414Z","shell.execute_reply.started":"2021-11-19T20:26:20.283267Z","shell.execute_reply":"2021-11-19T20:26:23.943641Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# The last one standing:\n`The most common brain cells are neurons and non-neuron cells called glia.` Astrocytes, the most abundant glial cell type in the brain,And for a fact these cell types are well known to provide metabolic and trophic support to neurons and modulate synaptic activity. I bet you did not know that probably dont care about that either, like me `XD`. Anyways looking at the visualizations it feels like these are the cells which are bigger in size and probably annotated more precisely.  \n\n\n<!-- might sound like im a doctor, but... -->","metadata":{}},{"cell_type":"code","source":"show_img_subplot(23,39,\"All astrocyte type cells\",'astro')","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:23.945894Z","iopub.execute_input":"2021-11-19T20:26:23.946698Z","iopub.status.idle":"2021-11-19T20:26:25.750185Z","shell.execute_reply.started":"2021-11-19T20:26:23.946655Z","shell.execute_reply":"2021-11-19T20:26:25.749572Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Below are the augmentations visualizations for the astrocyte cells.  ","metadata":{}},{"cell_type":"code","source":"plot(aug_img3,2,2)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:25.751375Z","iopub.execute_input":"2021-11-19T20:26:25.751721Z","iopub.status.idle":"2021-11-19T20:26:29.476781Z","shell.execute_reply.started":"2021-11-19T20:26:25.751681Z","shell.execute_reply":"2021-11-19T20:26:29.476061Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import imgaug.augmenters as iaa\nide = \"fe3e30f849f4\"\nsample_image_df = df[df['id'] == ide]\nsample_path = f\"../input/sartorius-cell-instance-segmentation/train/{sample_image_df['id'].iloc[0]}.png\"\nsample_img = cv2.imread(sample_path,cv2.IMREAD_COLOR)\n# sample_img = plt.imread(sample_path)\n# sample_img = sample_img.astype('float16')\naug = iaa.BlendAlpha(\n    (0.0, 1.0),\n    iaa.Affine(rotate=(-20, 20)),\n    per_channel=0.5)(image=sample_img)\n\nplt.figure(figsize=(14,14))\nplt.subplot(1,2,1)\nplt.imshow(sample_img,cmap=\"seismic\")\nplt.title(\"normal\")\nplt.subplot(1,2,2)\nplt.imshow(aug,cmap=\"seismic\")\nplt.title(\"BlendAlpha\")","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:29.47824Z","iopub.execute_input":"2021-11-19T20:26:29.478767Z","iopub.status.idle":"2021-11-19T20:26:30.107161Z","shell.execute_reply.started":"2021-11-19T20:26:29.478728Z","shell.execute_reply":"2021-11-19T20:26:30.106536Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Data pipeline:\nThe load_data function loads the dataset and returns the training, validation and testing dataset. It takes the dataset path and the dataset split value as the argument. I gives the list of directories for train and test. After that we will be using `tf.data` for loading the data on the fly.","metadata":{}},{"cell_type":"code","source":"def load_data(mask_path, split=0.1):\n    img_path = \"../input/sartorius-cell-instance-segmentation/train\"\n    images = [img_path + \"/\"+ i for i in sorted(os.listdir(\"../input/sartorious-nb-1-data-preprocessing-visualization/masks/img\"))]\n    masks = [mask_path + \"/\"+ i for i in sorted(os.listdir(\"../input/sartorious-nb-1-data-preprocessing-visualization/masks/img\"))]\n\n    train_x, valid_x = train_test_split(images, test_size=split, random_state=42)\n    train_y, valid_y = train_test_split(masks, test_size=split, random_state=42)\n\n\n    return (train_x, train_y), (valid_x, valid_y)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:30.108548Z","iopub.execute_input":"2021-11-19T20:26:30.108979Z","iopub.status.idle":"2021-11-19T20:26:30.116016Z","shell.execute_reply.started":"2021-11-19T20:26:30.108944Z","shell.execute_reply":"2021-11-19T20:26:30.115254Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"mask_path = \"../input/sartorious-nb-1-data-preprocessing-visualization/masks/img\"\n(train_x, train_y), (valid_x, valid_y) = load_data(mask_path)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:30.117399Z","iopub.execute_input":"2021-11-19T20:26:30.117857Z","iopub.status.idle":"2021-11-19T20:26:30.236058Z","shell.execute_reply.started":"2021-11-19T20:26:30.117823Z","shell.execute_reply":"2021-11-19T20:26:30.235427Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_x = np.array(train_x)\ntrain_y = np.array(train_y)\nvalid_x = np.array(valid_x)\nvalid_y = np.array(valid_y)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:30.238892Z","iopub.execute_input":"2021-11-19T20:26:30.239081Z","iopub.status.idle":"2021-11-19T20:26:30.24476Z","shell.execute_reply.started":"2021-11-19T20:26:30.239058Z","shell.execute_reply":"2021-11-19T20:26:30.243269Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def load_image_and_label_from_path(image_path, mask_path):\n    img = tf.io.read_file(image_path)\n#     print(f\"image: {img}\")\n\n    img = tf.image.decode_png(img, channels=3)\n    \n    mask = tf.io.read_file(mask_path)\n#     print(f\"mask: {mask}\")\n    mask = tf.image.decode_png(mask, channels=1)\n#     print(image_path)\n#     img = read_image(image_path)\n#     mask = read_mask(mask_path)\n    \n    \n    \n    return img, mask","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:30.245994Z","iopub.execute_input":"2021-11-19T20:26:30.246827Z","iopub.status.idle":"2021-11-19T20:26:30.253079Z","shell.execute_reply.started":"2021-11-19T20:26:30.246788Z","shell.execute_reply":"2021-11-19T20:26:30.252394Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"training_data = tf.data.Dataset.from_tensor_slices((train_x, train_y))\n\n# training_data = tf.data.Dataset.from_tensor_slices((df[\"img_path\"].values, df[\"mask_path\"].values))\n# validation_data = tf.data.Dataset.from_tensor_slices((val_data[\"filepath\"].values, val_data[\"label\"].values))\nvalidation_data = tf.data.Dataset.from_tensor_slices((valid_x, valid_y))\n\ntraining_data = training_data.map(load_image_and_label_from_path, num_parallel_calls=AUTOTUNE)\nvalidation_data = validation_data.map(load_image_and_label_from_path, num_parallel_calls=AUTOTUNE)\n# for i in training_data:\n#     print(i[0].numpy().decode(\"utf-8\") )\n#     break\n","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:30.255121Z","iopub.execute_input":"2021-11-19T20:26:30.255721Z","iopub.status.idle":"2021-11-19T20:26:32.687732Z","shell.execute_reply.started":"2021-11-19T20:26:30.255678Z","shell.execute_reply":"2021-11-19T20:26:32.687009Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def augment_train_data(train_ds):\n    transforms = A.Compose([\n            A.RandomResizedCrop(image_size, image_size),\n            A.Transpose(p=0.5),\n            A.HorizontalFlip(p=0.5),\n            A.VerticalFlip(p=0.5),\n            A.ShiftScaleRotate(p=0.5),\n            A.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n            A.RandomBrightnessContrast(brightness_limit=(-0.1, 0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n            A.CoarseDropout(p=0.5),\n            A.Cutout(p=0.5),\n            ], p=1)\n    \n    def aug_fn(image,mask):\n#         data_img = {\"image\":image}\n#         aug_data_img = transforms(**data_img)\n        \n#         data_mask = {'mask':mask}\n#         aug_data_mask = transforms(**data_mask)\n\n        aug_data_all = transforms(image=image,mask=mask)\n        aug_img = aug_data_all[\"image\"]\n        aug_img = tf.cast(aug_img, tf.float32)\n        \n        aug_mask = aug_data_all[\"mask\"]\n        aug_mask = tf.cast(aug_mask, tf.float32)\n        return aug_img,aug_mask\n\n    def process_data(image, mask):\n        aug_img,aug_mask = tf.numpy_function(func=aug_fn, inp=[image,mask], Tout=[tf.float32,tf.float32])\n        return aug_img, aug_mask\n    \n    def set_shapes(img, mask, img_shape=(image_size,image_size,3)):\n        img.set_shape(img_shape)\n        mask.set_shape([img_shape[0],img_shape[1],1])\n        return img, mask\n    \n    ds_alb = train_ds.map(partial(process_data), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE)\n    ds_alb = ds_alb.map(set_shapes, num_parallel_calls=AUTOTUNE)\n    ds_alb = ds_alb.repeat()\n    ds_alb = ds_alb.batch(batch)\n    return ds_alb","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.688833Z","iopub.execute_input":"2021-11-19T20:26:32.689601Z","iopub.status.idle":"2021-11-19T20:26:32.70103Z","shell.execute_reply.started":"2021-11-19T20:26:32.689562Z","shell.execute_reply":"2021-11-19T20:26:32.700179Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def augment_val_data(val_ds):\n    transforms = A.Compose([\n                A.CenterCrop(image_size, image_size),\n                ], p=1)\n#     transforms = A.Compose([\n#             A.RandomResizedCrop(image_size, image_size),\n#             A.Transpose(p=0.5),\n#             A.HorizontalFlip(p=0.5),\n#             A.VerticalFlip(p=0.5),\n#             A.ShiftScaleRotate(p=0.5),\n#             A.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n#             A.RandomBrightnessContrast(brightness_limit=(-0.1, 0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n#             A.CoarseDropout(p=0.5),\n#             A.Cutout(p=0.5),\n#             ], p=1)\n    \n    def aug_fn(image,mask):\n#         data_img = {\"image\":image}\n#         aug_data_img = transforms(**data_img)\n        \n#         data_mask = {'mask':mask}\n#         aug_data_mask = transforms(**data_mask)\n\n        aug_data_all = transforms(image=image,mask=mask)\n        aug_img = aug_data_all[\"image\"]\n        aug_img = tf.cast(aug_img, tf.float32)\n        \n        aug_mask = aug_data_all[\"mask\"]\n        aug_mask = tf.cast(aug_mask, tf.float32)\n        return aug_img,aug_mask\n\n    def process_data(image, mask):\n        aug_img,aug_mask = tf.numpy_function(func=aug_fn, inp=[image,mask], Tout=[tf.float32,tf.float32])\n        return aug_img, aug_mask\n    \n    def set_shapes(img, mask, img_shape=(image_size,image_size,3)):\n        img.set_shape(img_shape)\n        mask.set_shape([img_shape[0],img_shape[1],1])\n        return img, mask\n    \n    ds_alb = val_ds.map(partial(process_data), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE)\n    ds_alb = ds_alb.map(set_shapes, num_parallel_calls=AUTOTUNE).batch(batch)\n    return ds_alb","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.702244Z","iopub.execute_input":"2021-11-19T20:26:32.703005Z","iopub.status.idle":"2021-11-19T20:26:32.714937Z","shell.execute_reply.started":"2021-11-19T20:26:32.702945Z","shell.execute_reply":"2021-11-19T20:26:32.714181Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from functools import partial\n\nbatch = 4\nimage_size = 256\ninput_shape = (image_size, image_size, 3)\ntrain_alb = augment_train_data(training_data)\nval_alb = augment_val_data(validation_data)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.716257Z","iopub.execute_input":"2021-11-19T20:26:32.716581Z","iopub.status.idle":"2021-11-19T20:26:32.905191Z","shell.execute_reply.started":"2021-11-19T20:26:32.716545Z","shell.execute_reply":"2021-11-19T20:26:32.904502Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":">The above two functions tf_parse and tf_dataset are used to build the dataset pipeline.\nThe tf_dataset function create a tf.data pipeline which takes a list of images, masks paths and the batch size. The tf_parse function parses a single image and mask path.","metadata":{}},{"cell_type":"markdown","source":"# Attention based Residual Unet:\nOkay! Now we are talking some fancy names xD. I have seen some NBs and found people saying that using Unet is not worth it, and I agree on that. Using MRCNN or detectron looks promising. I did not realise that earlie. So, at the last hour i though of using Unet attention based Unet,but after that I saw some code on residual attention Unet. So, I decided to give that a shot. And also there is an algorithm called `watershed` algorithm which helps to perform instance segmentation using Unet, Im thinking of doing that in future.  \n\n## Reasons for using Attention:\n<!-- ![sdfsdf](https://miro.medium.com/max/700/1*J3Rlq9K-ldxdJ5XWa4cD9g.png) -->\n<p align=\"center\">\n<img width = \"970\" src=\"https://miro.medium.com/max/700/1*J3Rlq9K-ldxdJ5XWa4cD9g.png\">\n</p>\n\n\nWe all know about the Unet model, its a encoder and decoder based architecture with multi-stage cascaded convolutional neural networks. It also takes feature representation from encoder/downsampling path into consideration by concating the it the decoder/upsampling path. And after that it uses a dense layer for prediction. But, if you think carefully, it will come to your mind that the feature vectors from the starting of the downsampling path is not robust, and using that with the same level upsampling does not give much improvements. So, to address this issue we bring `attention` into the picture. `Attention` uses the `same level` downsampling feature map and also the `\"feature map from one level below\"` and pass that through an attention gate, which helps to extract better feature map for concating with the cooresponding upsampling level.It also helps to focus on the important part of the image that is relevant. That drags down the Time per epoch. Check out the below gradcam visualization,\n\n<br>\n<p align=\"center\">\n<img width = \"970\" src=\"https://www.researchgate.net/publication/330877339/figure/fig6/AS:863282093096962@1582833898320/The-figure-shows-the-attention-coefficients-a-l-s-2-a-l-s-3-across-different.jpg\">\n</p>\n\n\nNow, the topic boiles down to how does `attention gates` work? Lets, Check out the below image, \n\n<p align=\"center\">\n<img width = \"970\" src=\"https://miro.medium.com/max/700/1*kEuDd2mCqTZkyNmkTGdg9Q.png\">\n</p>\n\n> In the W_g, we have strib of 1 and for the W_x there is stribe of 2. But for both of them the number of filters are same. W_g and W_x helps to match the size of g and x. As g is coming from one level lower than the current one, thats why it has smaller shape. Now, that we have same size for both, we can go ahead and add them. Because of the addition aligned weights gets larger but the unaligned weight gets even smaller. After that we send that through `relu` activation which will make all the weights >=0. After that we pass that through a Conv block of stribe = 1 and filters = 1. Then we add a `sigmoid` layer which helps to convert the values in between the limit of 0 to 1. Then we resample it to the size of x. and multiply the x and the resampled result. This becomes the final feature map which gets concatinated with the current upsample later.\n\n\n\n## Reasons for using Residual/skip connections:\nWell when we talk about Deep Learning, the models that comes to the mind is BERT, transformer , YOLO, MRCNN etc. All these models are big in size where duzons of layers stacked together. But some times making a model too big comes with a price. Which is nothing but the `vanishing gradient problem`. It happens when the model is too big, and at the time of backpropagation the updating of the model weights becomes smaller. For dealing with this problem ResidualNet was introduced. The main point of residualnet was the skip connections. Even if the weights get smaller because we have skip connection it can't reduce the feature. Check out the below image,\n\n<p align=\"center\">\n<img width = \"500\" src=\"https://i.imgur.com/xBk3ZGT.png\">\n</p>\n\n> Think of the (a) image a conv block for Unet which is consist of two `conv+relu` layer. (b) image is same too, just with a skip connection. If the Unet model is having only the Skip connections then the model would look like this,\n\n<br>\n<br>\n<p align=\"center\">\n<img width = \"970\" src=\"https://www.researchgate.net/profile/Lei-Xiang/publication/327748708/figure/fig2/AS:698665882619906@1543586335266/Illustration-of-the-proposed-Res-Unet-architecture-as-the-generator.png\">\n</p>\n","metadata":{}},{"cell_type":"code","source":"def conv_block(x, filter_size, size, dropout, batch_norm=False):\n    \n    conv = layers.Conv2D(size, (filter_size, filter_size), padding=\"same\")(x)\n    if batch_norm is True:\n        conv = layers.BatchNormalization(axis=3)(conv)\n    conv = layers.Activation(\"relu\")(conv)\n\n    conv = layers.Conv2D(size, (filter_size, filter_size), padding=\"same\")(conv)\n    if batch_norm is True:\n        conv = layers.BatchNormalization(axis=3)(conv)\n    conv = layers.Activation(\"relu\")(conv)\n    \n    if dropout > 0:\n        conv = layers.Dropout(dropout)(conv)\n\n    return conv\n\n\ndef repeat_elem(tensor, rep):\n\n     return layers.Lambda(lambda x, repnum: K.repeat_elements(x, repnum, axis=3),\n                          arguments={'repnum': rep})(tensor)\n\n\ndef res_conv_block(x, filter_size, size, dropout, batch_norm=False):\n    \n\n    conv = layers.Conv2D(size, (filter_size, filter_size), padding='same')(x)\n    if batch_norm is True:\n        conv = layers.BatchNormalization(axis=3)(conv)\n    conv = layers.Activation('relu')(conv)\n    \n    conv = layers.Conv2D(size, (filter_size, filter_size), padding='same')(conv)\n    if batch_norm is True:\n        conv = layers.BatchNormalization(axis=3)(conv)\n    #conv = layers.Activation('relu')(conv)    #Activation before addition with shortcut\n    if dropout > 0:\n        conv = layers.Dropout(dropout)(conv)\n\n    shortcut = layers.Conv2D(size, kernel_size=(1, 1), padding='same')(x)\n    if batch_norm is True:\n        shortcut = layers.BatchNormalization(axis=3)(shortcut)\n\n    res_path = layers.add([shortcut, conv])\n    res_path = layers.Activation('relu')(res_path)    #Activation after addition with shortcut (Original residual block)\n    return res_path\n\ndef gating_signal(input, out_size, batch_norm=False):\n    \n    x = layers.Conv2D(out_size, (1, 1), padding='same')(input)\n    if batch_norm:\n        x = layers.BatchNormalization()(x)\n    x = layers.Activation('relu')(x)\n    return x\n\ndef attention_block(x, gating, inter_shape):\n    shape_x = K.int_shape(x)\n    shape_g = K.int_shape(gating)\n\n# Getting the x signal to the same shape as the gating signal\n    theta_x = layers.Conv2D(inter_shape, (2, 2), strides=(2, 2), padding='same')(x)  # 16\n    shape_theta_x = K.int_shape(theta_x)\n\n# Getting the gating signal to the same number of filters as the inter_shape\n    phi_g = layers.Conv2D(inter_shape, (1, 1), padding='same')(gating)\n    upsample_g = layers.Conv2DTranspose(inter_shape, (3, 3),\n                                 strides=(shape_theta_x[1] // shape_g[1], shape_theta_x[2] // shape_g[2]),\n                                 padding='same')(phi_g)  # 16\n\n    concat_xg = layers.add([upsample_g, theta_x])\n    act_xg = layers.Activation('relu')(concat_xg)\n    psi = layers.Conv2D(1, (1, 1), padding='same')(act_xg)\n    sigmoid_xg = layers.Activation('sigmoid')(psi)\n    shape_sigmoid = K.int_shape(sigmoid_xg)\n    upsample_psi = layers.UpSampling2D(size=(shape_x[1] // shape_sigmoid[1], shape_x[2] // shape_sigmoid[2]))(sigmoid_xg)  # 32\n\n    upsample_psi = repeat_elem(upsample_psi, shape_x[3])\n\n    y = layers.multiply([upsample_psi, x])\n\n    result = layers.Conv2D(shape_x[3], (1, 1), padding='same')(y)\n    result_bn = layers.BatchNormalization()(result)\n    return result_bn\n\n\n\ndef Attention_ResUNet(input_shape, NUM_CLASSES=1, dropout_rate=0.0, batch_norm=True):\n    '''\n    attention  baed Rsidual UNet\n    \n    '''\n    # network structure\n    FILTER_NUM = 64 # number of basic filters for the first layer\n    FILTER_SIZE = 3 # size of the convolutional filter\n    UP_SAMP_SIZE = 2 # size of upsampling filters\n    # input data\n    # dimension of the image depth\n    inputs = layers.Input(input_shape, dtype=tf.float32)\n    axis = 3\n\n    # Downsampling layers\n    # DownRes 1, double residual convolution + pooling\n    conv_128 = res_conv_block(inputs, FILTER_SIZE, FILTER_NUM, dropout_rate, batch_norm)\n    pool_64 = layers.MaxPooling2D(pool_size=(2,2))(conv_128)\n    # DownRes 2\n    conv_64 = res_conv_block(pool_64, FILTER_SIZE, 2*FILTER_NUM, dropout_rate, batch_norm)\n    pool_32 = layers.MaxPooling2D(pool_size=(2,2))(conv_64)\n    # DownRes 3\n    conv_32 = res_conv_block(pool_32, FILTER_SIZE, 4*FILTER_NUM, dropout_rate, batch_norm)\n    pool_16 = layers.MaxPooling2D(pool_size=(2,2))(conv_32)\n    # DownRes 4\n    conv_16 = res_conv_block(pool_16, FILTER_SIZE, 8*FILTER_NUM, dropout_rate, batch_norm)\n    pool_8 = layers.MaxPooling2D(pool_size=(2,2))(conv_16)\n    # DownRes 5, convolution only\n    conv_8 = res_conv_block(pool_8, FILTER_SIZE, 16*FILTER_NUM, dropout_rate, batch_norm)\n\n    # Upsampling layers\n    # UpRes 6, attention gated concatenation + upsampling + double residual convolution\n    gating_16 = gating_signal(conv_8, 8*FILTER_NUM, batch_norm)\n    att_16 = attention_block(conv_16, gating_16, 8*FILTER_NUM)\n    up_16 = layers.UpSampling2D(size=(UP_SAMP_SIZE, UP_SAMP_SIZE), data_format=\"channels_last\")(conv_8)\n    up_16 = layers.concatenate([up_16, att_16], axis=axis)\n    up_conv_16 = res_conv_block(up_16, FILTER_SIZE, 8*FILTER_NUM, dropout_rate, batch_norm)\n    # UpRes 7\n    gating_32 = gating_signal(up_conv_16, 4*FILTER_NUM, batch_norm)\n    att_32 = attention_block(conv_32, gating_32, 4*FILTER_NUM)\n    up_32 = layers.UpSampling2D(size=(UP_SAMP_SIZE, UP_SAMP_SIZE), data_format=\"channels_last\")(up_conv_16)\n    up_32 = layers.concatenate([up_32, att_32], axis=axis)\n    up_conv_32 = res_conv_block(up_32, FILTER_SIZE, 4*FILTER_NUM, dropout_rate, batch_norm)\n    # UpRes 8\n    gating_64 = gating_signal(up_conv_32, 2*FILTER_NUM, batch_norm)\n    att_64 = attention_block(conv_64, gating_64, 2*FILTER_NUM)\n    up_64 = layers.UpSampling2D(size=(UP_SAMP_SIZE, UP_SAMP_SIZE), data_format=\"channels_last\")(up_conv_32)\n    up_64 = layers.concatenate([up_64, att_64], axis=axis)\n    up_conv_64 = res_conv_block(up_64, FILTER_SIZE, 2*FILTER_NUM, dropout_rate, batch_norm)\n    # UpRes 9\n    gating_128 = gating_signal(up_conv_64, FILTER_NUM, batch_norm)\n    att_128 = attention_block(conv_128, gating_128, FILTER_NUM)\n    up_128 = layers.UpSampling2D(size=(UP_SAMP_SIZE, UP_SAMP_SIZE), data_format=\"channels_last\")(up_conv_64)\n    up_128 = layers.concatenate([up_128, att_128], axis=axis)\n    up_conv_128 = res_conv_block(up_128, FILTER_SIZE, FILTER_NUM, dropout_rate, batch_norm)\n\n    # 1*1 convolutional layers\n    \n    conv_final = layers.Conv2D(NUM_CLASSES, kernel_size=(1,1))(up_conv_128)\n    conv_final = layers.BatchNormalization(axis=axis)(conv_final)\n    conv_final = layers.Activation('sigmoid')(conv_final)  #Change to softmax for multichannel\n\n    # Model integration\n    model = models.Model(inputs, conv_final, name=\"AttentionResUNet\")\n    return model\n\n","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.90652Z","iopub.execute_input":"2021-11-19T20:26:32.906927Z","iopub.status.idle":"2021-11-19T20:26:32.93852Z","shell.execute_reply.started":"2021-11-19T20:26:32.90689Z","shell.execute_reply":"2021-11-19T20:26:32.937774Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Model Training:","metadata":{}},{"cell_type":"code","source":"def iou(y_true, y_pred):\n    def f(y_true, y_pred):\n        \n        intersection = (y_true * y_pred).sum()\n        union = y_true.sum() + y_pred.sum() - intersection\n        x = (intersection + 1e-15) / (union + 1e-15)\n        x = x.astype(np.float32)\n        return x\n    return tf.numpy_function(f, [y_true, y_pred], tf.float32)\n\ndef dice_loss(y_true, y_pred):\n    smooth = 1.\n    y_true_f = K.flatten(y_true)\n    y_pred_f = K.flatten(y_pred)\n    intersection = y_true_f * y_pred_f\n    score = (2. * K.sum(intersection) + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n    return 1. - score\n\ndef bce_dice_loss(y_true, y_pred):\n    return binary_crossentropy(tf.cast(y_true, tf.float32), y_pred) + 0.5 * dice_loss(tf.cast(y_true, tf.float32), y_pred)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.940966Z","iopub.execute_input":"2021-11-19T20:26:32.941247Z","iopub.status.idle":"2021-11-19T20:26:32.952823Z","shell.execute_reply.started":"2021-11-19T20:26:32.941192Z","shell.execute_reply":"2021-11-19T20:26:32.952064Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model_output = \"./model_unet.h5\"\n# \"save the Keras model or model weights at some frequency\"\nmodel_checkpoint = tf.keras.callbacks.ModelCheckpoint(\n    model_output,\n    save_best_only=True,\n    save_weights_only=True,\n)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.953969Z","iopub.execute_input":"2021-11-19T20:26:32.954795Z","iopub.status.idle":"2021-11-19T20:26:32.961941Z","shell.execute_reply.started":"2021-11-19T20:26:32.954728Z","shell.execute_reply":"2021-11-19T20:26:32.961172Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Initialize wandb with your project name\nrun = wandb.init(project='kaggle sartorius',entity=\"somusan\",\n                 config={  # and include hyperparameters and metadata\n                     \"learning_rate\": 1e-4,\n                     \"epochs\": 60,\n                     \"batch_size\": 4,\n                     \"loss_function\": \"bce_dice_loss\",\n                     \"architecture\": \"attention based residual Unet\",\n                     \"dataset\": \"sartorius:brain cell i/s\"\n                 })\nconfig = wandb.config  # We'll use this to configure our experiment\n\n\nconfig={ \n     \"learning_rate\": 1e-4,\n     \"epochs\": 60,\n     \"batch_size\": 4,\n     \"loss_function\": \"bce_dice_loss\",\n     \"architecture\": \"attention based residual Unet\",\n     \"dataset\": \"sartorius:brain cell i/s\"\n}","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:26:32.963218Z","iopub.execute_input":"2021-11-19T20:26:32.963747Z","iopub.status.idle":"2021-11-19T20:26:39.812082Z","shell.execute_reply.started":"2021-11-19T20:26:32.963711Z","shell.execute_reply":"2021-11-19T20:26:39.811306Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import segmentation_models as sm\nIMG_HEIGHT = 256\nIMG_WIDTH  = 256\nIMG_CHANNELS = 3\nnum_labels = 1\ninput_shape = (IMG_HEIGHT,IMG_WIDTH,IMG_CHANNELS)\nbatch = config[\"batch_size\"]\n\n\ntrain_steps = len(train_x)//batch\nvalid_steps = len(valid_x)//batch\n\nif len(train_x) % batch != 0:\n    train_steps += 1\nif len(valid_x) % batch != 0:\n    valid_steps += 1\n    \n\n\natt_res_unet_model = Attention_ResUNet(input_shape,dropout_rate=0.3)\n\nopt = tf.keras.optimizers.Adam(config[\"learning_rate\"])\n# metrics = [\"acc\", iou]\nmetrics = [sm.metrics.IOUScore(threshold=0.5), sm.metrics.FScore(threshold=0.5)]\natt_res_unet_model.compile(loss=bce_dice_loss, optimizer=opt, metrics=metrics)\n\n\n# att_res_unet_model.compile(optimizer=Adam(lr = 1e-3), loss='binary_crossentropy', \n#               metrics=['accuracy', jacard_coef])\n\nprint(att_res_unet_model.summary())","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-11-19T20:26:39.813745Z","iopub.execute_input":"2021-11-19T20:26:39.814143Z","iopub.status.idle":"2021-11-19T20:26:40.994972Z","shell.execute_reply.started":"2021-11-19T20:26:39.814106Z","shell.execute_reply":"2021-11-19T20:26:40.968318Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"att_res_unet_history = att_res_unet_model.fit(train_alb, \n                    verbose=1,\n                    batch_size = batch,\n                    validation_data=val_alb, \n                    steps_per_epoch=train_steps,\n                    validation_steps=valid_steps,\n                    shuffle=False,\n                    epochs=config[\"epochs\"],# \n                    callbacks=[model_checkpoint,WandbCallback()]) # WandbCallback()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-11-19T20:26:41.007894Z","iopub.execute_input":"2021-11-19T20:26:41.008108Z","iopub.status.idle":"2021-11-19T20:27:43.885917Z","shell.execute_reply.started":"2021-11-19T20:26:41.008081Z","shell.execute_reply":"2021-11-19T20:27:43.884127Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"att_res_unet_model.save('./attn_res_unet_60_tta_aug_1e4_metrics_change.h5')","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.887226Z","iopub.status.idle":"2021-11-19T20:27:43.88766Z","shell.execute_reply.started":"2021-11-19T20:27:43.887435Z","shell.execute_reply":"2021-11-19T20:27:43.887459Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"wandb.save(\"./attn_res_unet_60_tta_aug_1e4_metrics_change.h5\")","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.888977Z","iopub.status.idle":"2021-11-19T20:27:43.889484Z","shell.execute_reply.started":"2021-11-19T20:27:43.88924Z","shell.execute_reply":"2021-11-19T20:27:43.889265Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Model performance: ","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\nloss = att_res_unet_history.history['loss']\nval_loss = att_res_unet_history.history['val_loss']\nepochs = range(1, len(loss) + 1)\nplt.plot(epochs, loss, 'y', label='Training loss')\nplt.plot(epochs, val_loss, 'r', label='Validation loss')\nplt.title('Training and validation loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.890728Z","iopub.status.idle":"2021-11-19T20:27:43.891573Z","shell.execute_reply.started":"2021-11-19T20:27:43.891331Z","shell.execute_reply":"2021-11-19T20:27:43.891356Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# att_res_unet_history.history","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.89319Z","iopub.status.idle":"2021-11-19T20:27:43.893719Z","shell.execute_reply.started":"2021-11-19T20:27:43.893433Z","shell.execute_reply":"2021-11-19T20:27:43.893471Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\nacc = att_res_unet_history.history['iou_score']\nval_acc = att_res_unet_history.history['val_iou_score']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'y', label='Training IoU')\nplt.plot(epochs, val_acc, 'r', label='Validation IoU')\nplt.title('Training and validation IoU')\nplt.xlabel('Epochs')\nplt.ylabel('IOU')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.895088Z","iopub.status.idle":"2021-11-19T20:27:43.895769Z","shell.execute_reply.started":"2021-11-19T20:27:43.895501Z","shell.execute_reply":"2021-11-19T20:27:43.895528Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(15,5))\nacc = att_res_unet_history.history['f1-score']\nval_acc = att_res_unet_history.history['val_f1-score']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'y', label='Training F1')\nplt.plot(epochs, val_acc, 'r', label='Validation F1')\nplt.title('Training and validation F1')\nplt.xlabel('Epochs')\nplt.ylabel('IOU')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.897421Z","iopub.status.idle":"2021-11-19T20:27:43.898012Z","shell.execute_reply.started":"2021-11-19T20:27:43.897639Z","shell.execute_reply":"2021-11-19T20:27:43.897664Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Inference time:\n<!-- #### Loading another model trained on 100 epoch, you can also replace the the current trained one  -->","metadata":{}},{"cell_type":"code","source":"# with CustomObjectScope({'iou': iou}):\n#     model = Attention_ResUNet(input_shape,dropout_rate=0.3)\n#     model.load_weights(\"./model_unet.h5\")\nmodel = tf.keras.models.load_model(\"./attn_res_unet_60_tta_aug_1e4_metrics_change.h5\",custom_objects={\"bce_dice_loss\":bce_dice_loss,\"iou_score\": sm.metrics.IOUScore(threshold=0.5),\"f1-score\": sm.metrics.FScore(threshold=0.5)})","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.900079Z","iopub.status.idle":"2021-11-19T20:27:43.901007Z","shell.execute_reply.started":"2021-11-19T20:27:43.900757Z","shell.execute_reply":"2021-11-19T20:27:43.900782Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# # restore the model file \"model.h5\" from a specific run by user \"lavanyashukla\"\n# # in project \"save_and_restore\" from run \"10pr4joa\"\n# best_model = wandb.restore('attn_res_unet_100_tta_aug_1e4.h5', run_path=\"somusan/kaggle sartorius/5335czpx\")\n\n# # use the \"name\" attribute of the returned object if your framework expects a filename, e.g. as in Keras\n# model = Attention_ResUNet(input_shape,dropout_rate=0.3)\n# model.load_weights(best_model.name)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.902099Z","iopub.status.idle":"2021-11-19T20:27:43.902991Z","shell.execute_reply.started":"2021-11-19T20:27:43.902747Z","shell.execute_reply":"2021-11-19T20:27:43.902772Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def read_image(path):\n    x = cv2.imread(path)\n    x = cv2.resize(x, (256, 256))\n#     x = np.expand_dims(x, axis=-1)\n    x = x.astype(np.double)\n    return x\n\ndef read_mask(path):\n    x = cv2.imread(path, cv2.IMREAD_GRAYSCALE)\n    x = cv2.resize(x, (256, 256))\n    x = np.expand_dims(x, axis=-1)\n    x = x.astype(np.double)\n    return x","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.904083Z","iopub.status.idle":"2021-11-19T20:27:43.904967Z","shell.execute_reply.started":"2021-11-19T20:27:43.904721Z","shell.execute_reply":"2021-11-19T20:27:43.904747Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Inference on some validation data:","metadata":{}},{"cell_type":"code","source":"\ny_data = \"../input/sartorious-nb-1-data-preprocessing-visualization\"\nids = ['fe3e30f849f4','ffdb3cc02eef','0140b3c8f445','529f53145d75','b89f9cca5384','fe3e30f849f4']\nsample_image_df = df[df['id'] == ids[4]]\nx = f\"../input/sartorius-cell-instance-segmentation/train/{sample_image_df['id'].iloc[0]}.png\"\nx = read_image(x)\ny_file = f\"../input/sartorious-nb-1-data-preprocessing-visualization/masks/img/{sample_image_df['id'].iloc[0]}.png\"\n\ny = read_mask(y_file)\n# print(\"y shape\",y.shape)\ny_pred = model.predict(np.expand_dims(x, axis=0))[0]>0.5\n# print(\"y_pred shape\",y_pred.shape)\nh, w, _ = x.shape\nwhite_line = np.ones((h, 10, 3)) * 255.0\n\n\n# wandb.init(project='kaggle sartorius',entity=\"somusan\")\n           \n           \nheight = 50\nwidth = 50\nfig, (ax1,ax2) = plt.subplots(nrows = 1,ncols = 2,figsize=(width,height))\nax = (ax1,ax2)\n\n\nax[0].imshow(y)\nax[0].set_xlabel('gt',fontsize=50)\nax[1].imshow(y_pred)\nax[1].set_xlabel('prediction',fontsize=50)\n\n# wandb.log({\"plot\": fig})\n# wandb.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.906066Z","iopub.status.idle":"2021-11-19T20:27:43.906917Z","shell.execute_reply.started":"2021-11-19T20:27:43.906686Z","shell.execute_reply":"2021-11-19T20:27:43.90671Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def post_process(probability, threshold=0.5, min_size=300):\n    mask = cv2.threshold(probability, threshold, 1, cv2.THRESH_BINARY)[1]\n    num_component, component = cv2.connectedComponents(mask.astype(np.uint8))\n    predictions = []\n    for c in range(1, num_component):\n        p = (component == c)\n        if p.sum() > min_size:\n            a_prediction = np.zeros((520, 704), np.float32)\n            a_prediction[p] = 1\n            predictions.append(a_prediction)\n    return predictions","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.908Z","iopub.status.idle":"2021-11-19T20:27:43.9089Z","shell.execute_reply.started":"2021-11-19T20:27:43.908653Z","shell.execute_reply":"2021-11-19T20:27:43.908678Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def rle_encoding(x):\n    dots = np.where(x.flatten() == 1)[0]\n    run_lengths = []\n    prev = -2\n    for b in dots:\n        if (b>prev+1): run_lengths.extend((b + 1, 0))\n        run_lengths[-1] += 1\n        prev = b\n    return ' '.join(map(str, run_lengths))\n\n# print(rle_encoding(predictions[0]))","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.910003Z","iopub.status.idle":"2021-11-19T20:27:43.911618Z","shell.execute_reply.started":"2021-11-19T20:27:43.91136Z","shell.execute_reply":"2021-11-19T20:27:43.911387Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sub_file = \"../input/sartorius-cell-instance-segmentation/sample_submission.csv\"\nsub_df = pd.read_csv(sub_file)\nsub_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.912986Z","iopub.status.idle":"2021-11-19T20:27:43.913422Z","shell.execute_reply.started":"2021-11-19T20:27:43.913183Z","shell.execute_reply":"2021-11-19T20:27:43.913206Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Basic TTA","metadata":{}},{"cell_type":"code","source":"def tta_preds(image_dataset):\n#Now that we know the transformations are working, let us extend to all predictions\n    predictions = []\n    for image in image_dataset:\n        plt.figure(figsize=(20,20))\n        pred_original = model.predict(np.expand_dims(image, axis=0))\n        plt.subplot(1,5,1)\n        plt.imshow(np.reshape(pred_original,(256,256)))\n        \n        pred_lr = model.predict(np.expand_dims(np.fliplr(image), axis=0))\n        pred_lr = np.fliplr(pred_lr.squeeze(0))\n        plt.subplot(1,5,2)\n        plt.imshow(np.reshape(pred_lr,(256,256)))\n\n        \n        pred_ud = model.predict(np.expand_dims(np.flipud(image), axis=0))\n        pred_ud = np.flipud(pred_ud.squeeze(0))\n        plt.subplot(1,5,3)\n        plt.imshow(np.reshape(pred_ud,(256,256)))\n        \n        \n        \n        pred_lr_ud = model.predict(np.expand_dims(np.fliplr(np.flipud(image)), axis=0))\n        pred_lr_ud = np.fliplr(np.flipud(pred_lr_ud).squeeze(0))\n        plt.subplot(1,5,4)\n        plt.imshow(np.reshape(pred_lr_ud,(256,256)))\n\n        \n        \n        preds = (pred_original + pred_lr + pred_ud + pred_lr_ud) / 4\n#         plt.subplot(1,5,4)\n#         plt.imshow(np.reshape(preds,(256,256)))\n        \n#         break\n        predictions.append(preds)\n\n\n    predictions = np.array(predictions)\n\n    threshold = 0.5\n    predictions_th = predictions > threshold\n    \n    return predictions_th\n\ntest_dir = \"../input/sartorius-cell-instance-segmentation/test\"\nids = sub_df[\"id\"].tolist()\n\nsub_list = []\nsub_img = []\nfor i in ids:\n    file_path = test_dir + \"/\" + i + \".png\"\n    x = read_image(file_path)\n    sub_img.append(x)\n\nsub_preds = tta_preds(sub_img)\n\n\nfor ide,j in enumerate(sub_preds):\n#     print(j.astype(np.float32).shape)\n#     break\n    y_pred = np.reshape(j.astype(np.uint16),(256,256))\n#     print(j.astype(np.float32).shape)\n\n    y_pred = cv2.resize(y_pred,(704,520),interpolation = cv2.INTER_AREA)\n#     print(y_pred.shape)\n    predictions = post_process(y_pred)\n    for k in predictions:\n        sub_list.append((ids[ide],rle_encoding(k)))","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.914777Z","iopub.status.idle":"2021-11-19T20:27:43.915185Z","shell.execute_reply.started":"2021-11-19T20:27:43.914966Z","shell.execute_reply":"2021-11-19T20:27:43.914987Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sub = pd.DataFrame(sub_list,columns=['id','predicted'])\nsub.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.916567Z","iopub.status.idle":"2021-11-19T20:27:43.916988Z","shell.execute_reply.started":"2021-11-19T20:27:43.916765Z","shell.execute_reply":"2021-11-19T20:27:43.916788Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sub.to_csv(\"submission.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.918039Z","iopub.status.idle":"2021-11-19T20:27:43.918448Z","shell.execute_reply.started":"2021-11-19T20:27:43.918215Z","shell.execute_reply":"2021-11-19T20:27:43.918251Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# ref: https://www.kaggle.com/inversion/run-length-decoding-quick-start\ndef rle_decode(mask_rle, shape, color=1):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height, width, channels) of array to return\n    color: color for the mask\n    Returns numpy array (mask)\n\n    '''\n    s = mask_rle.split()\n\n    starts = list(map(lambda x: int(x) - 1, s[0::2]))\n    lengths = list(map(int, s[1::2]))\n    ends = [x + y for x, y in zip(starts, lengths)]\n    if len(shape)==3:\n        img = np.zeros((shape[0] * shape[1], shape[2]), dtype=np.float32)\n    else:\n        img = np.zeros(shape[0] * shape[1], dtype=np.float32)\n    for start, end in zip(starts, ends):\n        img[start : end] = color\n\n    return img.reshape(shape)\n\n\n\ndef build_masks(df_train, image_id, input_shape):\n    height, width = input_shape\n    labels = df_train[df_train[\"id\"] == image_id][\"predicted\"].tolist()\n    mask = np.zeros((height, width))\n    for label in labels:\n        mask += rle_decode(label, shape=(height, width))\n#         plt.imshow(mask)\n#         break\n    mask = mask.clip(0, 1)\n    return mask","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.919985Z","iopub.status.idle":"2021-11-19T20:27:43.921645Z","shell.execute_reply.started":"2021-11-19T20:27:43.921395Z","shell.execute_reply":"2021-11-19T20:27:43.921421Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sub[\"id\"].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.922885Z","iopub.status.idle":"2021-11-19T20:27:43.923433Z","shell.execute_reply.started":"2021-11-19T20:27:43.92319Z","shell.execute_reply":"2021-11-19T20:27:43.923215Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sample_filename = '7ae19de7bc2a'\nsample_path = f\"../input/sartorius-cell-instance-segmentation/test/{sample_filename}.png\"\nsample_img = plt.imread(sample_path)\n# sample_rles = sample_image_df['annotation'].values\n\nCOLOR_MAP = 'seismic'\nsample_masks1=build_masks(sub,sample_filename,input_shape=(520, 704))\n# sample_masks2=build_masks(sample_filename,input_shape=(520, 704), colors=True)\n\nfig, axs = plt.subplots(1,2, figsize=(18, 18))\n\naxs[0].imshow(sample_img,cmap='seismic')\naxs[0].set_title('original image', fontsize=16)\n\n\naxs[1].imshow(sample_masks1,cmap='gray')\naxs[1].set_title('mask image', fontsize=16)\n\nplt.imshow(sample_masks1)","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.924811Z","iopub.status.idle":"2021-11-19T20:27:43.925665Z","shell.execute_reply.started":"2021-11-19T20:27:43.925433Z","shell.execute_reply":"2021-11-19T20:27:43.925457Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Checking overlapping:","metadata":{}},{"cell_type":"code","source":"def remove_overlapping_pixels(mask, other_masks):\n    for other_mask in other_masks:\n        if np.sum(np.logical_and(mask, other_mask)) > 0:\n            mask[np.logical_and(mask, other_mask)] = 0\n    return mask\n\ndef combine_masks(masks, mask_threshold):\n    \"\"\"\n    combine masks into one image\n    \"\"\"\n    maskimg = np.zeros((HEIGHT, WIDTH))\n    \n    for m, mask in enumerate(masks,1):\n        maskimg[mask>mask_threshold] = m\n    return maskimg\n\n\ndef tta_preds(image_dataset):\n#Now that we know the transformations are working, let us extend to all predictions\n    predictions = []\n    for image in image_dataset:\n#         plt.figure(figsize=(20,20))\n        pred_original = model.predict(np.expand_dims(image, axis=0))\n#         plt.subplot(1,5,1)\n#         plt.imshow(np.reshape(pred_original,(256,256)))\n        \n        pred_lr = model.predict(np.expand_dims(np.fliplr(image), axis=0))\n        pred_lr = np.fliplr(pred_lr.squeeze(0))\n#         plt.subplot(1,5,2)\n#         plt.imshow(np.reshape(pred_lr,(256,256)))\n\n        \n        pred_ud = model.predict(np.expand_dims(np.flipud(image), axis=0))\n        pred_ud = np.flipud(pred_ud.squeeze(0))\n#         plt.subplot(1,5,3)\n#         plt.imshow(np.reshape(pred_ud,(256,256)))\n        \n        \n        \n        pred_lr_ud = model.predict(np.expand_dims(np.fliplr(np.flipud(image)), axis=0))\n        pred_lr_ud = np.fliplr(np.flipud(pred_lr_ud).squeeze(0))\n#         plt.subplot(1,5,4)\n#         plt.imshow(np.reshape(pred_lr_ud,(256,256)))\n\n        \n        \n        preds = (pred_original + pred_lr + pred_ud + pred_lr_ud) / 4\n#         plt.subplot(1,5,4)\n#         plt.imshow(np.reshape(preds,(256,256)))\n        \n#         break\n        predictions.append(preds)\n\n\n    predictions = np.array(predictions)\n\n    threshold = 0.5\n    predictions_th = predictions > threshold\n    \n    return predictions_th\n\ntest_dir = \"../input/sartorius-cell-instance-segmentation/test\"\nids = sub_df[\"id\"].tolist()\n\nsub_list = []\nsub_img = []\nfor i in ids:\n    file_path = test_dir + \"/\" + i + \".png\"\n    x = read_image(file_path)\n    sub_img.append(x)\n\nsub_preds = tta_preds(sub_img)\n\n\nfor ide,j in enumerate(sub_preds):\n#     print(j.astype(np.float32).shape)\n#     break\n    y_pred = np.reshape(j.astype(np.uint16),(256,256))\n#     print(j.astype(np.float32).shape)\n\n    y_pred = cv2.resize(y_pred,(704,520),interpolation = cv2.INTER_AREA)\n#     print(y_pred.shape)\n    predictions = post_process(y_pred)\n#     for k in predictions:\n#         sub_list.append((ids[ide],rle_encoding(k)))\n    mask_1 = []\n\n    for i in predictions:\n        b_mask = remove_overlapping_pixels(i,mask_1)\n        mask_1.append(b_mask)\n        sub_list.append((ids[ide],rle_encoding(b_mask)))\n\n\n        \nsub_df_man = pd.DataFrame(sub_list,columns=['id','predicted'])\nsub_df_man.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-19T20:27:43.92687Z","iopub.status.idle":"2021-11-19T20:27:43.927707Z","shell.execute_reply.started":"2021-11-19T20:27:43.927466Z","shell.execute_reply":"2021-11-19T20:27:43.92749Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### Seems like the same as previous df","metadata":{}},{"cell_type":"markdown","source":"# Credits:\n1. https://github.com/MoleImg/Attention_UNet\n2. https://www.kaggle.com/aramos/sartorius-competition-training-keras-unet\n3. https://www.kaggle.com/ammarnassanalhajali/sartorius-segmentation-keras-u-net-training","metadata":{}}]}