{"cells":[{"metadata":{},"cell_type":"markdown","source":"In this notebook I will be using [Weights and Biases](https://wandb.ai/site) overlay visualization tool to interactively visualize all the segmentation mask generated using the provided HPASegmentation tool.\n\nHere's the W&B report summarizing the results: http://bit.ly/play-with-segmentation-masks"},{"metadata":{},"cell_type":"markdown","source":"![](https://i.imgur.com/9kNLI4L.gif)"},{"metadata":{},"cell_type":"markdown","source":"In this competition, we are provided with image-level labels, and the task is to classify each cell in a given image into one or multiple labels.\n\n* Thus, each image has multiple numbers of cells. \n\n* Each cell consists of multiple [organelles](https://www.genome.gov/genetics-glossary/Organelle). In the previous [HPA competition](https://www.kaggle.com/c/human-protein-atlas-image-classification), 28 organelles were used as labels, and the task was to predict image-level labels (given input image predict the organelles). \n\n* In this competition, the task is to predict cell-level labels using signals from image-level labels. That's what makes it a more challenging problem statement. \n\n* **But what are we predicting?** There's a specific _protein of interest_(signal in the green channel) that can be present in an organelle or multiple organelles in each cell. The image-level labels point to the presence of that protein in the cells *in general*. Thus at the cell-level, \nthe protein might not be present in the ground truth organelle. Interesting!\n\n* This calls for cell segmentation. We have to know the presence of the cells in an image. But we also need to differentiate one cell from another cell. Thus it's instance segmentation. \n\n* The authors have provided with [HPA-Cell-Segmentation](https://github.com/CellProfiling/HPA-Cell-Segmentation) tool. **Is it good?** In a discussion thread, I read that it can accurately segment cells in ~90% of test set images. That's an excellent baseline to start with and focus on cell-level classification. "},{"metadata":{},"cell_type":"markdown","source":"# Imports and Setups"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture\n# Install Weights and Biases.\n!pip install wandb -q\n\n# Install HPA Cell Segmentation tool.\n!pip install https://github.com/CellProfiling/HPA-Cell-Segmentation/archive/master.zip","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\n# To silent W&B logs\nos.environ['WANDB_SILENT'] = 'true'\n\nimport re\nimport cv2\nimport glob\nimport imageio\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nfrom tqdm.notebook import tqdm\nimport matplotlib.pyplot as plt\nfrom skimage.transform import resize\n\n%matplotlib inline\n\n# HPA Segmentation tool related imports\nimport hpacellseg.cellsegmentator as cellsegmentator\nfrom hpacellseg.utils import label_cell, label_nuclei","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### W&B Setup\n\n* Install wandb. ✔️\n* Create an account on https://wandb.ai (it's free)\n* Input your personal authentication token key. You can get your auth key [here](https://wandb.ai/authorize)."},{"metadata":{"trusted":true},"cell_type":"code","source":"import wandb\n\nfrom kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\nwandb_api = user_secrets.get_secret(\"wandb_api\")\n\nwandb.login(key=wandb_api)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### (Hyper)parameters"},{"metadata":{"trusted":true},"cell_type":"code","source":"WORKING_DIR_PATH = '../input/hpa-single-cell-image-classification/'\nVISUALIZE_SAMPLES = 32","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Prepare Dataset"},{"metadata":{},"cell_type":"markdown","source":"### Get `train.csv`"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(WORKING_DIR_PATH+'train.csv')\ndf_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"label_names= {\n0: \"Nucleoplasm\",\n1: \"Nuclear membrane\",\n2: \"Nucleoli\",\n3: \"Nucleoli fibrillar center\",\n4: \"Nuclear speckles\",\n5: \"Nuclear bodies\",\n6: \"Endoplasmic reticulum\",\n7: \"Golgi apparatus\",\n8: \"Intermediate filaments\",\n9: \"Actin filaments\",\n10: \"Microtubules\",\n11: \"Mitotic spindle\",\n12: \"Centrosome\",\n13: \"Plasma membrane\",\n14: \"Mitochondria\",\n15: \"Aggresome\",\n16: \"Cytosol\",\n17: \"Vesicles and punctate cytosolic patterns\",\n18: \"Negative\"\n}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Path to images"},{"metadata":{"trusted":true},"cell_type":"code","source":"red_images = sorted(glob.glob(WORKING_DIR_PATH+'train/*_red.png'))\ngreen_images = sorted(glob.glob(WORKING_DIR_PATH+'train/*_green.png'))\nblue_images = sorted(glob.glob(WORKING_DIR_PATH+'train/*_blue.png'))\nyellow_images = sorted(glob.glob(WORKING_DIR_PATH+'train/*_yellow.png'))\n\nprint(len(red_images), len(green_images), len(blue_images), len(yellow_images))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mt = red_images[:VISUALIZE_SAMPLES] \ner = yellow_images[:VISUALIZE_SAMPLES]\nnu = blue_images[:VISUALIZE_SAMPLES]\npr = green_images[:VISUALIZE_SAMPLES]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* We will not be needing the green channel to get the segmentation mask as shown in [this kernel](https://www.kaggle.com/lnhtrang/hpa-public-data-download-and-hpacellseg).\n\n* There are a total of 21,806 training images. We will visualize a small fraction of the images. Feel free to use Weights and Biases overlay tool to visualize more images."},{"metadata":{},"cell_type":"markdown","source":"# Initialize Segmentation Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"NUC_MODEL = \"./nuclei-model.pth\"\nCELL_MODEL = \"./cell-model.pth\"\n\nsegmentator = cellsegmentator.CellSegmentator(\n    NUC_MODEL,\n    CELL_MODEL,\n    scale_factor=0.25,\n    device=\"cuda\",\n    padding=False,\n    multi_channel_model=True,\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Log Segmentation Masks"},{"metadata":{"trusted":true},"cell_type":"code","source":"# utility function that returns W&B Image. \n# Learn more about overlay logging here: https://docs.wandb.ai/library/log#images-and-overlays\ndef wandb_mask(bg_img, pred_mask):\n  return wandb.Image(bg_img, masks={\n      \"prediction\" : {\n          \"mask_data\" : pred_mask,\n      }\n    }\n  )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i in tqdm(range(VISUALIZE_SAMPLES)):\n    # Image ID \n    image_id = re.findall(r'[^\\/]+(?=\\_.)', mt[i])[0]\n    \n    # Initialize W&B\n    run = wandb.init(project='hpa-segmentation-mask', name=image_id)\n\n    # Get red, blue and yellow channel images. \n    microtubule = np.array(Image.open(mt[i]))\n    endoplasmicrec = np.array(Image.open(er[i]))\n    nuclei = np.array(Image.open(nu[i]))\n    protein = np.array(Image.open(pr[i]))\n    # Stack the channels to form image.\n    image = np.dstack((microtubule, endoplasmicrec, nuclei))\n    \n    # Get the label\n    labels = df_train.loc[df_train['ID'] == image_id].Label.values[0]\n    labels = labels.replace(\"|\", \" \").split()\n    labels = '-'.join([label_names[int(label)] for label in labels])\n    \n    # For nuclei segmentation only blue channel is required.\n    nuc_segmentation = segmentator.pred_nuclei([nu[i]])\n    # For full cells all the three reference(except green) channels are required.\n    cell_segmentation = segmentator.pred_cells([[mt[i]], [er[i]], [nu[i]]])\n    # get cell mask\n    nuclei_mask, cell_mask = label_cell(nuc_segmentation[0], cell_segmentation[0])\n    \n    # resize mask and image \n    image = cv2.resize(image, (512,512), interpolation=cv2.INTER_NEAREST)\n    cell_mask = cv2.resize(cell_mask, (512,512), interpolation=cv2.INTER_NEAREST)\n    protein = cv2.resize(protein, (512,512), interpolation=cv2.INTER_NEAREST)\n        \n    # log the image as well as the mask\n    wandb.log({f\"mask_{image_id}\" : [wandb_mask(image, cell_mask)]})\n    \n    # log green channel \n    wandb.log({f\"protein_{image_id}\": [wandb.Image(protein, caption=f\"{labels}\")]})\n    \n    # Close W&B run\n    run.finish()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After the above cell is executed head over to the W&B project to visualize the segmentation masks and other visualizations. \n\n**Note**: Since we have silenced the W&B logs head over to your W&B profile page and open `hpa-segmentation-mask` project.\n\nHere's the link to my W&B project: https://wandb.ai/ayush-thakur/hpa-segmentation-mask\n\n![](https://i.imgur.com/iw3rETK.png)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}