{"cells":[{"metadata":{},"cell_type":"markdown","source":"# HOWTO upload module and perform simple image classification\n\nIn this notebook, I'll explain some stuff that needed some time for me to figure out. It uses the HPA Single Cell Classification competition as an example.\n"},{"metadata":{},"cell_type":"markdown","source":"\n# Default folders and how to use them\n\nKaggle's notebook environment's basic directory structure is split into the folder `/kaggle/input/` where the user can upload files and `/kaggle/output/` for outputting files from notebook run.\n\nThe notebook itself is run from the `/kaggle/working/` folder. So to use the module, we have to put it in the working folder. This folder doesn't let users upload files directly, so we need to copy files from within the code.\n\nLet's say we need to upload a module that we want to use in our notebook. First, we upload the module as a data set. This is then uploaded in the folder `/kaggle/input/`. After that, we copy the module explicitly in the working directory to import it into the notebook. Later you can update the module, and click **More Actions -> Check for Updates** to refresh the module and start using the updated version."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom shutil import copyfile, copytree\nimport matplotlib.pyplot as plt\nimport cv2 as cv\n\n# We load explicitly md so we can print markdown from code\nfrom IPython.display import Markdown as md\nimport os\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\n# import os\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We use `copyfile` to copy the module."},{"metadata":{"trusted":true},"cell_type":"code","source":"# copy our file into the working directory\ncopyfile(src = \"/kaggle/input/hpamodule1/module_1.py\", dst = \"/kaggle/working/module_1.py\")\nif os.path.exists(\"../input/howto-upload-module-and-simple-classification/nuclei-model.pth\"):\n    copyfile(src = \"../input/howto-upload-module-and-simple-classification/nuclei-model.pth\", dst = \"./nuclei-model.pth\")\n    copyfile(src = \"../input/howto-upload-module-and-simple-classification/cell-model.pth\", dst = \"./cell-model.pth\")\n\n# copytree(src = \"/kaggle/input/hpa-cell-segmentation/hpacellseg\", dst = \"/kaggle/working/hpacellseg\")\nimport module_1 as hpm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets see how the module works by loading the row from the submission file and using the `test` folder for the image source. You can also use the train as a source, if you set `img_folder = 'train'`, in that case the csv file that is used is train.csv."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Set data folders and files\nhpa_data = hpm.HPA(data_folder='/kaggle/input/hpa-single-cell-image-classification', img_folder='test')\nprint(hpa_data.sample_sub_pd.loc[5, :])\nhpa_data.get_rgby_images(hpa_data.sample_sub_pd.loc[5, 'ID'])\n\n# hpa_data = hpm.HPA(data_folder='/kaggle/input/hpa-single-cell-image-classification', img_folder='train')\n# print(hpa_data.train_csv_pd.loc[10, :])\n# hpa_data.get_rgby_images(hpa_data.train_csv_pd.loc[10, 'ID'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can show the image which is uploaded as a tuple, per channel, in the `self.img` member variable."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(23,23))\nax1 = fig.add_subplot(141)\nax1.imshow(hpa_data.img[0])\nax1.title.set_text(\"Original red channel\")\nax2 = fig.add_subplot(142)\nax2.imshow(hpa_data.img[1])\nax2.title.set_text(\"Original green channel\")\nax3 = fig.add_subplot(143)\nax3.imshow(hpa_data.img[2])\nax3.title.set_text(\"Original blue channel\")\nax4 = fig.add_subplot(144)\nax4.imshow(hpa_data.img[3])\nax4.title.set_text(\"Original yellow channel\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the end, everything that is written in the working folder is transferred to the output folder, so is the uploaded module."},{"metadata":{},"cell_type":"markdown","source":"# Download only specific files on your PC\n\nThis competition has large data set, around 160GB. If you want to use only some of the files, then you can install the kaggle API and download only specific files on your local machine. The Kaggle API is located on GitHub (https://github.com/Kaggle/kaggle-api), where you can also find instructions on using it. After installing it, create a new token (check this for info https://www.kaggle.com/docs/api). Then you can use the following function to download specific file on your PC."},{"metadata":{"trusted":true},"cell_type":"code","source":"'''\nfrom kaggle.api.kaggle_api_extended import KaggleApi\nimport os\n\ndef download_file(api, fname):\n    if os.path.exists(f\"./data/train/{fname}_red.png\"):\n        return\n\n    for ch in ch_names:\n        api.competition_download_file(\n            'hpa-single-cell-image-classification', \n            f'train/{fname}_{ch}.png', \n            path='./data/train'\n        )\n        \napi = KaggleApi()\napi.authenticate()\ndownload_file(api, '5c27f04c-bb99-11e8-b2b9-ac1f6b6435d0')\n'''","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Dilation\n\nOur first step of processing the loaded image is to perform dilation of the images. We use the module's function `dilate` which performs the following steps to every channel: **Gaussian blur -> Otsu's threshold -> dilation with 5x5 kernel**. "},{"metadata":{"trusted":true},"cell_type":"code","source":"hpa_data.dilate()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25,12))\nax1 = fig.add_subplot(241)\nax1.imshow(hpa_data.img[0])\nax1.title.set_text(\"Original red channel\")\nax2 = fig.add_subplot(245, sharex=ax1, sharey=ax1)\nax2.imshow(hpa_data.img_dilate[0])\nax2.title.set_text(\"Dilate red channel\")\nax3 = fig.add_subplot(242)\nax3.imshow(hpa_data.img[1])\nax3.title.set_text(\"Original green channel\")\nax4 = fig.add_subplot(246, sharex=ax1, sharey=ax1)\nax4.imshow(hpa_data.img_dilate[1])\nax4.title.set_text(\"Dilate green channel\")\nax5 = fig.add_subplot(243)\nax5.imshow(hpa_data.img[2])\nax5.title.set_text(\"Original blue channel\")\nax6 = fig.add_subplot(247, sharex=ax1, sharey=ax1)\nax6.imshow(hpa_data.img_dilate[2])\nax6.title.set_text(\"Dilate blue channel\")\nax7 = fig.add_subplot(244)\nax7.imshow(hpa_data.img[3])\nax7.title.set_text(\"Original yellow channel\")\nax8 = fig.add_subplot(248, sharex=ax1, sharey=ax1)\nax8.imshow(hpa_data.img_dilate[3])\nax8.title.set_text(\"Dilate yellow channel\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Connected components and the blue channel\n\nNext we use connected components (cc) analysis on the blue channel, since this channel represents cells' nucleus. The nucleus is in the center of the cell and we can easly spot individual cells by performing `connectedComponentsWithStats` from the `cv` library. We run the cc on the dilated images from the previous step."},{"metadata":{"trusted":true},"cell_type":"code","source":"hpa_data.connected_components()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can show the detected regions by drawing rectangles and centroids with the data given by the `cc_stat` variables. Variables are generated after calling `connected_components`. We start counting stats from 1, since 0 is reserved for the background as a connected region."},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.patches as pch\n\nfig = plt.figure(figsize=(25,12))\nax1 = fig.add_subplot(111)\nax1.imshow(hpa_data.cc_labels)\nfor i in range(1, hpa_data.cc_numLabels):\n    left = hpa_data.cc_stats[i, cv.CC_STAT_LEFT]\n    top = hpa_data.cc_stats[i, cv.CC_STAT_TOP]\n    width = hpa_data.cc_stats[i, cv.CC_STAT_WIDTH]\n    height = hpa_data.cc_stats[i, cv.CC_STAT_HEIGHT]\n    rect = pch.Rectangle((left, top), \n        width, height, linewidth=1, edgecolor='r', facecolor='none')\n    circ = pch.Circle((hpa_data.cc_centroids[i]), \n        5, linewidth=1, edgecolor='none', facecolor='r')\n    ax1.add_patch(rect)\n    ax1.add_patch(circ)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Segmentation using CellProfiling/HPA-Cell-Segmentation code\n\nAfter trying a couple of segmentation methods without much success, I accepted the popular and widely used professional algorithm for cell segmentation found on GitHub https://github.com/CellProfiling/HPA-Cell-Segmentation/. Before using, we need to install the packet, so we run the zipped folder's pip install."},{"metadata":{"trusted":true},"cell_type":"code","source":"#!pip install https://github.com/CellProfiling/HPA-Cell-Segmentation/archive/master.zip\n!pip install ../input/pytorch-zoo/pytorch_zoo-master\n!pip install ../input/hpacellsegmentation/HPA-Cell-Segmentation-master","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After the first run, the module will download two files with pretrained models of nuclei and cells. If this is not a first run and you have added the output files from a previous run, the following code will load that files."},{"metadata":{"trusted":true},"cell_type":"code","source":"import hpacellseg.cellsegmentator as cellsegmentator\nfrom PIL import Image\nfrom hpacellseg.utils import label_cell, label_nuclei\n\nNUC_MODEL = \"./nuclei-model.pth\"\nCELL_MODEL = \"./cell-model.pth\"\nsegmentator = cellsegmentator.CellSegmentator(\n    NUC_MODEL,\n    CELL_MODEL,\n    scale_factor=0.25,\n    device=\"cuda\",\n    padding=True,\n    multi_channel_model=True,\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We detect nuclei and cells."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Nuclei segmentation\n\nnuc_segments = segmentator.pred_nuclei([hpa_data.b_img])\n\nnuc_mask = label_nuclei(nuc_segments[0])\n\nfig = plt.figure(figsize=(24, 24))\nax1 = fig.add_subplot(131)\nax1.imshow(hpa_data.b_img)\nax1.set_title('Original nuclei')\nax2 = fig.add_subplot(132)\nax2.imshow(nuc_segments[0])\nax2.set_title('HPA segmented nuclei')\nax3 = fig.add_subplot(133)\nax3.imshow(nuc_mask)\nax3.set_title('HPA labeled nuclei')\nplt.show()\n\n# Cell segmentation\n\ncell_segments = segmentator.pred_cells([[hpa_data.r_img], [hpa_data.y_img], [hpa_data.b_img]])\nnuc_mask, cell_mask = label_cell(nuc_segments[0], cell_segments[0])\n\nrgb_img = np.stack([\n    cv.convertScaleAbs(hpa_data.r_img, alpha=(255.0/65535.0)), \n    cv.convertScaleAbs(hpa_data.g_img, alpha=(255.0/65535.0)), \n    cv.convertScaleAbs(hpa_data.b_img, alpha=(255.0/65535.0))], 2)\n\nfig = plt.figure(figsize=(24, 24))\nax1 = fig.add_subplot(131)\nax1.imshow(rgb_img)\nax1.set_title('Original cell rgb')\nax2 = fig.add_subplot(132)\nax2.imshow(cell_mask)\nax2.set_title('HPA segmented cells')\nax3 = fig.add_subplot(133)\nax3.imshow(rgb_img)\nax3.imshow(cell_mask, alpha=0.3)\nax3.set_title('HPA cell rgb + cell label')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Submit segmented cell masks with label 0\n\nHere we can even make a submission file by labeling all segments with 0."},{"metadata":{"trusted":true},"cell_type":"code","source":"# !pip install pycocotools\n!pip install ../input/pycocotools/dist/pycocotools-2.0.2.tar\n    \nimport base64\nfrom pycocotools import _mask as coco_mask\nimport typing as t\nimport zlib\n\nsub_file = open('submission.csv', 'w')\n\nsub_file.write(\"ID,ImageWidth,ImageHeight,PredictionString\\n\")\n\ncnt = 0\nfor id in hpa_data.sample_sub_pd.loc[:, 'ID']:\n    print(f\"{cnt}: {id}\")\n    hpa_data.get_rgby_images(id)\n    nuc_segments = segmentator.pred_nuclei([hpa_data.b_img])\n    nuc_mask = label_nuclei(nuc_segments[0])\n    cell_segments = segmentator.pred_cells([[hpa_data.r_img], [hpa_data.y_img], [hpa_data.b_img]])\n    nuc_mask, cell_mask = label_cell(nuc_segments[0], cell_segments[0])\n    sub_str = f\"{id},{hpa_data.b_img.shape[0]},{hpa_data.b_img.shape[1]},\"\n    for ci in range(1, cell_mask.max() + 1):\n        mask = np.squeeze((cell_mask == ci))\n        mask_to_encode = mask.reshape(mask.shape[0], mask.shape[1], 1)\n        mask_to_encode = mask_to_encode.astype(np.uint8)\n        mask_to_encode = np.asfortranarray(mask_to_encode)\n        encoded_mask = coco_mask.encode(mask_to_encode)[0][\"counts\"]\n        binary_str = zlib.compress(encoded_mask, zlib.Z_BEST_COMPRESSION)\n        base64_str = base64.b64encode(binary_str)\n        if ci == cell_mask.max():\n            sub_str += f\"0 1 {base64_str.decode()}\\n\"\n        else:\n            sub_str += f\"0 1 {base64_str.decode()} \"\n    sub_file.write(sub_str)\n    cnt += 1\n    \nsub_file.close()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}