{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Masks for catheter segmentation\n\nIn this notebook you will find:\n* Code snippets for masks creation\n* Examples of usage of already created masks\n* Dataset with created masks"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport ast\nimport random\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\nfrom typing import Dict\n\nimport cv2\nfrom PIL import Image\nfrom scipy import interpolate\nfrom torch.utils.data import Dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def set_seed(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    \nset_seed(123)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_DATA = '../input/ranzcr-clip-catheter-line-classification/train'\nTEST_DATA = '../input/ranzcr-clip-catheter-line-classification/test'\n\nTRAIN_CSV = '../input/ranzcr-clip-catheter-line-classification/train.csv'\nTRAIN_ANNOT_CSV = '../input/ranzcr-clip-catheter-line-classification/train_annotations.csv'\nSUBMISSION = '../input/ranzcr-clip-catheter-line-classification/sample_submission.csv'\n\nMASKS = '../input/ranzcr-catheter-and-line-masks/train_masks'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df = pd.read_csv(TRAIN_CSV)\ntrain_annot_df = pd.read_csv(TRAIN_ANNOT_CSV)\nsubmission = pd.read_csv(SUBMISSION)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the competition dataset, about 30% of data has manual annotations of catheter and line positions. <br> \nFor the test set, there is no annotation available."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_imgs = set(train_df['StudyInstanceUID'].unique())\nannotated_imgs = set(train_annot_df['StudyInstanceUID'].unique())\ntest_imgs = set(submission['StudyInstanceUID'].unique())\n\nprint('Annotated images in train set: ', len(train_imgs.intersection(annotated_imgs)))\nprint('Annotated images in test set: ', len(test_imgs.intersection(annotated_imgs)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"One of the most straightforward ways to use provided annotation data is to train the segmentation model, then predict catheter masks for train and test sets then use it as additional data in the downstream classification model. <br>\nTo train such a segmentation model catheter masks should be used. <br>\nTo create a segmentation target masks one should:\n* Interpolate annotated points as a continuous line\n* Define classes for masks\n\nDifferent strategies for class definition can be used:\n* 1 class - find any catheter in the image.\n* 4 classes - find and classify one of the catheter types regardless of their positioning (e.g. CVC, NGT, etc.).\n* 11 classes - classify catheters with respect to their positioning (e.g. CVC-Normal, ETT-Abnormal, etc.).\n* N classes - any other combination you may find useful.\n\n1 class segmentation should be the simplest one to train but it provides less information, 11 classes it the opposite. As a tradeoff between model complexity and provided information, I decided to use 4 class segmentation. <br>\nIf you want to change it you can adjust classes in a `labels_dict` below and run `create_masks` function. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_dict = {\n    'CVC - Normal': 1,\n    'CVC - Borderline': 1,\n    'CVC - Abnormal': 1,\n    'NGT - Normal': 2,\n    'NGT - Incompletely Imaged': 2,\n    'NGT - Borderline': 2,\n    'NGT - Abnormal': 2,\n    'ETT - Normal': 3,\n    'ETT - Borderline': 3,\n    'ETT - Abnormal': 3, \n    'Swan Ganz Catheter Present': 4,\n}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Data preparation\nFirst, let's check consistency between `train` and `train_annotation` labels:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Check consistensy between train and train_annot\n\nfor idx in train_annot_df.index:\n  uid = train_annot_df.loc[idx, 'StudyInstanceUID']\n  label = train_annot_df.loc[idx, 'label']\n  train_label_value = train_df[train_df['StudyInstanceUID'] == uid][label].values[0]\n  assert train_label_value == 1\nprint('Labels are consistent.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Then let's correct some label mistakes: <br>\nA correction based on this [discussion topic](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/210064)."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Errors correction \n\nto_correct = [\n\t[3589,\t'1.2.826.0.1.3680043.8.498.57005638787237813934531972491254580369',\t'CVC - Borderline',\t'NGT - Borderline'],\n\t[4344,\t'1.2.826.0.1.3680043.8.498.93345761486297843389996628528592497280',\t'ETT - Abnormal',\t'CVC - Abnormal'],\n\t[6294,\t'1.2.826.0.1.3680043.8.498.50891603479257167332052859560303996365',\t'NGT - Normal',\t'CVC - Normal'],\n\t[7558,\t'1.2.826.0.1.3680043.8.498.32665013930528750130301395098139968929',\t'NGT - Borderline',\t'CVC - Borderline'],\n\t[8457,\t'1.2.826.0.1.3680043.8.498.47822809495672253227315400926882161159',\t'NGT - Borderline',\t'CVC - Borderline'],\n\t[8586,\t'1.2.826.0.1.3680043.8.498.55171965195784371324650309161724846475',\t'NGT - Borderline',\t'CVC - Borderline'],\n\t[8589,\t'1.2.826.0.1.3680043.8.498.29639870594803047496855371142714987539',\t'ETT - Normal',\t'CVC - Normal'],\n\t[9908,\t'1.2.826.0.1.3680043.8.498.52422864792637441690285442425747003963',\t'NGT - Normal',\t'ETT - Normal'],\n\t[10889,\t'1.2.826.0.1.3680043.8.498.51277351337858188519077141427236143108',\t'NGT - Normal',\t'CVC - Normal'],\n\t[10963,\t'1.2.826.0.1.3680043.8.498.33011244702337270174558484639492100815',\t'CVC - Normal',\t'NGT - Normal'],\n\t[11902,\t'1.2.826.0.1.3680043.8.498.10505287747515183956922280117689383476',\t'NGT - Normal',\t'CVC - Normal'],\n\t[12041,\t'1.2.826.0.1.3680043.8.498.43340424479611237895060478106689360500',\t'NGT - Normal',\t'CVC - Normal'],\n\t[12782,\t'1.2.826.0.1.3680043.8.498.12545979153892772426852721449004507757',\t'NGT - Abnormal',\t'CVC - Abnormal'],\n\t[13513,\t'1.2.826.0.1.3680043.8.498.83700037297895094021306651705503600111',\t'NGT - Normal',\t'ETT - Normal'],\n\t[14226,\t'1.2.826.0.1.3680043.8.498.35772244095675958072394978496245125294',\t'NGT - Normal',\t'ETT - Normal'],\n\t[15750,\t'1.2.826.0.1.3680043.8.498.96130195933728659348647733812659169362',\t'CVC - Abnormal',\t'NGT - Abnormal'],\n\t[15779,\t'1.2.826.0.1.3680043.8.498.75269816256944932004789976844599885553',\t'NGT - Abnormal',\t'CVC - Abnormal'],\n\t[16629,\t'1.2.826.0.1.3680043.8.498.11935284122896798228836385959451625327',\t'NGT - Abnormal',\t'CVC - Abnormal'],\n\t[17501,\t'1.2.826.0.1.3680043.8.498.83574817573978660270935463700320068005',\t'NGT - Abnormal',\t'CVC - Abnormal']\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"for case in to_correct:\n  train_df.loc[train_df.StudyInstanceUID==case[1], case[2]] = 0\n  train_df.loc[train_df.StudyInstanceUID==case[1], case[3]] = 1\n  train_annot_df.loc[case[0], 'label'] = case[3]\n\nprint('Labels are corrected.')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Masks creation\nFunction bellow performs the piecewise linear interpolation of the annotated points, then makes interpolated line thicker and returns the result as `np.array`. <br>\nLabels on the resulted mask correspond to `labels_dict`, label `0` - corresponds to the background."},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_mask(img_name, data_path, df, labels_dict, thin_scale=145):\n\n  img_path = os.path.join(data_path, img_name + '.jpg')\n  img = np.asanyarray(Image.open(img_path), dtype='uint8')\n  img_data = df[df['StudyInstanceUID'] == img_name]\n\n  mask = np.zeros_like(img)\n\n  for idx in img_data.index:\n    data = np.array(ast.literal_eval(img_data.loc[idx, 'data']))\n    label = img_data.loc[idx, 'label']\n    label_id = labels_dict[label]\n    x, y = data[:, 0], data[:, 1]\n\n    for i in range(data.shape[0]-1):\n      xi, yi = np.array([x[i], x[i+1]]), np.array([y[i], y[i+1]])\n      f1, f2 = interpolate.interp1d(xi, yi), interpolate.interp1d(yi, xi)\n      x_new, y_new = np.arange(xi.min(), xi.max(), 1), np.arange(yi.min(), yi.max(), 1)\n      y_inter, x_inter = f1(x_new), f2(y_new)\n      \n      y_mask = y_inter.astype(np.int32).clip(0, mask.shape[0]-1)  \n      x_mask = x_inter.astype(np.int32).clip(0, mask.shape[1]-1)  \n\n      mask[y_mask, x_new] = label_id\n      mask[y_new, x_mask] = label_id\n\n  ks = max(mask.shape) // thin_scale\n  kernel = cv2.getStructuringElement(cv2.MORPH_OPEN, (ks, ks))\n  mask = cv2.dilate(mask, kernel, iterations=1)\n\n  return mask.astype(np.int16)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Visual test**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def plot_sample(img_sample, train_annot_df, train_df, images_path, labels_dict):\n  fig, ax = plt.subplots(1, 2, figsize=(14, 14))\n\n  img_path = os.path.join(images_path, img_sample + '.jpg')\n  img = np.asanyarray(Image.open(img_path), dtype=np.uint16)\n\n  sample = train_annot_df[train_annot_df['StudyInstanceUID'] == img_sample]\n  annots_data = np.array(ast.literal_eval(sample['data'].values[0]))\n\n  sample = train_annot_df[train_annot_df['StudyInstanceUID'] == img_sample]\n  mask = create_mask(img_sample, images_path, train_annot_df, labels_dict)\n\n  print(sample.label)\n  ax[0].imshow(img)\n  ax[0].scatter(annots_data[:, 0], annots_data[:, 1])\n  ax[1].imshow(mask)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_sample = train_annot_df['StudyInstanceUID'].sample().values[0]\n\nplot_sample(img_sample, train_annot_df, train_df, TRAIN_DATA, labels_dict)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can use masks from [provided dataset](https://www.kaggle.com/glebkum/ranzcr-catheter-and-line-masks) or create your own, for example with other class labeling using `create_masks` function below:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# PATH_TO_SAVE_MASKS = # define path\n\ndef create_masks(train_annot_df, train_data_path, labels_dict, path_so_save):\n    # Create and save masks for train in `.npz` format\n    for img_name in tqdm(train_annot_df['StudyInstanceUID'].unique()):\n      mask = create_mask(img_name, train_data_path, train_annot_df, labels_dict)\n      f_name = os.path.join(path_so_save, img_name)\n      np.savez_compressed(f_name, mask)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Usage example\nIn order to train the segmentation model, one can use already created masks. <br>\nAn example of the Dataloader class for that task is provided below."},{"metadata":{"trusted":true},"cell_type":"code","source":"class SegmDataset(Dataset):\n    def __init__(self,\n                 data_df_path: str,\n                 data_path: str,\n                 data_masks_path: str,\n                 transforms=None):\n        super().__init__()\n        self.data_df = pd.read_csv(data_df_path)\n        self.data_path = data_path\n        self.data_masks_path = data_masks_path\n        self.transforms = transforms\n        self.unique_images = self.data_df['StudyInstanceUID'].unique()\n\n    def __len__(self):\n        return len(self.unique_images)\n\n    def __getitem__(self, idx):\n        img_uid = self.unique_images[idx]\n        img_path = os.path.join(self.data_path, img_uid + '.jpg')\n        mask_path = os.path.join(self.data_masks_path, img_uid + '.npz')\n        img = np.asarray(Image.open(img_path), dtype='uint8')\n        mask = np.load(mask_path)['arr_0']\n\n        if self.transforms:\n            transformed = self.transforms(image=img, mask=mask)\n            img = transformed['image']\n            mask = transformed['mask']\n\n        data = {\n            'image': img,\n            'mask': mask\n        }\n\n        return data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataloader = SegmDataset(TRAIN_ANNOT_CSV,\n                         TRAIN_DATA,\n                         MASKS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(3, 2, figsize=(12, 12))\n\nfor i in range(3):\n    rand_idx = random.randint(0, len(dataloader)) \n    data_sample = dataloader[rand_idx]\n    ax[i][0].imshow(data_sample['image'])\n    ax[i][1].imshow(data_sample['mask'])\n    \nax[0][0].set_title('Original image')\nax[0][1].set_title('Target mask')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}