{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## **Contents**\n-  [Introduction](#i)\n-  [1.Importing Libraries](#1)\n-  [2.Helper Functions](#2)\n-  [3.Dataset Managament](#3)\n    -  [3.1.Dataset and DataLoaders](#3.1)\n    -  [3.2.Visualizing Dataset](#3.2)\n-  [4.Initializing pre-trained model](#4)\n-  [5.Training](#5)\n-  [6.Plotting Graphs](#6)\n    -  [6.1.Plotting Loss vs Epoch](#6.1)\n-  [7.Loading and Testing](#7)\n-  [8.Predictions](#8)","metadata":{"execution":{"iopub.execute_input":"2021-10-22T02:20:53.048499Z","iopub.status.busy":"2021-10-22T02:20:53.047964Z","iopub.status.idle":"2021-10-22T02:20:55.442888Z","shell.execute_reply":"2021-10-22T02:20:55.442186Z","shell.execute_reply.started":"2021-10-22T02:20:53.048407Z"}}},{"cell_type":"markdown","source":"## **Introduction** <a class=\"anchor\" id=\"i\"></a>\n\n\nIn this notebook I am visualizing the dataset and also training [U-Net](https://arxiv.org/abs/1505.04597) model using [Segmentation model](https://github.com/qubvel/segmentation_models.pytorch) library which is based on [Pytorch](https://pytorch.org/) library.  \nResnet34 net is being utilized for the U-Net trained in this notebook.\n\nU-Net is an architecture for semantic segmentation. It consists of a contracting path and an expansive path. The contracting path follows the typical architecture of a convolutional network. It consists of the repeated application of two 3x3 convolutions (unpadded convolutions), each followed by a rectified linear unit (ReLU) and a 2x2 max pooling operation with stride 2 for downsampling. At each downsampling step we double the number of feature channels. Every step in the expansive path consists of an upsampling of the feature map followed by a 2x2 convolution (“up-convolution”) that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3x3 convolutions, each followed by a ReLU. The cropping is necessary due to the loss of border pixels in every convolution. At the final layer a 1x1 convolution is used to map each 64-component feature vector to the desired number of classes. In total the network has 23 convolutional layers.\n\n\nRead more about U-Net [here](https://towardsdatascience.com/unet-line-by-line-explanation-9b191c76baf5)\n\nSome of the helper functions is derived from [this](https://www.kaggle.com/julian3833/sartorius-starter-baseline-torch-u-net/notebook#Model:-U-net) notebook.\n\n<img src= \"https://miro.medium.com/max/953/1*Z98NhzbVISHa4CoemZS4Kw.png\"  style='width: 700px;'>","metadata":{}},{"cell_type":"markdown","source":"### **1.Importing Libraries** <a class=\"anchor\" id=\"1\"></a>","metadata":{}},{"cell_type":"code","source":"%%capture\n!pip install ../input/segmentation-models-wheels/efficientnet_pytorch-0.6.3-py3-none-any.whl\n!pip install ../input/segmentation-models-wheels/pretrainedmodels-0.7.4-py3-none-any.whl\n!pip install ../input/segmentation-models-wheels/timm-0.3.2-py3-none-any.whl\n!pip install ../input/segmentation-models-wheels/segmentation_models_pytorch-0.1.3-py3-none-any.whl","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:16:57.404839Z","iopub.execute_input":"2023-04-01T18:16:57.405583Z","iopub.status.idle":"2023-04-01T18:17:26.400201Z","shell.execute_reply.started":"2023-04-01T18:16:57.405542Z","shell.execute_reply":"2023-04-01T18:17:26.399040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport torch\nimport random\nimport numpy as np\nimport torchvision\nimport pandas as pd\nimport torch.nn as nn\nimport seaborn as sns\nfrom PIL import Image\nfrom PIL import Image\nfrom sklearn import cluster\nimport torch.nn.functional as F\nimport matplotlib.pyplot as plt\nfrom albumentations.pytorch import ToTensorV2\nfrom torch.utils.data import Dataset as Dataset\nfrom torch.utils.data import DataLoader as DataLoader\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\nfrom torchvision.models.detection.mask_rcnn import MaskRCNNPredictor\nfrom albumentations import (HorizontalFlip, VerticalFlip, ShiftScaleRotate, Normalize, Resize, Compose, GaussNoise)\n\ntorch.cuda.empty_cache()\nrandom.seed(0)\nnp.random.seed(0)\ntorch.manual_seed(0)\n\nsns.set_style(\"darkgrid\")","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:26.403848Z","iopub.execute_input":"2023-04-01T18:17:26.404642Z","iopub.status.idle":"2023-04-01T18:17:26.415587Z","shell.execute_reply.started":"2023-04-01T18:17:26.404596Z","shell.execute_reply":"2023-04-01T18:17:26.414867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **2.Helper Functions** <a class=\"anchor\" id=\"2\"></a>","metadata":{}},{"cell_type":"code","source":"IMG_SIZE = (224,224)\nroot = \"../input/sartorius-cell-instance-segmentation/\"\n\ndef rle_decode(mask_rle, shape, color=1):\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    \n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0] * shape[1], dtype=np.float32)\n    for lo, hi in zip(starts, ends):\n        img[lo : hi] = color\n    return img.reshape(shape)\n\ndef build_masks(df_train, image_id, input_shape):\n    height, width = input_shape\n    labels = df_train[df_train[\"id\"] == image_id][\"annotation\"].tolist()\n    mask = np.zeros((height, width))\n    for label in labels:\n        mask += rle_decode(label, shape=(height, width))\n    mask = mask.clip(0, 1)\n    return mask\n\ndef post_process(probability, threshold=0.5, min_size=300):\n    mask = cv2.threshold(probability, threshold, 1, cv2.THRESH_BINARY)[1]\n    num_component, component = cv2.connectedComponents(mask.astype(np.uint8))\n    predictions = []\n    for c in range(1, num_component):\n        p = (component == c)\n        if p.sum() > min_size:\n            a_prediction = np.zeros((520, 704), np.float32)\n            a_prediction[p] = 1\n            predictions.append(a_prediction)\n    return predictions\n\n# Stolen from: https://www.kaggle.com/arunamenon/cell-instance-segmentation-unet-eda\n# Run-length encoding stolen from https://www.kaggle.com/rakhlin/fast-run-length-encoding-python\n# Modified by me\n\ndef rle_encoding(x):\n    dots = np.where(x.flatten() == 1)[0]\n    run_lengths = []\n    prev = -2\n    for b in dots:\n        if (b>prev+1): run_lengths.extend((b + 1, 0))\n        run_lengths[-1] += 1\n        prev = b\n    return ' '.join(map(str, run_lengths))\n\ndef check_is_run_length(mask_rle):\n    if not mask_rle:\n        return True\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    start_prev = starts[0]\n    ok = True\n    for start in starts[1:]:\n        ok = ok and start > start_prev\n        start_prev = start\n        if not ok:\n            return False\n    return True\n\ndef create_empty_submission():\n    fs = os.listdir(\"../input/sartorius-cell-instance-segmentation/test\")\n    df = pd.DataFrame([(f[:-4], \"\") for f in fs], columns=['id', 'predicted'])\n    df.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:26.417043Z","iopub.execute_input":"2023-04-01T18:17:26.417530Z","iopub.status.idle":"2023-04-01T18:17:26.434134Z","shell.execute_reply.started":"2023-04-01T18:17:26.417491Z","shell.execute_reply":"2023-04-01T18:17:26.433402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport skimage\nimport numpy as np\nimport pandas as pd\nimport skimage.segmentation\nimport matplotlib.pyplot as plt\n\ndef rles_to_mask(encs, shape):\n    \"\"\"\n    Decodes a rle.\n\n    Args:\n        encs (list of str): Rles for each class.\n        shape (tuple [2]): Mask size.\n\n    Returns:\n        np array [shape]: Mask.\n    \"\"\"\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint)\n    for m, enc in enumerate(encs):\n        if isinstance(enc, np.float) and np.isnan(enc):\n            continue\n        enc_split = enc.split()\n        for i in range(len(enc_split) // 2):\n            start = int(enc_split[2 * i]) - 1\n            length = int(enc_split[2 * i + 1])\n            img[start: start + length] = 1 + m\n    return img.reshape(shape)\n\ndef compute_iou(labels, y_pred):\n\n    \"\"\"\n    Computes the IoU for instance labels and predictions.\n\n    Args:\n        labels (np array): Labels.\n        y_pred (np array): predictions\n\n    Returns:\n        np array: IoU matrix, of size true_objects x pred_objects.\n    \"\"\"\n\n    true_objects = len(np.unique(labels))\n    pred_objects = len(np.unique(y_pred))\n\n    # Compute intersection between all objects\n    intersection = np.histogram2d(\n        labels.flatten(), y_pred.flatten(), bins=(true_objects, pred_objects)\n    )[0]\n\n    # Compute areas (needed for finding the union between all objects)\n    area_true = np.histogram(labels, bins=true_objects)[0]\n    area_pred = np.histogram(y_pred, bins=pred_objects)[0]\n    area_true = np.expand_dims(area_true, -1)\n    area_pred = np.expand_dims(area_pred, 0)\n\n    # Compute union\n    union = area_true + area_pred - intersection\n    iou = intersection / union\n    \n    return iou[1:, 1:]  # exclude background\n\ndef precision_at(threshold, iou):\n    \"\"\"\n    Computes the precision at a given threshold.\n\n    Args:\n        threshold (float): Threshold.\n        iou (np array [n_truths x n_preds]): IoU matrix.\n\n    Returns:\n        int: Number of true positives,\n        int: Number of false positives,\n        int: Number of false negatives.\n    \"\"\"\n    matches = iou > threshold\n    true_positives = np.sum(matches, axis=1) >= 1  # Correct objects\n    false_negatives = np.sum(matches, axis=1) == 0  # Missed objects\n    false_positives = np.sum(matches, axis=0) == 0  # Extra objects\n    tp, fp, fn = (\n        np.sum(true_positives),\n        np.sum(false_positives),\n        np.sum(false_negatives),\n    )\n    return tp, fp, fn\n\ndef iou_map(truths, preds, verbose=0):\n    \"\"\"\n    Computes the metric for the competition.\n    Masks contain the segmented pixels where each object has one value associated,\n    and 0 is the background.\n\n    Args:\n        truths (list of masks): Ground truths.\n        preds (list of masks): Predictions.\n        verbose (int, optional): Whether to print infos. Defaults to 0.\n\n    Returns:\n        float: mAP.\n    \"\"\"\n    ious = [compute_iou(truth, pred) for truth, pred in zip(truths, preds)]\n    \n    print(ious[0].shape)\n\n    if verbose:\n        print(\"Thresh\\tTP\\tFP\\tFN\\tPrec.\")\n\n    prec = []\n    for t in np.arange(0.5, 1.0, 0.05):\n        tps, fps, fns = 0, 0, 0\n        for iou in ious:\n            tp, fp, fn = precision_at(t, iou)\n            tps += tp\n            fps += fp\n            fns += fn\n\n        p = tps / (tps + fps + fns)\n        prec.append(p)\n\n        if verbose:\n            print(\"{:1.3f}\\t{}\\t{}\\t{}\\t{:1.3f}\".format(t, tps, fps, fns, p))\n\n    if verbose:\n        print(\"AP\\t-\\t-\\t-\\t{:1.3f}\".format(np.mean(prec)))\n\n    return np.mean(prec)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:26.436557Z","iopub.execute_input":"2023-04-01T18:17:26.436927Z","iopub.status.idle":"2023-04-01T18:17:26.454096Z","shell.execute_reply.started":"2023-04-01T18:17:26.436867Z","shell.execute_reply":"2023-04-01T18:17:26.453362Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **3.Dataset Managament** <a class=\"anchor\" id=\"3\"></a>","metadata":{}},{"cell_type":"markdown","source":"#### 1.Downloading and Extracting Dataset <a class=\"anchor\" id=\"3.1\"></a>","metadata":{}},{"cell_type":"code","source":"TRAIN_CSV = \"/kaggle/input/sartorius-cell-instance-segmentation/train.csv\"\nTRAIN_PATH=\"/kaggle/input/sartorius-cell-instance-segmentation/train\"\ndf_train = pd.read_csv(TRAIN_CSV)\ntotal = 0\nids = df_train['id'].unique()\nfor i in ids[::-1]:\n    total+=len(df_train[df_train['id']==i])\n    \nprint(total)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:26.455841Z","iopub.execute_input":"2023-04-01T18:17:26.456059Z","iopub.status.idle":"2023-04-01T18:17:29.537098Z","shell.execute_reply.started":"2023-04-01T18:17:26.456035Z","shell.execute_reply":"2023-04-01T18:17:29.535419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"all_imgs = []\n\nfor ctype in df_train['cell_type'].unique():\n    all_imgs += list(df_train[df_train['cell_type']==ctype]['id'].unique()[:20])","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:29.538340Z","iopub.execute_input":"2023-04-01T18:17:29.538617Z","iopub.status.idle":"2023-04-01T18:17:29.575737Z","shell.execute_reply.started":"2023-04-01T18:17:29.538577Z","shell.execute_reply":"2023-04-01T18:17:29.575027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from collections import Counter\nCounter(all_imgs)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:29.576851Z","iopub.execute_input":"2023-04-01T18:17:29.577611Z","iopub.status.idle":"2023-04-01T18:17:29.585835Z","shell.execute_reply.started":"2023-04-01T18:17:29.577567Z","shell.execute_reply":"2023-04-01T18:17:29.584983Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_val = df_train[df_train.id.isin(all_imgs)]\ndf_val","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:29.587703Z","iopub.execute_input":"2023-04-01T18:17:29.588426Z","iopub.status.idle":"2023-04-01T18:17:29.612672Z","shell.execute_reply.started":"2023-04-01T18:17:29.588383Z","shell.execute_reply":"2023-04-01T18:17:29.611998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = df_train[~df_train.id.isin(all_imgs)]\ndf_train","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:29.613957Z","iopub.execute_input":"2023-04-01T18:17:29.614226Z","iopub.status.idle":"2023-04-01T18:17:29.641287Z","shell.execute_reply.started":"2023-04-01T18:17:29.614167Z","shell.execute_reply":"2023-04-01T18:17:29.640458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class SatoriusDataset(torch.utils.data.Dataset):\n    def __init__(self, df, transforms, root = 'data/',train = True):\n        self.root = root\n        self.transforms = transforms\n        self.w , self.h = 520 , 704\n        \n        info = df\n        self.df =df\n        info = info.groupby('id')['annotation'].agg(lambda x: list(x)).reset_index()\n        validation =  30\n        self.data_info = info\n\n        \n            \n    def __getitem__(self, idx):\n        img = Image.open(self.root+'train/'+self.data_info['id'][idx]+'.png').convert(\"RGB\")\n        labels = self.data_info['annotation'][idx]\n        mask = np.zeros((self.w , self.h))\n        for label in labels:\n            mask = mask + rle_decode(label, shape=(self.w , self.h))\n        mask = mask.clip(0, 1)\n        mask = (mask >= 1).astype('float32')\n        augmented = self.transforms(image=np.array(img), mask=mask)\n        img = augmented['image']\n        mask = augmented['mask']\n        return img, mask.reshape((1, IMG_SIZE[0], IMG_SIZE[1]))\n\n    def __len__(self):\n        return len(self.data_info)\n\n\n    \ndef get_transform(train):\n    transforms = []\n    # converts the image, a PIL image, into a PyTorch Tensor\n    transforms.append(Resize(IMG_SIZE[0], IMG_SIZE[1]))\n    transforms.append(Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)))\n    if train:\n        # during training, randomly flip the training images\n        # and ground-truth for data augmentation\n        transforms.append(HorizontalFlip(0.5))\n        transforms.append(VerticalFlip(0.5))\n        \n    transforms.append(ToTensorV2())\n    return Compose(transforms)\n\n\ntrain_dataset = SatoriusDataset(df_train, transforms=get_transform(train=True),root =root,train = True)\ntrain_dataloader = DataLoader(train_dataset, batch_size=64, shuffle=True)\n\nvalidation_dataset = SatoriusDataset(df_val, transforms=get_transform(train=False),root =root,train = False)\nvalidation_dataloader = DataLoader(validation_dataset, batch_size=30, shuffle=False)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-04-01T18:17:29.644352Z","iopub.execute_input":"2023-04-01T18:17:29.644614Z","iopub.status.idle":"2023-04-01T18:17:29.683746Z","shell.execute_reply.started":"2023-04-01T18:17:29.644577Z","shell.execute_reply":"2023-04-01T18:17:29.683078Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### 2.Visualizing Dataset <a class=\"anchor\" id=\"3.2\"></a>","metadata":{}},{"cell_type":"code","source":"train_df = train_dataset.data_info\nvalidation_df = validation_dataset.data_info\n\npd.set_option('max_colwidth', 125)\ndisplay(train_df.head())\ndisplay(validation_df)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:29.684981Z","iopub.execute_input":"2023-04-01T18:17:29.685256Z","iopub.status.idle":"2023-04-01T18:17:29.747001Z","shell.execute_reply.started":"2023-04-01T18:17:29.685221Z","shell.execute_reply":"2023-04-01T18:17:29.746229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.set_palette(\"pastel\")\n\nn_annotations = [[],[]] #[[train], [validation]]\nfor i in train_df['annotation']:\n    n_annotations[0].append(len(i))\n    \nfor i in validation_df['annotation']:\n    n_annotations[1].append(len(i))\n\nx_axis,y_axis = 'Number Of Annotations' , 'Number Of Images'\n\nfig = plt.figure(figsize=(8,8))\np = sns.histplot(data = n_annotations[0])\np.set_xlabel(x_axis, fontsize = 15)\np.set_ylabel(y_axis, fontsize = 15)\n\n\nplt.show()\n\nfig = plt.figure(figsize=(8,8))\np = sns.histplot(data = n_annotations[1])\np.set_xlabel(x_axis, fontsize = 15)\np.set_ylabel(x_axis, fontsize = 15)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:29.748372Z","iopub.execute_input":"2023-04-01T18:17:29.748623Z","iopub.status.idle":"2023-04-01T18:17:30.328736Z","shell.execute_reply.started":"2023-04-01T18:17:29.748590Z","shell.execute_reply":"2023-04-01T18:17:30.328020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import cv2\n\nimg, mask = next(iter(validation_dataloader))\nprint(f'Shape of Image : {img.shape} and Shape of Mask : {mask.shape}')\nimage = np.array(img[0])[0]\nmask = mask[0].reshape((IMG_SIZE[0], IMG_SIZE[1]))\n\nplt.figure(figsize=(10, 10))\nplt.imshow(image,cmap = 'plasma')\nplt.axis(\"off\")\nplt.figure(figsize=(10, 10))\nplt.imshow(image)\nplt.imshow(mask,alpha = 0.3,cmap = 'plasma')\nplt.axis(\"off\")\n    \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:30.330968Z","iopub.execute_input":"2023-04-01T18:17:30.331794Z","iopub.status.idle":"2023-04-01T18:17:33.074581Z","shell.execute_reply.started":"2023-04-01T18:17:30.331753Z","shell.execute_reply":"2023-04-01T18:17:33.073993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def clustered_img(x):\n    kmeans = cluster.KMeans(2)\n    dims = np.shape(x)\n    pixel_matrix = np.reshape(x, (dims[0] * dims[1], dims[2]))\n    clustered = kmeans.fit_predict(pixel_matrix)\n    clustered_img = np.reshape(clustered, (dims[0], dims[1]))\n    return clustered_img\n\nfor i in range(1):\n    fig = plt.figure(figsize=(10,10))\n    img, mask = next(iter(validation_dataloader))\n    img = img[0]\n    mask = mask[0].reshape((IMG_SIZE[0], IMG_SIZE[1]))\n    plt.imshow(clustered_img(img.numpy().transpose((1,2,0))))\n    plt.grid(None)\n    plt.title(\"K-Means\")\n    plt.show()\n\n    fig = plt.figure(figsize=(10,10))\n    plt.imshow(img.numpy()[0],cmap = 'plasma')\n    plt.imshow(mask, alpha=0.3)\n    plt.grid(None)\n    plt.title(\"Target\")\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:33.075743Z","iopub.execute_input":"2023-04-01T18:17:33.076510Z","iopub.status.idle":"2023-04-01T18:17:36.181636Z","shell.execute_reply.started":"2023-04-01T18:17:33.076472Z","shell.execute_reply":"2023-04-01T18:17:36.181013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **4.Initializing pre-trained model** <a class=\"anchor\" id=\"4\"></a>","metadata":{}},{"cell_type":"code","source":"!mkdir -p /root/.cache/torch/hub/checkpoints/\n!cp ../input/pytorch-pretrained-image-models/resnet34.pth /root/.cache/torch/hub/checkpoints/resnet34-333f7ec4.pth\n\nimport torch\nimport collections.abc as container_abcs\ntorch._six.container_abcs = container_abcs\nimport segmentation_models_pytorch as smp","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:36.182969Z","iopub.execute_input":"2023-04-01T18:17:36.183432Z","iopub.status.idle":"2023-04-01T18:17:38.586253Z","shell.execute_reply.started":"2023-04-01T18:17:36.183394Z","shell.execute_reply":"2023-04-01T18:17:38.585222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import segmentation_models_pytorch as smp\nmodel = smp.Unet(\"resnet34\", encoder_weights=\"imagenet\", activation=None)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:38.588413Z","iopub.execute_input":"2023-04-01T18:17:38.588702Z","iopub.status.idle":"2023-04-01T18:17:39.034351Z","shell.execute_reply.started":"2023-04-01T18:17:38.588664Z","shell.execute_reply":"2023-04-01T18:17:39.033570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = 'cuda'\nmodel = model.to(device)\noptimizer = torch.optim.Adam(model.parameters(), lr=0.0005)\n\nepoch = 25\nloss_history = [[],[]]\niou_history = [[],[]]\ntrain_n_minibatches = train_dataloader.__len__()\nvalidation_n_minibatches = validation_dataloader.__len__()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:39.035826Z","iopub.execute_input":"2023-04-01T18:17:39.036110Z","iopub.status.idle":"2023-04-01T18:17:39.077227Z","shell.execute_reply.started":"2023-04-01T18:17:39.036071Z","shell.execute_reply":"2023-04-01T18:17:39.076550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def dice_loss(input, target):\n    input = torch.sigmoid(input)\n    smooth = 1.0\n    iflat = input.view(-1)\n    tflat = target.view(-1)\n    intersection = (iflat * tflat).sum()\n    return ((2.0 * intersection + smooth) / (iflat.sum() + tflat.sum() + smooth))\n\n\nclass FocalLoss(nn.Module):\n    def __init__(self, gamma):\n        super().__init__()\n        self.gamma = gamma\n\n    def forward(self, input, target):\n        if not (target.size() == input.size()):\n            raise ValueError(\"Target size ({}) must be the same as input size ({})\"\n                             .format(target.size(), input.size()))\n        max_val = (-input).clamp(min=0)\n        loss = input - input * target + max_val + \\\n            ((-max_val).exp() + (-input - max_val).exp()).log()\n        invprobs = F.logsigmoid(-input * (target * 2.0 - 1.0))\n        loss = (invprobs * self.gamma).exp() * loss\n        return loss.mean()\n\n\nclass MixedLoss(nn.Module):\n    def __init__(self, alpha, gamma):\n        super().__init__()\n        self.alpha = alpha\n        self.focal = FocalLoss(gamma)\n\n    def forward(self, input, target):\n        loss = self.alpha*self.focal(input, target) - torch.log(dice_loss(input, target))\n        return loss.mean()\n    \ncriterion = MixedLoss(10.0, 2.0)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:39.078612Z","iopub.execute_input":"2023-04-01T18:17:39.078878Z","iopub.status.idle":"2023-04-01T18:17:39.089084Z","shell.execute_reply.started":"2023-04-01T18:17:39.078842Z","shell.execute_reply":"2023-04-01T18:17:39.088158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"DEVICE = 'cuda'\nfrom tqdm.auto import tqdm\ndef analyze_train_iou(model, ds_train, df):\n    model.eval()\n    ds_train.transforms = get_transform(False)\n    df = df.groupby('id').agg(list).reset_index()\n    all_true = []\n    all_pred = []\n\n    for col in df.columns[2:]:\n        df[col] = df[col].apply(\n                lambda x: np.unique(x)[0] if len(np.unique(x)) == 1 else np.unique(x)\n        )\n    for sample_index in tqdm(range(len(ds_train))):\n\n        img, targets = ds_train[sample_index]\n\n        model.eval()\n        with torch.no_grad():\n            preds = model(img.unsqueeze(0).to(DEVICE))[0]\n#         print(preds.shape)\n#         probability_mask = cv2.resize(preds[0].cpu().numpy(), dsize=(704, 520), interpolation=cv2.INTER_LINEAR)\n#         predictions = cv2.threshold(probability_mask, 0.5, 1, cv2.THRESH_BINARY)[1]\n#         print(predictions.shape)\n        rles = df['annotation'].values[sample_index]\n        shape = df[['height', 'width']].values[0]\n        masks = rles_to_mask(rles, shape).astype(np.uint16)\n        \n#         print(preds.shape)\n        probability_mask = cv2.resize(preds[0].cpu().numpy(), dsize=(704, 520), interpolation=cv2.INTER_LINEAR)\n#         print(probability_mask.shape)\n        predictions = post_process(probability_mask)\n        indv_pred = []\n        for i in predictions:\n            indv_pred.append(rle_encoding(i))\n        predictions = rles_to_mask(indv_pred, shape).astype(np.uint16)\n#         print(predictions.shape)\n#         print(np.unique(masks))\n#         print(masks.shape)\n#         plt.imshow(masks)\n#         plt.show()\n#         plt.imshow(predictions)\n#         plt.show()\n\n\n        all_true.append(masks)\n        all_pred.append(predictions)\n        \n        \n#         print(iou)\n# #         break\n#         print(iou_map([masks],[predictions], verbose=1))\n#         print(np.unique(masks))\n    \n    ds_train.transforms = get_transform(True)\n    return iou_map(all_true, all_pred, verbose=1)\n\n\n\n\n\n# analyze_train_iou(model, train_dataset, df_train)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:39.090920Z","iopub.execute_input":"2023-04-01T18:17:39.091191Z","iopub.status.idle":"2023-04-01T18:17:39.101777Z","shell.execute_reply.started":"2023-04-01T18:17:39.091143Z","shell.execute_reply":"2023-04-01T18:17:39.100662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **5.Training** <a class=\"anchor\" id=\"5\"></a>","metadata":{}},{"cell_type":"code","source":"model.train()\nlog_idx = 3\nimport time\ntotal_time = 0\nstart = time.time()\nfor e in range(epoch):\n    for batch_idx , (images ,masks) in enumerate(train_dataloader):\n        model.train()\n        optimizer.zero_grad()\n        images, masks = images.to(device),  masks.to(device)\n        outputs = model(images)\n        loss = criterion(outputs, masks)\n        loss.backward()\n        optimizer.step()\n        loss_history[0].append(float(loss.detach()))\n        total_time += time.time() - start\n        start = time.time()\n        if batch_idx % log_idx == 0:\n            # Printing Log\n            print(f'LOSS for EPOCH {e+1} BATCH {batch_idx+1}/{train_n_minibatches} TRAIN LOSS : {loss_history[0][-1]}',end = ' ')\n            with torch.no_grad():\n                # Calculating loss and accuracy for validation\n                for _batch_idx_ , (images ,masks) in enumerate(validation_dataloader):\n                    images, masks = images.to(device),  masks.to(device)\n                    outputs = model(images)\n                    validation_loss = criterion(outputs, masks)\n                    loss_history[1].append(float(validation_loss.detach()))\n                                      \n                print(f'VALIDATION LOSS : {sum(loss_history[1][-1:-validation_n_minibatches-1:-1])/validation_n_minibatches}')\n\n    torch.save(model.state_dict(),'unet_ss')\n    #Log for e+1th epoch\n    print(f'---------------------------------------EPOCH {e+1}-------------------------------------------')\n    print(f'Loss for EPOCH {e+1}  TRAIN LOSS : {sum(loss_history[0][-1:-train_n_minibatches-1:-1])/train_n_minibatches}')\n    n_validation_losses = int(train_n_minibatches/log_idx)*validation_n_minibatches\n    print(f'VALIDATION LOSS for EPOCH {e+1} : {sum(loss_history[1][-1:-1*n_validation_losses-1:-1])/n_validation_losses}',end = '\\n')\n    print(\"TRAIN IOU:\")\n    iou_history[0].append(analyze_train_iou(model, train_dataset, df_train))\n    print(\"VAL IOU\")\n    iou_history[1].append(analyze_train_iou(model, validation_dataset, df_val))\n    print('---------------------------------------------------------------------------------------------')","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:17:39.103394Z","iopub.execute_input":"2023-04-01T18:17:39.103869Z","iopub.status.idle":"2023-04-01T18:19:53.838434Z","shell.execute_reply.started":"2023-04-01T18:17:39.103833Z","shell.execute_reply":"2023-04-01T18:19:53.837066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Total time taken for training : {total_time} seconds')","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.839784Z","iopub.status.idle":"2023-04-01T18:19:53.840246Z","shell.execute_reply.started":"2023-04-01T18:19:53.839993Z","shell.execute_reply":"2023-04-01T18:19:53.840017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **6.Plotting Graphs** <a class=\"anchor\" id=\"6\"></a>","metadata":{}},{"cell_type":"markdown","source":"#### 1.Plotting Loss vs Epoch<a class=\"anchor\" id=\"6.1\"></a>","metadata":{}},{"cell_type":"code","source":"# Plotting Loss per epoch\nloss_per_epoch = [[],[]]\nfor i in range(epoch):\n    temp = 0\n    for j in loss_history[0][i*train_n_minibatches:(i+1)*train_n_minibatches]:\n        temp = temp + j\n    loss_per_epoch[0].append(temp/train_n_minibatches)\n    temp = 0\n    for j in loss_history[1][i*n_validation_losses:(i+1)*n_validation_losses]:\n        temp = temp + j\n    loss_per_epoch[1].append(temp/n_validation_losses)    \n\nsns.lineplot(x=range(len(loss_per_epoch[0])),y=loss_per_epoch[0])\nsns.lineplot(x=range(len(loss_per_epoch[1])),y=loss_per_epoch[1])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.841813Z","iopub.status.idle":"2023-04-01T18:19:53.842455Z","shell.execute_reply.started":"2023-04-01T18:19:53.842214Z","shell.execute_reply":"2023-04-01T18:19:53.842241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.lineplot(x=range(len(iou_history[0])),y=iou_history[0])\nsns.lineplot(x=range(len(iou_history[1])),y=iou_history[1])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.843413Z","iopub.status.idle":"2023-04-01T18:19:53.844251Z","shell.execute_reply.started":"2023-04-01T18:19:53.843982Z","shell.execute_reply":"2023-04-01T18:19:53.844013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **7.Loading and Testing**<a class=\"anchor\" id=\"7\"></a>","metadata":{}},{"cell_type":"code","source":"# model = smp.Unet(\"resnet34\", encoder_weights=\"imagenet\", activation=None)\n# model.to(device)\n# model.load_state_dict(torch.load('unet_ss', map_location='cuda'))\nwith torch.no_grad():\n    for _batch_idx_ , (images ,masks) in enumerate(validation_dataloader):\n        images, masks_target = images.to(device),  masks.to(device)\n        masks_out = model(images)\n\nimages = images.cpu().detach().numpy()\nmasks_out = masks_out.cpu().detach().numpy()\nmasks_target = masks_target.cpu().detach().numpy()\n\nfor i in range(1):\n    image = images[i][0]\n    mask_out = masks_out[i].reshape((IMG_SIZE[0], IMG_SIZE[1]))\n    mask_target = masks_target[i].reshape((IMG_SIZE[0], IMG_SIZE[1]))\n    \n    plt.figure(figsize=(8, 8))\n    plt.imshow(image , cmap = 'bone')\n    plt.title(\"Image\")\n    plt.axis(\"off\")\n    plt.show()\n    \n    plt.figure(figsize=(8, 8))\n    plt.imshow(mask_target , cmap = 'bone')\n    plt.title(\"Ground Truth\")\n    plt.axis(\"off\")\n    plt.show()\n    \n    plt.figure(figsize=(8, 8))\n    plt.imshow(mask_out , cmap = 'bone')\n    plt.title(\"Prediction\")\n    plt.axis(\"off\")\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.845477Z","iopub.status.idle":"2023-04-01T18:19:53.846308Z","shell.execute_reply.started":"2023-04-01T18:19:53.846032Z","shell.execute_reply":"2023-04-01T18:19:53.846064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **8.Predictions**<a class=\"anchor\" id=\"8\"></a>","metadata":{}},{"cell_type":"code","source":"class TestDataset(Dataset):\n    def __init__(self, root,transforms = None):\n        self.root = root\n        self.transforms = transforms\n        self.img_name = []\n        for i in os.listdir(root+'test/'):\n            self.img_name.append(i[:-4])\n        self.w , self.h = 520 , 704\n\n            \n    def __getitem__(self, idx):\n        image = Image.open(self.root+'test/'+self.img_name[idx]+'.png').convert(\"RGB\")\n        if self.transforms is not None:\n            image = self.transforms(image=np.array(image))['image']\n        \n        return {'image': image, 'id': self.img_name[idx]}\n\n    def __len__(self):\n        return len(self.img_name)\n    \n\ntd = TestDataset(root, transforms=get_transform(train=False))\ntest_loader = DataLoader(td, batch_size=3, shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.847523Z","iopub.status.idle":"2023-04-01T18:19:53.848352Z","shell.execute_reply.started":"2023-04-01T18:19:53.848081Z","shell.execute_reply":"2023-04-01T18:19:53.848114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.eval()\n\nsubmission = []\nfor i, batch in enumerate(test_loader):\n    preds = torch.sigmoid(model(batch['image'].cuda()))\n    preds = preds.detach().cpu().numpy()[:, 0, :, :] # (batch_size, 1, size, size) -> (batch_size, size, size)\n    for image_id, probability_mask in zip(batch['id'], preds):\n        try:\n            #if probability_mask.shape != IMAGE_RESIZE:\n            #    probability_mask = cv2.resize(probability_mask, dsize=IMAGE_RESIZE, interpolation=cv2.INTER_LINEAR)\n            probability_mask = cv2.resize(probability_mask, dsize=(704, 520), interpolation=cv2.INTER_LINEAR)\n            predictions = post_process(probability_mask)\n            for prediction in predictions:\n                #plt.imshow(prediction)\n                #plt.show()\n                try:\n                    submission.append((image_id, rle_encoding(prediction)))\n                except:\n                    print(\"Error in RL encoding\")\n        except Exception as e:\n            print(f\"Exception for img: {image_id}: {e}\")\n        \n        # Fill images with no predictions\n        image_ids = [image_id for image_id, preds in submission]\n        if image_id not in image_ids:\n            submission.append((image_id, \"\"))\n            \ndf_submission = pd.DataFrame(submission, columns=['id', 'predicted'])\ndf_submission.to_csv('submission.csv', index=False)\n\nif df_submission['predicted'].apply(check_is_run_length).mean() != 1:\n    print(\"Check run lenght failed\")\n    create_empty_submission()\n\n","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.849579Z","iopub.status.idle":"2023-04-01T18:19:53.850396Z","shell.execute_reply.started":"2023-04-01T18:19:53.850138Z","shell.execute_reply":"2023-04-01T18:19:53.850186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_submission.groupby('id')['predicted'].agg(lambda x: list(x)).reset_index()","metadata":{"execution":{"iopub.status.busy":"2023-04-01T18:19:53.851569Z","iopub.status.idle":"2023-04-01T18:19:53.852307Z","shell.execute_reply.started":"2023-04-01T18:19:53.852046Z","shell.execute_reply":"2023-04-01T18:19:53.852072Z"},"trusted":true},"execution_count":null,"outputs":[]}]}