{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🦠 Sartorius - Starter Torch Mask R-CNN\n### A self-contained, simple, pure Torch Mask R-CNN implementation, with `LB=0.273`\n\n![](https://storage.googleapis.com/kaggle-competitions/kaggle/30201/logos/header.png)\n\nFollowing [this discussion thread](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790), in this notebook we build a base starter Mask R-CNN with pytorch.\n\nThe code is an adapted version from [this notebook](https://www.kaggle.com/abhishek/mask-rcnn-using-torchvision-0-17/) by the first quadruple kaggle grandmaster [Abishek](https://www.kaggle.com/abhishek).\n\nThe [previous U-net model](https://www.kaggle.com/julian3833/sartorius-starter-baseline-torch-u-net), which I was expecting to enter a steep improvement regime with quick-wins, hit a ceiling at `0.03`, no matter what changes I performed 🥲.\nData augmentation, changes in the architecture, and other changes didn't work. The suggestion that semantic segmentation doesn't work seems reasonable, since the individuals cannot be split by connected components, as they overlap heavily.\n\nThis is a follow up notebook with a Mask R-CNN, which was proposed by one of the top competitors ([Inoichan](https://www.kaggle.com/inoueu1)) as a more suitable architecture for this task.\n\nI'm not very familiar with the architecture, but it seems that it is the state-of-the art for \"instance segmentation\".\nIt classifies individuals, gets bounding boxes around them and, most importantly, provides a separated mask for each of them.\n\nYou can read more about it [here](https://viso.ai/deep-learning/mask-r-cnn/).\n\n\nThis model predicts different masks for different individual, rather that an unique mask for the whole picture and thus is better to address the problem at hand.\n\nAt the end, any overlapping pixel is removed, to ensure the non-overlapping policy. That wasn't required with the U-net, since the output was only one unique mask and therefore no overlap could have happened.\n\n\n## Please _DO_ upvote!\n\n\n<h3 style=\"text-align:center; background-color:#C8FF33;padding:40px;border-radius: 30px;\">\nSee also this notebook: <b><a href=\"https://www.kaggle.com/julian3833/sartorius-classifier-mask-r-cnn-lb-0-28\">🦠 Sartorius - Classifier + Mask R-CNN [LB=0.28]</a></b> using this model along with a simple Resnet Classifier to achieve 0.28\n</h3>\n\n\n\n\n### Changelog\n\n|| Version | Comments | LB |\n|---|  --- | --- | --- |\n|**Best**|33| Roll back to `V31`. Best conf from [here](https://www.kaggle.com/julian3833/sartorius-classifier-mask-r-cnn-lb-0-28). | `0.273` |\n||32| A lot of epochs | `0.273` |\n|**Best**|31| `MIN_SCORE=0.59`. `BOX_DETECTIONS_PER_IMG = 539`. Best conf from [here](https://www.kaggle.com/julian3833/sartorius-classifier-mask-r-cnn-lb-0-28). |`0.273` |\n||30| Version 18 with `MIN_SCORE=0.5`. Remove validation. | `0.27` |\n||28| V27 but pick best epoch using mask-only validation loss. 18 epochs. | `0.205` |\n||27| V18 + 7.5% validation (`PCT_IMAGES_VALIDATION`) w/best epoch for pred. Added `BOX_DETECTIONS_PER_IMG` and `MIN_SCORE` but not used yet. | `0.178` |\n||24| 8 epochs. With Scheduler. | `0.195` |\n||23| 8 epochs. Mask loss only. | `0.036` |\n||22| V18 + Normalize. (7 epochs = `0.189`) | `0.202`|\n||19| 3 epochs size 25%. 3 epochs size 50%. 6 epochs full sized| `0.178` |\n| |__18__| __8 epochs. Added vertical flip. Full sized.__ Tidied-up code.|  `0.202` |\n||15| 12 -> 15 epochs. Setup classification head with classes. Bugfix in `analyze_train_sample`|  `0.172` |\n|| 14 | 12 epochs. Full sized |`0.173` |\n|| 8 | 12 epochs. Resize to (256, 256) |`0.057` |\n\n","metadata":{"execution":{"iopub.status.busy":"2021-10-23T04:12:16.649005Z","iopub.execute_input":"2021-10-23T04:12:16.6494Z","iopub.status.idle":"2021-10-23T04:12:16.726081Z","shell.execute_reply.started":"2021-10-23T04:12:16.649296Z","shell.execute_reply":"2021-10-23T04:12:16.723107Z"}}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"# The notebooks is self-contained\n# It has very few imports\n# No external dependencies (only the model weights)\n# No train - inference notebooks\n# We only rely on Pytorch\nimport os\nimport time\nimport random\nimport collections\n\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\n\nimport torch\nimport torchvision\nfrom torchvision.transforms import ToPILImage\nfrom torchvision.transforms import functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\nfrom torchvision.models.detection.mask_rcnn import MaskRCNNPredictor","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-10-24T04:11:52.137885Z","iopub.execute_input":"2021-10-24T04:11:52.138229Z","iopub.status.idle":"2021-10-24T04:11:54.525318Z","shell.execute_reply.started":"2021-10-24T04:11:52.138145Z","shell.execute_reply":"2021-10-24T04:11:54.524552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fix randomness\n\ndef fix_all_seeds(seed):\n    np.random.seed(seed)\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    \nfix_all_seeds(2021)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.527041Z","iopub.execute_input":"2021-10-24T04:11:54.52764Z","iopub.status.idle":"2021-10-24T04:11:54.536241Z","shell.execute_reply.started":"2021-10-24T04:11:54.527586Z","shell.execute_reply":"2021-10-24T04:11:54.535451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Configuration","metadata":{}},{"cell_type":"code","source":"TRAIN_CSV = \"../input/sartorius-cell-instance-segmentation/train.csv\"\nTRAIN_PATH = \"../input/sartorius-cell-instance-segmentation/train\"\nTEST_PATH = \"../input/sartorius-cell-instance-segmentation/test\"\n\nWIDTH = 704\nHEIGHT = 520\n\n# Reduced the train dataset to 5000 rows\nTEST = False\n\nDEVICE = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\n\nRESNET_MEAN = (0.485, 0.456, 0.406)\nRESNET_STD = (0.229, 0.224, 0.225)\n\nBATCH_SIZE = 2\n\n# No changes tried with the optimizer yet.\nMOMENTUM = 0.9\nLEARNING_RATE = 0.001\nWEIGHT_DECAY = 0.0005\n\n# Changes the confidence required for a pixel to be kept for a mask. \n# Only used 0.5 till now.\nMASK_THRESHOLD = 0.5\n\n# Normalize to resnet mean and std if True.\nNORMALIZE = False \n\n\n# Use a StepLR scheduler if True. Not tried yet.\nUSE_SCHEDULER = False\n\n# Number of epochs\nNUM_EPOCHS = 8\n\n\nBOX_DETECTIONS_PER_IMG = 539\n\n\nMIN_SCORE = 0.59","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.537778Z","iopub.execute_input":"2021-10-24T04:11:54.538334Z","iopub.status.idle":"2021-10-24T04:11:54.594792Z","shell.execute_reply.started":"2021-10-24T04:11:54.538293Z","shell.execute_reply":"2021-10-24T04:11:54.593987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Traning Dataset","metadata":{}},{"cell_type":"markdown","source":"## Utilities\n\n\n### Transformations\n\nJust Horizontal and Vertical Flip for now.\n\nNormalization to Resnet's mean and std can be performed using the parameter `NORMALIZE` in the top cell. Haven't tested it yet.\n\nThe first 3 transformations come from [this](https://www.kaggle.com/abhishek/maskrcnn-utils) utils package by Abishek, `VerticalFlip` is my adaption of HorizontalFlip, and `Normalize` is of my own.","metadata":{}},{"cell_type":"code","source":"# These are slight redefinitions of torch.transformation classes\n# The difference is that they handle the target and the mask\n# Copied from Abishek, added new ones\nclass Compose:\n    def __init__(self, transforms):\n        self.transforms = transforms\n\n    def __call__(self, image, target):\n        for t in self.transforms:\n            image, target = t(image, target)\n        return image, target\n\nclass VerticalFlip:\n    def __init__(self, prob):\n        self.prob = prob\n\n    def __call__(self, image, target):\n        if random.random() < self.prob:\n            height, width = image.shape[-2:]\n            image = image.flip(-2)\n            bbox = target[\"boxes\"]\n            bbox[:, [1, 3]] = height - bbox[:, [3, 1]]\n            target[\"boxes\"] = bbox\n            target[\"masks\"] = target[\"masks\"].flip(-2)\n        return image, target\n\nclass HorizontalFlip:\n    def __init__(self, prob):\n        self.prob = prob\n\n    def __call__(self, image, target):\n        if random.random() < self.prob:\n            height, width = image.shape[-2:]\n            image = image.flip(-1)\n            bbox = target[\"boxes\"]\n            bbox[:, [0, 2]] = width - bbox[:, [2, 0]]\n            target[\"boxes\"] = bbox\n            target[\"masks\"] = target[\"masks\"].flip(-1)\n        return image, target\n\nclass Normalize:\n    def __call__(self, image, target):\n        image = F.normalize(image, RESNET_MEAN, RESNET_STD)\n        return image, target\n\nclass ToTensor:\n    def __call__(self, image, target):\n        image = F.to_tensor(image)\n        return image, target\n    \n\ndef get_transform(train):\n    transforms = [ToTensor()]\n    if NORMALIZE:\n        transforms.append(Normalize())\n    \n    # Data augmentation for train\n    if train: \n        transforms.append(HorizontalFlip(0.5))\n        transforms.append(VerticalFlip(0.5))\n\n    return Compose(transforms)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.597209Z","iopub.execute_input":"2021-10-24T04:11:54.597753Z","iopub.status.idle":"2021-10-24T04:11:54.613644Z","shell.execute_reply.started":"2021-10-24T04:11:54.597715Z","shell.execute_reply":"2021-10-24T04:11:54.612907Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def rle_decode(mask_rle, shape, color=1):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height,width) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0] * shape[1], dtype=np.float32)\n    for lo, hi in zip(starts, ends):\n        img[lo : hi] = color\n    return img.reshape(shape)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.616175Z","iopub.execute_input":"2021-10-24T04:11:54.617563Z","iopub.status.idle":"2021-10-24T04:11:54.627593Z","shell.execute_reply.started":"2021-10-24T04:11:54.617522Z","shell.execute_reply":"2021-10-24T04:11:54.626863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training Dataset and DataLoader","metadata":{}},{"cell_type":"code","source":"class CellDataset(Dataset):\n    def __init__(self, image_dir, df, transforms=None, resize=False):\n        self.transforms = transforms\n        self.image_dir = image_dir\n        self.df = df\n        \n        self.should_resize = resize is not False\n        if self.should_resize:\n            self.height = int(HEIGHT * resize)\n            self.width = int(WIDTH * resize)\n        else:\n            self.height = HEIGHT\n            self.width = WIDTH\n        \n        self.image_info = collections.defaultdict(dict)\n        temp_df = self.df.groupby('id')['annotation'].agg(lambda x: list(x)).reset_index()\n        for index, row in temp_df.iterrows():\n            self.image_info[index] = {\n                    'image_id': row['id'],\n                    'image_path': os.path.join(self.image_dir, row['id'] + '.png'),\n                    'annotations': row[\"annotation\"]\n                    }\n    \n    def get_box(self, a_mask):\n        ''' Get the bounding box of a given mask '''\n        pos = np.where(a_mask)\n        xmin = np.min(pos[1])\n        xmax = np.max(pos[1])\n        ymin = np.min(pos[0])\n        ymax = np.max(pos[0])\n        return [xmin, ymin, xmax, ymax]\n\n    def __getitem__(self, idx):\n        ''' Get the image and the target'''\n        \n        img_path = self.image_info[idx][\"image_path\"]\n        img = Image.open(img_path).convert(\"RGB\")\n        \n        if self.should_resize:\n            img = img.resize((self.width, self.height), resample=Image.BILINEAR)\n\n        info = self.image_info[idx]\n\n        n_objects = len(info['annotations'])\n        masks = np.zeros((len(info['annotations']), self.height, self.width), dtype=np.uint8)\n        boxes = []\n        \n        for i, annotation in enumerate(info['annotations']):\n            a_mask = rle_decode(annotation, (HEIGHT, WIDTH))\n            a_mask = Image.fromarray(a_mask)\n            \n            if self.should_resize:\n                a_mask = a_mask.resize((self.width, self.height), resample=Image.BILINEAR)\n            \n            a_mask = np.array(a_mask) > 0\n            masks[i, :, :] = a_mask\n            \n            boxes.append(self.get_box(a_mask))\n\n        # dummy labels\n        labels = [1 for _ in range(n_objects)]\n        \n        boxes = torch.as_tensor(boxes, dtype=torch.float32)\n        labels = torch.as_tensor(labels, dtype=torch.int64)\n        masks = torch.as_tensor(masks, dtype=torch.uint8)\n\n        image_id = torch.tensor([idx])\n        area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])\n        iscrowd = torch.zeros((n_objects,), dtype=torch.int64)\n\n        # This is the required target for the Mask R-CNN\n        target = {\n            'boxes': boxes,\n            'labels': labels,\n            'masks': masks,\n            'image_id': image_id,\n            'area': area,\n            'iscrowd': iscrowd\n        }\n\n        if self.transforms is not None:\n            img, target = self.transforms(img, target)\n\n        return img, target\n\n    def __len__(self):\n        return len(self.image_info)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.629175Z","iopub.execute_input":"2021-10-24T04:11:54.629771Z","iopub.status.idle":"2021-10-24T04:11:54.650706Z","shell.execute_reply.started":"2021-10-24T04:11:54.629731Z","shell.execute_reply":"2021-10-24T04:11:54.649929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = pd.read_csv(TRAIN_CSV, nrows=5000 if TEST else None)\nds_train = CellDataset(TRAIN_PATH, df_train, resize=False, transforms=get_transform(train=True))\ndl_train = DataLoader(ds_train, batch_size=BATCH_SIZE, shuffle=True, \n                      num_workers=2, collate_fn=lambda x: tuple(zip(*x)))","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.725523Z","iopub.execute_input":"2021-10-24T04:11:54.725784Z","iopub.status.idle":"2021-10-24T04:11:54.75018Z","shell.execute_reply.started":"2021-10-24T04:11:54.725752Z","shell.execute_reply":"2021-10-24T04:11:54.749557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train loop","metadata":{}},{"cell_type":"markdown","source":"## Model","metadata":{}},{"cell_type":"code","source":"# Override pythorch checkpoint with an \"offline\" version of the file\n!mkdir -p /root/.cache/torch/hub/checkpoints/\n!cp ../input/cocopre/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth /root/.cache/torch/hub/checkpoints/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:54.752349Z","iopub.execute_input":"2021-10-24T04:11:54.752731Z","iopub.status.idle":"2021-10-24T04:11:59.831649Z","shell.execute_reply.started":"2021-10-24T04:11:54.752696Z","shell.execute_reply":"2021-10-24T04:11:59.830487Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_model():\n    # This is just a dummy value for the classification head\n    NUM_CLASSES = 2\n    \n    if NORMALIZE:\n        model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True, \n                                                                   box_detections_per_img=BOX_DETECTIONS_PER_IMG,\n                                                                   image_mean=RESNET_MEAN, \n                                                                   image_std=RESNET_STD)\n    else:\n        model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True,\n                                                                  box_detections_per_img=BOX_DETECTIONS_PER_IMG)\n\n    # get the number of input features for the classifier\n    in_features = model.roi_heads.box_predictor.cls_score.in_features\n    # replace the pre-trained head with a new one\n    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, NUM_CLASSES)\n\n    # now get the number of input features for the mask classifier\n    in_features_mask = model.roi_heads.mask_predictor.conv5_mask.in_channels\n    hidden_layer = 256\n    # and replace the mask predictor with a new one\n    model.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask, hidden_layer, NUM_CLASSES)\n    return model\n\n\n# Get the Mask R-CNN model\n# The model does classification, bounding boxes and MASKs for individuals, all at the same time\n# We only care about MASKS\nmodel = get_model()\nmodel.to(DEVICE)\n\n# TODO: try removing this for\nfor param in model.parameters():\n    param.requires_grad = True\n    \nmodel.train();","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:11:59.835113Z","iopub.execute_input":"2021-10-24T04:11:59.835447Z","iopub.status.idle":"2021-10-24T04:12:03.585295Z","shell.execute_reply.started":"2021-10-24T04:11:59.835384Z","shell.execute_reply":"2021-10-24T04:12:03.584474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training loop!","metadata":{}},{"cell_type":"code","source":"params = [p for p in model.parameters() if p.requires_grad]\noptimizer = torch.optim.SGD(params, lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n\nlr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)\n\nn_batches = len(dl_train)\n\nfor epoch in range(1, NUM_EPOCHS + 1):\n    print(f\"Starting epoch {epoch} of {NUM_EPOCHS}\")\n    \n    time_start = time.time()\n    loss_accum = 0.0\n    loss_mask_accum = 0.0\n    \n    for batch_idx, (images, targets) in enumerate(dl_train, 1):\n    \n        # Predict\n        images = list(image.to(DEVICE) for image in images)\n        targets = [{k: v.to(DEVICE) for k, v in t.items()} for t in targets]\n\n        loss_dict = model(images, targets)\n        loss = sum(loss for loss in loss_dict.values())\n        \n        # Backprop\n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n        \n        # Logging\n        loss_mask = loss_dict['loss_mask'].item()\n        loss_accum += loss.item()\n        loss_mask_accum += loss_mask\n        \n        if batch_idx % 50 == 0:\n            print(f\"    [Batch {batch_idx:3d} / {n_batches:3d}] Batch train loss: {loss.item():7.3f}. Mask-only loss: {loss_mask:7.3f}\")\n    \n    if USE_SCHEDULER:\n        lr_scheduler.step()\n    \n    # Train losses\n    train_loss = loss_accum / n_batches\n    train_loss_mask = loss_mask_accum / n_batches\n    \n    \n    elapsed = time.time() - time_start\n    \n    \n    torch.save(model.state_dict(), f\"pytorch_model-e{epoch}.bin\")\n    prefix = f\"[Epoch {epoch:2d} / {NUM_EPOCHS:2d}]\"\n    print(f\"{prefix} Train mask-only loss: {train_loss_mask:7.3f}\")\n    print(f\"{prefix} Train loss: {train_loss:7.3f}. [{elapsed:.0f} secs]\")\n     ","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:12:03.586712Z","iopub.execute_input":"2021-10-24T04:12:03.587259Z","iopub.status.idle":"2021-10-24T04:12:27.97856Z","shell.execute_reply.started":"2021-10-24T04:12:03.587219Z","shell.execute_reply":"2021-10-24T04:12:27.976819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Analyze prediction results for train set","metadata":{}},{"cell_type":"code","source":"# Plots: the image, The image + the ground truth mask, The image + the predicted mask\ndef analyze_train_sample(model, ds_train, sample_index):\n    \n    img, targets = ds_train[sample_index]\n    plt.imshow(img.numpy().transpose((1,2,0)))\n    plt.title(\"Image\")\n    plt.show()\n    \n    masks = np.zeros((HEIGHT, WIDTH))\n    for mask in targets['masks']:\n        masks = np.logical_or(masks, mask)\n    plt.imshow(img.numpy().transpose((1,2,0)))\n    plt.imshow(masks, alpha=0.3)\n    plt.title(\"Ground truth\")\n    plt.show()\n    \n    model.eval()\n    with torch.no_grad():\n        preds = model([img.to(DEVICE)])[0]\n\n    plt.imshow(img.cpu().numpy().transpose((1,2,0)))\n    all_preds_masks = np.zeros((HEIGHT, WIDTH))\n    for mask in preds['masks'].cpu().detach().numpy():\n        all_preds_masks = np.logical_or(all_preds_masks, mask[0] > MASK_THRESHOLD)\n    plt.imshow(all_preds_masks, alpha=0.4)\n    plt.title(\"Predictions\")\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:12:27.98033Z","iopub.execute_input":"2021-10-24T04:12:27.981559Z","iopub.status.idle":"2021-10-24T04:12:27.993269Z","shell.execute_reply.started":"2021-10-24T04:12:27.981524Z","shell.execute_reply":"2021-10-24T04:12:27.99243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# NOTE: It puts the model in eval mode!! Revert for re-training\nanalyze_train_sample(model, ds_train, 20)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:12:27.994437Z","iopub.execute_input":"2021-10-24T04:12:27.995025Z","iopub.status.idle":"2021-10-24T04:12:29.303689Z","shell.execute_reply.started":"2021-10-24T04:12:27.994981Z","shell.execute_reply":"2021-10-24T04:12:29.302906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"analyze_train_sample(model, ds_train, 100)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:12:29.305088Z","iopub.execute_input":"2021-10-24T04:12:29.305507Z","iopub.status.idle":"2021-10-24T04:12:29.448657Z","shell.execute_reply.started":"2021-10-24T04:12:29.305472Z","shell.execute_reply":"2021-10-24T04:12:29.447461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"analyze_train_sample(model, ds_train, 2)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:12:29.449907Z","iopub.status.idle":"2021-10-24T04:12:29.450401Z","shell.execute_reply.started":"2021-10-24T04:12:29.450158Z","shell.execute_reply":"2021-10-24T04:12:29.450182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction","metadata":{}},{"cell_type":"markdown","source":"## Test Dataset and DataLoader","metadata":{}},{"cell_type":"code","source":"class CellTestDataset(Dataset):\n    def __init__(self, image_dir, transforms=None):\n        self.transforms = transforms\n        self.image_dir = image_dir\n        self.image_ids = [f[:-4]for f in os.listdir(self.image_dir)]\n    \n    def __getitem__(self, idx):\n        image_id = self.image_ids[idx]\n        image_path = os.path.join(self.image_dir, image_id + '.png')\n        image = Image.open(image_path).convert(\"RGB\")\n\n        if self.transforms is not None:\n            image, _ = self.transforms(image=image, target=None)\n        return {'image': image, 'image_id': image_id}\n\n    def __len__(self):\n        return len(self.image_ids)","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:13:06.993109Z","iopub.execute_input":"2021-10-24T04:13:06.993372Z","iopub.status.idle":"2021-10-24T04:13:07.002454Z","shell.execute_reply.started":"2021-10-24T04:13:06.993343Z","shell.execute_reply":"2021-10-24T04:13:07.001575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_test = CellTestDataset(TEST_PATH, transforms=get_transform(train=False))\nds_test[0]","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:13:07.293984Z","iopub.execute_input":"2021-10-24T04:13:07.294642Z","iopub.status.idle":"2021-10-24T04:13:07.337214Z","shell.execute_reply.started":"2021-10-24T04:13:07.294603Z","shell.execute_reply":"2021-10-24T04:13:07.3363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Utilities","metadata":{}},{"cell_type":"code","source":"def rle_encoding(x):\n    dots = np.where(x.flatten() == 1)[0]\n    run_lengths = []\n    prev = -2\n    for b in dots:\n        if (b>prev+1): run_lengths.extend((b + 1, 0))\n        run_lengths[-1] += 1\n        prev = b\n    return ' '.join(map(str, run_lengths))\n\n\ndef remove_overlapping_pixels(mask, other_masks):\n    for other_mask in other_masks:\n        if np.sum(np.logical_and(mask, other_mask)) > 0:\n            mask[np.logical_and(mask, other_mask)] = 0\n    return mask","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:13:08.242844Z","iopub.execute_input":"2021-10-24T04:13:08.245743Z","iopub.status.idle":"2021-10-24T04:13:08.25477Z","shell.execute_reply.started":"2021-10-24T04:13:08.245668Z","shell.execute_reply":"2021-10-24T04:13:08.253818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Run predictions","metadata":{}},{"cell_type":"code","source":"model.eval();\n\nsubmission = []\nfor sample in ds_test:\n    img = sample['image']\n    image_id = sample['image_id']\n    with torch.no_grad():\n        result = model([img.to(DEVICE)])[0]\n    \n    previous_masks = []\n    for i, mask in enumerate(result[\"masks\"]):\n        \n        # Filter-out low-scoring results. Not tried yet.\n        score = result[\"scores\"][i].cpu().item()\n        if score < MIN_SCORE:\n            continue\n        \n        mask = mask.cpu().numpy()\n        # Keep only highly likely pixels\n        binary_mask = mask > MASK_THRESHOLD\n        binary_mask = remove_overlapping_pixels(binary_mask, previous_masks)\n        previous_masks.append(binary_mask)\n        rle = rle_encoding(binary_mask)\n        submission.append((image_id, rle))\n    \n    # Add empty prediction if no RLE was generated for this image\n    all_images_ids = [image_id for image_id, rle in submission]\n    if image_id not in all_images_ids:\n        submission.append((image_id, \"\"))\n\ndf_sub = pd.DataFrame(submission, columns=['id', 'predicted'])\ndf_sub.to_csv(\"submission.csv\", index=False)\ndf_sub.head()","metadata":{"execution":{"iopub.status.busy":"2021-10-24T04:13:11.081941Z","iopub.execute_input":"2021-10-24T04:13:11.08221Z","iopub.status.idle":"2021-10-24T04:13:23.967017Z","shell.execute_reply.started":"2021-10-24T04:13:11.082177Z","shell.execute_reply":"2021-10-24T04:13:23.966285Z"},"trusted":true},"execution_count":null,"outputs":[]}]}