**\n> \n> In this competition we are segmenting neuronal cells in images. The training annotations are provided as run length encoded masks, and the images are in PNG format. The number of images is small, but the number of annotated objects is quite high. The hidden test set is roughly 240 images.\n> \n> ### **
**\n> \n> **train.csv** - IDs and masks for all training objects. None of this metadata is provided for the test set.\n> \n> - `id` - unique identifier for object\n> \n> - `annotation` - run length encoded pixels for the identified neuronal cell\n> \n> - `width` - source image width\n> \n> - `height` - source image height\n> \n> - `cell_type` - the cell line\n> \n> - `plate_time` - time plate was created\n> \n> - `sample_date` - date sample was created\n> \n> - `sample_id` - sample identifier\n> \n> - `elapsed_timedelta` - time since first image taken of sample\n> \n> **sample_submission.csv** - a sample submission file in the correct format\n> \n> **train** - train images in PNG format\n> \n> **test** - test images in PNG format. Only a few test set images are available for download; the remainder can only be accessed by your notebooks when you submit.\n> \n> **train_semi_supervised** - unlabeled images offered in case you want to use additional data for a semi-supervised approach.\n> \n> **LIVECell_dataset_2021** - A mirror of the data from the LIVECell dataset. LIVECell is the predecessor dataset to this competition. You will find extra data for the SH-SHY5Y cell line, plus several other cell lines not covered in the competition dataset that may be of interest for transfer learning.\n>\n>---","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_csv(config.TRAIN_CSV)","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:32:59.947021Z","iopub.execute_input":"2021-11-04T08:32:59.947816Z","iopub.status.idle":"2021-11-04T08:33:00.466851Z","shell.execute_reply.started":"2021-11-04T08:32:59.947789Z","shell.execute_reply":"2021-11-04T08:33:00.466179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def getImagePaths(path):\n \"\"\"\n Function to Combine Directory Path with individual Image Paths\n \n parameters: path(string) - Path of directory\n returns: image_names(string) - Full Image Path\n \"\"\"\n image_names = []\n for dirname, _, filenames in os.walk(path):\n for filename in tqdm(filenames):\n fullpath = os.path.join(dirname, filename)\n image_names.append(fullpath)\n return image_names","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:00.468083Z","iopub.execute_input":"2021-11-04T08:33:00.468325Z","iopub.status.idle":"2021-11-04T08:33:00.475972Z","shell.execute_reply.started":"2021-11-04T08:33:00.468290Z","shell.execute_reply":"2021-11-04T08:33:00.475256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Get complete image paths for train and test datasets\ntrain_images_path = getImagePaths(config.TRAIN_PATH)\ntest_images_path = getImagePaths(config.TEST_PATH)\ntrain_semi_supervised_path = getImagePaths(config.TRAIN_SEMI_SUPERVISED_PATH)","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:00.477369Z","iopub.execute_input":"2021-11-04T08:33:00.477658Z","iopub.status.idle":"2021-11-04T08:33:02.304349Z","shell.execute_reply.started":"2021-11-04T08:33:00.477622Z","shell.execute_reply":"2021-11-04T08:33:02.303637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
U-Net Model
","metadata":{}},{"cell_type":"markdown","source":"There is large consent that successful training of **deep networks** requires many thousand annotated training samples. In the UNet paper, researchers presented a network and training strategy that relies on the strong use of **data augmentation** to use the available annotated samples more efficiently. \n \nThe architecture consists of a **contracting path** to capture context and a **symmetric expanding path** that enables precise localization. They showed that such a network can be trained end-to-end from very few images and outperforms the prior best method (*a sliding-window convolutional network*) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. \n \nMoreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. \n \n> The full\n> implementation (based on Caffe) and the trained networks are available\n> at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"## **
Introduction to UNet**\n\nIn the **UNet architecture** the upsampling part has a large number of feature channels, which allow the network to propagate context information to higher resolution layers. As a consequence, the **expansive path** is more or less symmetric to the **contracting path**, and yields a u-shaped architecture. \n \n>The network does not have any fully connected layers and only uses the valid part of each convolution, i.e., the segmentation map only contains the pixels, for which the full context is available in the input image.\n\n\n\nThis strategy allows the seamless segmentation of arbitrarily large images by an\n**overlap-tile strategy** (above image). \n \nTo predict the pixels in the border region of the image, the missing context is extrapolated by mirroring the input image. \n \n>This tiling strategy is important to apply the network to large images, since otherwise the resolution would be limited by the GPU memory.\n \nAs for the tasks there is very little training data available, researchers used excessive\n**data augmentation** by applying **elastic deformations** to the available training images. This allows the network to learn invariance to such deformations, without the need to see these transformations in the annotated image corpus. \n \nThis is particularly important in *biomedical segmentation*, since deformation used to\nbe the most common variation in tissue and realistic deformations can be simulated efficiently. \n\n\n\nAnother challenge in many cell segmentation tasks is the separation of touching objects of the same class (above image). To this end, researchers proposed the use of a **weighted loss**, where the separating background labels between touching cells obtain a large weight in the loss function.\n \nThe resulting network is applicable to various biomedical segmentation problems.","metadata":{}},{"cell_type":"markdown","source":"## **
UNet Architecture**","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"The network architecture is illustrated above. \n \nIt consists of a **Contracting Path** (left side) and an **Expansive Path** (right side). \n \n1.**Contracting Path:**\n \n>The contracting path follows the typical architecture of a convolutional network. It consists of the repeated application of two 3x3 convolutions (*unpadded convolutions*), each followed by\na rectified linear unit (ReLU) and a 2x2 max pooling operation with *stride 2\nfor downsampling*. \n \n>At each downsampling step we double the number of feature channels. \n\n2.**Expansive Path:**\n\n>Every step in the expansive path consists of an upsampling of the feature map followed by a 2x2 convolution (*up-convolution*) that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3x3 convolutions, each followed by a ReLU. \n \nThe cropping is necessary due to the loss of border pixels in every convolution. At the final layer a 1x1 convolution is used to map each 64-component feature vector to the desired number of classes. \n \n*In total the network has 23 convolutional layers.*\n \nTo allow a seamless tiling of the output segmentation map, it is important to select the input tile size such that all 2x2 max-pooling operations are applied to a layer with an even x- and y-size.","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"## **
Data Augmentation**\n\n- Data augmentation is essential to teach the network the desired **invariance** and **robustness** properties, when only few training samples are available. \n \n- In case of microscopical images we primarily need shift and rotation invariance as well as\nrobustness to deformations and gray value variations. \n \n- Especially **random elastic deformations** of the training samples seem to be the key concept to train a segmentation network with very few annotated images. \n \n- We generate smooth deformations using random **displacement vectors** on a coarse 3 by 3 grid. \n \n- The displacements are sampled from a **Gaussian distribution** with 10 pixels standard deviation. \n \n- Per-pixel displacements are then computed using **bicubic interpolation**. \n \n- Drop-out layers at the end of the contracting path perform further implicit data augmentation.\n","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"## **
Utilities**","metadata":{}},{"cell_type":"code","source":"def rle_decode(mask_rle, shape, color=1):\n '''\n mask_rle: run-length as string formated (start length)\n shape: (height,width) of array to return \n Returns numpy array, 1 - mask, 0 - background\n\n '''\n s = mask_rle.split()\n starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n starts -= 1\n ends = starts + lengths\n img = np.zeros(shape[0] * shape[1], dtype=np.float32)\n for lo, hi in zip(starts, ends):\n img[lo : hi] = color\n return img.reshape(shape)\n\n\ndef build_masks(df_train, image_id, input_shape):\n height, width = input_shape\n labels = df_train[df_train[\"id\"] == image_id][\"annotation\"].tolist()\n mask = np.zeros((height, width))\n for label in labels:\n mask += rle_decode(label, shape=(height, width))\n mask = mask.clip(0, 1)\n return mask","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:02.308126Z","iopub.execute_input":"2021-11-04T08:33:02.309698Z","iopub.status.idle":"2021-11-04T08:33:02.320820Z","shell.execute_reply.started":"2021-11-04T08:33:02.309658Z","shell.execute_reply":"2021-11-04T08:33:02.319936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **
Dataset Class**","metadata":{}},{"cell_type":"code","source":"class CellDataset(Dataset):\n def __init__(self, df):\n self.df = df\n self.base_path = config.TRAIN_PATH\n self.transforms = Compose([Resize(config.IMAGE_RESIZE[0], config.IMAGE_RESIZE[1]), \n Normalize(mean=config.RESNET_MEAN, std=config.RESNET_STD, p=1), \n HorizontalFlip(p=0.5),\n VerticalFlip(p=0.5),\n ToTensorV2()])\n self.gb = self.df.groupby('id')\n self.image_ids = df.id.unique().tolist()\n\n def __getitem__(self, idx):\n image_id = self.image_ids[idx]\n df = self.gb.get_group(image_id)\n annotations = df['annotation'].tolist()\n image_path = os.path.join(self.base_path, image_id + \".png\")\n image = cv2.imread(image_path)\n mask = build_masks(df_train, image_id, input_shape=(520, 704))\n mask = (mask >= 1).astype('float32')\n augmented = self.transforms(image=image, mask=mask)\n image = augmented['image']\n mask = augmented['mask']\n return image, mask.reshape((1, config.IMAGE_RESIZE[0], config.IMAGE_RESIZE[1]))\n\n def __len__(self):\n return len(self.image_ids) ","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:02.322142Z","iopub.execute_input":"2021-11-04T08:33:02.322583Z","iopub.status.idle":"2021-11-04T08:33:02.337481Z","shell.execute_reply.started":"2021-11-04T08:33:02.322543Z","shell.execute_reply":"2021-11-04T08:33:02.336634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Prepare Dataset\n\nds_train = CellDataset(df_train)\nimage, mask = ds_train[1]\nimage.shape, mask.shape","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:02.341103Z","iopub.execute_input":"2021-11-04T08:33:02.341558Z","iopub.status.idle":"2021-11-04T08:33:02.550926Z","shell.execute_reply.started":"2021-11-04T08:33:02.341518Z","shell.execute_reply":"2021-11-04T08:33:02.548476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(image[0], cmap='bone')\nplt.show()\nplt.imshow(mask[0], alpha=0.3)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:02.555534Z","iopub.execute_input":"2021-11-04T08:33:02.558073Z","iopub.status.idle":"2021-11-04T08:33:03.107007Z","shell.execute_reply.started":"2021-11-04T08:33:02.558033Z","shell.execute_reply":"2021-11-04T08:33:03.105808Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **
Dataset Loader**","metadata":{}},{"cell_type":"code","source":"# Prepare Dataloader\n\ndl_train = DataLoader(\n ds_train, \n batch_size=64, \n num_workers=4, \n pin_memory=True, \n shuffle=False\n)","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:03.108620Z","iopub.execute_input":"2021-11-04T08:33:03.109092Z","iopub.status.idle":"2021-11-04T08:33:03.114787Z","shell.execute_reply.started":"2021-11-04T08:33:03.109047Z","shell.execute_reply":"2021-11-04T08:33:03.113429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **
Losses**","metadata":{}},{"cell_type":"code","source":"def dice_loss(input, target):\n input = torch.sigmoid(input)\n smooth = 1.0\n iflat = input.view(-1)\n tflat = target.view(-1)\n intersection = (iflat * tflat).sum()\n return ((2.0 * intersection + smooth) / (iflat.sum() + tflat.sum() + smooth))","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:03.116920Z","iopub.execute_input":"2021-11-04T08:33:03.117230Z","iopub.status.idle":"2021-11-04T08:33:03.125843Z","shell.execute_reply.started":"2021-11-04T08:33:03.117191Z","shell.execute_reply":"2021-11-04T08:33:03.124676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **
1. Focal Loss**\n\nFocal loss was introduced by Lin et al. in [Focal Loss for Dense Object Detection](https://paperswithcode.com/paper/focal-loss-for-dense-object-detection)\n\n- A **Focal Loss** function addresses class imbalance during training in tasks like object detection. \n \n- Focal loss applies a modulating term to the cross entropy loss in order to focus learning on hard negative examples. \n \n- It is a dynamically scaled cross entropy loss, where the scaling factor decays to zero as confidence in the correct class increases. \n \n- Intuitively, this scaling factor can automatically down-weight the contribution of easy examples during training and rapidly focus the model on hard examples.\n\n","metadata":{}},{"cell_type":"code","source":"class FocalLoss(nn.Module):\n def __init__(self, gamma):\n super().__init__()\n self.gamma = gamma\n\n def forward(self, input, target):\n if not (target.size() == input.size()):\n raise ValueError(\"Target size ({}) must be the same as input size ({})\"\n .format(target.size(), input.size()))\n max_val = (-input).clamp(min=0)\n loss = input - input * target + max_val + \\\n ((-max_val).exp() + (-input - max_val).exp()).log()\n invprobs = F.logsigmoid(-input * (target * 2.0 - 1.0))\n loss = (invprobs * self.gamma).exp() * loss\n return loss.mean()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:03.128004Z","iopub.execute_input":"2021-11-04T08:33:03.128652Z","iopub.status.idle":"2021-11-04T08:33:03.139237Z","shell.execute_reply.started":"2021-11-04T08:33:03.128605Z","shell.execute_reply":"2021-11-04T08:33:03.138305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **
2. Focal Loss**","metadata":{}},{"cell_type":"code","source":"class MixedLoss(nn.Module):\n def __init__(self, alpha, gamma):\n super().__init__()\n self.alpha = alpha\n self.focal = FocalLoss(gamma)\n\n def forward(self, input, target):\n loss = self.alpha*self.focal(input, target) - torch.log(dice_loss(input, target))\n return loss.mean()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:03.141876Z","iopub.execute_input":"2021-11-04T08:33:03.142111Z","iopub.status.idle":"2021-11-04T08:33:03.151352Z","shell.execute_reply.started":"2021-11-04T08:33:03.142079Z","shell.execute_reply":"2021-11-04T08:33:03.150466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **
Model**","metadata":{}},{"cell_type":"code","source":"model = smp.Unet(\"resnet34\", encoder_weights=\"imagenet\", activation=None)","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:33:03.153191Z","iopub.execute_input":"2021-11-04T08:33:03.153493Z","iopub.status.idle":"2021-11-04T08:33:03.645985Z","shell.execute_reply.started":"2021-11-04T08:33:03.153457Z","shell.execute_reply":"2021-11-04T08:33:03.645255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **
Training**","metadata":{}},{"cell_type":"code","source":"torch.set_default_tensor_type(\"torch.cuda.FloatTensor\")\nn_batches = len(dl_train)\n\nmodel.cuda()\nmodel.train()\n\ncriterion = MixedLoss(10.0, 2.0)\noptimizer = optim.Adam(model.parameters(), lr=config.LEARNING_RATE)\n\n# Initialize W&B\nrun = wandb.init(project='sartorius-unet', config=WANDB_CONFIG)\n\nfor epoch in range(1, config.EPOCHS + 1):\n print(f\"Starting epoch: {epoch} / {config.EPOCHS}\")\n running_loss = 0.0\n optimizer.zero_grad()\n \n for batch_idx, batch in enumerate(dl_train):\n \n # Predict\n images, masks = batch\n images, masks = images.cuda(), masks.cuda()\n outputs = model(images)\n loss = criterion(outputs, masks)\n wandb.log({\"loss\": loss})\n \n # Back prop\n loss.backward()\n optimizer.step()\n optimizer.zero_grad()\n running_loss += loss.item()\n\n epoch_loss = running_loss / n_batches\n print(f\"Epoch: {epoch} - Train Loss {epoch_loss:.4f}\")\n \nwandb.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T08:42:40.499967Z","iopub.execute_input":"2021-11-04T08:42:40.500229Z","iopub.status.idle":"2021-11-04T08:49:31.066312Z","shell.execute_reply.started":"2021-11-04T08:42:40.500200Z","shell.execute_reply":"2021-11-04T08:49:31.065567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## [Check out the run page here $\\rightarrow$](https://wandb.ai/ishandutta/sartorius-unet?workspace=) ","metadata":{}},{"cell_type":"markdown","source":"
\n
\n
References
","metadata":{}},{"cell_type":"markdown","source":">- [🦠 Sartorius - Starter Baseline Torch U-net](https://www.kaggle.com/julian3833/sartorius-starter-baseline-torch-u-net)\n>- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://paperswithcode.com/paper/u-net-convolutional-networks-for-biomedical)\n>\n>---","metadata":{}},{"cell_type":"markdown","source":"
More Plots and Models coming soon!
\n\n
![](\"https://static.wixstatic.com/media/5f8fae_7581e21a24a1483085024f88b0949a9d~mv2.jpg/v1/fill/w_934,h_379,al_c,q_90/5f8fae_7581e21a24a1483085024f88b0949a9d~mv2.jpg\")
","metadata":{}},{"cell_type":"markdown","source":"--- \n\n## **
Let's have a Talk!**\n> ### Reach out to me on [LinkedIn](https://www.linkedin.com/in/ishandutta0098)\n\n---","metadata":{}}]}
![](\"https://www.innovationnewsnetwork.com/wp-content/uploads/2021/02/%C2%A9-iStock-peterschreiber.media_-696x392.jpg\")
![](\"https://i.imgur.com/1sm6x8P.png\")