{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n

[Inference] - FastAI Baseline

\n
","metadata":{}},{"cell_type":"markdown","source":"
\n\n
","metadata":{}},{"cell_type":"markdown","source":"# Description \n\nWelcome to Human BioMolecular Atlas Program (HuBMAP) + Human Protein Atlas (HPA) competition. \nThe objective of this challenge is segmentation of functional tissue units (FTU. e.g., glomeruli in kidney or alveoli in the lung) in biopsy slides from several different organs. \nThe underlying data includes imagery from different sources prepared with different protocols at a variety of resolutions, reflecting typical challenges for working with medical data.\n\nThis notebook provides a fast.ai starter Pytorch code based on a U-shape network (UneXt50) that was used on multiple competitions in the past and includes several tricks from the previous segmentation competitions.\nIt is [dividing the images into tiles](https://www.kaggle.com/code/thedevastator/converting-to-256x256), selection of tiles with tissue, evaluation of the predictions of multiple models with TTA, combining the tile masks back into image level masks, and conversion into RLE. The [inference](https://www.kaggle.com/code/thedevastator/inference-fastai-baseline) is performed based on models trained in the [fast.ai training notebook](https://www.kaggle.com/code/thedevastator/training-fastai-baseline).\n\n**Training & Dataset Creation**\n\n#### - Training Notebook [here](https://www.kaggle.com/code/thedevastator/training-fastai-baseline). \n#### - Dataset Creation [here](https://www.kaggle.com/code/thedevastator/converting-to-256x256). \n\n**Precomputed Datasets**\n\n##### - [Dataset (512 x 512)](https://www.kaggle.com/datasets/thedevastator/hubmap-2022-512x512/)\n\n##### - [Dataset (256 x 256)](https://www.kaggle.com/datasets/thedevastator/hubmap-2022-256x256/)\n\n##### - [Dataset (128 x 128)](https://www.kaggle.com/datasets/thedevastator/hubmap-2022-128x128/settings)\n\n____\n\n#### Everything is based on the excellent [notebooks](https://www.kaggle.com/code/iafoss/hubmap-pytorch-fast-ai-starter) by [iafoss](https://www.kaggle.com/iafoss) \nAll credit to belongs to the original author!\n____","metadata":{}},{"cell_type":"markdown","source":"# Description\n","metadata":{"papermill":{"duration":0.00991,"end_time":"2021-03-12T06:33:14.88117","exception":false,"start_time":"2021-03-12T06:33:14.87126","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\"\"\"\nLovasz-Softmax and Jaccard hinge loss in PyTorch\nMaxim Berman 2018 ESAT-PSI KU Leuven (MIT License)\n\"\"\"\n\nimport torch\nfrom torch.autograd import Variable\nimport torch.nn.functional as F\nimport numpy as np\ntry:\n from itertools import ifilterfalse\nexcept ImportError: # py3k\n from itertools import filterfalse\n\n\ndef lovasz_grad(gt_sorted):\n \"\"\"\n Computes gradient of the Lovasz extension w.r.t sorted errors\n See Alg. 1 in paper\n \"\"\"\n p = len(gt_sorted)\n gts = gt_sorted.sum()\n intersection = gts - gt_sorted.float().cumsum(0)\n union = gts + (1 - gt_sorted).float().cumsum(0)\n jaccard = 1. - intersection / union\n if p > 1: # cover 1-pixel case\n jaccard[1:p] = jaccard[1:p] - jaccard[0:-1]\n return jaccard\n\n\ndef iou_binary(preds, labels, EMPTY=1., ignore=None, per_image=True):\n \"\"\"\n IoU for foreground class\n binary: 1 foreground, 0 background\n \"\"\"\n if not per_image:\n preds, labels = (preds,), (labels,)\n ious = []\n for pred, label in zip(preds, labels):\n intersection = ((label == 1) & (pred == 1)).sum()\n union = ((label == 1) | ((pred == 1) & (label != ignore))).sum()\n if not union:\n iou = EMPTY\n else:\n iou = float(intersection) / union\n ious.append(iou)\n iou = f_mean(ious) # mean accross images if per_image\n return 100 * iou\n\n\ndef iou(preds, labels, C, EMPTY=1., ignore=None, per_image=False):\n \"\"\"\n Array of IoU for each (non ignored) class\n \"\"\"\n if not per_image:\n preds, labels = (preds,), (labels,)\n ious = []\n for pred, label in zip(preds, labels):\n iou = [] \n for i in range(C):\n if i != ignore: # The ignored label is sometimes among predicted classes (ENet - CityScapes)\n intersection = ((label == i) & (pred == i)).sum()\n union = ((label == i) | ((pred == i) & (label != ignore))).sum()\n if not union:\n iou.append(EMPTY)\n else:\n iou.append(float(intersection) / union)\n ious.append(iou)\n ious = map(f_mean, zip(*ious)) # mean accross images if per_image\n return 100 * np.array(ious)\n\n\n# --------------------------- BINARY LOSSES ---------------------------\n\n\ndef lovasz_hinge(logits, labels, per_image=True, ignore=None):\n \"\"\"\n Binary Lovasz hinge loss\n logits: [B, H, W] Variable, logits at each pixel (between -\\infty and +\\infty)\n labels: [B, H, W] Tensor, binary ground truth masks (0 or 1)\n per_image: compute the loss per image instead of per batch\n ignore: void class id\n \"\"\"\n if per_image:\n loss = f_mean(lovasz_hinge_flat(*flatten_binary_scores(log.unsqueeze(0), lab.unsqueeze(0), ignore))\n for log, lab in zip(logits, labels))\n else:\n loss = lovasz_hinge_flat(*flatten_binary_scores(logits, labels, ignore))\n return loss\n\n\ndef lovasz_hinge_flat(logits, labels):\n \"\"\"\n Binary Lovasz hinge loss\n logits: [P] Variable, logits at each prediction (between -\\infty and +\\infty)\n labels: [P] Tensor, binary ground truth labels (0 or 1)\n ignore: label to ignore\n \"\"\"\n if len(labels) == 0:\n # only void pixels, the gradients should be 0\n return logits.sum() * 0.\n signs = 2. * labels.float() - 1.\n errors = (1. - logits * Variable(signs))\n errors_sorted, perm = torch.sort(errors, dim=0, descending=True)\n perm = perm.data\n gt_sorted = labels[perm]\n grad = lovasz_grad(gt_sorted)\n #loss = torch.dot(F.relu(errors_sorted), Variable(grad))\n loss = torch.dot(F.elu(errors_sorted)+1, Variable(grad))\n return loss\n\n\ndef flatten_binary_scores(scores, labels, ignore=None):\n \"\"\"\n Flattens predictions in the batch (binary case)\n Remove labels equal to 'ignore'\n \"\"\"\n scores = scores.view(-1)\n labels = labels.view(-1)\n if ignore is None:\n return scores, labels\n valid = (labels != ignore)\n vscores = scores[valid]\n vlabels = labels[valid]\n return vscores, vlabels\n\n\nclass StableBCELoss(torch.nn.modules.Module):\n def __init__(self):\n super(StableBCELoss, self).__init__()\n def forward(self, input, target):\n neg_abs = - input.abs()\n loss = input.clamp(min=0) - input * target + (1 + neg_abs.exp()).log()\n return loss.mean()\n\n\ndef binary_xloss(logits, labels, ignore=None):\n \"\"\"\n Binary Cross entropy loss\n logits: [B, H, W] Variable, logits at each pixel (between -\\infty and +\\infty)\n labels: [B, H, W] Tensor, binary ground truth masks (0 or 1)\n ignore: void class id\n \"\"\"\n logits, labels = flatten_binary_scores(logits, labels, ignore)\n loss = StableBCELoss()(logits, Variable(labels.float()))\n return loss\n\n\n# --------------------------- MULTICLASS LOSSES ---------------------------\n\n\ndef lovasz_softmax(probas, labels, only_present=False, per_image=False, ignore=None):\n \"\"\"\n Multi-class Lovasz-Softmax loss\n probas: [B, C, H, W] Variable, class probabilities at each prediction (between 0 and 1)\n labels: [B, H, W] Tensor, ground truth labels (between 0 and C - 1)\n only_present: average only on classes present in ground truth\n per_image: compute the loss per image instead of per batch\n ignore: void class labels\n \"\"\"\n if per_image:\n loss = f_mean(lovasz_softmax_flat(*flatten_probas(prob.unsqueeze(0), lab.unsqueeze(0), ignore), only_present=only_present)\n for prob, lab in zip(probas, labels))\n else:\n loss = lovasz_softmax_flat(*flatten_probas(probas, labels, ignore), only_present=only_present)\n return loss\n\n\ndef lovasz_softmax_flat(probas, labels, only_present=False):\n \"\"\"\n Multi-class Lovasz-Softmax loss\n probas: [P, C] Variable, class probabilities at each prediction (between 0 and 1)\n labels: [P] Tensor, ground truth labels (between 0 and C - 1)\n only_present: average only on classes present in ground truth\n \"\"\"\n C = probas.size(1)\n losses = []\n for c in range(C):\n fg = (labels == c).float() # foreground for class c\n if only_present and fg.sum() == 0:\n continue\n errors = (Variable(fg) - probas[:, c]).abs()\n errors_sorted, perm = torch.sort(errors, 0, descending=True)\n perm = perm.data\n fg_sorted = fg[perm]\n losses.append(torch.dot(errors_sorted, Variable(lovasz_grad(fg_sorted))))\n return f_mean(losses)\n\n\ndef flatten_probas(probas, labels, ignore=None):\n \"\"\"\n Flattens predictions in the batch\n \"\"\"\n B, C, H, W = probas.size()\n probas = probas.permute(0, 2, 3, 1).contiguous().view(-1, C) # B * H * W, C = P, C\n labels = labels.view(-1)\n if ignore is None:\n return probas, labels\n valid = (labels != ignore)\n vprobas = probas[valid.nonzero().squeeze()]\n vlabels = labels[valid]\n return vprobas, vlabels\n\ndef xloss(logits, labels, ignore=None):\n \"\"\"\n Cross entropy loss\n \"\"\"\n return F.cross_entropy(logits, Variable(labels), ignore_index=255)\n\n\n# --------------------------- HELPER FUNCTIONS ---------------------------\n\ndef f_mean(l, ignore_nan=False, empty=0):\n \"\"\"\n nanmean compatible with generators.\n \"\"\"\n l = iter(l)\n if ignore_nan:\n l = ifilterfalse(np.isnan, l)\n try:\n n = 1\n acc = next(l)\n except StopIteration:\n if empty == 'raise':\n raise ValueError('Empty mean')\n return empty\n for n, v in enumerate(l, 2):\n acc += v\n if n == 1:\n return acc\n return acc / n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-17T08:53:18.765433Z","iopub.execute_input":"2022-07-17T08:53:18.766391Z","iopub.status.idle":"2022-07-17T08:53:21.031119Z","shell.execute_reply.started":"2022-07-17T08:53:18.766272Z","shell.execute_reply":"2022-07-17T08:53:21.029845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport gc\nimport cv2\nimport rasterio\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nfrom tqdm import tqdm\nimport tifffile as tiff\nimport matplotlib.pyplot as plt\nfrom fastai.vision.all import *\nfrom rasterio.windows import Window\nfrom torch.utils.data import Dataset, DataLoader\nimport warnings; warnings.filterwarnings(\"ignore\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":3.066435,"end_time":"2021-03-12T06:33:17.956368","exception":false,"start_time":"2021-03-12T06:33:14.889933","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:21.033017Z","iopub.execute_input":"2022-07-17T08:53:21.033798Z","iopub.status.idle":"2022-07-17T08:53:23.5438Z","shell.execute_reply.started":"2022-07-17T08:53:21.033754Z","shell.execute_reply":"2022-07-17T08:53:23.542575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bs = 64\nsz = 256 # the size of tiles\nreduce = 4 # reduce the original images by 4 times\nTH = 0.225 # threshold for positive predictions\nDATA = '../input/hubmap-organ-segmentation/test_images/'\nMODELS = [f'../input/hubmap-models/hubmap_models/run_0/model_{i}.pth' for i in range(4)] + [f'../input/hubmap-models/hubmap_models/run_1/model_{i}.pth' for i in range(4)] + [f'../input/training-fastai-baseline/model_{i}.pth' for i in range(4)]\ndf_sample = pd.read_csv('../input/hubmap-organ-segmentation/sample_submission.csv')\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')","metadata":{"papermill":{"duration":0.024698,"end_time":"2021-03-12T06:33:17.991398","exception":false,"start_time":"2021-03-12T06:33:17.9667","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.549964Z","iopub.execute_input":"2022-07-17T08:53:23.553107Z","iopub.status.idle":"2022-07-17T08:53:23.580153Z","shell.execute_reply.started":"2022-07-17T08:53:23.55306Z","shell.execute_reply":"2022-07-17T08:53:23.578594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data","metadata":{"papermill":{"duration":0.008314,"end_time":"2021-03-12T06:33:18.008555","exception":false,"start_time":"2021-03-12T06:33:18.000241","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# functions to convert encoding to mask and mask to encoding\ndef enc2mask(encs, shape):\n img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n for m,enc in enumerate(encs):\n if isinstance(enc,np.float) and np.isnan(enc): continue\n s = enc.split()\n for i in range(len(s)//2):\n start = int(s[2*i]) - 1\n length = int(s[2*i+1])\n img[start:start+length] = 1 + m\n return img.reshape(shape).T\n\ndef mask2enc(mask, n=1):\n pixels = mask.T.flatten()\n encs = []\n for i in range(1,n+1):\n p = (pixels == i).astype(np.int8)\n if p.sum() == 0: encs.append(np.nan)\n else:\n p = np.concatenate([[0], p, [0]])\n runs = np.where(p[1:] != p[:-1])[0] + 1\n runs[1::2] -= runs[::2]\n encs.append(' '.join(str(x) for x in runs))\n return encs\n\n#https://www.kaggle.com/bguberfain/memory-aware-rle-encoding\n#with transposed mask\ndef rle_encode_less_memory(img):\n #the image should be transposed\n pixels = img.T.flatten()\n \n # This simplified method requires first and last pixel to be zero\n pixels[0] = 0\n pixels[-1] = 0\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 2\n runs[1::2] -= runs[::2]\n \n return ' '.join(str(x) for x in runs)","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.026676,"end_time":"2021-03-12T06:33:18.043775","exception":false,"start_time":"2021-03-12T06:33:18.017099","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.594133Z","iopub.execute_input":"2022-07-17T08:53:23.59461Z","iopub.status.idle":"2022-07-17T08:53:23.630255Z","shell.execute_reply.started":"2022-07-17T08:53:23.594563Z","shell.execute_reply":"2022-07-17T08:53:23.628458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# https://www.kaggle.com/datasets/thedevastator/hubmap-2022-256x256\nmean = np.array([0.7720342, 0.74582646, 0.76392896])\nstd = np.array([0.24745085, 0.26182273, 0.25782376])\n\ns_th = 40 #saturation blancking threshold\np_th = 1000*(sz//256)**2 #threshold for the minimum number of pixels\nidentity = rasterio.Affine(1, 0, 0, 0, 1, 0)\n\ndef img2tensor(img,dtype:np.dtype=np.float32):\n if img.ndim==2 : img = np.expand_dims(img,2)\n img = np.transpose(img,(2,0,1))\n return torch.from_numpy(img.astype(dtype, copy=False))\n\nclass HuBMAPDataset(Dataset):\n def __init__(self, idx, sz=sz, reduce=reduce):\n self.data = rasterio.open(os.path.join(DATA,idx+'.tiff'), transform = identity,\n num_threads='all_cpus')\n # some images have issues with their format \n # and must be saved correctly before reading with rasterio\n if self.data.count != 3:\n subdatasets = self.data.subdatasets\n self.layers = []\n if len(subdatasets) > 0:\n for i, subdataset in enumerate(subdatasets, 0):\n self.layers.append(rasterio.open(subdataset))\n self.shape = self.data.shape\n self.reduce = reduce\n self.sz = reduce*sz\n self.pad0 = (self.sz - self.shape[0]%self.sz)%self.sz\n self.pad1 = (self.sz - self.shape[1]%self.sz)%self.sz\n self.n0max = (self.shape[0] + self.pad0)//self.sz\n self.n1max = (self.shape[1] + self.pad1)//self.sz\n \n def __len__(self):\n return self.n0max*self.n1max\n \n def __getitem__(self, idx):\n # the code below may be a little bit difficult to understand,\n # but the thing it does is mapping the original image to\n # tiles created with adding padding, as done in\n # https://www.kaggle.com/iafoss/256x256-images ,\n # and then the tiles are loaded with rasterio\n # n0,n1 - are the x and y index of the tile (idx = n0*self.n1max + n1)\n n0,n1 = idx//self.n1max, idx%self.n1max\n # x0,y0 - are the coordinates of the lower left corner of the tile in the image\n # negative numbers correspond to padding (which must not be loaded)\n x0,y0 = -self.pad0//2 + n0*self.sz, -self.pad1//2 + n1*self.sz\n # make sure that the region to read is within the image\n p00,p01 = max(0,x0), min(x0+self.sz,self.shape[0])\n p10,p11 = max(0,y0), min(y0+self.sz,self.shape[1])\n img = np.zeros((self.sz,self.sz,3),np.uint8)\n # mapping the loade region to the tile\n if self.data.count == 3:\n img[(p00-x0):(p01-x0),(p10-y0):(p11-y0)] = np.moveaxis(self.data.read([1,2,3],\n window=Window.from_slices((p00,p01),(p10,p11))), 0, -1)\n else:\n for i,layer in enumerate(self.layers):\n img[(p00-x0):(p01-x0),(p10-y0):(p11-y0),i] =\\\n layer.read(1,window=Window.from_slices((p00,p01),(p10,p11)))\n \n if self.reduce != 1:\n img = cv2.resize(img,(self.sz//reduce,self.sz//reduce),\n interpolation = cv2.INTER_AREA)\n #check for empty imges\n hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n h,s,v = cv2.split(hsv)\n if (s>s_th).sum() <= p_th or img.sum() <= p_th:\n #images with -1 will be skipped\n return img2tensor((img/255.0 - mean)/std), -1\n else: return img2tensor((img/255.0 - mean)/std), idx","metadata":{"papermill":{"duration":0.037945,"end_time":"2021-03-12T06:33:18.090462","exception":false,"start_time":"2021-03-12T06:33:18.052517","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.635916Z","iopub.execute_input":"2022-07-17T08:53:23.636328Z","iopub.status.idle":"2022-07-17T08:53:23.692796Z","shell.execute_reply.started":"2022-07-17T08:53:23.636287Z","shell.execute_reply":"2022-07-17T08:53:23.689806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#iterator like wrapper that returns predicted masks\nclass Model_pred:\n def __init__(self, models, dl, tta:bool=True, half:bool=False):\n self.models = models\n self.dl = dl\n self.tta = tta\n self.half = half\n \n def __iter__(self):\n count=0\n with torch.no_grad():\n for x,y in iter(self.dl):\n if ((y>=0).sum() > 0): #exclude empty images\n x = x[y>=0].to(device)\n y = y[y>=0]\n if self.half: x = x.half()\n py = None\n for model in self.models:\n p = model(x)\n p = torch.sigmoid(p).detach()\n if py is None: py = p\n else: py += p\n if self.tta:\n #x,y,xy flips as TTA\n flips = [[-1],[-2],[-2,-1]]\n for f in flips:\n xf = torch.flip(x,f)\n for model in self.models:\n p = model(xf)\n p = torch.flip(p,f)\n py += torch.sigmoid(p).detach()\n py /= (1+len(flips)) \n py /= len(self.models)\n\n py = F.upsample(py, scale_factor=reduce, mode=\"bilinear\")\n py = py.permute(0,2,3,1).float().cpu()\n \n batch_size = len(py)\n for i in range(batch_size):\n yield py[i],y[i]\n count += 1\n \n def __len__(self):\n return len(self.dl.dataset)","metadata":{"papermill":{"duration":0.030274,"end_time":"2021-03-12T06:33:18.129546","exception":false,"start_time":"2021-03-12T06:33:18.099272","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.69728Z","iopub.execute_input":"2022-07-17T08:53:23.700957Z","iopub.status.idle":"2022-07-17T08:53:23.74247Z","shell.execute_reply.started":"2022-07-17T08:53:23.700921Z","shell.execute_reply":"2022-07-17T08:53:23.739888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{"papermill":{"duration":0.008902,"end_time":"2021-03-12T06:33:18.153045","exception":false,"start_time":"2021-03-12T06:33:18.144143","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class FPN(nn.Module):\n def __init__(self, input_channels:list, output_channels:list):\n super().__init__()\n self.convs = nn.ModuleList(\n [nn.Sequential(nn.Conv2d(in_ch, out_ch*2, kernel_size=3, padding=1),\n nn.ReLU(inplace=True), nn.BatchNorm2d(out_ch*2),\n nn.Conv2d(out_ch*2, out_ch, kernel_size=3, padding=1))\n for in_ch, out_ch in zip(input_channels, output_channels)])\n \n def forward(self, xs:list, last_layer):\n hcs = [F.interpolate(c(x),scale_factor=2**(len(self.convs)-i),mode='bilinear') \n for i,(c,x) in enumerate(zip(self.convs, xs))]\n hcs.append(last_layer)\n return torch.cat(hcs, dim=1)\n\nclass UnetBlock(Module):\n def __init__(self, up_in_c:int, x_in_c:int, nf:int=None, blur:bool=False,\n self_attention:bool=False, **kwargs):\n super().__init__()\n self.shuf = PixelShuffle_ICNR(up_in_c, up_in_c//2, blur=blur, **kwargs)\n self.bn = nn.BatchNorm2d(x_in_c)\n ni = up_in_c//2 + x_in_c\n nf = nf if nf is not None else max(up_in_c//2,32)\n self.conv1 = ConvLayer(ni, nf, norm_type=None, **kwargs)\n self.conv2 = ConvLayer(nf, nf, norm_type=None,\n xtra=SelfAttention(nf) if self_attention else None, **kwargs)\n self.relu = nn.ReLU(inplace=True)\n\n def forward(self, up_in:Tensor, left_in:Tensor) -> Tensor:\n s = left_in\n up_out = self.shuf(up_in)\n cat_x = self.relu(torch.cat([up_out, self.bn(s)], dim=1))\n return self.conv2(self.conv1(cat_x))\n \nclass _ASPPModule(nn.Module):\n def __init__(self, inplanes, planes, kernel_size, padding, dilation, groups=1):\n super().__init__()\n self.atrous_conv = nn.Conv2d(inplanes, planes, kernel_size=kernel_size,\n stride=1, padding=padding, dilation=dilation, bias=False, groups=groups)\n self.bn = nn.BatchNorm2d(planes)\n self.relu = nn.ReLU()\n\n self._init_weight()\n\n def forward(self, x):\n x = self.atrous_conv(x)\n x = self.bn(x)\n\n return self.relu(x)\n\n def _init_weight(self):\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n torch.nn.init.kaiming_normal_(m.weight)\n elif isinstance(m, nn.BatchNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_()\n\nclass ASPP(nn.Module):\n def __init__(self, inplanes=512, mid_c=256, dilations=[6, 12, 18, 24], out_c=None):\n super().__init__()\n self.aspps = [_ASPPModule(inplanes, mid_c, 1, padding=0, dilation=1)] + \\\n [_ASPPModule(inplanes, mid_c, 3, padding=d, dilation=d,groups=4) for d in dilations]\n self.aspps = nn.ModuleList(self.aspps)\n self.global_pool = nn.Sequential(nn.AdaptiveMaxPool2d((1, 1)),\n nn.Conv2d(inplanes, mid_c, 1, stride=1, bias=False),\n nn.BatchNorm2d(mid_c), nn.ReLU())\n out_c = out_c if out_c is not None else mid_c\n self.out_conv = nn.Sequential(nn.Conv2d(mid_c*(2+len(dilations)), out_c, 1, bias=False),\n nn.BatchNorm2d(out_c), nn.ReLU(inplace=True))\n self.conv1 = nn.Conv2d(mid_c*(2+len(dilations)), out_c, 1, bias=False)\n self._init_weight()\n\n def forward(self, x):\n x0 = self.global_pool(x)\n xs = [aspp(x) for aspp in self.aspps]\n x0 = F.interpolate(x0, size=xs[0].size()[2:], mode='bilinear', align_corners=True)\n x = torch.cat([x0] + xs, dim=1)\n return self.out_conv(x)\n \n def _init_weight(self):\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n torch.nn.init.kaiming_normal_(m.weight)\n elif isinstance(m, nn.BatchNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_()","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.04647,"end_time":"2021-03-12T06:33:18.208385","exception":false,"start_time":"2021-03-12T06:33:18.161915","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.748981Z","iopub.execute_input":"2022-07-17T08:53:23.751771Z","iopub.status.idle":"2022-07-17T08:53:23.79563Z","shell.execute_reply.started":"2022-07-17T08:53:23.751728Z","shell.execute_reply":"2022-07-17T08:53:23.793916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from torchvision.models.resnet import ResNet, Bottleneck\nclass UneXt50(nn.Module):\n def __init__(self, stride=1, **kwargs):\n super().__init__()\n #encoder\n m = ResNet(Bottleneck, [3, 4, 6, 3], groups=32, width_per_group=4)\n #m = torch.hub.load('facebookresearch/semi-supervised-ImageNet1K-models',\n # 'resnext50_32x4d_ssl')\n self.enc0 = nn.Sequential(m.conv1, m.bn1, nn.ReLU(inplace=True))\n self.enc1 = nn.Sequential(nn.MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1),\n m.layer1) #256\n self.enc2 = m.layer2 #512\n self.enc3 = m.layer3 #1024\n self.enc4 = m.layer4 #2048\n #aspp with customized dilatations\n self.aspp = ASPP(2048,256,out_c=512,dilations=[stride*1,stride*2,stride*3,stride*4])\n self.drop_aspp = nn.Dropout2d(0.5)\n #decoder\n self.dec4 = UnetBlock(512,1024,256)\n self.dec3 = UnetBlock(256,512,128)\n self.dec2 = UnetBlock(128,256,64)\n self.dec1 = UnetBlock(64,64,32)\n self.fpn = FPN([512,256,128,64],[16]*4)\n self.drop = nn.Dropout2d(0.1)\n self.final_conv = ConvLayer(32+16*4, 1, ks=1, norm_type=None, act_cls=None)\n \n def forward(self, x):\n enc0 = self.enc0(x)\n enc1 = self.enc1(enc0)\n enc2 = self.enc2(enc1)\n enc3 = self.enc3(enc2)\n enc4 = self.enc4(enc3)\n enc5 = self.aspp(enc4)\n dec3 = self.dec4(self.drop_aspp(enc5),enc3)\n dec2 = self.dec3(dec3,enc2)\n dec1 = self.dec2(dec2,enc1)\n dec0 = self.dec1(dec1,enc0)\n x = self.fpn([enc5, dec3, dec2, dec1], dec0)\n x = self.final_conv(self.drop(x))\n x = F.interpolate(x,scale_factor=2,mode='bilinear')\n return x","metadata":{"papermill":{"duration":0.028491,"end_time":"2021-03-12T06:33:18.245935","exception":false,"start_time":"2021-03-12T06:33:18.217444","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.797364Z","iopub.execute_input":"2022-07-17T08:53:23.79978Z","iopub.status.idle":"2022-07-17T08:53:23.82531Z","shell.execute_reply.started":"2022-07-17T08:53:23.799736Z","shell.execute_reply":"2022-07-17T08:53:23.824192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"models = []\nfor path in MODELS:\n state_dict = torch.load(path,map_location=torch.device('cpu'))\n model = UneXt50()\n model.load_state_dict(state_dict)\n model.float()\n model.eval()\n model.to(device)\n models.append(model)\n\ndel state_dict","metadata":{"papermill":{"duration":13.838863,"end_time":"2021-03-12T06:33:32.093854","exception":false,"start_time":"2021-03-12T06:33:18.254991","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:23.828642Z","iopub.execute_input":"2022-07-17T08:53:23.8297Z","iopub.status.idle":"2022-07-17T08:53:24.254612Z","shell.execute_reply.started":"2022-07-17T08:53:23.829657Z","shell.execute_reply":"2022-07-17T08:53:24.252874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction","metadata":{"papermill":{"duration":0.009141,"end_time":"2021-03-12T06:33:32.112738","exception":false,"start_time":"2021-03-12T06:33:32.103597","status":"completed"},"tags":[]}},{"cell_type":"code","source":"names,preds = [],[]\nfor idx,row in tqdm(df_sample.iterrows(),total=len(df_sample)):\n idx = str(row['id'])\n ds = HuBMAPDataset(idx)\n #rasterio cannot be used with multiple workers\n dl = DataLoader(ds,bs,num_workers=0,shuffle=False,pin_memory=True)\n mp = Model_pred(models,dl)\n #generate masks\n mask = torch.zeros(len(ds),ds.sz,ds.sz,dtype=torch.int8)\n for p,i in iter(mp): mask[i.item()] = p.squeeze(-1) > TH\n \n #reshape tiled masks into a single mask and crop padding\n mask = mask.view(ds.n0max,ds.n1max,ds.sz,ds.sz).\\\n permute(0,2,1,3).reshape(ds.n0max*ds.sz,ds.n1max*ds.sz)\n mask = mask[ds.pad0//2:-(ds.pad0-ds.pad0//2) if ds.pad0 > 0 else ds.n0max*ds.sz,\n ds.pad1//2:-(ds.pad1-ds.pad1//2) if ds.pad1 > 0 else ds.n1max*ds.sz]\n \n #convert to rle\n #https://www.kaggle.com/bguberfain/memory-aware-rle-encoding\n rle = rle_encode_less_memory(mask.numpy())\n names.append(idx)\n preds.append(rle)\n del mask, ds, dl\n gc.collect()","metadata":{"_kg_hide-output":true,"papermill":{"duration":638.710533,"end_time":"2021-03-12T06:44:10.832427","exception":false,"start_time":"2021-03-12T06:33:32.121894","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:24.255705Z","iopub.status.idle":"2022-07-17T08:53:24.256704Z","shell.execute_reply.started":"2022-07-17T08:53:24.25642Z","shell.execute_reply":"2022-07-17T08:53:24.256446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame({'id':names,'rle':preds})\ndf.to_csv('submission.csv',index=False)","metadata":{"papermill":{"duration":0.419953,"end_time":"2021-03-12T06:44:11.262501","exception":false,"start_time":"2021-03-12T06:44:10.842548","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:24.25791Z","iopub.status.idle":"2022-07-17T08:53:24.258799Z","shell.execute_reply.started":"2022-07-17T08:53:24.258526Z","shell.execute_reply":"2022-07-17T08:53:24.258551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df","metadata":{"papermill":{"duration":0.010126,"end_time":"2021-03-12T06:44:11.283196","exception":false,"start_time":"2021-03-12T06:44:11.27307","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-07-17T08:53:24.260206Z","iopub.status.idle":"2022-07-17T08:53:24.261222Z","shell.execute_reply.started":"2022-07-17T08:53:24.260958Z","shell.execute_reply":"2022-07-17T08:53:24.260984Z"},"trusted":true},"execution_count":null,"outputs":[]}]}