{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# ※ We didn't use any Hand-labeling for this competition\n### In this notebook, we introduce our findings, and solutions to get best score for public LB (we got 2nd place at former public LB with this model).","metadata":{}},{"cell_type":"markdown","source":"# Agenda\n## ・Magic1: Head-Shot Post Processing Stage1\n## ・Head-Shot Post Processing Stage2\n## ・Magic2: Brightness-based PreProcessing\n## ・Training Information\n## ・Pseudo Labeling\n## ・Idea for advance: Rotation based Head-Shot Stage2","metadata":{}},{"cell_type":"markdown","source":"# Magic 1: Head-Shot Post Processing system\n### Whole system is like below","metadata":{}},{"cell_type":"code","source":"from PIL import Image\nim = Image.open('../input/headshotpp-diagram/hubmap_diagram0.png')\nim","metadata":{"_kg_hide-output":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Stage 1\n### In this system, at first we predict each tile in the raw image.\n### If image size==(X,Y), we predict (X//1024) * (Y//1024) tiles, and map them into 1 tiff-wise prediction by EfficientUnet-B5. We repeat & add this for 4 times. (x-shift(0,512), y-shift(0,512), total 2x2=4)\n### we can roughly find each glomeruli's place in the tiff image like below.","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/stage-1/stage_1.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Stage 2\n### Nextly, we extract center positions of each glomeruli by using pseudo-morphological transformation.\n### After that, we create a tile which center is same as each glomeruli, and make a prediction for each glomeruli.\n### This system gave us significant improvement (+0.005〜) at last public LB(before updated)","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/stage2/stage2.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### By adding Stage2, we can fix weird prediction caused by random tile place which doesn't depend on glomeruli's place.","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/pseudo-c/pseudo_comparison.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Magic 2: Brightness-based PreProcessing\n### Below image shows this preprocessing","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/bright-01/bright_01.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### When we checked model predictions, we found there were 2 types of area that are really hard to predict, dark / bright place like below.","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/bright-dark/bd_00.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Then we've created classifiers that judges whether tile is too bright/dark or not, based on mean/std for r, g, b of tile and made bright tile darker, dark tile brighter.\n### Those CLS predictions are like below (black place are predicted to be too bright/dark).\n### This preprocessing made us huge jump (+0.015〜) at last public LB","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/bd-cls/bd_cls.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### and below are predictions of before/after this preprocessing","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/prep-ba/prep_before_after.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### And below is the inference code(Brightness Preprocess + Headshot Post Process)","metadata":{}},{"cell_type":"code","source":"'''\nimport glob\nimport gc\nimport rasterio\nfrom rasterio.windows import Window\nimport pathlib\n\nidentity = rasterio.Affine(1, 0, 0, 0, 1, 0)\n\np = pathlib.Path('../input/')\nsubm = {}\ntest_transform = albumentations.Compose([])\n\nfor i, filename in enumerate(p.glob('test/*.tiff')):\n \n print(f'{i+1} Predicting {filename.stem}')\n \n dataset = rasterio.open(filename.as_posix(), transform=identity)\n slices = make_grid(dataset.shape, window=1024)\n preds = np.zeros(dataset.shape, dtype=np.uint8)\n shape = dataset.shape\n \n for (x1, x2, y1, y2) in tqdm(slices):\n \n # shifted ensemble\n shift_x = [-1, 1]\n shift_y = [-1, 1]\n flags_x = [True, True]\n flags_y = [True, True]\n \n if x1//1024 == 0:\n flags_x[0] = False\n if y1//1024 == 0:\n flags_y[0] = False\n if x2//1024 == shape[0]//1024:\n flags_x[1] = False\n if y2//1024 == shape[1]//1024:\n flags_y[1] = False\n \n pred = np.zeros((1024, 1024)).astype(float)\n devide = np.ones((1024, 1024)).astype(float)\n \n raw_image = dataset.read([1, 2, 3], window=Window.from_slices((x1,x2),(y1,y2)))\n raw_image = np.moveaxis(raw_image, 0, -1)\n \n image = cv2.resize(raw_image, (512, 512), interpolation=cv2.INTER_AREA)\n image_mean = image.mean(-1)\n \n if ((image_mean==0).sum()>1000):\n continue\n \n image = image.astype(np.float32)\n \n # Dark-Bright CLS\n \n m = image.mean(0).mean(0)\n st = image.reshape(-1, 3).std(0)\n \n dark = m.mean()<100\n light = (((195 BTH).astype(np.uint8)\n if dark:\n preds[x1:x2, y1:y2] += (pred > DTH).astype(np.uint8)\n if (not light) & (not dark):\n preds[x1:x2, y1:y2] += (pred > THRESHOLD).astype(np.uint8)\n \n \n pred = np.zeros((1024, 1024)).astype(float)\n \n \n for n_, f1 in enumerate(flags_x):\n \n if f1==True:\n if n_==0:\n raw_image = dataset.read([1, 2, 3], window=Window.from_slices((x1-512, x2-512), (y1, y2)))\n if n_==1:\n raw_image = dataset.read([1, 2, 3], window=Window.from_slices((x1+512, x2+512), (y1, y2)))\n raw_image = np.moveaxis(raw_image, 0, -1)\n \n image = cv2.resize(raw_image, (512, 512), interpolation=cv2.INTER_AREA)\n image_mean = image.mean(-1)\n\n if ((image_mean==0).sum()>1000):\n continue\n\n image = image.astype(np.float32)\n \n # Dark-Bright CLS\n\n m = image.mean(0).mean(0)\n st = image.reshape(-1, 3).std(0)\n\n dark = m.mean()<100\n light = (((195 BTH).astype(np.uint8)\n if dark:\n preds[x1:x2, y1:y2] += (pred > DTH).astype(np.uint8)\n if (not light) & (not dark):\n preds[x1:x2, y1:y2] += (pred > THRESHOLD).astype(np.uint8)\n\n \n pred = np.zeros((1024, 1024)).astype(float)\n \n for n_, f1 in enumerate(flags_y):\n \n if f1==True:\n if n_==0:\n raw_image = dataset.read([1, 2, 3], window=Window.from_slices((x1, x2),(y1-512, y2-512)))\n if n_==1:\n raw_image = dataset.read([1, 2, 3], window=Window.from_slices((x1, x2),(y1+512, y2+512)))\n raw_image = np.moveaxis(raw_image, 0, -1)\n \n image = cv2.resize(raw_image, (512, 512), interpolation=cv2.INTER_AREA)\n image_mean = image.mean(-1)\n\n if ((image_mean==0).sum()>1000):\n continue\n\n image = image.astype(np.float32)\n \n # Dark-Bright CLS\n\n m = image.mean(0).mean(0)\n st = image.reshape(-1,3).std(0)\n\n dark = m.mean()<100\n light = (((195 BTH).astype(np.uint8)\n if dark:\n preds[x1:x2, y1:y2] += (pred > DTH).astype(np.uint8)\n if (not light) & (not dark):\n preds[x1:x2, y1:y2] += (pred > THRESHOLD).astype(np.uint8)\n \n \n # headshot post processing\n \n scale_factor = 8\n preds_small = cv2.resize(preds, (shape[1]//scale_factor, shape[0]//scale_factor))\n preds = np.zeros(dataset.shape, dtype=np.uint8)\n ret, preds_small= cv2.threshold((preds_small.astype(int) * 255.).astype(np.uint8), 127, 255, 0)\n contours, hierarchy = cv2.findContours(preds_small, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)\n \n centers = []\n for j, c in enumerate(contours):\n center = c.reshape(-1, 2).mean(0)\n centers.append([int(center[1]*scale_factor), int(center[0]*scale_factor)])\n \n for c0, c1 in tqdm(centers):\n \n if c0 // 2 - 256 < 0:\n c0 = 512\n if shape[0] < c0 + 512:\n c0 = shape[0] - 512\n if c1 // 2 - 256 < 0:\n c1 = 512\n if shape[1] < c1 + 512:\n c1 = shape[1] - 512\n \n x1, y1, x2, y2 = c0-512, c1-512, c0+512, c1+512\n \n pred = np.zeros((1024, 1024)).astype(float)\n pred_2 = np.zeros((1024, 1024)).astype(float)\n raw_image = dataset.read([1, 2, 3], window=Window.from_slices((x1, x2), (y1, y2)))\n raw_image = np.moveaxis(raw_image, 0, -1)\n \n image = cv2.resize(raw_image, (512, 512), interpolation=cv2.INTER_AREA)\n image_mean = image.mean(-1)\n \n if ((image_mean==0).sum()>1000):\n continue\n \n image = image.astype(np.float32)\n \n # Dark-Bright CLS\n \n m = image.mean(0).mean(0)\n st = image.reshape(-1, 3).std(0)\n \n dark = m.mean()<100\n light = (((195 BTH).astype(np.uint8)\n #pred_2 = (pred_2 > BTH).astype(np.uint8)\n if dark:\n pred = (pred > DTH).astype(np.uint8)\n #pred_2 = (pred_2 > DTH).astype(np.uint8)\n if (not light) & (not dark):\n pred = (pred > S2_TH).astype(np.uint8)\n #pred_2 = (pred_2 > S2_TH).astype(np.uint8)\n \n preds[x1+128:x2-128, y1+128:y2-128] += pred[128:-128, 128:-128]\n \n # clip duplicate\n preds = np.clip(preds, 0, 1)\n \n subm[i] = {'id':filename.stem, 'predicted': rle_encode_less_memory(preds)}\n del preds\n gc.collect()\n'''\n0","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Information","metadata":{}},{"cell_type":"markdown","source":"## Model\n### TF Efficientnet B5 Unet\n### For each model, TF was always better than pytorch (average +0.04~6)","metadata":{}},{"cell_type":"markdown","source":"## Augmentation\n### As data augmentation for training, we applied \n### ・Virtical/Horizontal flip\n### ・Make small % of image grayscale\n### ・random saturation\n### ・random contrast\n### ・brightness augmentation\n### They made us about +0.005~0.001 on LB","metadata":{}},{"cell_type":"markdown","source":"# Pseudo Labeling\n### Hard-label based pseudo labeling, which means we concatenate predicted->thresholded public test masks to train dataset and use whole data for training.\n### Instead of hand label, after we trained all fold models, we've created pseudo label for public test dataset. this pseudo label made us a little jump on public LB.","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/pseudo-00/pseudo_00.png')\nim.resize((int(321*1.5),int(261*1.5)))","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ・Idea for advance: Rotation based Head-Shot Stage2\n### We couldn't include this for last submission because of inference time, but found it effective test-time augmentation\n### for Stage2 of Head-Shot Prediction, we rotate tile 90 x N degree (N = 0, 1, 2, 3), predict them with model, and back rotate prediction as it'll be like base tile prediction.","metadata":{}},{"cell_type":"code","source":"im = Image.open('../input/rotation-00/rotation_00.png')\nim","metadata":{"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion\n## In this notebook, we proposed our magics / solutions for this competition.\n## We hope you enjoy our solution.","metadata":{}}]}