{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# HuBMAP - Efficient Sampling Baseline (deepflash2, pytorch, fastai) [train]\n\n> Kernel for model training with efficient region based sampling.\n\nRequires deepflash2 (git version), zarr, and segmentation-models-pytorch\n","metadata":{"id":"OcsetTMwKXqC"}},{"cell_type":"markdown","source":"## Overview\n\n1. Installation and package loading\n2. Helper functions and patches\n3. Configuration\n4. Training\n\n### Inputs\n- https://www.kaggle.com/matjes/hubmap-zarr converted images (downscaled with factor 2)\n- https://www.kaggle.com/matjes/hubmap-labels-pdf-0-5-0-25-0-01 masks and weights for sampling\n\n### Versions\n- V7: Fixed augmentations in deepflash2 `RandomTileDataset` config (random zoom) - LB 0.913\n- V8: Adding *albumentations* transforms, switching to Cross-entropy loss","metadata":{}},{"cell_type":"markdown","source":"## Motivation\n\n### Background\n\nA glomerulus is a network of small blood vessels located at the beginning of a nephron in the kidney ([Wikipedia](https://en.wikipedia.org/wiki/Glomerulus_(kidney))\n)). Glomeruli are mainly found in the renal **cortex**, while the renal **medulla** contains mainly the renal tubule. Since we are dealing with biological structures, the separation is not not absolute and the transitions are not always perfectly sharp.\n\n![Diagram of a nephron](http://s3-us-west-2.amazonaws.com/courses-images/wp-content/uploads/sites/1842/2017/05/26234530/m9skcbftjqzrokkkopam.png)\n[Diagram of a nephron from libretexts.org, Introductory and General Biology](https://bio.libretexts.org/Bookshelves/Introductory_and_General_Biology/Book%3A_General_Biology_(Boundless)/41%3A_Osmotic_Regulation_and_the_Excretory_System/41.4%3A_Human_Osmoregulatory_and_Excretory_Systems/41.4B%3A_Nephron%3A_The_Functional_Unit_of_the_Kidney)\n\n### Key Idea\n\nA common approach to deal with the very large (>500MB - 5GB) TIFF files in the dataset is to decompose the images in smaller patches/tiles, for instance by using a sliding window apporach.\n> **Knowing that the glomeruli are mainly found in the cortex, we should focus on this region during training**. \n\nInstead of preprocessing the images and saving them into fixed tiles, we sample tiles from the entire images with a higher probability on tiles that contain glumeroli and cortex. Have a look at [this kernel](https://www.kaggle.com/matjes/hubmap-labels-pdf-0-5-0-25-0-01) for more details.\n\n\n## Advantages of this approach\n\nIn combination with [deepflash2](https://github.com/matjesg/deepflash2/tree/master/) and the deepflash2 [pytorch datasets](https://matjesg.github.io/deepflash2/data.html#Datasets) in particular, this approach has several advantages:\n- no preprocessing of the data (only saving them to .zarr files for memory efficient loading)\n    - flexible tile shapes (input shapes, e.g. 1024, 512, 256) at runtime\n    - flexible scaling (e.g., by facors of 2,3,4)\n- faster convergence during traing (~30 min for training a competitive model)\n    - focusing on the relevant regions (e.g., tiles that contain glumeroli and cortex)\n    - \"additional\" data augmentation from random sampling (compared to fixed windows)","metadata":{}},{"cell_type":"markdown","source":"### Installation and package loading","metadata":{}},{"cell_type":"code","source":"!pip install -q git+https://github.com/matjesg/deepflash2.git\n!pip install segmentation_models_pytorch","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# imports\nimport zarr, cv2\nimport numpy as np, pandas as pd, segmentation_models_pytorch as smp\nfrom fastai.vision.all import *\nfrom deepflash2.all import *\nimport albumentations as alb","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Helper functions and patches","metadata":{}},{"cell_type":"markdown","source":"![](http://)Patches for deepflash2 classes, see https://fastcore.fast.ai/basics.html#patch","metadata":{}},{"cell_type":"code","source":"BN = 'b1' # b1, b2, b3, b4\nSAMP = '050025' # 05025001\nRANDOM = '' \n\nPSEUDO = ''\n\nPSEUDO_PATH = ''\n\n\nEPOCHS = 20 # raw12 / 20 / 25 / 30 / 35\n#################\n# BATCH_SIZE = 28 # B1-28  ，B2-24  ， B3-24 , Raw B4-16\nif BN=='b1':\n    BATCH_SIZE = 28\nelif BN=='b2':\n    BATCH_SIZE = 24\nelif BN=='b3':\n    BATCH_SIZE = 24\nelse:\n    BATCH_SIZE = 16\n    \nVERSION = f'{PSEUDO}-{SAMP}-{RANDOM}' # deepflash\n\nprint(VERSION)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"@patch\ndef read_img(self:BaseDataset, file, *args, **kwargs):\n    return zarr.open(str(file), mode='r')\n\n@patch\ndef _name_fn(self:BaseDataset, g):\n    \"Name of preprocessed and compressed data.\"\n    return f'{g}'\n\n@patch\ndef apply(self:DeformationField, data, offset=(0, 0), pad=(0, 0), order=1):\n    \"Apply deformation field to image using interpolation\"\n    outshape = tuple(int(s - p) for (s, p) in zip(self.shape, pad))\n    coords = [np.squeeze(d).astype('float32').reshape(*outshape) for d in self.get(offset, pad)]\n    # Get slices to avoid loading all data (.zarr files)\n    sl = []\n    for i in range(len(coords)):\n        cmin, cmax = int(coords[i].min()), int(coords[i].max())\n        dmax = data.shape[i]\n        if cmin<0: \n            cmax = max(-cmin, cmax)\n            cmin = 0 \n        elif cmax>dmax:\n            cmin = min(cmin, 2*dmax-cmax)\n            cmax = dmax\n            coords[i] -= cmin\n        else: coords[i] -= cmin\n        sl.append(slice(cmin, cmax))    \n    if len(data.shape) == len(self.shape) + 1:\n        tile = np.empty((*outshape, data.shape[-1]))\n        for c in range(data.shape[-1]):\n            # Adding divide\n            tile[..., c] = cv2.remap(data[sl[0],sl[1], c]/255, coords[1],coords[0], interpolation=order, borderMode=cv2.BORDER_REFLECT)\n    else:\n        tile = cv2.remap(data[sl[0], sl[1]], coords[1], coords[0], interpolation=order, borderMode=cv2.BORDER_REFLECT)\n    return tile","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Config","metadata":{}},{"cell_type":"code","source":"class CONFIG():\n    \n    # data paths\n    data_path = Path('../input/hubmap-kidney-segmentation')\n    data_path_zarr = Path('../input/hubmap-zarr/images_scale2') \n    mask_preproc_dir = '../input/hubmap-labels-pdf-0-5-0-25-0-01'\n    \n    mask_preproc_dir += '/masks_scale2'\n    # deepflash2 dataset\n    scale = 1.5 # data is already downscaled to 2, so absulute downscale is 3\n    tile_shape = (512, 512)\n    padding = (0,0) # Border overlap for prediction\n    n_jobs = 1\n    sample_mult = 100 # Sample 100 tiles from each image, per epoch\n    val_length = 500 # Randomly sample 500 validation tiles\n    stats = np.array([0.61561477, 0.5179343 , 0.64067212]), np.array([0.2915353 , 0.31549066, 0.28647661])\n    \n    # deepflash2 augmentation options\n    zoom_sigma = 0.1\n    flip = True\n    max_rotation = 360\n    deformation_grid_size = (150,150)\n    deformation_magnitude = (10,10)\n    \n\n    # pytorch model (segmentation_models_pytorch)\n    encoder_name = f\"efficientnet-{BN}\"\n    encoder_weights = 'imagenet'\n    in_channels = 3\n    classes = 2\n    \n    # fastai Learner \n    mixed_precision_training = True\n    batch_size = BATCH_SIZE\n    weight_decay = 0.01\n    loss_func = CrossEntropyLossFlat(axis=1)\n    metrics = [Iou(), Dice_f1()]\n    optimizer = ranger\n    max_learning_rate = 1e-3\n    epochs = EPOCHS\n    \ncfg = CONFIG()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Albumentations augmentations\n# Inspired by https://www.kaggle.com/iafoss/hubmap-pytorch-fast-ai-starter\n# deepflash2 augmentations are only affine transformations\ntfms = alb.OneOf([\n    alb.HueSaturationValue(10,15,10),\n    alb.CLAHE(clip_limit=2),\n    alb.RandomBrightnessContrast(),            \n    ], p=0.3)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = pd.read_csv(cfg.data_path/'train.csv')\n\ndf_test = pd.read_csv(PSEUDO_PATH)\ndf_test.columns = df_train.columns\ndf_train = pd.concat([df_train, df_test]).reset_index(drop=True)\n\n\ndf_info = pd.read_csv(cfg.data_path/'HuBMAP-20-dataset_information.csv')\n\nfiles = [x for x in cfg.data_path_zarr.iterdir() if x.is_dir() if not x.name.startswith('.')]\nlabel_fn = lambda o: o","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training","metadata":{}},{"cell_type":"code","source":"# Model\nmodel = smp.Unet(encoder_name=cfg.encoder_name, \n                 encoder_weights=cfg.encoder_weights, \n                 in_channels=cfg.in_channels, \n                 classes=cfg.classes)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Datasets\nds_kwargs = {\n    'tile_shape':cfg.tile_shape,\n    'padding':cfg.padding,\n    'scale': cfg.scale,\n    'n_jobs': cfg.n_jobs, \n    'preproc_dir': cfg.mask_preproc_dir, \n    'val_length':cfg.val_length, \n    'sample_mult':cfg.sample_mult,\n    'loss_weights':False,\n    'zoom_sigma': cfg.zoom_sigma,\n    'flip' : cfg.flip,\n    'max_rotation': cfg.max_rotation,\n    'deformation_grid_size' : cfg.deformation_grid_size,\n    'deformation_magnitude' : cfg.deformation_magnitude,\n    'albumentations_tfms': tfms\n}\n\ntrain_ds = RandomTileDataset(files, label_fn=label_fn, **ds_kwargs)\nvalid_ds = TileDataset(files, label_fn=label_fn, **ds_kwargs, is_zarr=True)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Dataloader and learner\ndls = DataLoaders.from_dsets(train_ds, valid_ds, bs=cfg.batch_size, after_batch=Normalize.from_stats(*cfg.stats))\nif torch.cuda.is_available(): dls.cuda(), model.cuda()\ncbs = [SaveModelCallback(monitor='iou'), ElasticDeformCallback]\nlearn = Learner(dls, model, metrics=cfg.metrics, wd=cfg.weight_decay, loss_func=cfg.loss_func, opt_func=ranger, cbs=cbs)\nif cfg.mixed_precision_training: learn.to_fp16()\n  ","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fit\nlearn.fit_one_cycle(cfg.epochs, lr_max=cfg.max_learning_rate)\nlearn.recorder.plot_metrics()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Save Model\nstate = {'model': learn.model.state_dict(), 'stats':cfg.stats}\ntorch.save(state, f'{VERSION}_unet_{cfg.encoder_name}.pth', pickle_protocol=2, _use_new_zipfile_serialization=False)","metadata":{},"execution_count":null,"outputs":[]}]}