{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Tensorflow HuBMAP - Hacking the Kidney competition starter kit:\n* https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-tfrecs (how to create training and inference tfrecords)\n* https://www.kaggle.com/wrrosa/hubmap-tf-with-tpu-efficientunet-512x512-train (training pipeline)\n* this notebook (inference with submission)\n\n# Versions\n* V1 (V7 train notebook) 4-CV efficientunetb0 512x512 (**LB .834**)\n* V2 (V8 train notebook) loss bce (LB .835)\n* V3 (V9 train notebook) efficientunetb1 (CV .871, LB .830)\n* V4 (V10 train notebook) efficientunetb4 (CV .874, **LB .839**)\n* V5 (V12 train notebook) efficientunetb7 (CV .858, LB .835)\n* V6 (V13 train notebook) efficientunetb4 (CV .877, LB .836)\n* V7 (V14 train notebook) efficientunetb4 with overlapped train data, summing preds in inference (CV .879, **LB .843**)\n* V8 (V14 train notebook) efficientunetb4  THRESHOLD=0.4, interpolation = cv2.INTER_AREA, rle_encode_less_memory (**LB .846**)\n* V9 (V14 train notebook) efficientunetb4, MIN_OVERLAP = 300 (**LB 0.848**)\n* V10 (V14 train notebook) efficientunetb4, checksum mask before modifications (1h 11m, no need to score)\n* V11 (V14 train notebook) efficientunetb4, SUBMISSION_MODE added (generate submission from public tfrec files, almost 20m = 3.5 times faster!)\n* V12 (V14 train notebook) efficientunetb4, CHECKSUM = False (**LB 0.848**)\n* V13 (V15 train notebook) efficientunetb4, (**LB 0.849**)\n* V14 (V15 train notebook) efficientunetb4, switch public tfrec files path: from nb output to dataset (**LB 0.849**)\n* V15 (V15 train notebook) efficientunetb4, competition data update, model not updated  (**LB 0.903**)\n* V16 (V15 train notebook) efficientunetb4, fixed issue with new data and FULL mode, full submission took about 3hrs compared to 1hr submission on public test set. It looks like private test set is 2 times larger then public. (**LB 0.903**)\n* V17 (V15 train notebook) efficientunetb4, back to PUBLIC_TFREC mode without CHECKSUM, with THRESHOLD = 0.5 (**LB 0.897**)\n* V18 (V16 train notebook) efficientunetb4, model trained with new competition data (**LB 0.890**)\n* V19 (V16 train notebook) efficientunetb4, THRESHOLD = 0.4 (...)","metadata":{}},{"cell_type":"markdown","source":"# Refferences:\n* https://www.kaggle.com/joshi98kishan/hubmap-keras-pipeline-training-inference\n* https://www.kaggle.com/bguberfain/memory-aware-rle-encoding/\n* https://www.kaggle.com/leighplt/pytorch-fcn-resnet50\n* https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186","metadata":{}},{"cell_type":"markdown","source":"# Parameters\nRead parameteres from notebook output, actually only **DIM** is used:","metadata":{}},{"cell_type":"code","source":"mod_path = '/kaggle/input/hubmap-tf-with-tpu-efficientunet-512x512-train/'\nimport yaml\nimport pprint\nwith open(mod_path+'params.yaml') as file:\n    P = yaml.load(file, Loader=yaml.FullLoader)\n    pprint.pprint(P)\n    \nTHRESHOLD = 0.4 # preds > THRESHOLD\nWINDOW = 1024\nMIN_OVERLAP = 300\nNEW_SIZE = P['DIM']\n \nSUBMISSION_MODE = 'FULL' # PUBLIC_TFREC or FULL\n# 'PUBLIC_TFREC' = use created tfrecords for public test set with MIN_OVERLAP = 300 tiling 1024-512, ignore other (private test) data\n# 'FULL' do not use tfrecords, just full submission \n\nCHECKSUM = False # compute mask sum for each image\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Metrics","metadata":{}},{"cell_type":"code","source":"import json\n\nwith open(mod_path + 'metrics.json') as json_file:\n    M = json.load(json_file)\nprint('Model run datetime: '+M['datetime'])\nprint('OOF val_dice_coe: ' + str(M['oof_dice_coe']))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Packages","metadata":{}},{"cell_type":"code","source":"! pip install ../input/kerasapplications/keras-team-keras-applications-3b180cb -f ./ --no-index -q\n! pip install ../input/efficientnet/efficientnet-1.1.0/ -f ./ --no-index -q\nimport numpy as np\nimport pandas as pd\nimport os\nimport glob\nimport gc\n\nimport rasterio\nfrom rasterio.windows import Window\n\nimport pathlib\nfrom tqdm.notebook import tqdm\nimport cv2\n\nimport tensorflow as tf\nimport efficientnet as efn\nimport efficientnet.tfkeras","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Functions","metadata":{}},{"cell_type":"code","source":"def rle_encode_less_memory(img):\n    pixels = img.T.flatten()\n    pixels[0] = 0\n    pixels[-1] = 0\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 2\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\ndef make_grid(shape, window=256, min_overlap=32):\n    \"\"\"\n        Return Array of size (N,4), where N - number of tiles,\n        2nd axis represente slices: x1,x2,y1,y2 \n    \"\"\"\n    x, y = shape\n    nx = x // (window - min_overlap) + 1\n    x1 = np.linspace(0, x, num=nx, endpoint=False, dtype=np.int64)\n    x1[-1] = x - window\n    x2 = (x1 + window).clip(0, x)\n    ny = y // (window - min_overlap) + 1\n    y1 = np.linspace(0, y, num=ny, endpoint=False, dtype=np.int64)\n    y1[-1] = y - window\n    y2 = (y1 + window).clip(0, y)\n    slices = np.zeros((nx,ny, 4), dtype=np.int64)\n    \n    for i in range(nx):\n        for j in range(ny):\n            slices[i,j] = x1[i], x2[i], y1[j], y2[j]    \n    return slices.reshape(nx*ny,4), [x1, x2, y1, y2]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Models","metadata":{}},{"cell_type":"code","source":"identity = rasterio.Affine(1, 0, 0, 0, 1, 0)\nfold_models = []\nfor fold_model_path in glob.glob(mod_path+'*.h5'):\n    fold_models.append(tf.keras.models.load_model(fold_model_path,compile = False))\nprint(len(fold_models))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Tfrecords functions","metadata":{}},{"cell_type":"code","source":"AUTO = tf.data.experimental.AUTOTUNE\nimage_feature = {\n    'image': tf.io.FixedLenFeature([], tf.string),\n    'x1': tf.io.FixedLenFeature([], tf.int64),\n    'y1': tf.io.FixedLenFeature([], tf.int64)\n}\ndef _parse_image(example_proto):\n    example = tf.io.parse_single_example(example_proto, image_feature)\n    image = tf.reshape( tf.io.decode_raw(example['image'],out_type=np.dtype('uint8')), (P['DIM'],P['DIM'], 3))\n    return image, example['x1'], example['y1']\n\ndef load_dataset(filenames, ordered=True):\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False\n    dataset = tf.data.TFRecordDataset(filenames)\n    dataset = dataset.with_options(ignore_order)\n    dataset = dataset.map(_parse_image)\n    return dataset\n\ndef get_dataset(FILENAME):\n    dataset = load_dataset(FILENAME)\n    dataset  = dataset.batch(64)\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Results","metadata":{}},{"cell_type":"code","source":"def pred_TTA(model, image):\n    p = np.squeeze(model.predict(image))\n    #p += np.squeeze(tf.image.flip_left_right(model.predict(tf.image.flip_left_right(image))))\n    #p += np.squeeze(tf.image.flip_up_down(model.predict(tf.image.flip_up_down(image))))\n    p += np.squeeze(tf.image.flip_left_right(tf.image.flip_up_down(model.predict(tf.image.flip_up_down(tf.image.flip_left_right(image))))))\n    return p / 2 #4","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"p = pathlib.Path('../input/hubmap-kidney-segmentation')\nsubm = {}\n\nfor i, filename in tqdm(enumerate(p.glob('test/*.tiff')), \n                        total = len(list(p.glob('test/*.tiff')))):\n    \n    print(f'{i+1} Predicting {filename.stem}')\n    \n    dataset = rasterio.open(filename.as_posix(), transform = identity)\n    preds = np.zeros(dataset.shape, dtype=np.uint8)    \n    \n    if SUBMISSION_MODE == 'PUBLIC_TFREC' and MIN_OVERLAP == 300 and WINDOW == 1024 and NEW_SIZE == 512:\n        print('SUBMISSION_MODE: PUBLIC_TFREC')\n        fnames = glob.glob('/kaggle/input/hubmap-tfrecords-1024-512-test/test/'+filename.stem+'*.tfrec')\n        \n        if len(fnames)>0: # PUBLIC TEST SET\n            for FILENAME in fnames:\n                pred = None\n                for fold_model in fold_models:\n                    tmp = fold_model.predict(get_dataset(FILENAME))/len(fold_models)\n                    if pred is None:\n                        pred = tmp\n                    else:\n                        pred += tmp\n                    del tmp\n                    gc.collect()\n\n                pred = tf.cast((tf.image.resize(pred, (WINDOW,WINDOW)) > THRESHOLD),tf.bool).numpy().squeeze()\n\n                idx = 0\n                for img, X1, Y1 in get_dataset(FILENAME):\n                    for fi in range(X1.shape[0]):\n                        x1 = X1[fi].numpy()\n                        y1 = Y1[fi].numpy()\n                        preds[x1:(x1+WINDOW),y1:(y1+WINDOW)] += pred[idx]\n                        idx += 1\n                        \n        else: # IGNORE PRIVATE TEST SET (CREATE TFRECORDS IN FUTURE)\n            pass\n    else:\n        print('SUBMISSION_MODE: FULL')\n        slices, pos = make_grid(dataset.shape, window=WINDOW, min_overlap=MIN_OVERLAP)\n        X1 = pos[0][1:,]\n        X2 = pos[1][:-1,]\n        overlap_x = X2 - X1\n        Y1 = pos[2][1:,]\n        Y2 = pos[3][:-1,]\n        overlap_y = Y2 - Y1\n        preds = np.zeros(dataset.shape, dtype=np.float16)\n\n        if dataset.count != 3:\n            print('Image file with subdatasets as channels')\n            layers = [rasterio.open(subd) for subd in dataset.subdatasets]\n            \n        for (x1,x2,y1,y2) in slices:\n            if dataset.count == 3:\n                image = dataset.read([1,2,3], window=Window.from_slices((x1,x2),(y1,y2)))\n                image = np.moveaxis(image, 0, -1)\n            else:\n                image = np.zeros((WINDOW, WINDOW, 3), dtype=np.uint8)\n                for fl in range(3):\n                    image[:,:,fl] = layers[fl].read(window=Window.from_slices((x1,x2),(y1,y2)))\n                    \n            image = cv2.resize(image, (NEW_SIZE, NEW_SIZE),interpolation = cv2.INTER_AREA)\n            image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)\n            image = np.expand_dims(image, 0)\n            pred = None\n            for fold_model in fold_models:\n                if pred is None:\n                    pred = pred_TTA(fold_model, image)\n                else:\n                    pred += pred_TTA(fold_model, image)\n            pred = pred/len(fold_models)\n            pred = cv2.resize(pred, (WINDOW, WINDOW))\n            # probability mask for overlap\n            mask = np.ones([WINDOW,WINDOW])\n            if x1 > 0:\n                overlap = overlap_x[np.where(X1 == x1)[0][0]]\n                mask[:overlap,:] = mask[overlap,:] / 2\n            if x2 < dataset.shape[0]:\n                overlap = overlap_x[np.where(X2 == x2)[0][0]]\n                mask[-overlap:,:] = mask[-overlap:,:] / 2\n            if y1 > 0:\n                overlap = overlap_y[np.where(Y1 == y1)[0][0]]\n                mask[:,:overlap] = mask[:,:overlap] / 2\n            if y2 < dataset.shape[1]:\n                overlap = overlap_y[np.where(Y2 == y2)[0][0]]\n                mask[:,-overlap:] = mask[:,-overlap:] / 2\n            preds[x1:x2,y1:y2] += (mask * pred).astype(np.float16)\n\n    preds = (preds > THRESHOLD).astype(np.uint8)\n    subm[i] = {'id':filename.stem, 'predicted': rle_encode_less_memory(preds)}\n    if CHECKSUM:\n        print('Checksum: '+ str(np.sum(preds)))\n    del preds\n    gc.collect();","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Making submission","metadata":{}},{"cell_type":"code","source":"submission = pd.DataFrame.from_dict(subm, orient='index')\nsubmission.to_csv('submission.csv', index=False)\nsubmission.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}