{"cells":[{"metadata":{},"cell_type":"markdown","source":"# MAKE_GRID Function Explained\n\n**Description:**\n\nmake_grid is a function used in a number of public HuBMAP notebooks for inference and creating a submission.\nSeemed to me this function was essential to making a submission, and so I thought I would comb through the function line by line to get a better understanding of how it works.\n\n**References:**\n\nhttps://www.kaggle.com/leighplt/pytorch-fcn-resnet50\nhttps://www.kaggle.com/joshi98kishan/hubmap-keras-pipeline-training-inference\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/overview/supervised-ml-evaluation\n\n**Function Expectation:**\n\n - \"Return Array of size (N,4), where N - number of tiles, 2nd axis represente slices: x1,x2,y1,y2\"\n - The function breaks the image into blocks.  In our case 1024x1024 blocks (which are later resized to 256x256).\n - Technically, make_grid just specifies the blocks via a series of pixel coordinates ((x1,y1),(x2,y2)).\n - Later, we use the output to loop through each set of coordinates, extract the image, resize, predict, and store results."},{"metadata":{},"cell_type":"markdown","source":"# Imports"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# For reading tiff images in parts \nimport rasterio\nfrom rasterio.windows import Window\nimport pathlib\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Full 'Make_Grid' Function"},{"metadata":{"trusted":true},"cell_type":"code","source":"# https://www.kaggle.com/leighplt/pytorch-fcn-resnet50\ndef make_grid(shape, window=256, min_overlap=32):\n    \"\"\"\n        Return Array of size (N,4), where N = number of tiles,\n        2nd axis represente slices: x1,x2,y1,y2 \n    \"\"\"\n    x, y = shape\n    nx = x // (window - min_overlap) + 1\n    x1 = np.linspace(0, x, num=nx, endpoint=False, dtype=np.int64)\n    x1[-1] = x - window\n    x2 = (x1 + window).clip(0, x)\n    ny = y // (window - min_overlap) + 1\n    y1 = np.linspace(0, y, num=ny, endpoint=False, dtype=np.int64)\n    y1[-1] = y - window\n    y2 = (y1 + window).clip(0, y)\n    slices = np.zeros((nx,ny, 4), dtype=np.int64)\n    \n    for i in range(nx):\n        for j in range(ny):\n            slices[i,j] = x1[i], x2[i], y1[j], y2[j]    \n    return slices.reshape(nx*ny,4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Settings & Global Variables"},{"metadata":{"trusted":true},"cell_type":"code","source":"DATA_ORIG_PATH = '../input/hubmap-kidney-segmentation'\nidentity = rasterio.Affine(1, 0, 0, 0, 1, 0)\n# WINDOW is the size of the tile to be read by rasterio\nWINDOW = 1024\n# Tiles will have some overlap\nMIN_OVERLAP = 32\n# Tiles will be resized to NEW_SIZE, which is the size of the image\n# on which, we have trained our model.\nNEW_SIZE = 256","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"https://docs.python.org/3/library/pathlib.html\n\n\"This module offers classes representing filesystem paths with semantics appropriate for different operating systems. Path classes are divided between pure paths, which provide purely computational operations without I/O, and concrete paths, which inherit from pure paths but also provide I/O operations.\""},{"metadata":{"trusted":true},"cell_type":"code","source":"p = pathlib.Path(DATA_ORIG_PATH)\np","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"empty dictionary for submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"subm = {}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"we're going to iterate through each test image"},{"metadata":{"trusted":true},"cell_type":"code","source":"p.glob('test/*.tiff')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"let's take a look at the beginning of the inference loop.\n\n* We start by reading in an image\n* Next we run 'make_grid' on an image, and we get a numpy array returned of int values with shape 1710x4"},{"metadata":{"trusted":true},"cell_type":"code","source":"for i, filename in enumerate(p.glob('test/*.tiff')):\n    print(\"i: \", i)\n    print(\"filename: \", filename)\n    print(f'{i+1} Predicting {filename.stem}')\n    dataset = rasterio.open(filename.as_posix(), transform = identity)\n    print(\"dataset: \", dataset)\n    print(\"dataset shape: \", dataset.shape)\n    slices = make_grid(dataset.shape, window=WINDOW, min_overlap=MIN_OVERLAP)\n    print(\"slices shape: \", slices.shape)\n    print(\"slices type: \", type(slices))\n    print(\"first 5 values of slices: \", slices[:5])\n    break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So let's look at make_grid line by line.  As input the function takes...\n\n* the image size in form tuple\n* a window size which we've pre-selected\n* and a min_overlap value which we've pre-selected"},{"metadata":{"trusted":true},"cell_type":"code","source":"shape = (36800, 43780)\nwindow=WINDOW\nmin_overlap=MIN_OVERLAP","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we unwrap shape into values x and y"},{"metadata":{"trusted":true},"cell_type":"code","source":"x, y = shape\nprint(x)\nprint(y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"x: \", x)\nprint(\"window: \", window)\nprint(\"min_overlap: \", min_overlap)\nprint(\"window - min_overlap: \", window - min_overlap)\nprint(\"{} / {} = {}\".format(x,window - min_overlap,x / (window - min_overlap)))\nprint(\"{} // {} i.e. int division = {}\".format(x,window - min_overlap,x // (window - min_overlap)))\nnx = x // (window - min_overlap) + 1\nprint(\"nx i.e. int division result + 1 = \", nx)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"numpy.linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None, axis=0)[source]¶\nReturn evenly spaced numbers over a specified interval.\n\n### **Returns num evenly spaced samples, calculated over the interval [start, stop].**\n\nThe endpoint of the interval can optionally be excluded.\n\nChanged in version 1.16.0: Non-scalar start and stop are now supported.\n\nParameters\n\nstart\n\narray_like\nThe starting value of the sequence.\n\nstop\n\narray_like\nThe end value of the sequence, unless endpoint is set to False. In that case, the sequence consists of all but the last of num + 1 evenly spaced samples, so that stop is excluded. Note that the step size changes when endpoint is False.\n\nnum\n\nint, optional\nNumber of samples to generate. Default is 50. Must be non-negative.\n\nendpoint\n\nbool, optional\nIf True, stop is the last sample. Otherwise, it is not included. Default is True.\n\nretstep\n\nbool, optional\nIf True, return (samples, step), where step is the spacing between samples.\n\ndtyped\n\ntype, optional\nThe type of the output array. If dtype is not given, infer the data type from the other input arguments.\n\nNew in version 1.9.0.\n\naxisint, optional\nThe axis in the result to store the samples. Relevant only if start or stop are array-like. By default (0), the samples will be along a new axis inserted at the beginning. Use -1 to get an axis at the end.\n\nNew in version 1.16.0.\n\nReturns\nsamples\n\nndarray\nThere are num equally spaced samples in the closed interval [start, stop] or the half-open interval [start, stop) (depending on whether endpoint is True or False).\n\nstepfloat, optional\nOnly returned if retstep is True\n\nSize of spacing between samples."},{"metadata":{"trusted":true},"cell_type":"code","source":"x1 = np.linspace(0, x, num=nx, endpoint=False, dtype=np.int64)\nprint(\"len(x1): \", len(x1))\nprint(\"x1: \", x1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"now we take the last values in x1 and set it to x-window"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"original value of x1[-1]: \", x1[-1])\nprint(\"x: \", x)\nprint(\"window: \", window)\nx1[-1] = x - window\nprint(\"new value of x1[-1]\", x1[-1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"next we set x2 to x1 + window"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"x1: \", x1)\nprint(\"window:\", window)\nprint(\"x1 + window: \", x1 + window)\nx2 = (x1 + window).clip(0, x)\nprint(\"x2: \", x2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"then we complete the same process for 'y'"},{"metadata":{"trusted":true},"cell_type":"code","source":"ny = y // (window - min_overlap) + 1\nprint(\"ny: \", ny)\ny1 = np.linspace(0, y, num=ny, endpoint=False, dtype=np.int64)\ny1[-1] = y - window\nprint(\"y1: \", y1)\ny2 = (y1 + window).clip(0, y)\nprint(\"y2: \", y2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"initialize a numpy matrix 'slices' using nx & ny"},{"metadata":{"trusted":true},"cell_type":"code","source":"slices = np.zeros((nx,ny, 4), dtype=np.int64)\nprint(slices)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"fill slices with our values from x1, x2, y1, y2"},{"metadata":{"trusted":true},"cell_type":"code","source":"for i in range(nx):\n    for j in range(ny):\n        slices[i,j] = x1[i], x2[i], y1[j], y2[j]    \nprint(slices)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"reshape"},{"metadata":{"trusted":true},"cell_type":"code","source":"slices.reshape(nx*ny,4)\nprint(slices)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"the full loop!"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# for i, filename in tqdm(enumerate(p.glob('test/*.tiff')), \n#                         total = len(list(p.glob('test/*.tiff')))):\n    \n#     print(f'{i+1} Predicting {filename.stem}')\n    \n#     dataset = rasterio.open(filename.as_posix(), transform = identity)\n#     slices = make_grid(dataset.shape, window=WINDOW, min_overlap=MIN_OVERLAP)\n#     preds = np.zeros(dataset.shape, dtype=np.uint8)\n    \n#     for (x1,x2,y1,y2) in slices:\n#         image = dataset.read([1,2,3],\n#                     window=Window.from_slices((x1,x2),(y1,y2)))\n#         image = np.moveaxis(image, 0, -1)\n        \n#         image = tf.image.convert_image_dtype(image, \n#                                  tf.float32)\n#         image = cv2.resize(image.numpy(), (NEW_SIZE, NEW_SIZE))\n#         image = np.expand_dims(image, 0)\n        \n#         pred = None\n        \n#         for fold_model in trained_fold_models:\n#             if pred is None:\n#                 pred = np.squeeze(fold_model.predict(image))\n#             else:\n#                 pred += np.squeeze(fold_model.predict(image))\n        \n#         pred = pred/len(trained_fold_models)\n        \n#         pred = cv2.resize(pred, (WINDOW, WINDOW))\n#         preds[x1:x2,y1:y2] = (pred > 0.5).astype(np.uint8)\n            \n#     subm[i] = {'id':filename.stem, 'predicted': rle_numba_encode(preds)}\n#     del preds\n#     gc.collect();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# The End\n\nHope this helps some people as it helped me.  :-)"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}