{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Loading input data in the COCO format

\n
","metadata":{}},{"cell_type":"markdown","source":"## **Introduction**\n\nCOCO: https://cocodataset.org/ is a large, popular dataset for image object detection, segmentation, and captioning. It stores its annotations in the json format describing object classes, bounding boxes and bitmasks.\n\nI've created a dataset: https://www.kaggle.com/slawekbiel/sartorius-cell-instance-segmentation-coco that converts the input data given in the competition into the COCO format. This allows to easly explore the data with [pycocotools](https://github.com/cocodataset/cocoapi) and directly load it into [detectron](https://github.com/facebookresearch/detectron2)\n\nIn this notebook I'll show how we can use this to load images and annotations in just few lines of code","metadata":{"execution":{"iopub.status.busy":"2021-10-20T14:29:26.640407Z","iopub.execute_input":"2021-10-20T14:29:26.640713Z","iopub.status.idle":"2021-10-20T14:29:26.647283Z","shell.execute_reply.started":"2021-10-20T14:29:26.640684Z","shell.execute_reply":"2021-10-20T14:29:26.645577Z"}}},{"cell_type":"code","source":"!pip install pycocotools","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-10-21T14:39:48.544571Z","iopub.execute_input":"2021-10-21T14:39:48.54491Z","iopub.status.idle":"2021-10-21T14:40:06.210929Z","shell.execute_reply.started":"2021-10-21T14:39:48.544872Z","shell.execute_reply":"2021-10-21T14:40:06.209742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pycocotools.coco import COCO\nimport skimage.io as io\nimport matplotlib.pyplot as plt\nfrom pathlib import Path\nfrom PIL import Image","metadata":{"execution":{"iopub.status.busy":"2021-10-21T14:40:06.213437Z","iopub.execute_input":"2021-10-21T14:40:06.213864Z","iopub.status.idle":"2021-10-21T14:40:06.880392Z","shell.execute_reply.started":"2021-10-21T14:40:06.213816Z","shell.execute_reply":"2021-10-21T14:40:06.879406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load the annotations file into a COCO dataset","metadata":{"execution":{"iopub.status.busy":"2021-10-20T14:40:49.42228Z","iopub.execute_input":"2021-10-20T14:40:49.423207Z","iopub.status.idle":"2021-10-20T14:40:49.427388Z","shell.execute_reply.started":"2021-10-20T14:40:49.423161Z","shell.execute_reply":"2021-10-20T14:40:49.42652Z"}}},{"cell_type":"code","source":"dataDir=Path('../input/sartorius-cell-instance-segmentation')\nannFile = Path('../input/sartorius-cell-instance-segmentation-coco/annotations_all.json')\ncoco = COCO(annFile)\nimgIds = coco.getImgIds()\nimgs = coco.loadImgs(imgIds[-3:])","metadata":{"execution":{"iopub.status.busy":"2021-10-20T14:21:37.729073Z","iopub.execute_input":"2021-10-20T14:21:37.729925Z","iopub.status.idle":"2021-10-20T14:21:39.766226Z","shell.execute_reply.started":"2021-10-20T14:21:37.729874Z","shell.execute_reply":"2021-10-20T14:21:39.765394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load the first three images and display objects bitmasks and bounding boxes. This is done by the `COCO.showAnns` function","metadata":{}},{"cell_type":"code","source":"imgs = coco.loadImgs(imgIds[-3:])\n_,axs = plt.subplots(len(imgs),2,figsize=(40,15 * len(imgs)))\nfor img, ax in zip(imgs, axs):\n I = io.imread(dataDir/img['file_name'])\n annIds = coco.getAnnIds(imgIds=[img['id']])\n anns = coco.loadAnns(annIds)\n ax[0].imshow(I)\n ax[1].imshow(I)\n plt.sca(ax[1])\n coco.showAnns(anns, draw_bbox=True)","metadata":{"execution":{"iopub.status.busy":"2021-10-20T14:42:41.971096Z","iopub.execute_input":"2021-10-20T14:42:41.971455Z","iopub.status.idle":"2021-10-20T14:45:34.379989Z","shell.execute_reply.started":"2021-10-20T14:42:41.971417Z","shell.execute_reply":"2021-10-20T14:45:34.379309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **How is that generated**\n\n### **Update**: See improved version of the generation code by Adriano Passos here: https://www.kaggle.com/coldfir3/coco-dataset-generator It's faster and generates smaller files\n\n\nBelow are the functions I used to translate the original CSV dataset into the COCO formatted json file. \nNote that translation of RLE representations is done in a naive way, decoding into bitmasks and encoding it back. This makes the whole dataset take around 20 minutes to process. But since I only needed to do it once I didn't spend time on trying to optimize it.","metadata":{"execution":{"iopub.status.busy":"2021-10-20T14:29:26.640407Z","iopub.execute_input":"2021-10-20T14:29:26.640713Z","iopub.status.idle":"2021-10-20T14:29:26.647283Z","shell.execute_reply.started":"2021-10-20T14:29:26.640684Z","shell.execute_reply":"2021-10-20T14:29:26.645577Z"}}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom tqdm.notebook import tqdm\nimport json,itertools","metadata":{"execution":{"iopub.status.busy":"2021-10-21T14:42:23.241893Z","iopub.execute_input":"2021-10-21T14:42:23.242148Z","iopub.status.idle":"2021-10-21T14:42:23.246633Z","shell.execute_reply.started":"2021-10-21T14:42:23.242121Z","shell.execute_reply":"2021-10-21T14:42:23.245614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# From https://www.kaggle.com/stainsby/fast-tested-rle\ndef rle_decode(mask_rle, shape):\n '''\n mask_rle: run-length as string formated (start length)\n shape: (height,width) of array to return \n Returns numpy array, 1 - mask, 0 - background\n\n '''\n s = mask_rle.split()\n starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n starts -= 1\n ends = starts + lengths\n img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n for lo, hi in zip(starts, ends):\n img[lo:hi] = 1\n return img.reshape(shape) # Needed to align to RLE direction\n\n# From https://newbedev.com/encode-numpy-array-using-uncompressed-rle-for-coco-dataset\ndef binary_mask_to_rle(binary_mask):\n rle = {'counts': [], 'size': list(binary_mask.shape)}\n counts = rle.get('counts')\n for i, (value, elements) in enumerate(itertools.groupby(binary_mask.ravel(order='F'))):\n if i == 0 and value == 1:\n counts.append(0)\n counts.append(len(list(elements)))\n return rle","metadata":{"execution":{"iopub.status.busy":"2021-10-21T14:40:56.19028Z","iopub.execute_input":"2021-10-21T14:40:56.190583Z","iopub.status.idle":"2021-10-21T14:40:56.199274Z","shell.execute_reply.started":"2021-10-21T14:40:56.190548Z","shell.execute_reply":"2021-10-21T14:40:56.198407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def coco_structure(train_df):\n cat_ids = {name:id+1 for id, name in enumerate(train_df.cell_type.unique())} \n cats =[{'name':name, 'id':id} for name,id in cat_ids.items()]\n images = [{'id':id, 'width':row.width, 'height':row.height, 'file_name':f'train/{id}.png'} for id,row in train_df.groupby('id').agg('first').iterrows()]\n annotations=[]\n for idx, row in tqdm(train_df.iterrows()):\n mk = rle_decode(row.annotation, (row.height, row.width))\n ys, xs = np.where(mk)\n x1, x2 = min(xs), max(xs)\n y1, y2 = min(ys), max(ys)\n enc =binary_mask_to_rle(mk)\n seg = {\n 'segmentation':enc, \n 'bbox': [int(x1), int(y1), int(x2-x1+1), int(y2-y1+1)],\n 'area': int(np.sum(mk)),\n 'image_id':row.id, \n 'category_id':cat_ids[row.cell_type], \n 'iscrowd':0, \n 'id':idx\n }\n annotations.append(seg)\n return {'categories':cats, 'images':images,'annotations':annotations}","metadata":{"execution":{"iopub.status.busy":"2021-10-21T14:40:56.552639Z","iopub.execute_input":"2021-10-21T14:40:56.553131Z","iopub.status.idle":"2021-10-21T14:40:56.561852Z","shell.execute_reply.started":"2021-10-21T14:40:56.553077Z","shell.execute_reply":"2021-10-21T14:40:56.560967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## run it on first three images for demonstration:\ntrain_df = pd.read_csv('../input/sartorius-cell-instance-segmentation/train.csv')\nall_ids = train_df.id.unique()\ntrain_sample = train_df[train_df.id.isin(all_ids[:3])]\nroot = coco_structure(train_sample)\n\nwith open('annotations_sample.json', 'w', encoding='utf-8') as f:\n json.dump(root, f, ensure_ascii=True, indent=4)","metadata":{"execution":{"iopub.status.busy":"2021-10-21T14:42:42.129118Z","iopub.execute_input":"2021-10-21T14:42:42.130001Z","iopub.status.idle":"2021-10-21T14:43:11.64497Z","shell.execute_reply.started":"2021-10-21T14:42:42.129959Z","shell.execute_reply":"2021-10-21T14:43:11.643931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!head -n 100 annotations_sample.json","metadata":{"execution":{"iopub.status.busy":"2021-10-21T14:43:36.489686Z","iopub.execute_input":"2021-10-21T14:43:36.490026Z","iopub.status.idle":"2021-10-21T14:43:37.287643Z","shell.execute_reply.started":"2021-10-21T14:43:36.489986Z","shell.execute_reply":"2021-10-21T14:43:37.286454Z"},"trusted":true},"execution_count":null,"outputs":[]}]}