{"cells":[{"metadata":{},"cell_type":"markdown","source":"<span style=\"color: #000000; font-family: Tahoma; font-size: 3em;\">Convert Annotations to COCO Format</span>"},{"metadata":{},"cell_type":"markdown","source":"### **Hi Everyone! Welcome!**\n\n**This noteboook demonstrates the conversion of Custom Segmentation Annotation formats to the widely supported COCO format to use in any Object Detection or Segmentation Task.**\n\n\n#### With COCO Annotations, all Segmentation Frameworks such as AdelaiDet (BlendMask), Detectron2, TF Matterplot Mask RCNN etc. can be used.\n\n\nI have converted & registered the HuBMAP Dataset created in this notebook as **'HuBMap COCO Dataset 512x512 Tiled'**\n\n#### Please don't miss to visit and upvote if you find it useful\nDATASET LINK - https://www.kaggle.com/sreevishnudamodaran/hubmap-coco-dataset-512x512-tiled\n\n\nIt is properly structured with images split into directories and no downscaling has been done to preserve the whole information.\n\n#### Directory Format:\n\n```\n    - coco_train\n        - images(contains images in jpg format)\n            - original_tiff_image_name\n               - tile_column_number\n                   - image\n                   .\n                   .\n                   .\n                 .\n                 .\n                 .\n              .\n              .\n              .\n        - train.json (contains all the segmentation annotations in coco \n        -             format with proper relative path of the images)\n\n```\n\nA lot of thanks to Marcos Novaes (https://www.kaggle.com/marcosnovaes) for helping with the Tiling and TFRecords creation notebook and the TFRecords Dataset.\n\n\n##### I will keep updating this notebook with further details of the implementation. Thanks to the whole Kaggle community and staff for all the support!\n\n#### Please don't miss to upvote and comment if you like my work :)\n\n\n##### Hope everyone finds this useful!"},{"metadata":{"trusted":true,"_kg_hide-output":true,"collapsed":true},"cell_type":"code","source":"!pip install pycocotools","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%matplotlib inline\n\nimport numpy as np\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom matplotlib import rcParams\nsns.set(rc={\"font.size\":9,\"axes.titlesize\":15,\"axes.labelsize\":9,\n            \"axes.titlepad\":11, \"axes.labelpad\":9, \"legend.fontsize\":7, \"legend.title_fontsize\":7, 'axes.grid' : False})\nimport cv2\nimport json\nimport pandas as pd\nimport tensorflow as tf\nimport glob\nimport os.path as osp\nfrom path import Path\n\nimport re\nimport datetime\nimport numpy as np\nfrom skimage import measure\nfrom PIL import Image\nimport pycocotools\nfrom pycocotools import mask\n\nimport collections\nimport sys\nimport uuid\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BASE_PATH = \"/kaggle/input/hubmap-kidney-segmentation\"\ndf_info = pd.read_csv(\n    osp.join(BASE_PATH, \"HuBMAP-20-dataset_information.csv\")\n)\ndf_info.head(13)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## hubmap-tfrecord-512 Dataset Loading and Exploration\n\nWe wil be using the hubmap-tfrecord-512 Dataset which has the HuBMAP Kidney Segmenatation Dataset Images which are of very large varying sizes i.e 13kx18k pixels upto 49kx34k\n\n#### **No downscaling of images have been done in this notebook with the intention of preserving information**"},{"metadata":{"trusted":true},"cell_type":"code","source":"file_list = glob.glob('/kaggle/input/hubmap-tfrecord-512/train/*.csv')\nfile_list","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Combine Image Csv files to Dataframe"},{"metadata":{"trusted":true},"cell_type":"code","source":"li = []\nfor filename in file_list:\n    df = pd.read_csv(filename, index_col=None, header=0)\n    li.append(df)\n\ncoco_helper_df = pd.concat(li, axis=0, ignore_index=True)\ncoco_helper_df.drop('Unnamed: 0', axis=1, inplace=True)\ncoco_helper_df.head(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define Functions to Parse & Load TFRecords"},{"metadata":{"trusted":true},"cell_type":"code","source":"# read back a record to make sure it the decoding works\n# Create a dictionary describing the features.\nimage_feature_description = {\n    'img_index': tf.io.FixedLenFeature([], tf.int64),\n    'height': tf.io.FixedLenFeature([], tf.int64),\n    'width': tf.io.FixedLenFeature([], tf.int64),\n    'num_channels': tf.io.FixedLenFeature([], tf.int64),\n    'img_bytes': tf.io.FixedLenFeature([], tf.string),\n    'mask': tf.io.FixedLenFeature([], tf.string),\n    'tile_id': tf.io.FixedLenFeature([], tf.int64),\n    'tile_col_pos': tf.io.FixedLenFeature([], tf.int64),\n    'tile_row_pos': tf.io.FixedLenFeature([], tf.int64),\n}\n\ndef _parse_image_function(example_proto):\n  # Parse the input tf.Example proto using the dictionary above.\n    single_example = tf.io.parse_single_example(example_proto, image_feature_description)\n    img_index = single_example['img_index']\n    img_height = single_example['height']\n    img_width = single_example['width']\n    num_channels = single_example['num_channels']\n    \n    img_bytes =  tf.io.decode_raw(single_example['img_bytes'],out_type='uint8')\n   \n    img_array = tf.reshape( img_bytes, (img_height, img_width, num_channels))\n   \n    mask_bytes =  tf.io.decode_raw(single_example['mask'],out_type='bool')\n    \n    mask = tf.reshape(mask_bytes, (img_height,img_width))\n    mtd = dict()\n    mtd['img_index'] = single_example['img_index']\n    mtd['width'] = single_example['width']\n    mtd['height'] = single_example['height']\n    mtd['tile_id'] = single_example['tile_id']\n    mtd['tile_col_pos'] = single_example['tile_col_pos']\n    mtd['tile_row_pos'] = single_example['tile_row_pos']\n    struct = {\n        'img_array': img_array,\n        'mask': mask,\n        'mtd': mtd\n    } \n    return struct\n\ndef read_tf_dataset(storage_file_path):\n    encoded_image_dataset = tf.data.TFRecordDataset(storage_file_path, compression_type=\"GZIP\")\n    parsed_image_dataset = encoded_image_dataset.map(_parse_image_function)\n    return parsed_image_dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Exploration with a Sample"},{"metadata":{"trusted":true},"cell_type":"code","source":"BASE_TRAIN_PATH = \"/kaggle/input/hubmap-tfrecord-512/train\"\nsample_tile_path = osp.join(BASE_TRAIN_PATH,\n                            coco_helper_df.loc[850,\"img_id\"],\n                            'col'+str(coco_helper_df.loc[850,\"tile_col_num\"]),\n                            str(Path(coco_helper_df.loc[850,\"tile_rel_path\"]).name))\nsample_tile_path","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Convert Binary Mask to Polygons\n### Define Functions for the Conversion"},{"metadata":{"trusted":true},"cell_type":"code","source":"convert = lambda text: int(text) if text.isdigit() else text.lower()\nnatrual_key = lambda key: [ convert(c) for c in re.split('([0-9]+)', key) ]\n\ndef close_contour(contour):\n    if not np.array_equal(contour[0], contour[-1]):\n        contour = np.vstack((contour, contour[0]))\n    return contour\n\ndef binary_mask_to_polygon(binary_mask, tolerance=0):\n    \"\"\"Converts a binary mask to COCO polygon representation\n    Args:\n        binary_mask: a 2D binary numpy array where '1's represent the object\n        tolerance: Maximum distance from original points of polygon to approximated\n            polygonal chain. If tolerance is 0, the original coordinate array is returned.\n    \"\"\"\n    polygons = []\n    poly_nogrp = []\n    # pad mask to close contours of shapes which start and end at an edge\n    padded_binary_mask = np.pad(binary_mask, pad_width=1, mode='constant', constant_values=0)\n    contours = measure.find_contours(padded_binary_mask, 0.5)\n    contours = np.subtract(contours, 1)\n    for contour in contours:\n        contour = close_contour(contour)\n        contour = measure.approximate_polygon(contour, tolerance)\n        if len(contour) < 3:\n            continue\n        contour = np.flip(contour, axis=1)\n        segmentation = contour.ravel().tolist()\n        # after padding and subtracting 1 we may get -0.5 points in our segmentation \n        segmentation = [0 if i < 0 else i for i in segmentation]\n        poly_nogrp.append(segmentation)\n        it = iter(segmentation)\n        seg_grouped = list(map(list, zip(it, it)))\n        \n        polygons.append(seg_grouped)\n\n    return polygons, poly_nogrp","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualize Before and After Conversion"},{"metadata":{"trusted":true},"cell_type":"code","source":"ds = read_tf_dataset(sample_tile_path)\n\nfor struct in ds.as_numpy_iterator():\n    #struct = g_dataset.get_next()\n    img_mtd = struct[\"mtd\"]\n    img_array  = struct[\"img_array\"]\n    img_mask = struct[\"mask\"]\n\n    fig, ax = plt.subplots(1,3,figsize=(20,8))\n    \n    fig.suptitle(\"Tile ID = {} Xpos = {} Ypos = {}\".format(img_mtd['tile_id'],\n                  img_mtd['tile_col_pos'],img_mtd['tile_row_pos']), fontsize=25)\n\n    \n    ax[0].set_title(\"Tile Image\")\n    ax[0].imshow(img_array)\n    #ax[1].set_title(\"Pixelarray distribution\");\n    #sns.distplot(img_array.flatten(), ax=ax[1]);\n    ax[1].imshow(img_mask)\n    ax[1].set_title(\"Tile Mask From Tfrecord\")\n\n\n    poly, _ = binary_mask_to_polygon(img_mask)\n    im = np.zeros([512,512],dtype=np.uint8)\n    for mask_pts in poly:\n        pts = np.array(mask_pts, np.int32)\n        pts = pts.reshape((-1,1,2))\n        #cv2.polylines(im, pts, True, (0,255,255))\n        cv2.fillConvexPoly(im, pts, 255)\n    ax[2].imshow(im)\n    ax[2].set_title(\"Tile Mask After Polygon Conversion\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Create Output Directories"},{"metadata":{"trusted":true},"cell_type":"code","source":"input_dir = osp.join(BASE_PATH, \"train\")\noutput_dir = \"coco_train\"\n\nif osp.exists(output_dir):\n    print('Output directory already exists:', output_dir)\n    sys.exit(1)\nos.makedirs(output_dir)\nos.makedirs(osp.join(output_dir, 'images'))\nprint('Coco Dataset Directory:', output_dir)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Analyzing the Popular COCO Format\n\n"},{"metadata":{},"cell_type":"markdown","source":"## COCO Annotation Basic Syntax\n\n```python\n{\n  \"type\": \"instances\",\n  \"images\": [\n    {\n      \"file_name\": \"<image_name.jpg>\",\n      \"height\": \"<height>\",\n      \"width\": \"<width>\",\n      \"id\": \"<Used to reference each image and it should be unique for each image. This will be the 'image_id' used to tag each annotation>\"\n    }\n#    .\n#    .\n#    .\n      \n  ],\n\n  \"categories\": [\n    {\n      \"supercategory\": \"none\",\n      \"name\": \"<Class One>\",\n      \"id\": 0\n    },\n    {\n      \"supercategory\": \"none\",\n      \"name\": \"<Class Two>\",\n      \"id\": 2\n    }\n#    .\n#    .\n#    .\n\n  ],\n\n  \"annotations\": [\n    {\n      \"id\": 1,\n      \"bbox\": [\n        \"<xmin>\",\n        \"<ymin>\",\n        \"<xmax>\",\n        \"<xmax>\"\n      ],\n      \"image_id\": \"<id of the image from which the polygon annotation is from as defined in the 'images' block above>\",\n\n      \"segmentation\": [\n          \"<x1>\",\n          \"<y1>\",\n          \"<x2>\",\n          \"<y2>\"\n#          .\n#          .\n#          .\n\n      ],\n      \"ignore\": 0,\n      \"area\": \"<Area of the Polygon represented by the points in 'segmentation' block>\",\n      \"iscrowd\": 0,\n      \"category_id\": \"<Class category ID as an integer which will be defined below>\"\n    },\n\n  ],\n\"categories\": [\n    {\n        \"supercategory\": null,\n        \"id\": \"<Integer ID for the Class Label>\",\n        \"name\": \"<Class One Label as a String>\"\n    },\n#    .\n#    .\n#    .\n\n]\n}\n```"},{"metadata":{},"cell_type":"markdown","source":"## Defining Blocks in the Annotation\n### Adding the Glomerule Class as '0'"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels =  [\"__ignore__\",\n                \"glomerule\"\n                ]\nnow = datetime.datetime.now()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Define the Basic Structure"},{"metadata":{"trusted":true},"cell_type":"code","source":"data = dict(\n    info=dict(\n        description=None,\n        url=None,\n        version=None,\n        year=now.year,\n        contributor=None,\n        date_created=now.strftime('%Y-%m-%d %H:%M:%S.%f'),\n    ),\n    licenses=[dict(\n        url=None,\n        id=0,\n        name=None,\n    )],\n    images=[\n        # license, url, file_name, height, width, date_captured, id\n    ],\n    type='instances',\n    annotations=[\n        # segmentation, area, iscrowd, image_id, bbox, category_id, id\n    ],\n    categories=[\n        # supercategory, id, name\n    ],\n)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class_name_to_id = {}\nfor i, each_label in enumerate(labels):\n    class_id = i - 1  # starts with -1\n    class_name = each_label\n    if class_id == -1:\n        assert class_name == '__ignore__'\n        continue\n    class_name_to_id[class_name] = class_id\n    data['categories'].append(dict(\n        supercategory=None,\n        id=class_id,\n        name=class_name,\n    ))\ndata","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"out_ann_file = osp.join(output_dir, 'train.json')\ntf_folders = glob.glob('/kaggle/input/hubmap-tfrecord-512/train/*/')\ntf_folders","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Dict Before Adding Annotations"},{"metadata":{"trusted":true},"cell_type":"code","source":"data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Start the Conversion Job"},{"metadata":{"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"for i in range(coco_helper_df.shape[0]):\n#for i in range(850,852):  ##for testing\n    tile_path = osp.join(BASE_TRAIN_PATH,\n                        coco_helper_df.loc[i,\"img_id\"],\n                        'col'+str(coco_helper_df.loc[i,\"tile_col_num\"]),\n                        str(Path(coco_helper_df.loc[i,\"tile_rel_path\"]).name))\n    print(\"tile_path\", tile_path)\n\n    ds = read_tf_dataset(tile_path)\n\n    for struct in ds.as_numpy_iterator():\n        #struct = g_dataset.get_next()\n        img_mtd = struct[\"mtd\"]\n        img_array  = struct[\"img_array\"]\n        img_mask = struct[\"mask\"]\n\n# Code to Visualize Each Annotation  ##for testing\n#         fig, ax = plt.subplots(1,3,figsize=(20,8))\n#         fig.suptitle(\"Tile ID = {} Xpos = {} Ypos = {}\".format(img_mtd['tile_id'],\n#                      img_mtd['tile_col_pos'],img_mtd['tile_row_pos']), fontsize=25)\n\n#         ax[0].set_title(\"Tile Image\")\n#         ax[0].imshow(img_array)\n#         ax[1].imshow(img_mask)\n#         ax[1].set_title(\"Tile Mask From Tfrecord\")\n\n        poly, poly_nogrp = binary_mask_to_polygon(img_mask)\n\n#         im = np.zeros([512,512],dtype=np.uint8)\n#         for mask_pts in poly:\n#             pts = np.array(mask_pts, np.int32)\n#             pts = pts.reshape((-1,1,2))\n#             #cv2.polylines(im, pts, True, (0,255,255))\n#             cv2.fillConvexPoly(im, pts, 255)\n#         ax[2].imshow(im)\n#         ax[2].set_title(\"Tile Mask After Polygon Conversion\")\n        \n        print('Generating annotations from: {}'.format(coco_helper_df.loc[i,\"img_id\"]))\n\n        jpg_folder_path = osp.join(output_dir, 'images/', coco_helper_df.loc[i,\"img_id\"],\n                    'col'+str(coco_helper_df.loc[i,\"tile_col_num\"]))\n\n        if not osp.exists(jpg_folder_path):\n            os.makedirs(jpg_folder_path)\n\n        #str(Path(train_df.loc[i,\"tile_rel_path\"]).stem)\n        #print('JPG Directory:', jpg_folder_path)\n\n        out_img_file = osp.join(jpg_folder_path,\n                                str(Path(coco_helper_df.loc[i,\"tile_rel_path\"]).stem)+'.jpg'\n                                )\n        print('out_img_file:', out_img_file)\n        im_out = Image.fromarray(img_array)\n        im_out.save(out_img_file)\n\n        data['images'].append(dict(\n            license=0,\n            url=None,\n            file_name=osp.relpath(out_img_file, osp.dirname(out_ann_file)),\n            height=img_array.shape[0],\n            width=img_array.shape[1],\n            date_captured=None,\n            id=int(coco_helper_df.loc[i,'tile_id'])\n        ))\n\n        masks = {}\n\n        for shape in poly_nogrp:\n\n            group_id = uuid.uuid1()\n            mask = img_mask.copy()\n\n            mask = np.asfortranarray(mask.astype(np.uint8))\n            mask = pycocotools.mask.encode(mask)\n            area = float(pycocotools.mask.area(mask))\n            bbox = pycocotools.mask.toBbox(mask).flatten().tolist()\n\n            data['annotations'].append(dict(\n                id=len(data['annotations']),\n                image_id=int(coco_helper_df.loc[i,'tile_id']),\n                category_id='0',\n                segmentation=[shape],\n                area=area,\n                bbox=bbox,\n                iscrowd=0,\n            ))\n    \n    print(\"{}/{} Tile Image done.\".format(i,coco_helper_df.shape[0]))\n    \nwith open(out_ann_file, 'w') as f:\n    json.dump(data, f)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Verify Number of Files "},{"metadata":{"trusted":true},"cell_type":"code","source":"!find /kaggle/working/coco_train -type f | wc -l","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"coco_helper_df.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Package Dataset into Zip for Upload"},{"metadata":{"trusted":true},"cell_type":"code","source":"!zip -r hubmap-coco-512x512-tiled.zip ./coco_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!rm -R coco_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!ls -ahl","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}