{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Welcome to the world where fashion meets computer vision! This is a starter kernel that applies Mask R-CNN with COCO pretrained weights to the task of [iMaterialist (Fashion) 2019 at FGVC6](https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6).","metadata":{}},{"cell_type":"code","source":"# !pip install keras==2.2.5\n# %tensorflow_version 1.x","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport gc\nimport sys\nimport json\nimport glob\nimport random\nfrom pathlib import Path\n\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nimport itertools\nfrom tqdm import tqdm\n\nfrom imgaug import augmenters as iaa\nfrom sklearn.model_selection import StratifiedKFold, KFold","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:43:41.829390Z","iopub.execute_input":"2021-08-25T07:43:41.829649Z","iopub.status.idle":"2021-08-25T07:43:43.300602Z","shell.execute_reply.started":"2021-08-25T07:43:41.829604Z","shell.execute_reply":"2021-08-25T07:43:43.299665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"DATA_DIR = Path('/kaggle/input')\n\nROOT_DIR = Path('/kaggle/working')\n\n# For demonstration purpose, the classification ignores attributes (only categories),\n# and the image size is set to 512, which is the same as the size of submission masks\nNUM_CATS = 46\nIMAGE_SIZE = 512","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:43:47.733791Z","iopub.execute_input":"2021-08-25T07:43:47.734088Z","iopub.status.idle":"2021-08-25T07:43:47.741286Z","shell.execute_reply.started":"2021-08-25T07:43:47.734038Z","shell.execute_reply":"2021-08-25T07:43:47.740248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dowload Libraries and Pretrained Weights","metadata":{}},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:43:55.752060Z","iopub.execute_input":"2021-08-25T07:43:55.752349Z","iopub.status.idle":"2021-08-25T07:43:55.759690Z","shell.execute_reply.started":"2021-08-25T07:43:55.752301Z","shell.execute_reply":"2021-08-25T07:43:55.758459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# git config --global --unset https.proxy","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:43:56.049078Z","iopub.execute_input":"2021-08-25T07:43:56.049361Z","iopub.status.idle":"2021-08-25T07:43:56.052563Z","shell.execute_reply.started":"2021-08-25T07:43:56.049309Z","shell.execute_reply":"2021-08-25T07:43:56.051858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!git clone https://www.github.com/matterport/Mask_RCNN.git\nos.chdir('Mask_RCNN')\n\n# no idea\n!rm -rf .git # to prevent an error when the kernel is committed\n!rm -rf images assets # to prevent displaying images at the bottom of a kernel","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.status.busy":"2021-08-25T07:43:56.261572Z","iopub.execute_input":"2021-08-25T07:43:56.261847Z","iopub.status.idle":"2021-08-25T07:44:04.824197Z","shell.execute_reply.started":"2021-08-25T07:43:56.261795Z","shell.execute_reply":"2021-08-25T07:44:04.823346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:04.827649Z","iopub.execute_input":"2021-08-25T07:44:04.827915Z","iopub.status.idle":"2021-08-25T07:44:04.836823Z","shell.execute_reply.started":"2021-08-25T07:44:04.827855Z","shell.execute_reply":"2021-08-25T07:44:04.836111Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sys.path.append(ROOT_DIR/'Mask_RCNN')","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:04.837830Z","iopub.execute_input":"2021-08-25T07:44:04.840107Z","iopub.status.idle":"2021-08-25T07:44:04.846229Z","shell.execute_reply.started":"2021-08-25T07:44:04.840058Z","shell.execute_reply":"2021-08-25T07:44:04.845422Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from mrcnn.config import Config\nfrom mrcnn import utils\nimport mrcnn.model as modellib\nfrom mrcnn import visualize\nfrom mrcnn.model import log","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","execution":{"iopub.status.busy":"2021-08-25T07:44:04.847213Z","iopub.execute_input":"2021-08-25T07:44:04.847485Z","iopub.status.idle":"2021-08-25T07:44:05.768457Z","shell.execute_reply.started":"2021-08-25T07:44:04.847435Z","shell.execute_reply":"2021-08-25T07:44:05.767654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Download COCO pre-trained weights\n!wget --quiet https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5\n!ls -lh mask_rcnn_coco.h5\n\nCOCO_WEIGHTS_PATH = 'mask_rcnn_coco.h5'","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:05.770082Z","iopub.execute_input":"2021-08-25T07:44:05.770610Z","iopub.status.idle":"2021-08-25T07:44:10.539787Z","shell.execute_reply.started":"2021-08-25T07:44:05.770356Z","shell.execute_reply":"2021-08-25T07:44:10.538822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Directory to save logs and model checkpoints\nDEFAULT_LOGS_DIR = os.path.join(ROOT_DIR, \"logs\")\n\n# now I do not have to start my training from the beginning.\n# I can simply change my weights path to the last weights file e.g. ‘mask_rcnn_object_0003.h5’.\n\n# Skip detections with < 90% confidence\n# DETECTION_MIN_CONFIDENCE = 0.9","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.541432Z","iopub.execute_input":"2021-08-25T07:44:10.542007Z","iopub.status.idle":"2021-08-25T07:44:10.547232Z","shell.execute_reply.started":"2021-08-25T07:44:10.541693Z","shell.execute_reply":"2021-08-25T07:44:10.546271Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Set Config","metadata":{}},{"cell_type":"markdown","source":"Mask R-CNN has a load of hyperparameters. I only adjust some of them.","metadata":{}},{"cell_type":"code","source":"class FashionConfig(Config):\n    NAME = \"fashion\"\n    NUM_CLASSES = NUM_CATS + 1 # +1 for the background class\n    \n    GPU_COUNT = 1\n    IMAGES_PER_GPU = 4 # a memory error occurs when IMAGES_PER_GPU is too high\n    \n    BACKBONE = 'resnet50'\n    \n    IMAGE_MIN_DIM = IMAGE_SIZE\n    IMAGE_MAX_DIM = IMAGE_SIZE    \n    IMAGE_RESIZE_MODE = 'none'\n    \n    RPN_ANCHOR_SCALES = (16, 32, 64, 128, 256)\n    #DETECTION_NMS_THRESHOLD = 0.0\n    \n    # STEPS_PER_EPOCH should be the number of instances \n    # divided by (GPU_COUNT*IMAGES_PER_GPU), and so should VALIDATION_STEPS;\n    # however, due to the time limit, I set them so that this kernel can be run in 9 hours\n    STEPS_PER_EPOCH = 1000\n    VALIDATION_STEPS = 200\n    \nconfig = FashionConfig()\nconfig.display()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.548856Z","iopub.execute_input":"2021-08-25T07:44:10.549522Z","iopub.status.idle":"2021-08-25T07:44:10.562222Z","shell.execute_reply.started":"2021-08-25T07:44:10.549198Z","shell.execute_reply":"2021-08-25T07:44:10.561031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Make Datasets","metadata":{}},{"cell_type":"code","source":"DATA_DIR","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.564119Z","iopub.execute_input":"2021-08-25T07:44:10.564707Z","iopub.status.idle":"2021-08-25T07:44:10.577600Z","shell.execute_reply.started":"2021-08-25T07:44:10.564545Z","shell.execute_reply":"2021-08-25T07:44:10.576711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.578757Z","iopub.execute_input":"2021-08-25T07:44:10.579192Z","iopub.status.idle":"2021-08-25T07:44:10.589281Z","shell.execute_reply.started":"2021-08-25T07:44:10.578998Z","shell.execute_reply":"2021-08-25T07:44:10.588484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir(\"../input\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.591992Z","iopub.execute_input":"2021-08-25T07:44:10.592272Z","iopub.status.idle":"2021-08-25T07:44:10.920853Z","shell.execute_reply.started":"2021-08-25T07:44:10.592226Z","shell.execute_reply":"2021-08-25T07:44:10.859346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir(\"/kaggle/input/\")\n","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.860140Z","iopub.status.idle":"2021-08-25T07:44:10.860750Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.861801Z","iopub.status.idle":"2021-08-25T07:44:10.862614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(\"../input/imaterialist-fashion-2019-FGVC6/label_descriptions.json\") as f:\n    print(\"a\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.863593Z","iopub.status.idle":"2021-08-25T07:44:10.864271Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(DATA_DIR/\"label_descriptions.json\") as f:\n    label_descriptions = json.load(f)\n\nlabel_names = [x['name'] for x in label_descriptions['categories']]","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:45:16.107482Z","iopub.execute_input":"2021-08-25T07:45:16.107999Z","iopub.status.idle":"2021-08-25T07:45:16.119430Z","shell.execute_reply.started":"2021-08-25T07:45:16.107908Z","shell.execute_reply":"2021-08-25T07:45:16.118485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label_descriptions","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.867037Z","iopub.status.idle":"2021-08-25T07:44:10.867785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label_names","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:10.868627Z","iopub.status.idle":"2021-08-25T07:44:10.869382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"segment_df = pd.read_csv(DATA_DIR/\"train.csv\")\n\nmultilabel_percent = len(segment_df[segment_df['ClassId'].str.contains('_')])/len(segment_df)*100\nprint(f\"Segments that have attributes: {multilabel_percent:.2f}%\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:11.517956Z","iopub.execute_input":"2021-08-25T07:44:11.518630Z","iopub.status.idle":"2021-08-25T07:44:38.175477Z","shell.execute_reply.started":"2021-08-25T07:44:11.518577Z","shell.execute_reply":"2021-08-25T07:44:38.173382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"segment_df.tail(100)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:38.176647Z","iopub.execute_input":"2021-08-25T07:44:38.176946Z","iopub.status.idle":"2021-08-25T07:44:38.223701Z","shell.execute_reply.started":"2021-08-25T07:44:38.176899Z","shell.execute_reply":"2021-08-25T07:44:38.222752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Segments that contain attributes are only 3.46% of data, and [according to the host](https://www.kaggle.com/c/imaterialist-fashion-2019-FGVC6/discussion/90643#523135), 80% of images have no attribute. So, in the first step, we can only deal with categories to reduce the complexity of the task.","metadata":{}},{"cell_type":"code","source":"segment_df['CategoryId'] = segment_df['ClassId'].str.split('_').str[0]\n\nprint(\"Total segments: \", len(segment_df))\nsegment_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:38.225052Z","iopub.execute_input":"2021-08-25T07:44:38.225524Z","iopub.status.idle":"2021-08-25T07:44:39.082256Z","shell.execute_reply.started":"2021-08-25T07:44:38.225308Z","shell.execute_reply":"2021-08-25T07:44:39.081481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"segment_df.tail(60)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:39.083425Z","iopub.execute_input":"2021-08-25T07:44:39.083693Z","iopub.status.idle":"2021-08-25T07:44:39.118900Z","shell.execute_reply.started":"2021-08-25T07:44:39.083639Z","shell.execute_reply":"2021-08-25T07:44:39.117964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Rows with the same image are grouped together because the subsequent operations perform in an image level.","metadata":{}},{"cell_type":"code","source":"image_df = segment_df.groupby('ImageId')['EncodedPixels', 'CategoryId'].agg(lambda x: list(x))\nsize_df = segment_df.groupby('ImageId')['Height', 'Width'].mean()\nimage_df = image_df.join(size_df, on='ImageId')\n\nprint(\"Total images: \", len(image_df))\nimage_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:39.120106Z","iopub.execute_input":"2021-08-25T07:44:39.120541Z","iopub.status.idle":"2021-08-25T07:44:47.783232Z","shell.execute_reply.started":"2021-08-25T07:44:39.120358Z","shell.execute_reply":"2021-08-25T07:44:47.782259Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is the custom function that resizes an image.","metadata":{}},{"cell_type":"code","source":"def resize_image(image_path):\n    img = cv2.imread(image_path)\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    img = cv2.resize(img, (IMAGE_SIZE, IMAGE_SIZE), interpolation=cv2.INTER_AREA)  \n    return img","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:47.784555Z","iopub.execute_input":"2021-08-25T07:44:47.784978Z","iopub.status.idle":"2021-08-25T07:44:47.789815Z","shell.execute_reply.started":"2021-08-25T07:44:47.784796Z","shell.execute_reply":"2021-08-25T07:44:47.788895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The crucial part is to create a dataset for this task.","metadata":{}},{"cell_type":"code","source":"class FashionDataset(utils.Dataset):\n\n    def __init__(self, df):\n        super().__init__(self)\n        \n        # Add classes\n        for i, name in enumerate(label_names):\n            self.add_class(\"fashion\", i+1, name)\n        \n        # Add images \n        for i, row in df.iterrows():\n            self.add_image(\"fashion\", \n                           image_id=row.name, \n                           path=str(DATA_DIR/'train'/row.name), \n                           labels=row['CategoryId'],\n                           annotations=row['EncodedPixels'], \n                           height=row['Height'], width=row['Width'])\n\n    def image_reference(self, image_id):\n        info = self.image_info[image_id]\n        return info['path'], [label_names[int(x)] for x in info['labels']]\n    \n    def load_image(self, image_id):\n        return resize_image(self.image_info[image_id]['path'])\n\n    def load_mask(self, image_id):\n        info = self.image_info[image_id]\n                \n        mask = np.zeros((IMAGE_SIZE, IMAGE_SIZE, len(info['annotations'])), dtype=np.uint8)\n        labels = []\n        \n        for m, (annotation, label) in enumerate(zip(info['annotations'], info['labels'])):\n            sub_mask = np.full(info['height']*info['width'], 0, dtype=np.uint8)\n            annotation = [int(x) for x in annotation.split(' ')]\n            \n            for i, start_pixel in enumerate(annotation[::2]):\n                sub_mask[start_pixel: start_pixel+annotation[2*i+1]] = 1\n\n            sub_mask = sub_mask.reshape((info['height'], info['width']), order='F')\n            sub_mask = cv2.resize(sub_mask, (IMAGE_SIZE, IMAGE_SIZE), interpolation=cv2.INTER_NEAREST)\n            \n            mask[:, :, m] = sub_mask\n            labels.append(int(label)+1)\n            \n        return mask, np.array(labels)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:47.791001Z","iopub.execute_input":"2021-08-25T07:44:47.791493Z","iopub.status.idle":"2021-08-25T07:44:47.805657Z","shell.execute_reply.started":"2021-08-25T07:44:47.791444Z","shell.execute_reply":"2021-08-25T07:44:47.804771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's visualize some random images and their masks.","metadata":{}},{"cell_type":"code","source":"image","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:47.806986Z","iopub.execute_input":"2021-08-25T07:44:47.807535Z","iopub.status.idle":"2021-08-25T07:44:47.894675Z","shell.execute_reply.started":"2021-08-25T07:44:47.807346Z","shell.execute_reply":"2021-08-25T07:44:47.830931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:47.831558Z","iopub.status.idle":"2021-08-25T07:44:47.832019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from PIL import Image","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:47.832591Z","iopub.status.idle":"2021-08-25T07:44:47.833039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image.open(path)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:44:47.834036Z","iopub.status.idle":"2021-08-25T07:44:47.834742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for ImageId, width, height in image.values:\n    image_path = os.path.join(IMAGES_TRAIN_DIR, ImageId)\n    (real_width, real_height) = read_image_dimensions(image_path)\n    if real_width != width or real_height!=height:\n        images_with_incorrect_size[ImageId] = (real_width, real_height)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def read_image_dimensions(path):\n    \"returns real width and height\"\n    with Image.open(path) as image:\n        dimensions = image.size\n    return dimensions\n\nimages_with_incorrect_size = {}\nfor ImageId, width, height in image.values:\n    image_path = os.path.join(IMAGES_TRAIN_DIR, ImageId)\n    (real_width, real_height) = read_image_dimensions(image_path)\n    if real_width != width or real_height!=height:\n        images_with_incorrect_size[ImageId] = (real_width, real_height)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T06:46:46.037788Z","iopub.execute_input":"2021-08-25T06:46:46.038170Z","iopub.status.idle":"2021-08-25T06:46:46.187455Z","shell.execute_reply.started":"2021-08-25T06:46:46.038090Z","shell.execute_reply":"2021-08-25T06:46:46.186153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def read_image_dimensions(path):\n    \"returns real width and height\"\n    with Image.open(path) as image:\n        dimensions = image.size\n    return dimensions\n\nimages_with_incorrect_size = {}\nfor ImageId, width, height in image.values:\n    image_path = os.path.join(\"../input/imaterialist-fashion-2019-FGVC6/train\", ImageId)\n    (real_width, real_height) = read_image_dimensions(image_path)\n    if real_width != width or real_height!=height:\n        images_with_incorrect_size[ImageId] = (real_width, real_height)","metadata":{"execution":{"iopub.status.busy":"2021-08-23T08:07:34.226577Z","iopub.status.idle":"2021-08-23T08:07:34.227167Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset = FashionDataset(image_df)\ndataset.prepare()\n","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:45:44.860042Z","iopub.execute_input":"2021-08-25T07:45:44.860344Z","iopub.status.idle":"2021-08-25T07:45:50.976561Z","shell.execute_reply.started":"2021-08-25T07:45:44.860296Z","shell.execute_reply":"2021-08-25T07:45:50.975754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(6):\n    image_id = random.choice(dataset.image_ids)\n    print(dataset.image_reference(image_id))\n    \n    image = dataset.load_image(image_id)\n    mask, class_ids = dataset.load_mask(image_id)\n    visualize.display_top_masks(image, mask, class_ids, dataset.class_names, limit=4)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:45:50.977960Z","iopub.execute_input":"2021-08-25T07:45:50.978442Z","iopub.status.idle":"2021-08-25T07:45:54.616160Z","shell.execute_reply.started":"2021-08-25T07:45:50.978232Z","shell.execute_reply":"2021-08-25T07:45:54.615475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, the data are partitioned into train and validation sets.","metadata":{}},{"cell_type":"code","source":"image_df.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:45:54.617444Z","iopub.execute_input":"2021-08-25T07:45:54.617718Z","iopub.status.idle":"2021-08-25T07:45:54.623471Z","shell.execute_reply.started":"2021-08-25T07:45:54.617670Z","shell.execute_reply":"2021-08-25T07:45:54.622688Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_df=image_df[0:5000]","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:45:54.624707Z","iopub.execute_input":"2021-08-25T07:45:54.625285Z","iopub.status.idle":"2021-08-25T07:45:54.634838Z","shell.execute_reply.started":"2021-08-25T07:45:54.625232Z","shell.execute_reply":"2021-08-25T07:45:54.634111Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_df.shape","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:10.499809Z","iopub.execute_input":"2021-08-25T07:46:10.500093Z","iopub.status.idle":"2021-08-25T07:46:10.505499Z","shell.execute_reply.started":"2021-08-25T07:46:10.500043Z","shell.execute_reply":"2021-08-25T07:46:10.504582Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This code partially supports k-fold training, \n# you can specify the fold to train and the total number of folds here\n\n# # change folds back to 5\nFOLD = 0\nN_FOLDS = 2\n\nkf = KFold(n_splits=N_FOLDS, random_state=42, shuffle=True)\nsplits = kf.split(image_df) # ideally, this should be multilabel stratification\n\ndef get_fold():    \n    for i, (train_index, valid_index) in enumerate(splits):\n        if i == FOLD:\n            return image_df.iloc[train_index], image_df.iloc[valid_index]\n        \ntrain_df, valid_df = get_fold()\n\ntrain_dataset = FashionDataset(train_df)\ntrain_dataset.prepare()\n\nvalid_dataset = FashionDataset(valid_df)\nvalid_dataset.prepare()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:13.953409Z","iopub.execute_input":"2021-08-25T07:46:13.953698Z","iopub.status.idle":"2021-08-25T07:46:14.585583Z","shell.execute_reply.started":"2021-08-25T07:46:13.953649Z","shell.execute_reply":"2021-08-25T07:46:14.584889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's visualize class distributions of the train and validation data.","metadata":{}},{"cell_type":"code","source":"train_segments = np.concatenate(train_df['CategoryId'].values).astype(int)\nprint(\"Total train images: \", len(train_df))\nprint(\"Total train segments: \", len(train_segments))\n\nplt.figure(figsize=(12, 3))\nvalues, counts = np.unique(train_segments, return_counts=True)\nplt.bar(values, counts)\nplt.xticks(values, label_names, rotation='vertical')\nplt.show()\n\nvalid_segments = np.concatenate(valid_df['CategoryId'].values).astype(int)\nprint(\"Total train images: \", len(valid_df))\nprint(\"Total validation segments: \", len(valid_segments))\n\nplt.figure(figsize=(12, 3))\nvalues, counts = np.unique(valid_segments, return_counts=True)\nplt.bar(values, counts)\nplt.xticks(values, label_names, rotation='vertical')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:19.357007Z","iopub.execute_input":"2021-08-25T07:46:19.357317Z","iopub.status.idle":"2021-08-25T07:46:20.572995Z","shell.execute_reply.started":"2021-08-25T07:46:19.357267Z","shell.execute_reply":"2021-08-25T07:46:20.572384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train","metadata":{}},{"cell_type":"code","source":"# Note that any hyperparameters here, such as LR, may still not be optimal\nLR = 1e-4\nEPOCHS = [6, 8]\n\nimport warnings \nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:20.619467Z","iopub.execute_input":"2021-08-25T07:46:20.619687Z","iopub.status.idle":"2021-08-25T07:46:20.623576Z","shell.execute_reply.started":"2021-08-25T07:46:20.619644Z","shell.execute_reply":"2021-08-25T07:46:20.622632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This section creates a Mask R-CNN model and specifies augmentations to be used.","metadata":{}},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:23.172741Z","iopub.execute_input":"2021-08-25T07:46:23.173037Z","iopub.status.idle":"2021-08-25T07:46:23.178635Z","shell.execute_reply.started":"2021-08-25T07:46:23.172988Z","shell.execute_reply":"2021-08-25T07:46:23.177513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir(\"/kaggle/working/Mask_RCNN\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:25.642654Z","iopub.execute_input":"2021-08-25T07:46:25.642948Z","iopub.status.idle":"2021-08-25T07:46:25.649189Z","shell.execute_reply.started":"2021-08-25T07:46:25.642897Z","shell.execute_reply":"2021-08-25T07:46:25.648414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:37:00.889096Z","iopub.execute_input":"2021-08-25T07:37:00.889555Z","iopub.status.idle":"2021-08-25T07:37:01.703968Z","shell.execute_reply.started":"2021-08-25T07:37:00.889498Z","shell.execute_reply":"2021-08-25T07:37:01.702799Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Loading weights for Mask R-CNN model…\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:29:37.644641Z","iopub.execute_input":"2021-08-25T07:29:37.644932Z","iopub.status.idle":"2021-08-25T07:29:37.649126Z","shell.execute_reply.started":"2021-08-25T07:29:37.644891Z","shell.execute_reply":"2021-08-25T07:29:37.648018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"./Mask_RCNN/mask_rcnn_coco.h5","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# model = modellib.MaskRCNN(mode=\"inference\", config=config, model_dir=\"./\")","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:37:44.967249Z","iopub.execute_input":"2021-08-25T07:37:44.967587Z","iopub.status.idle":"2021-08-25T07:37:49.453512Z","shell.execute_reply.started":"2021-08-25T07:37:44.967530Z","shell.execute_reply":"2021-08-25T07:37:49.451736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"./Mask_RCNN/mask_rcnn_coco.h5","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:50.642066Z","iopub.execute_input":"2021-08-25T07:46:50.642373Z","iopub.status.idle":"2021-08-25T07:46:50.649612Z","shell.execute_reply.started":"2021-08-25T07:46:50.642320Z","shell.execute_reply":"2021-08-25T07:46:50.648867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:46:50.855493Z","iopub.execute_input":"2021-08-25T07:46:50.855722Z","iopub.status.idle":"2021-08-25T07:46:50.860765Z","shell.execute_reply.started":"2021-08-25T07:46:50.855679Z","shell.execute_reply":"2021-08-25T07:46:50.860072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = modellib.MaskRCNN(mode='training', config=config, model_dir=\"./\")\n\nmodel.load_weights(COCO_WEIGHTS_PATH, by_name=True, exclude=[\n    'mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask'])","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:47:01.437677Z","iopub.execute_input":"2021-08-25T07:47:01.437968Z","iopub.status.idle":"2021-08-25T07:47:10.701699Z","shell.execute_reply.started":"2021-08-25T07:47:01.437919Z","shell.execute_reply":"2021-08-25T07:47:10.700796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"augmentation = iaa.Sequential([\n    iaa.Fliplr(0.5) # only horizontal flip here\n])","metadata":{"execution":{"iopub.status.busy":"2021-08-15T13:30:46.958773Z","iopub.status.idle":"2021-08-15T13:30:46.959483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First, we train only the heads.","metadata":{}},{"cell_type":"code","source":"%%time\nmodel.train(train_dataset, valid_dataset,\n            learning_rate=LR*2, # train heads with higher lr to speedup learning\n            epochs=EPOCHS[0],\n            layers='heads',\n            augmentation=None)\n\nhistory = model.keras_model.history.history","metadata":{"execution":{"iopub.status.busy":"2021-08-25T07:47:10.703115Z","iopub.execute_input":"2021-08-25T07:47:10.703436Z","iopub.status.idle":"2021-08-25T11:24:20.906271Z","shell.execute_reply.started":"2021-08-25T07:47:10.703388Z","shell.execute_reply":"2021-08-25T11:24:20.900735Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then, all layers are trained.","metadata":{}},{"cell_type":"code","source":"def save(self,save_path):\n    self.keras_model.save(save_path)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:29:09.682780Z","iopub.execute_input":"2021-08-25T11:29:09.683258Z","iopub.status.idle":"2021-08-25T11:29:09.691094Z","shell.execute_reply.started":"2021-08-25T11:29:09.683086Z","shell.execute_reply":"2021-08-25T11:29:09.690280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:29:12.269652Z","iopub.execute_input":"2021-08-25T11:29:12.269935Z","iopub.status.idle":"2021-08-25T11:29:12.293084Z","shell.execute_reply.started":"2021-08-25T11:29:12.269886Z","shell.execute_reply":"2021-08-25T11:29:12.291724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow import keras\nmodel = keras.models.load_model('path/to/location')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nmodel.train(train_dataset, valid_dataset,\n            learning_rate=LR,\n            epochs=EPOCHS[1],\n            layers='all',\n            augmentation=augmentation)\n\nnew_history = model.keras_model.history.history\nfor k in new_history: history[k] = history[k] + new_history[k]","metadata":{"execution":{"iopub.status.busy":"2021-08-08T20:39:07.301103Z","iopub.execute_input":"2021-08-08T20:39:07.301427Z","iopub.status.idle":"2021-08-08T23:33:05.986073Z","shell.execute_reply.started":"2021-08-08T20:39:07.30137Z","shell.execute_reply":"2021-08-08T23:33:05.98431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Afterwards, we reduce LR and train again.","metadata":{}},{"cell_type":"code","source":"%%time\nmodel.train(train_dataset, valid_dataset,\n            learning_rate=LR/5,\n            epochs=EPOCHS[2],\n            layers='all',\n            augmentation=augmentation)\n\nnew_history = model.keras_model.history.history\nfor k in new_history: history[k] = history[k] + new_history[k]","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's visualize training history and choose the best epoch.","metadata":{}},{"cell_type":"code","source":"epochs = range(EPOCHS[-1])\n\nplt.figure(figsize=(18, 6))\n\nplt.subplot(131)\nplt.plot(epochs, history['loss'], label=\"train loss\")\nplt.plot(epochs, history['val_loss'], label=\"valid loss\")\nplt.legend()\nplt.subplot(132)\nplt.plot(epochs, history['mrcnn_class_loss'], label=\"train class loss\")\nplt.plot(epochs, history['val_mrcnn_class_loss'], label=\"valid class loss\")\nplt.legend()\nplt.subplot(133)\nplt.plot(epochs, history['mrcnn_mask_loss'], label=\"train mask loss\")\nplt.plot(epochs, history['val_mrcnn_mask_loss'], label=\"valid mask loss\")\nplt.legend()\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:35:03.797149Z","iopub.execute_input":"2021-08-25T11:35:03.797472Z","iopub.status.idle":"2021-08-25T11:35:04.106286Z","shell.execute_reply.started":"2021-08-25T11:35:03.797421Z","shell.execute_reply":"2021-08-25T11:35:04.104937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"best_epoch = np.argmin(history[\"val_loss\"]) + 1\nprint(\"Best epoch: \", best_epoch)\nprint(\"Valid loss: \", history[\"val_loss\"][best_epoch-1])","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:38:39.155506Z","iopub.execute_input":"2021-08-25T11:38:39.155799Z","iopub.status.idle":"2021-08-25T11:38:39.160862Z","shell.execute_reply.started":"2021-08-25T11:38:39.155750Z","shell.execute_reply":"2021-08-25T11:38:39.160027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict","metadata":{}},{"cell_type":"markdown","source":"The final step is to use our model to predict test data.","metadata":{}},{"cell_type":"code","source":"glob_list = glob.glob(f'/kaggle/working/fashion*/mask_rcnn_fashion_{best_epoch:04d}.h5')\nmodel_path = glob_list[0] if glob_list else ''","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:38:46.247017Z","iopub.execute_input":"2021-08-25T11:38:46.247343Z","iopub.status.idle":"2021-08-25T11:38:46.251883Z","shell.execute_reply.started":"2021-08-25T11:38:46.247291Z","shell.execute_reply":"2021-08-25T11:38:46.251168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This cell defines InferenceConfig and loads the best trained model.","metadata":{}},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:41:37.976373Z","iopub.execute_input":"2021-08-25T11:41:37.976666Z","iopub.status.idle":"2021-08-25T11:41:37.985877Z","shell.execute_reply.started":"2021-08-25T11:41:37.976616Z","shell.execute_reply":"2021-08-25T11:41:37.985144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir('/kaggle/working/Mask_RCNN/fashion20210825T0747/')","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:42:10.273412Z","iopub.execute_input":"2021-08-25T11:42:10.273703Z","iopub.status.idle":"2021-08-25T11:42:10.277530Z","shell.execute_reply.started":"2021-08-25T11:42:10.273654Z","shell.execute_reply":"2021-08-25T11:42:10.276625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:42:16.344287Z","iopub.execute_input":"2021-08-25T11:42:16.344581Z","iopub.status.idle":"2021-08-25T11:42:17.087775Z","shell.execute_reply.started":"2021-08-25T11:42:16.344532Z","shell.execute_reply":"2021-08-25T11:42:17.086839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import shutil\nshutil.make_archive('OUTPUT_NAME', 'zip', '/kaggle/working/Mask_RCNN/fashion20210825T0747/')","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:49:13.430396Z","iopub.execute_input":"2021-08-25T11:49:13.430688Z","iopub.status.idle":"2021-08-25T11:59:53.121246Z","shell.execute_reply.started":"2021-08-25T11:49:13.430636Z","shell.execute_reply":"2021-08-25T11:59:53.118568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class InferenceConfig(FashionConfig):\n    GPU_COUNT = 1\n    IMAGES_PER_GPU = 1\n\ninference_config = InferenceConfig()\n\nmodel = modellib.MaskRCNN(mode='inference', \n                          config=inference_config,\n                          model_dir='./')\n\nassert model_path != '', \"Provide path to trained weights\"\nprint(\"Loading weights from \", model_path)\nmodel.load_weights(model_path, by_name=True)","metadata":{"execution":{"iopub.status.busy":"2021-08-25T11:39:06.728612Z","iopub.execute_input":"2021-08-25T11:39:06.728909Z","iopub.status.idle":"2021-08-25T11:39:09.160890Z","shell.execute_reply.started":"2021-08-25T11:39:06.728856Z","shell.execute_reply":"2021-08-25T11:39:09.159642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then, load the submission data.","metadata":{}},{"cell_type":"code","source":"sample_df = pd.read_csv(DATA_DIR/\"sample_submission.csv\")\nsample_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is the main prediction steps, along with some helper functions.","metadata":{}},{"cell_type":"code","source":"# Convert data to run-length encoding\ndef to_rle(bits):\n    rle = []\n    pos = 0\n    for bit, group in itertools.groupby(bits):\n        group_list = list(group)\n        if bit:\n            rle.extend([pos, sum(group_list)])\n        pos += len(group_list)\n    return rle","metadata":{"execution":{"iopub.status.busy":"2021-08-25T12:13:53.140420Z","iopub.execute_input":"2021-08-25T12:13:53.140713Z","iopub.status.idle":"2021-08-25T12:13:53.145489Z","shell.execute_reply.started":"2021-08-25T12:13:53.140664Z","shell.execute_reply":"2021-08-25T12:13:53.144750Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Since the submission system does not permit overlapped masks, we have to fix them\ndef refine_masks(masks, rois):\n    areas = np.sum(masks.reshape(-1, masks.shape[-1]), axis=0)\n    mask_index = np.argsort(areas)\n    union_mask = np.zeros(masks.shape[:-1], dtype=bool)\n    for m in mask_index:\n        masks[:, :, m] = np.logical_and(masks[:, :, m], np.logical_not(union_mask))\n        union_mask = np.logical_or(masks[:, :, m], union_mask)\n    for m in range(masks.shape[-1]):\n        mask_pos = np.where(masks[:, :, m]==True)\n        if np.any(mask_pos):\n            y1, x1 = np.min(mask_pos, axis=1)\n            y2, x2 = np.max(mask_pos, axis=1)\n            rois[m, :] = [y1, x1, y2, x2]\n    return masks, rois","metadata":{"execution":{"iopub.status.busy":"2021-08-25T12:13:56.719696Z","iopub.execute_input":"2021-08-25T12:13:56.719970Z","iopub.status.idle":"2021-08-25T12:13:56.729249Z","shell.execute_reply.started":"2021-08-25T12:13:56.719919Z","shell.execute_reply":"2021-08-25T12:13:56.728505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"%%time\nsub_list = []\nmissing_count = 0\nfor i, row in tqdm(sample_df.iterrows(), total=len(sample_df)):\n    image = resize_image(str(DATA_DIR/'test'/row['ImageId']))\n    result = model.detect([image])[0]\n    if result['masks'].size > 0:\n        masks, _ = refine_masks(result['masks'], result['rois'])\n        for m in range(masks.shape[-1]):\n            mask = masks[:, :, m].ravel(order='F')\n            rle = to_rle(mask)\n            label = result['class_ids'][m] - 1\n            sub_list.append([row['ImageId'], ' '.join(list(map(str, rle))), label])\n    else:\n        # The system does not allow missing ids, this is an easy way to fill them \n        sub_list.append([row['ImageId'], '1 1', 23])\n        missing_count += 1","metadata":{"execution":{"iopub.status.busy":"2021-08-25T12:14:13.322797Z","iopub.execute_input":"2021-08-25T12:14:13.323096Z","iopub.status.idle":"2021-08-25T12:14:13.351074Z","shell.execute_reply.started":"2021-08-25T12:14:13.323043Z","shell.execute_reply":"2021-08-25T12:14:13.350100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The submission file is created, when all predictions are ready.","metadata":{}},{"cell_type":"code","source":"submission_df = pd.DataFrame(sub_list, columns=sample_df.columns.values)\nprint(\"Total image results: \", submission_df['ImageId'].nunique())\nprint(\"Missing Images: \", missing_count)\nsubmission_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.to_csv(\"submission.csv\", index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally, it's pleasing to visualize the results! Sample images contain both fashion models and predictions from the Mask R-CNN model.","metadata":{}},{"cell_type":"code","source":"for i in range(9):\n    image_id = sample_df.sample()['ImageId'].values[0]\n    image_path = str(DATA_DIR/'test'/image_id)\n    \n    img = cv2.imread(image_path)\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    \n    result = model.detect([resize_image(image_path)])\n    r = result[0]\n    \n    if r['masks'].size > 0:\n        masks = np.zeros((img.shape[0], img.shape[1], r['masks'].shape[-1]), dtype=np.uint8)\n        for m in range(r['masks'].shape[-1]):\n            masks[:, :, m] = cv2.resize(r['masks'][:, :, m].astype('uint8'), \n                                        (img.shape[1], img.shape[0]), interpolation=cv2.INTER_NEAREST)\n        \n        y_scale = img.shape[0]/IMAGE_SIZE\n        x_scale = img.shape[1]/IMAGE_SIZE\n        rois = (r['rois'] * [y_scale, x_scale, y_scale, x_scale]).astype(int)\n        \n        masks, rois = refine_masks(masks, rois)\n    else:\n        masks, rois = r['masks'], r['rois']\n        \n    visualize.display_instances(img, rois, masks, r['class_ids'], \n                                ['bg']+label_names, r['scores'],\n                                title=image_id, figsize=(12, 12))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"My code is largely based on [this Mask-RCNN kernel](https://www.kaggle.com/hmendonca/mask-rcnn-and-coco-transfer-learning-lb-0-155) and borrowed some ideas from [the U-Net Baseline kernel](https://www.kaggle.com/go1dfish/u-net-baseline-by-pytorch-in-fgvc6-resize). So, I would like to thank the kernel authors for sharing insights and programming techniques. Importantly, an image segmentation task can be accomplished with short code and good accuracy thanks to [Matterport's implementation](https://github.com/matterport/Mask_RCNN) and a deep learning line of researches culminating in [Mask R-CNN](https://arxiv.org/abs/1703.06870).\n\nI am sorry that I published this kernel quite late, beyond the halfway of a timeline. I just started working for this competition about a week ago, and to my surprise, the score fell in the range of silver medals at that time. I have no dedicated GPU and no time to further tune the model, so I decided to make this kernel public as a starter guide for anyone who is interested to join this delightful competition.\n\n","metadata":{}},{"cell_type":"code","source":"##<img src='https://i.imgur.com/j6LPLQc.png'>\n","metadata":{"execution":{"iopub.status.busy":"2021-08-25T12:17:25.832509Z","iopub.execute_input":"2021-08-25T12:17:25.832825Z","iopub.status.idle":"2021-08-25T12:17:25.839127Z","shell.execute_reply.started":"2021-08-25T12:17:25.832771Z","shell.execute_reply":"2021-08-25T12:17:25.838193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Hope you guys like this kernel. If there are any bugs, please let me know.\n\nP.S. When clicking 'Submit to Competition' button, I always run into 404 erros, so I have to save a submission file and upload it to the submission page for submitting. The public LB score of this kernel is around **0.07**.","metadata":{}}]}