{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n

","metadata":{}},{"cell_type":"markdown","source":"
\n\n#

YOLOX + YOLOv5 Weighted Boxes Fusion Ensemble 2.o
\n\n
\n\n","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":0.027817,"end_time":"2021-12-05T21:43:15.021008","exception":false,"start_time":"2021-12-05T21:43:14.993191","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### How to ensemble object detection models?\nThis notebook shows how to detect starfish objects (COTS dataset) using multiple models on Kaggle (YOLOX & YOLOv5 for example). \n\n
\n\n**Original Notebooks Credits:**\n\nDo upvote this notebooks: \n\nOriginal Notebook: [Great-Barrier-Reef: YOLOX + YOLOv5 Ensemble](https://www.kaggle.com/yamqwe/great-barrier-reef-yolox-yolov5-ensemble)\n\nI ensemble the models from [YoloX full training pipeline for COTS dataset](https://www.kaggle.com/remekkinas/yolox-full-training-pipeline-for-cots-dataset) and [Great-Barrier-Reef: YOLOv5 [infer]](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer).\n\n
","metadata":{}},{"cell_type":"markdown","source":"## Goal of this notebook:\n### Fixing the overlapping issue in orignal notebook and improve the score\n

\n","metadata":{}},{"cell_type":"code","source":"import warnings; warnings.filterwarnings(\"ignore\")\n\nimport os\nimport cv2\nimport ast\nimport sys\nimport glob\nimport torch\nimport shutil\nimport importlib\nimport traceback\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nfrom tqdm.notebook import tqdm\nimport matplotlib.pyplot as plt\nsys.path.append('../input/tensorflow-great-barrier-reef')\ntqdm.pandas()\n\nfrom PIL import Image\nfrom IPython.display import display","metadata":{"papermill":{"duration":1.669635,"end_time":"2021-12-05T21:43:16.721911","exception":false,"start_time":"2021-12-05T21:43:15.052276","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:40:15.007553Z","iopub.execute_input":"2022-01-25T05:40:15.007887Z","iopub.status.idle":"2022-01-25T05:40:16.574306Z","shell.execute_reply.started":"2022-01-25T05:40:15.007785Z","shell.execute_reply":"2022-01-25T05:40:16.573567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Installing YOLOX \n

Some Kaggle enviroment hacking :) due to competition limitation - no internet access during submission.
","metadata":{"papermill":{"duration":0.014038,"end_time":"2021-12-05T21:43:16.750638","exception":false,"start_time":"2021-12-05T21:43:16.7366","status":"completed"},"tags":[]}},{"cell_type":"code","source":"%cp -r /kaggle/input/yolox-cots-models /kaggle/working/\n%cd /kaggle/working/yolox-cots-models/yolox-dep","metadata":{"papermill":{"duration":17.792999,"end_time":"2021-12-05T21:43:34.592784","exception":false,"start_time":"2021-12-05T21:43:16.799785","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:40:16.576086Z","iopub.execute_input":"2022-01-25T05:40:16.576348Z","iopub.status.idle":"2022-01-25T05:40:41.518154Z","shell.execute_reply.started":"2022-01-25T05:40:16.576315Z","shell.execute_reply":"2022-01-25T05:40:41.517266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Install YOLOX required modules\n!pip install pip-21.3.1-py3-none-any.whl -f ./ --no-index\n!pip install loguru-0.5.3-py3-none-any.whl -f ./ --no-index\n!pip install ninja-1.10.2.3-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl -f ./ --no-index\n!pip install onnx-1.8.1-cp37-cp37m-manylinux2010_x86_64.whl -f ./ --no-index\n!pip install onnxruntime-1.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl -f ./ --no-index\n!pip install onnxoptimizer-0.2.6-cp37-cp37m-manylinux2014_x86_64.whl -f ./ --no-index\n!pip install thop-0.0.31.post2005241907-py3-none-any.whl -f ./ --no-index\n!pip install tabulate-0.8.9-py3-none-any.whl -f ./ --no-index","metadata":{"_kg_hide-output":true,"papermill":{"duration":74.488225,"end_time":"2021-12-05T21:44:49.096558","exception":false,"start_time":"2021-12-05T21:43:34.608333","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-25T05:40:41.519891Z","iopub.execute_input":"2022-01-25T05:40:41.520341Z","iopub.status.idle":"2022-01-25T05:41:56.237577Z","shell.execute_reply.started":"2022-01-25T05:40:41.5203Z","shell.execute_reply":"2022-01-25T05:41:56.236725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Install YOLOX\n%cd /kaggle/working/yolox-cots-models/YOLOX\n!pip install -r requirements.txt\n!pip install -v -e . ","metadata":{"_kg_hide-output":true,"papermill":{"duration":74.641611,"end_time":"2021-12-05T21:46:03.760925","exception":false,"start_time":"2021-12-05T21:44:49.119314","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:41:56.241509Z","iopub.execute_input":"2022-01-25T05:41:56.241738Z","iopub.status.idle":"2022-01-25T05:43:11.166431Z","shell.execute_reply.started":"2022-01-25T05:41:56.241711Z","shell.execute_reply":"2022-01-25T05:43:11.165496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Install CocoAPI tool\n%cd /kaggle/working/yolox-cots-models/yolox-dep/cocoapi/PythonAPI\n!make\n!make install\n!python setup.py install","metadata":{"_kg_hide-output":true,"papermill":{"duration":18.265912,"end_time":"2021-12-05T21:46:22.056603","exception":false,"start_time":"2021-12-05T21:46:03.790691","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-25T05:43:11.171297Z","iopub.execute_input":"2022-01-25T05:43:11.171673Z","iopub.status.idle":"2022-01-25T05:43:30.246067Z","shell.execute_reply.started":"2022-01-25T05:43:11.171636Z","shell.execute_reply":"2022-01-25T05:43:30.24521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pycocotools","metadata":{"papermill":{"duration":0.045381,"end_time":"2021-12-05T21:46:22.141156","exception":false,"start_time":"2021-12-05T21:46:22.095775","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:43:30.247754Z","iopub.execute_input":"2022-01-25T05:43:30.248642Z","iopub.status.idle":"2022-01-25T05:43:30.255245Z","shell.execute_reply.started":"2022-01-25T05:43:30.248601Z","shell.execute_reply":"2022-01-25T05:43:30.254536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test Model","metadata":{"papermill":{"duration":0.036659,"end_time":"2021-12-05T21:46:22.214827","exception":false,"start_time":"2021-12-05T21:46:22.178168","status":"completed"},"tags":[]}},{"cell_type":"code","source":"%cd /kaggle/working/yolox-cots-models/YOLOX\n\nCHECKPOINT_FILE = '/kaggle/working/yolox-cots-models/yx_l_003.pth'","metadata":{"papermill":{"duration":0.046709,"end_time":"2021-12-05T21:46:22.298652","exception":false,"start_time":"2021-12-05T21:46:22.251943","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:43:30.256639Z","iopub.execute_input":"2022-01-25T05:43:30.257174Z","iopub.status.idle":"2022-01-25T05:43:30.271751Z","shell.execute_reply.started":"2022-01-25T05:43:30.257137Z","shell.execute_reply":"2022-01-25T05:43:30.270773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"config_file_template = '''\n\n#!/usr/bin/env python3\n# -*- coding:utf-8 -*-\n# Copyright (c) Megvii, Inc. and its affiliates.\n\nimport os\n\nfrom yolox.exp import Exp as MyExp\n\n\nclass Exp(MyExp):\n def __init__(self):\n super(Exp, self).__init__()\n self.depth = 1\n self.width = 1\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(\".\")[0]\n self.num_classes = 1\n\n'''\n\nwith open('cots_config.py', 'w') as f:\n f.write(config_file_template)","metadata":{"papermill":{"duration":0.044275,"end_time":"2021-12-05T21:46:22.380801","exception":false,"start_time":"2021-12-05T21:46:22.336526","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:43:30.273539Z","iopub.execute_input":"2022-01-25T05:43:30.274052Z","iopub.status.idle":"2022-01-25T05:43:30.282644Z","shell.execute_reply.started":"2022-01-25T05:43:30.274015Z","shell.execute_reply":"2022-01-25T05:43:30.281878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from yolox.utils import postprocess\nfrom yolox.data.data_augment import ValTransform\n\nCOCO_CLASSES = (\n \"starfish\",\n)\n\n# get YOLOX experiment\ncurrent_exp = importlib.import_module('cots_config')\nexp = current_exp.Exp()\n\n# set inference parameters\ntest_size = (800, 1280)\nnum_classes = 1\nconfthre = 0.2\nnmsthre = 0.2\n\n\n# get YOLOX model\nyolox_model = exp.get_model()\nyolox_model.cuda()\nyolox_model.eval()\n\n# get custom trained checkpoint\nckpt_file = CHECKPOINT_FILE\nckpt = torch.load(ckpt_file, map_location=\"cpu\")\nyolox_model.load_state_dict(ckpt[\"model\"])","metadata":{"papermill":{"duration":4.184097,"end_time":"2021-12-05T21:46:26.601756","exception":false,"start_time":"2021-12-05T21:46:22.417659","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:43:30.284303Z","iopub.execute_input":"2022-01-25T05:43:30.284839Z","iopub.status.idle":"2022-01-25T05:43:34.462393Z","shell.execute_reply.started":"2022-01-25T05:43:30.284786Z","shell.execute_reply":"2022-01-25T05:43:34.461586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def yolox_inference(img, model, test_size): \n bboxes = []\n bbclasses = []\n scores = []\n \n preproc = ValTransform(legacy = False)\n\n tensor_img, _ = preproc(img, None, test_size)\n tensor_img = torch.from_numpy(tensor_img).unsqueeze(0)\n tensor_img = tensor_img.float()\n tensor_img = tensor_img.cuda()\n\n with torch.no_grad():\n outputs = model(tensor_img)\n outputs = postprocess(\n outputs, num_classes, confthre,\n nmsthre, class_agnostic=True\n )\n\n if outputs[0] is None:\n return [], [], []\n \n outputs = outputs[0].cpu()\n bboxes = outputs[:, 0:4]\n\n bboxes /= min(test_size[0] / img.shape[0], test_size[1] / img.shape[1])\n bbclasses = outputs[:, 6]\n scores = outputs[:, 4] * outputs[:, 5]\n \n return bboxes, bbclasses, scores","metadata":{"papermill":{"duration":0.051006,"end_time":"2021-12-05T21:46:26.690736","exception":false,"start_time":"2021-12-05T21:46:26.63973","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:43:34.465665Z","iopub.execute_input":"2022-01-25T05:43:34.465918Z","iopub.status.idle":"2022-01-25T05:43:34.475279Z","shell.execute_reply.started":"2022-01-25T05:43:34.465891Z","shell.execute_reply":"2022-01-25T05:43:34.474388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def draw_yolox_predictions(img, bboxes, scores, bbclasses, confthre, classes_dict):\n for i in range(len(bboxes)):\n box = bboxes[i]\n cls_id = int(bbclasses[i])\n score = scores[i]\n if score < confthre:\n continue\n x0 = int(box[0])\n y0 = int(box[1])\n x1 = int(box[2])\n y1 = int(box[3])\n\n cv2.rectangle(img, (x0, y0), (x1, y1), (0, 255, 0), 2)\n cv2.putText(img, '{}:{:.1f}%'.format(classes_dict[cls_id], score * 100), (x0, y0 - 3), cv2.FONT_HERSHEY_PLAIN, 0.8, (0,255,0), thickness = 1)\n return img","metadata":{"papermill":{"duration":0.046822,"end_time":"2021-12-05T21:46:26.774811","exception":false,"start_time":"2021-12-05T21:46:26.727989","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:43:34.476785Z","iopub.execute_input":"2022-01-25T05:43:34.477131Z","iopub.status.idle":"2022-01-25T05:43:34.486054Z","shell.execute_reply.started":"2022-01-25T05:43:34.477092Z","shell.execute_reply":"2022-01-25T05:43:34.48535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# YOLOv5 Model","metadata":{}},{"cell_type":"code","source":"CKPT_PATH = '/kaggle/input/reef-baseline-fold12/l6_3600_uflip_vm5_f12_up/f1/best.pt'\nIMG_SIZE = 9000\nCONF = 0.35\nIOU = 0.20\nAUGMENT = True","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:34.487474Z","iopub.execute_input":"2022-01-25T05:43:34.487975Z","iopub.status.idle":"2022-01-25T05:43:34.498317Z","shell.execute_reply.started":"2022-01-25T05:43:34.487935Z","shell.execute_reply":"2022-01-25T05:43:34.497379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_path(row):\n row['image_path'] = f'{ROOT_DIR}/train_images/video_{row.video_id}/{row.video_frame}.jpg'\n return row","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:34.49961Z","iopub.execute_input":"2022-01-25T05:43:34.500444Z","iopub.status.idle":"2022-01-25T05:43:34.507055Z","shell.execute_reply.started":"2022-01-25T05:43:34.500405Z","shell.execute_reply":"2022-01-25T05:43:34.506339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train Data\nROOT_DIR = '/kaggle/input/tensorflow-great-barrier-reef/'\ndf = pd.read_csv(f'{ROOT_DIR}/train.csv')\ndf = df.progress_apply(get_path, axis=1)\ndf['annotations'] = df['annotations'].progress_apply(lambda x: ast.literal_eval(x))","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:34.508691Z","iopub.execute_input":"2022-01-25T05:43:34.509479Z","iopub.status.idle":"2022-01-25T05:43:50.83592Z","shell.execute_reply.started":"2022-01-25T05:43:34.50944Z","shell.execute_reply":"2022-01-25T05:43:50.834109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['num_bbox'] = df['annotations'].progress_apply(lambda x: len(x))\ndata = (df.num_bbox>0).value_counts()/len(df)*100","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:50.837118Z","iopub.execute_input":"2022-01-25T05:43:50.837679Z","iopub.status.idle":"2022-01-25T05:43:50.937709Z","shell.execute_reply.started":"2022-01-25T05:43:50.837639Z","shell.execute_reply":"2022-01-25T05:43:50.93689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def voc2yolo(bboxes, image_height=720, image_width=1280):\n \"\"\"\n voc => [x1, y1, x2, y1]\n yolo => [xmid, ymid, w, h] (normalized)\n \"\"\"\n \n bboxes = bboxes.copy().astype(float) # otherwise all value will be 0 as voc_pascal dtype is np.int\n \n bboxes[..., [0, 2]] = bboxes[..., [0, 2]]/ image_width\n bboxes[..., [1, 3]] = bboxes[..., [1, 3]]/ image_height\n \n w = bboxes[..., 2] - bboxes[..., 0]\n h = bboxes[..., 3] - bboxes[..., 1]\n \n bboxes[..., 0] = bboxes[..., 0] + w/2\n bboxes[..., 1] = bboxes[..., 1] + h/2\n bboxes[..., 2] = w\n bboxes[..., 3] = h\n \n return bboxes\n\ndef yolo2voc(bboxes, image_height=720, image_width=1280):\n \"\"\"\n yolo => [xmid, ymid, w, h] (normalized)\n voc => [x1, y1, x2, y1]\n \n \"\"\" \n bboxes = bboxes.copy().astype(float) # otherwise all value will be 0 as voc_pascal dtype is np.int\n \n bboxes[..., [0, 2]] = bboxes[..., [0, 2]]* image_width\n bboxes[..., [1, 3]] = bboxes[..., [1, 3]]* image_height\n \n bboxes[..., [0, 1]] = bboxes[..., [0, 1]] - bboxes[..., [2, 3]]/2\n bboxes[..., [2, 3]] = bboxes[..., [0, 1]] + bboxes[..., [2, 3]]\n \n return bboxes\n\ndef coco2yolo(bboxes, image_height=720, image_width=1280):\n \"\"\"\n coco => [xmin, ymin, w, h]\n yolo => [xmid, ymid, w, h] (normalized)\n \"\"\"\n \n bboxes = bboxes.copy().astype(float) # otherwise all value will be 0 as voc_pascal dtype is np.int\n \n # normolizinig\n bboxes[..., [0, 2]]= bboxes[..., [0, 2]]/ image_width\n bboxes[..., [1, 3]]= bboxes[..., [1, 3]]/ image_height\n \n # converstion (xmin, ymin) => (xmid, ymid)\n bboxes[..., [0, 1]] = bboxes[..., [0, 1]] + bboxes[..., [2, 3]]/2\n \n return bboxes\n\ndef yolo2coco(bboxes, image_height=720, image_width=1280):\n \"\"\"\n yolo => [xmid, ymid, w, h] (normalized)\n coco => [xmin, ymin, w, h]\n \n \"\"\" \n bboxes = bboxes.copy().astype(float) # otherwise all value will be 0 as voc_pascal dtype is np.int\n \n # denormalizing\n bboxes[..., [0, 2]]= bboxes[..., [0, 2]]* image_width\n bboxes[..., [1, 3]]= bboxes[..., [1, 3]]* image_height\n \n # converstion (xmid, ymid) => (xmin, ymin) \n bboxes[..., [0, 1]] = bboxes[..., [0, 1]] - bboxes[..., [2, 3]]/2\n \n return bboxes\n\ndef voc2coco(bboxes, image_height=720, image_width=1280):\n bboxes = voc2yolo(bboxes, image_height, image_width)\n bboxes = yolo2coco(bboxes, image_height, image_width)\n return bboxes\n\ndef coco2voc(bboxes, image_height=720, image_width=1280):\n bboxes = coco2yolo(bboxes, image_height, image_width)\n bboxes = yolo2voc(bboxes, image_height, image_width)\n return bboxes\n\ndef load_image(image_path):\n return cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)\n\n\ndef plot_one_box(x, img, color=None, label=None, line_thickness=None,score=None):\n # Plots one bounding box on image img\n tl = line_thickness or round(0.002 * (img.shape[0] + img.shape[1]) / 2) + 1 # line/font thickness\n color = color or [random.randint(0, 255) for _ in range(3)]\n c1, c2 = (int(x[0]), int(x[1])), (int(x[2]), int(x[3]))\n cv2.rectangle(img, c1, c2, color, thickness=tl, lineType=cv2.LINE_AA)\n label=label+\"{:.2f}%\".format(score)\n if label:\n tf = max(tl - 1, 1) # font thickness\n t_size = cv2.getTextSize(label, 0, fontScale=tl / 3, thickness=tf)[0]\n c2 = c1[0] + t_size[0], c1[1] - t_size[1] - 3\n cv2.rectangle(img, c1, c2, color, -1, cv2.LINE_AA) # filled\n cv2.putText(img, label, (c1[0], c1[1] - 2), 0, tl / 3, [225, 255, 255], thickness=tf, lineType=cv2.LINE_AA)\n\ndef draw_bboxes(img, bboxes, classes, class_ids, colors = None, show_classes = None, bbox_format = 'yolo', class_name = False, line_thickness = 2,scores=None): \n \n image = img.copy()\n show_classes = classes if show_classes is None else show_classes\n colors = (0, 255 ,0) if colors is None else colors\n \n if bbox_format == 'yolo':\n \n for idx in range(len(bboxes)): \n \n bbox = bboxes[idx]\n cls = classes[idx]\n score = scores[idx]\n cls_id = class_ids[idx]\n color = colors[cls_id] if type(colors) is list else colors\n \n if cls in show_classes:\n \n x1 = round(float(bbox[0])*image.shape[1])\n y1 = round(float(bbox[1])*image.shape[0])\n w = round(float(bbox[2])*image.shape[1]/2) #w/2 \n h = round(float(bbox[3])*image.shape[0]/2)\n\n voc_bbox = (x1-w, y1-h, x1+w, y1+h)\n plot_one_box(voc_bbox, \n image,\n color = color,\n label = cls if class_name else str(get_label(cls)),\n line_thickness = line_thickness,score=score)\n \n elif bbox_format == 'coco':\n \n for idx in range(len(bboxes)): \n \n bbox = bboxes[idx]\n cls = classes[idx]\n cls_id = class_ids[idx]\n score = scores[idx]\n color = colors[cls_id] if type(colors) is list else colors\n \n if cls in show_classes: \n x1 = int(round(bbox[0]))\n y1 = int(round(bbox[1]))\n w = int(round(bbox[2]))\n h = int(round(bbox[3]))\n\n voc_bbox = (x1, y1, x1+w, y1+h)\n plot_one_box(voc_bbox, \n image,\n color = color,\n label = cls if class_name else str(cls_id),\n line_thickness = line_thickness,score=score)\n\n elif bbox_format == 'voc_pascal':\n \n for idx in range(len(bboxes)): \n \n bbox = bboxes[idx]\n cls = classes[idx]\n cls_id = class_ids[idx]\n score = scores[idx]\n color = colors[cls_id] if type(colors) is list else colors\n \n if cls in show_classes: \n x1 = int(round(bbox[0]))\n y1 = int(round(bbox[1]))\n x2 = int(round(bbox[2]))\n y2 = int(round(bbox[3]))\n voc_bbox = (x1, y1, x2, y2)\n plot_one_box(voc_bbox, \n image,\n color = color,\n label = cls if class_name else str(cls_id),\n line_thickness = line_thickness,score=score)\n else:\n raise ValueError('wrong bbox format')\n\n return image\n\ndef get_bbox(annots):\n bboxes = [list(annot.values()) for annot in annots]\n return bboxes\n\ndef get_imgsize(row):\n row['width'], row['height'] = imagesize.get(row['image_path'])\n return row\n\nnp.random.seed(32)\ncolors = [(np.random.randint(255), np.random.randint(255), np.random.randint(255))\\\n for idx in range(1)]","metadata":{"pycharm":{"name":"#%%\n"},"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-25T05:43:50.93941Z","iopub.execute_input":"2022-01-25T05:43:50.939681Z","iopub.status.idle":"2022-01-25T05:43:50.981904Z","shell.execute_reply.started":"2022-01-25T05:43:50.939644Z","shell.execute_reply":"2022-01-25T05:43:50.981062Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mkdir -p /root/.config/Ultralytics\n!cp /kaggle/input/yolov5-font/Arial.ttf /root/.config/Ultralytics/","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:50.983319Z","iopub.execute_input":"2022-01-25T05:43:50.983744Z","iopub.status.idle":"2022-01-25T05:43:51.305927Z","shell.execute_reply.started":"2022-01-25T05:43:50.983706Z","shell.execute_reply":"2022-01-25T05:43:51.304934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def predict(model, img, size=768, augment=False):\n height, width = img.shape[:2]\n results = model(img, size=size, augment=augment) # custom inference size\n preds = results.pandas().xyxy[0]\n bboxes = preds[['xmin','ymin','xmax','ymax']].values\n if len(bboxes):\n bboxes = voc2coco(bboxes,height,width).astype(int)\n confs = preds.confidence.values\n return bboxes, confs\n else:\n return [],[]\n\ndef format_prediction(bboxes, confs):\n annot = ''\n if len(bboxes)>0:\n for idx in range(len(bboxes)):\n xmin, ymin, w, h = bboxes[idx]\n conf = confs[idx]\n annot += f'{conf} {xmin} {ymin} {w} {h}'\n annot +=' '\n annot = annot.strip(' ')\n return annot\n\n\ndef show_img(img, bboxes, bbox_format='yolo',scores=None):\n names = ['starfish']*len(bboxes)\n labels = [0]*len(bboxes)\n img = draw_bboxes(img = img,\n bboxes = bboxes, \n classes = names,\n class_ids = labels,\n class_name = True, \n colors = colors, \n bbox_format = bbox_format,\n line_thickness = 2,scores=scores)\n return Image.fromarray(img).resize((800, 400))","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:51.308345Z","iopub.execute_input":"2022-01-25T05:43:51.308901Z","iopub.status.idle":"2022-01-25T05:43:51.321118Z","shell.execute_reply.started":"2022-01-25T05:43:51.308858Z","shell.execute_reply":"2022-01-25T05:43:51.320162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_model(ckpt_path, conf=0.15, iou=0.2):\n model = torch.hub.load('/kaggle/input/yolov5-lib-ds','custom',path=ckpt_path,\n source='local',force_reload=True) # local repo\n model.conf = conf # NMS confidence threshold\n model.iou = iou # NMS IoU threshold\n model.classes = None # (optional list) filter by class, i.e. = [0, 15, 16] for persons, cats and dogs\n model.multi_label = False # NMS multiple labels per box\n model.max_det = 1000 # maximum number of detections per image\n return model","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:51.322758Z","iopub.execute_input":"2022-01-25T05:43:51.323263Z","iopub.status.idle":"2022-01-25T05:43:51.336528Z","shell.execute_reply.started":"2022-01-25T05:43:51.323224Z","shell.execute_reply":"2022-01-25T05:43:51.335654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Ensembling","metadata":{}},{"cell_type":"markdown","source":"## Let's look at some train images!","metadata":{}},{"cell_type":"code","source":"import sys; sys.path.append('/kaggle/input/weightedboxesfusion/')\n# !pip install --no-deps '/kaggle/input/weightedboxesfusion/' > /dev/null","metadata":{"execution":{"iopub.status.busy":"2022-01-25T05:43:51.337804Z","iopub.execute_input":"2022-01-25T05:43:51.338141Z","iopub.status.idle":"2022-01-25T05:43:51.345426Z","shell.execute_reply.started":"2022-01-25T05:43:51.338101Z","shell.execute_reply":"2022-01-25T05:43:51.344625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TEST_IMAGE_PATH = \"/kaggle/input/tensorflow-great-barrier-reef/train_images/video_0/9674.jpg\"\n#import torchvision.ops.boxes as bops\nfrom ensemble_boxes import *\n\ndef run_wbf(bboxes, confs, image_size=1280, iou_thr=0.2, skip_box_thr=0.001, weights=None):\n boxes = [bbox/(image_size) for bbox in bboxes]\n \n scores = [conf for conf in confs]\n labels = [np.ones(conf.shape[0]) for conf in confs]\n \n boxes, scores, labels = weighted_boxes_fusion(boxes, scores, labels, weights=[1,1], iou_thr=iou_thr, skip_box_thr=skip_box_thr)\n \n boxes = boxes*(image_size-1)\n boxes=voc2coco(boxes).astype(int)\n #print(\"voc:coco\",boxes)\n return boxes, scores, labels\n\nmodel = load_model(CKPT_PATH, conf=CONF, iou=IOU)\nimage_paths = df[df.num_bbox>1].sample(200).image_path.tolist()\nfor idx, path in enumerate(image_paths):\n img = cv2.imread(path)[...,::-1]\n \n bboxes_1, bbclasses, scores = yolox_inference(img[...,::-1], yolox_model, test_size) \n bboxes_2, confis = predict(model, img, size=IMG_SIZE, augment=AUGMENT) \n #pred_1, pred_2 = voc2coco(bboxes_1.detach().numpy()).astype(int), bboxes_2\n \n if len(bboxes_1) > 0 and len(bboxes_2) > 0: \n pred_1, pred_2 = bboxes_1.detach().numpy(), coco2voc(bboxes_2).astype(int)\n boxes, scores1, labels = run_wbf([pred_1, pred_2], [scores, confis], image_size = 1280)\n elif len(bboxes_1) > 0: boxes, scores1 = bboxes_1.detach().numpy(), scores\n elif len(bboxes_2) > 0: boxes, scores1 = voc2coco(bboxes_2,img.shape[1],img.shape[2]).astype(int), confis\n \n if len(bboxes_1) > 0: \n print('\\n\\nYOLOX Predictions: ')\n display(show_img(img, voc2coco(bboxes_1.detach().numpy(),img.shape[1],img.shape[2]).astype(int), bbox_format='coco',scores=scores))\n else: \n print('\\n\\nYOLOX Predictions: ')\n display(show_img(img, [], bbox_format='coco',scores=scores))\n if len(bboxes_2) > 0:\n print('\\n\\nYoloV5 Predictions: ')\n display(show_img(img, bboxes_2, bbox_format='coco',scores=confis))\n else: \n print('\\n\\nYOLOV5 Predictions: ')\n display(show_img(img, [], bbox_format='coco',scores=confis))\n if len(bboxes_1) > 0 and len(bboxes_2) > 0: \n print('\\n\\nEnsemble (WBF) Predictions: ')\n display(show_img(img, boxes, bbox_format='coco',scores=scores1))\n else: \n print('\\n\\nEnsemble (WBF) Predictions: ')\n display(show_img(img, [], bbox_format='coco',scores=scores1))\n \n if idx>5:\n break","metadata":{"collapsed":false,"pycharm":{"name":"#%%\n"},"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-01-25T05:43:51.346972Z","iopub.execute_input":"2022-01-25T05:43:51.347302Z","iopub.status.idle":"2022-01-25T05:44:09.87696Z","shell.execute_reply.started":"2022-01-25T05:43:51.347267Z","shell.execute_reply":"2022-01-25T05:44:09.876081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## SUBMIT PREDICTION TO COMPETITION","metadata":{"papermill":{"duration":0.095501,"end_time":"2021-12-05T21:46:32.303063","exception":false,"start_time":"2021-12-05T21:46:32.207562","status":"completed"},"tags":[]}},{"cell_type":"code","source":"%cd /kaggle/working/","metadata":{"papermill":{"duration":0.104486,"end_time":"2021-12-05T21:46:32.501117","exception":false,"start_time":"2021-12-05T21:46:32.396631","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:44:09.878644Z","iopub.execute_input":"2022-01-25T05:44:09.879205Z","iopub.status.idle":"2022-01-25T05:44:09.889585Z","shell.execute_reply.started":"2022-01-25T05:44:09.879161Z","shell.execute_reply":"2022-01-25T05:44:09.888867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import greatbarrierreef\n\nenv = greatbarrierreef.make_env() # initialize the environment\niter_test = env.iter_test() ","metadata":{"papermill":{"duration":0.133428,"end_time":"2021-12-05T21:46:32.729017","exception":false,"start_time":"2021-12-05T21:46:32.595589","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:44:09.890889Z","iopub.execute_input":"2022-01-25T05:44:09.89162Z","iopub.status.idle":"2022-01-25T05:44:09.937188Z","shell.execute_reply.started":"2022-01-25T05:44:09.891582Z","shell.execute_reply":"2022-01-25T05:44:09.936382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_dict = {\n 'id': [],\n 'prediction_string': [],\n}\n\nfor (image_np, sample_prediction_df) in iter_test:\n bboxes_1, bbclasses, scores = yolox_inference(image_np[...,::-1], yolox_model, test_size)\n bboxes_2, confis = predict(model, image_np, size=IMG_SIZE, augment=AUGMENT) \n\n if len(bboxes_1) > 0 and len(bboxes_2) > 0: \n pred_1, pred_2 = bboxes_1.detach().numpy(), coco2voc(bboxes_2).astype(int)\n boxes, scores1, labels = run_wbf([pred_1, pred_2], [scores, confis], image_size = 1280)\n elif len(bboxes_1) > 0: boxes, scores1 = voc2coco(bboxes_1.detach().numpy()).astype(int), scores\n elif len(bboxes_2) > 0: boxes, scores1 = bboxes_2, confis\n else: boxes = []\n # display(show_img(image_np, bboxes, bbox_format='coco'))\n predictions = []\n for i in range(len(boxes)):\n box = boxes[i] \n score = scores1[i]\n if score > 0.1:\n x_min = int(box[0])\n y_min = int(box[1])\n bbox_width = int(box[2])\n bbox_height = int(box[3])\n predictions.append('{:.2f} {} {} {} {}'.format(score, x_min, y_min, bbox_width, bbox_height))\n\n prediction_str = ' '.join(predictions)\n sample_prediction_df['annotations'] = prediction_str\n env.predict(sample_prediction_df)\n\n print('Prediction:', prediction_str)","metadata":{"papermill":{"duration":0.674293,"end_time":"2021-12-05T21:46:33.496659","exception":false,"start_time":"2021-12-05T21:46:32.822366","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:44:09.938776Z","iopub.execute_input":"2022-01-25T05:44:09.93917Z","iopub.status.idle":"2022-01-25T05:44:13.543394Z","shell.execute_reply.started":"2022-01-25T05:44:09.939131Z","shell.execute_reply":"2022-01-25T05:44:13.542361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df = pd.read_csv('submission.csv')\nsub_df.head()","metadata":{"papermill":{"duration":0.1231,"end_time":"2021-12-05T21:46:33.726446","exception":false,"start_time":"2021-12-05T21:46:33.603346","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-01-25T05:44:13.545153Z","iopub.execute_input":"2022-01-25T05:44:13.54548Z","iopub.status.idle":"2022-01-25T05:44:13.561796Z","shell.execute_reply.started":"2022-01-25T05:44:13.545438Z","shell.execute_reply":"2022-01-25T05:44:13.560798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

This notebook is just bug fixing ⚙🛠!

\n

If you find this notebook useful, please do Upvote to Original Notebook 🙏:)

\n\n#

\n\n

Tip: Add tracking to same notebook will boost score!...

","metadata":{"papermill":{"duration":0.103554,"end_time":"2021-12-05T21:46:33.93281","exception":false,"start_time":"2021-12-05T21:46:33.829256","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"
\n\n**Original Notebooks Credits:**\n\n[Great-Barrier-Reef: YOLOX + YOLOv5 Ensemble](https://www.kaggle.com/yamqwe/great-barrier-reef-yolox-yolov5-ensemble)\n\n[YoloX full training pipeline for COTS dataset](https://www.kaggle.com/remekkinas/yolox-full-training-pipeline-for-cots-dataset)\n\n[Great-Barrier-Reef: YOLOv5 [infer]](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer).\n\n
","metadata":{}}]}