{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Inference YoloR on COTS dataset (PART 2) - as simple as possible to help people start with YoloR and develop object detection solutions on Kaggle\n\nThis notebook introduces YOLOR on Kaggle and TensorFlow - Help Protect the Great Barrier Reef competition. It shows how to make inference using custom YoloR object detection model (COTS dataset). It could be good starting point for build own custom model based on YoloR detector and make object detection. Full github repository you can find here - [YOLOR](https://github.com/WongKinYiu/yolor). First part - TRAINING is here: https://www.kaggle.com/remekkinas/yolor-p6-w6-one-more-yolo-on-kaggle-train.\n\nSteps covered in this INFERENCE notebook:\n\n* Install modules\n* Run YoloR inference on test images\n* Submit to competition\n\n<div class=\"alert alert-warning\">I found that there is no reference custom YoloR model training and inference notebook on Kaggle. Since we have such an opportunity this is my contribution to this competition. Feel free to use it and enjoy! I really appreciate if you upvote this notebook. Thank you!</div>\n\n<div class=\"alert alert-success\" role=\"alert\">\nI introduced YoloX in TensorFlow - Help Protect the Great Barrier Reef competition as well. You can find these notebooks here:      \n    <ul>\n        <li> <a href=\"https://www.kaggle.com/remekkinas/yolox-full-training-pipeline-for-cots-dataset\">YoloX full training pipeline for COTS dataset</a></li>\n        <li> <a href=\"https://www.kaggle.com/remekkinas/yolox-inference-on-kaggle-for-cots-lb-0-507\">YoloX detections submission made on COTS dataset</a></li>\n    </ul>\n    \n</div>","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-01-04T17:56:51.894801Z","iopub.execute_input":"2022-01-04T17:56:51.895406Z","iopub.status.idle":"2022-01-04T17:56:51.930399Z","shell.execute_reply.started":"2022-01-04T17:56:51.895293Z","shell.execute_reply":"2022-01-04T17:56:51.929088Z"}}},{"cell_type":"markdown","source":"## Used, ran, forked? Please vote. It is a GREAT gift for me and a sign of thanks.","metadata":{}},{"cell_type":"markdown","source":"<div align=\"center\"><img width=\"640\" src=\"https://github.com/WongKinYiu/yolor/raw/main/figure/unifued_network.png\"/></div>\n\n<div align=\"center\"><img width=\"640\" src=\"https://github.com/WongKinYiu/yolor/raw/main/figure/performance.png\"/></div>","metadata":{}},{"cell_type":"markdown","source":"## 0. MODULES INSTALATION ","metadata":{}},{"cell_type":"code","source":"%cp -r ../input/yolor-kaggle-hub/yolor-lib .\n%cp -r ../input/yolor-kaggle-hub/cocoapi .","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:03:50.268507Z","iopub.execute_input":"2022-01-16T12:03:50.269232Z","iopub.status.idle":"2022-01-16T12:03:52.227773Z","shell.execute_reply.started":"2022-01-16T12:03:50.269132Z","shell.execute_reply":"2022-01-16T12:03:52.226882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd /kaggle/working/cocoapi/PythonAPI\n\n!make\n!make install\n!python setup.py install","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-16T12:03:52.230758Z","iopub.execute_input":"2022-01-16T12:03:52.231257Z","iopub.status.idle":"2022-01-16T12:04:16.522446Z","shell.execute_reply.started":"2022-01-16T12:03:52.231215Z","shell.execute_reply":"2022-01-16T12:04:16.521693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pycocotools\n\n%cd /kaggle/working/yolor-lib","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:16.524889Z","iopub.execute_input":"2022-01-16T12:04:16.525534Z","iopub.status.idle":"2022-01-16T12:04:16.532296Z","shell.execute_reply.started":"2022-01-16T12:04:16.525500Z","shell.execute_reply":"2022-01-16T12:04:16.531624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nimport cv2\n\nimport pandas as pd\nimport numpy as np\n\nfrom utils.torch_utils import select_device\nfrom PIL import Image\n\n# YoloR methods\nfrom models.models import Darknet\nfrom utils.datasets import letterbox\nfrom utils.general import non_max_suppression, scale_coords","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:16.533697Z","iopub.execute_input":"2022-01-16T12:04:16.534189Z","iopub.status.idle":"2022-01-16T12:04:18.273763Z","shell.execute_reply.started":"2022-01-16T12:04:16.534155Z","shell.execute_reply":"2022-01-16T12:04:18.273016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%cd /kaggle/working/","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:18.276026Z","iopub.execute_input":"2022-01-16T12:04:18.276233Z","iopub.status.idle":"2022-01-16T12:04:18.281773Z","shell.execute_reply.started":"2022-01-16T12:04:18.276207Z","shell.execute_reply":"2022-01-16T12:04:18.281079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. YoloR PREDICTION","metadata":{}},{"cell_type":"code","source":"def show_prediction(img, bboxes, scores):\n\n    for box, score in zip(bboxes, scores):\n        cv2.rectangle(img, (int(box[0]), int(box[1])), (int(box[2]), int(box[3])), (0,0,255), 2)\n        cv2.putText(img, f'{score:.2f}', (int(box[0]), int(box[1]-3)), cv2.FONT_HERSHEY_SIMPLEX, 0.4, (0,0,255), 1, cv2.LINE_AA)\n    \n    img = img[:,:,::-1]\n    img = Image.fromarray(img).resize((1280, 720))\n    return img\n\ndef load_yolor_model(model_path, cfg_path, img_size, device):\n    \n    model = Darknet(cfg_path).cuda()\n    model.load_state_dict(torch.load(model_path, map_location=device)['model'])\n    model.to(device).eval()\n    \n    init = torch.zeros((1, 3, img_size, img_size), device=device)\n    _ = model(init) if device.type != 'cpu' else None \n    return model\n\ndef predict_img(model, img_in, img_size, conf_thres, iou_thres, device):\n    bboxes = []\n    scores = []\n    classes = []\n    \n    img = letterbox(img_in, new_shape=img_size, auto_size=64)[0]\n    img = np.ascontiguousarray(img[:, :, ::-1].transpose(2, 0, 1))\n    img = (torch.from_numpy(img).to(device).float() / 255.0).unsqueeze(0)\n    \n    pred = model(img)[0]\n    \n    if pred is not None:\n        pred = non_max_suppression(pred, conf_thres=conf_thres, iou_thres=iou_thres, merge=False, classes=None, agnostic=False)\n\n    for i, det in enumerate(pred):\n        if det is not None and len(det):\n            det[:, :4] = scale_coords(img.shape[2:], det[:, :4], img_in.shape).round()    \n    \n    det = det.cpu().detach().numpy()\n    \n    bboxes = det[:, :4] \n    scores = det[:, 4] \n    classes = det[:, 5]\n    \n    return bboxes, scores, classes","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:18.283338Z","iopub.execute_input":"2022-01-16T12:04:18.283958Z","iopub.status.idle":"2022-01-16T12:04:18.300438Z","shell.execute_reply.started":"2022-01-16T12:04:18.283920Z","shell.execute_reply":"2022-01-16T12:04:18.299703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### A. MODEL PARAMETERS","metadata":{}},{"cell_type":"code","source":"DATASET_PATH = '/kaggle/input/tensorflow-great-barrier-reef/train_images/'\ncfg_path = '/kaggle/working/yolor-lib/cfg/yolor_p6.cfg'\nmodel_path = '/kaggle/input/yolor-kaggle-hub/yr_002.pt'\n\nCONF = 0.01\nIOU = 0.4\nimg_size = 2432","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:05:47.372591Z","iopub.execute_input":"2022-01-16T12:05:47.372897Z","iopub.status.idle":"2022-01-16T12:05:47.377129Z","shell.execute_reply.started":"2022-01-16T12:05:47.372864Z","shell.execute_reply":"2022-01-16T12:05:47.376221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### B. LOAD MODEL","metadata":{}},{"cell_type":"code","source":"device = select_device()\nmodel = load_yolor_model(model_path, cfg_path, img_size, device)","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:05:49.784049Z","iopub.execute_input":"2022-01-16T12:05:49.784598Z","iopub.status.idle":"2022-01-16T12:05:50.871623Z","shell.execute_reply.started":"2022-01-16T12:05:49.784559Z","shell.execute_reply":"2022-01-16T12:05:50.870837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### C. TEST ON IMAGES","metadata":{}},{"cell_type":"code","source":"dir = f'{DATASET_PATH}'\nimgs = [dir + f for f in ('video_1/4775.jpg', 'video_0/9794.jpg', 'video_0/4502.jpg', 'video_0/9651.jpg', 'video_0/9700.jpg',  'video_0/9674.jpg','video_0/20.jpg', 'video_0/17.jpg',)]\n\nfor fileimg in imgs:\n    img = cv2.imread(fileimg)\n    bboxes, scores, classes = predict_img(model, img, img_size, conf_thres = CONF, iou_thres = IOU, device = device)\n    display(show_prediction(img, bboxes, scores))","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:05:52.304363Z","iopub.execute_input":"2022-01-16T12:05:52.305204Z","iopub.status.idle":"2022-01-16T12:05:56.795188Z","shell.execute_reply.started":"2022-01-16T12:05:52.305148Z","shell.execute_reply":"2022-01-16T12:05:56.794413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. SUBMISSION","metadata":{}},{"cell_type":"code","source":"import greatbarrierreef\nenv = greatbarrierreef.make_env()# initialize the environment\niter_test = env.iter_test()      # an iterator which loops over the test set and sample submission","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:34.296942Z","iopub.execute_input":"2022-01-16T12:04:34.297355Z","iopub.status.idle":"2022-01-16T12:04:34.317101Z","shell.execute_reply.started":"2022-01-16T12:04:34.297318Z","shell.execute_reply":"2022-01-16T12:04:34.316442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for (image_np, sample_prediction_df) in iter_test:\n    \n    bboxes, scores, _ = predict_img(model, image_np[:,:,::-1], img_size, conf_thres = CONF, iou_thres = IOU, device = device)\n    \n    predictions = []\n    detects = []\n    \n    for i in range(len(bboxes)):\n        box = bboxes[i]\n        score = scores[i]\n        x_min = int(box[0])\n        y_min = int(box[1])\n        x_max = int(box[2])\n        y_max = int(box[3])\n        \n        bbox_width = x_max - x_min\n        bbox_height = y_max - y_min\n        detects.append([x_min, y_min, x_max, y_max, score])\n        \n        predictions.append('{:.2f} {} {} {} {}'.format(score, x_min, y_min, bbox_width, bbox_height))\n    \n    prediction_str = ' '.join(predictions)\n    sample_prediction_df['annotations'] = prediction_str\n    env.predict(sample_prediction_df)","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:34.318374Z","iopub.execute_input":"2022-01-16T12:04:34.318830Z","iopub.status.idle":"2022-01-16T12:04:34.896992Z","shell.execute_reply.started":"2022-01-16T12:04:34.318794Z","shell.execute_reply":"2022-01-16T12:04:34.896128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df = pd.read_csv('submission.csv')\nsub_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-16T12:04:34.898165Z","iopub.execute_input":"2022-01-16T12:04:34.898553Z","iopub.status.idle":"2022-01-16T12:04:34.913447Z","shell.execute_reply.started":"2022-01-16T12:04:34.898509Z","shell.execute_reply":"2022-01-16T12:04:34.912576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Glad you got to the end of this notebook. Isn't it easy? Good luck! Remember to vote - it's important to me and it motivates me to share more ...","metadata":{}}]}