{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# [Tensorflow - Help Protect the Great Barrier Reef](https://www.kaggle.com/c/tensorflow-great-barrier-reef)\n> Detect crown-of-thorns starfish in underwater image data\n\n","metadata":{}},{"cell_type":"markdown","source":"## 📒 Notebooks:\n* Train: [Great-Barrier-Reef: YOLOv5 [train] 🌊](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-train)\n* Infer: [Great-Barrier-Reef: YOLOv5 [infer] 🌊](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer)","metadata":{}},{"cell_type":"markdown","source":"# 🚩 Version Info\n| Version | Model | Conf | IoU | LB | Comment\n|---|---|---|---|---|---|\n| v25 | YOLOv5s | 0.15 | 0.50 | 0.453 | |\n| v30 | YOLOv5s | 0.19 | 0.45 | 0.433 | tta |\n","metadata":{}},{"cell_type":"markdown","source":"# 🛠 Install Libraries","metadata":{}},{"cell_type":"code","source":"# bbox-utility, check https://github.com/awsaf49/bbox for source code\n!pip install -q /kaggle/input/loguru-lib-ds/loguru-0.5.3-py3-none-any.whl\n!pip install -q /kaggle/input/bbox-lib-ds","metadata":{"_kg_hide-output":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-01-24T15:15:03.322381Z","iopub.execute_input":"2022-01-24T15:15:03.322977Z","iopub.status.idle":"2022-01-24T15:16:01.217311Z","shell.execute_reply.started":"2022-01-24T15:15:03.322883Z","shell.execute_reply":"2022-01-24T15:16:01.216360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📚 Import Libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np\nfrom tqdm.notebook import tqdm\ntqdm.pandas()\nimport pandas as pd\nimport os\nimport cv2\nimport matplotlib.pyplot as plt\nimport glob\nimport shutil\nimport sys\nsys.path.append('../input/tensorflow-great-barrier-reef')\nimport torch\nfrom PIL import Image","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-01-24T15:16:01.220689Z","iopub.execute_input":"2022-01-24T15:16:01.220954Z","iopub.status.idle":"2022-01-24T15:16:02.993883Z","shell.execute_reply.started":"2022-01-24T15:16:01.220917Z","shell.execute_reply":"2022-01-24T15:16:02.993090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📌 Key-Points\n* One have to submit prediction using the provided **python time-series API**, which makes this competition different from previous Object Detection Competitions.\n* Each prediction row needs to include all bounding boxes for the image. Submission is format seems also **COCO** which means `[x_min, y_min, width, height]`\n* Copmetition metric `F2` tolerates some false positives(FP) in order to ensure very few starfish are missed. Which means tackling **false negatives(FN)** is more important than false positives(FP). \n$$F2 = 5 \\cdot \\frac{precision \\cdot recall}{4\\cdot precision + recall}$$","metadata":{}},{"cell_type":"markdown","source":"## Please Upvote if you find this Helpful","metadata":{}},{"cell_type":"markdown","source":"# 📖 Meta Data\n* `train_images/` - Folder containing training set photos of the form `video_{video_id}/{video_frame}.jpg`.\n\n* `[train/test].csv` - Metadata for the images. As with other test files, most of the test metadata data is only available to your notebook upon submission. Just the first few rows available for download.\n\n* `video_id` - ID number of the video the image was part of. The video ids are not meaningfully ordered.\n* `video_frame` - The frame number of the image within the video. Expect to see occasional gaps in the frame number from when the diver surfaced.\n* `sequence` - ID of a gap-free subset of a given video. The sequence ids are not meaningfully ordered.\n* `sequence_frame` - The frame number within a given sequence.\n* `image_id` - ID code for the image, in the format `{video_id}-{video_frame}`\n* `annotations` - The bounding boxes of any starfish detections in a string format that can be evaluated directly with Python. Does not use the same format as the predictions you will submit. Not available in test.csv. A bounding box is described by the pixel coordinate `(x_min, y_min)` of its lower left corner within the image together with its `width` and `height` in pixels --> (COCO format).","metadata":{}},{"cell_type":"code","source":"ROOT_DIR = '/kaggle/input/tensorflow-great-barrier-reef/'\n# CKPT_DIR = '/kaggle/input/greatbarrierreef-yolov5-train-ds'\nCKPT_PATH = '/kaggle/input/reef-baseline-fold12/l6_3600_uflip_vm5_f12_up/f1/best.pt' # by @steamedsheep\nIMG_SIZE = 9000\nCONF = 0.25\nIOU = 0.40\nAUGMENT = True","metadata":{"execution":{"iopub.status.busy":"2022-01-24T15:16:02.999288Z","iopub.execute_input":"2022-01-24T15:16:02.999702Z","iopub.status.idle":"2022-01-24T15:16:03.004179Z","shell.execute_reply.started":"2022-01-24T15:16:02.999663Z","shell.execute_reply":"2022-01-24T15:16:03.003508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train Data\ndf = pd.read_csv(f'{ROOT_DIR}/train.csv')\ndf['image_path'] = f'{ROOT_DIR}/train_images/video_'+df.video_id.astype(str)+'/'+df.video_frame.astype(str)+'.jpg'\ndf['annotations'] = df['annotations'].progress_apply(eval)\ndisplay(df.head(2))","metadata":{"execution":{"iopub.status.busy":"2022-01-24T15:16:03.005376Z","iopub.execute_input":"2022-01-24T15:16:03.005820Z","iopub.status.idle":"2022-01-24T15:16:03.464101Z","shell.execute_reply.started":"2022-01-24T15:16:03.005778Z","shell.execute_reply":"2022-01-24T15:16:03.463418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Number of BBoxes","metadata":{}},{"cell_type":"code","source":"df['num_bbox'] = df['annotations'].progress_apply(lambda x: len(x))\ndata = (df.num_bbox>0).value_counts()/len(df)*100\nprint(f\"No BBox: {data[0]:0.2f}% | With BBox: {data[1]:0.2f}%\")","metadata":{"execution":{"iopub.status.busy":"2022-01-24T15:16:03.465386Z","iopub.execute_input":"2022-01-24T15:16:03.465790Z","iopub.status.idle":"2022-01-24T15:16:03.566166Z","shell.execute_reply.started":"2022-01-24T15:16:03.465752Z","shell.execute_reply":"2022-01-24T15:16:03.565485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔨 Helper","metadata":{}},{"cell_type":"code","source":"# check https://github.com/awsaf49/bbox for source code of following utility functions\nfrom bbox.utils import coco2yolo, coco2voc, voc2yolo, voc2coco\nfrom bbox.utils import draw_bboxes, load_image\nfrom bbox.utils import clip_bbox, str2annot, annot2str\n\ndef get_bbox(annots):\n bboxes = [list(annot.values()) for annot in annots]\n return bboxes\n\ndef get_imgsize(row):\n row['width'], row['height'] = imagesize.get(row['image_path'])\n return row\n\nnp.random.seed(32)\ncolors = [(np.random.randint(255), np.random.randint(255), np.random.randint(255))\\\n for idx in range(1)]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:03.567527Z","iopub.execute_input":"2022-01-24T15:16:03.567969Z","iopub.status.idle":"2022-01-24T15:16:04.218451Z","shell.execute_reply.started":"2022-01-24T15:16:03.567933Z","shell.execute_reply":"2022-01-24T15:16:04.217745Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📦 [YOLOv5](https://github.com/ultralytics/yolov5/)\n","metadata":{}},{"cell_type":"code","source":"!mkdir -p /root/.config/Ultralytics\n!cp /kaggle/input/yolov5-font/Arial.ttf /root/.config/Ultralytics/","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:04.219779Z","iopub.execute_input":"2022-01-24T15:16:04.220035Z","iopub.status.idle":"2022-01-24T15:16:05.598348Z","shell.execute_reply.started":"2022-01-24T15:16:04.220000Z","shell.execute_reply":"2022-01-24T15:16:05.597315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_model(ckpt_path, conf=0.25, iou=0.50):\n model = torch.hub.load('/kaggle/input/yolov5-lib-ds',\n 'custom',\n path=ckpt_path,\n source='local',\n force_reload=True) # local repo\n model.conf = conf # NMS confidence threshold\n model.iou = iou # NMS IoU threshold\n model.classes = None # (optional list) filter by class, i.e. = [0, 15, 16] for persons, cats and dogs\n model.multi_label = False # NMS multiple labels per box\n model.max_det = 1000 # maximum number of detections per image\n return model","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:05.600273Z","iopub.execute_input":"2022-01-24T15:16:05.600558Z","iopub.status.idle":"2022-01-24T15:16:05.609001Z","shell.execute_reply.started":"2022-01-24T15:16:05.600521Z","shell.execute_reply":"2022-01-24T15:16:05.608292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔭 Inference","metadata":{}},{"cell_type":"markdown","source":"## Helper","metadata":{}},{"cell_type":"code","source":"def predict(model, img, size=768, augment=False):\n height, width = img.shape[:2]\n results = model(img, size=size, augment=augment) # custom inference size\n preds = results.pandas().xyxy[0]\n bboxes = preds[['xmin','ymin','xmax','ymax']].values\n if len(bboxes):\n bboxes = voc2coco(bboxes,height,width).astype(int)\n confs = preds.confidence.values\n return bboxes, confs\n else:\n return [],[]\n \ndef format_prediction(bboxes, confs):\n annot = ''\n if len(bboxes)>0:\n for idx in range(len(bboxes)):\n xmin, ymin, w, h = bboxes[idx]\n conf = confs[idx]\n annot += f'{conf} {xmin} {ymin} {w} {h}'\n annot +=' '\n annot = annot.strip(' ')\n return annot\n\ndef show_img(img, bboxes, bbox_format='yolo'):\n names = ['starfish']*len(bboxes)\n labels = [0]*len(bboxes)\n img = draw_bboxes(img = img,\n bboxes = bboxes, \n classes = names,\n class_ids = labels,\n class_name = True, \n colors = colors, \n bbox_format = bbox_format,\n line_thickness = 2)\n return Image.fromarray(img).resize((800, 400))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:05.612263Z","iopub.execute_input":"2022-01-24T15:16:05.612460Z","iopub.status.idle":"2022-01-24T15:16:05.625300Z","shell.execute_reply.started":"2022-01-24T15:16:05.612435Z","shell.execute_reply":"2022-01-24T15:16:05.624531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Run Inference on **Train**","metadata":{}},{"cell_type":"code","source":"model = load_model(CKPT_PATH, conf=CONF, iou=IOU)\nimage_paths = df[df.num_bbox>1].sample(100).image_path.tolist()\nfor idx, path in enumerate(image_paths):\n img = cv2.imread(path)[...,::-1]\n bboxes, confis = predict(model, img, size=IMG_SIZE, augment=AUGMENT)\n display(show_img(img, bboxes, bbox_format='coco'))\n if idx>5:\n break","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:05.626782Z","iopub.execute_input":"2022-01-24T15:16:05.627119Z","iopub.status.idle":"2022-01-24T15:16:25.513970Z","shell.execute_reply.started":"2022-01-24T15:16:05.627079Z","shell.execute_reply":"2022-01-24T15:16:25.512926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Init `Env`","metadata":{}},{"cell_type":"code","source":"import greatbarrierreef\nenv = greatbarrierreef.make_env()# initialize the environment\niter_test = env.iter_test() # an iterator which loops over the test set and sample submission","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:25.515330Z","iopub.execute_input":"2022-01-24T15:16:25.515753Z","iopub.status.idle":"2022-01-24T15:16:25.538745Z","shell.execute_reply.started":"2022-01-24T15:16:25.515718Z","shell.execute_reply":"2022-01-24T15:16:25.537949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Run Inference on **Test**","metadata":{}},{"cell_type":"code","source":"model = load_model(CKPT_PATH, conf=CONF, iou=IOU)\nfor idx, (img, pred_df) in enumerate(tqdm(iter_test)):\n bboxes, confs = predict(model, img, size=IMG_SIZE, augment=AUGMENT)\n annot = format_prediction(bboxes, confs)\n pred_df['annotations'] = annot\n env.predict(pred_df)\n if idx<3:\n display(show_img(img, bboxes, bbox_format='coco'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-24T15:16:25.540033Z","iopub.execute_input":"2022-01-24T15:16:25.540510Z","iopub.status.idle":"2022-01-24T15:16:30.033134Z","shell.execute_reply.started":"2022-01-24T15:16:25.540472Z","shell.execute_reply":"2022-01-24T15:16:30.032342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 👀 Check Submission","metadata":{}},{"cell_type":"code","source":"sub_df = pd.read_csv('submission.csv')\nsub_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T15:16:30.034540Z","iopub.execute_input":"2022-01-24T15:16:30.034998Z","iopub.status.idle":"2022-01-24T15:16:30.047018Z","shell.execute_reply.started":"2022-01-24T15:16:30.034961Z","shell.execute_reply":"2022-01-24T15:16:30.046370Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Please Upvote if you find this Helpful","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}}]}