{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Tensorflow - Help Protect the Great Barrier Reef\nFor fulfiliing the goal to identify the starfish in real-time by building an object detection model trained on underwater videos of coral reefs, we will try to work with best model fit for image recognition. In the Australia's stunningly beautiful Great Barrier Reef there is world’s largest coral reef and home to 1,500 species of fish, 400 species of corals, 130 species of sharks, rays, and a massive variety of other sea life.\n","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nimport sys\nimport time\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:54:04.783948Z","iopub.execute_input":"2021-12-02T14:54:04.785033Z","iopub.status.idle":"2021-12-02T14:54:04.81287Z","shell.execute_reply.started":"2021-12-02T14:54:04.784925Z","shell.execute_reply":"2021-12-02T14:54:04.812107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Install Library Support\nHere are the supported libraries to be installed before proceeding with YOLOv5 algorithm.\n\nSpecial thanks to [Awsaf](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-train) for clearing the concept for YOLOv5 algorithm.","metadata":{}},{"cell_type":"code","source":"# Install UBUNTU based configuration required in YOLOv5\n\n!pip install -qU wandb\n!add-apt-repository ppa:ubuntu-toolchain-r/test -y\n!apt-get update\n!apt-get upgrade libstdc++6 -y\n!pip install -q imagesize","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:54:04.814593Z","iopub.execute_input":"2021-12-02T14:54:04.814858Z","iopub.status.idle":"2021-12-02T14:58:21.303385Z","shell.execute_reply.started":"2021-12-02T14:54:04.814819Z","shell.execute_reply":"2021-12-02T14:58:21.302554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Great Barrier Reef Data Details\nFile contains following type of data:\n\n* `train/` - Folder containing training set photos of the form `video_{video_id}/{video_frame_number}.jpg`\n* `train/test.csv` - Metadata for the images.\n* `video_id` - ID number of the video the image was part of.\n* `video_frame` - The frame number of the image within the video.\n* `sequence` - ID of a gap-free subset of a given video.\n* `sequence_frame` - The frame number within a given sequence.\n* `image_id` - ID code for the image, in the format `{video_id}-{video_frame}`\n* `annotations` - The bounding boxes of any starfish detections in a string format that can be evaluated directly providing the coordinates only. Bounding box is described by the pixel coordinate (`x_min`, `y_min`) of its lower left corner within the image together with its width and height in pixels. This also known as COCO format.\n\nFor this work, reference is taken from this [notbook](https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer#%F0%9F%94%AD-Inference).","metadata":{}},{"cell_type":"code","source":"# Call to GBR Directory\ngbr_dir = '../input/tensorflow-great-barrier-reef/'\nsys.path.insert(0, gbr_dir)\n\n# Import Great Barrier Reef library for submission\nimport greatbarrierreef\nimport cv2 # Open CV library\nimport matplotlib.pyplot as plt # For visualization plotting\n\n# For Progerss of pandas processsing\nfrom tqdm.notebook import tqdm\ntqdm.pandas()\nimport pandas as pd\n\n# Import Abstract Syntax Trees Library\nimport ast\n\n# Weights & Biases  (optional)\nimport wandb\nwandb.login(anonymous='must')\n\nimport torch\nfrom PIL import Image","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:58:21.305628Z","iopub.execute_input":"2021-12-02T14:58:21.305934Z","iopub.status.idle":"2021-12-02T14:58:24.094021Z","shell.execute_reply.started":"2021-12-02T14:58:21.305891Z","shell.execute_reply":"2021-12-02T14:58:24.093265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Base for YOLOv5 Implementation","metadata":{}},{"cell_type":"code","source":"# In work directory, create directories for images & lables\nimage_dir = '/kaggle/working/images' \nlabel_dir = '/kaggle/working/labels'\n\n# Create Directories\n!mkdir -p {image_dir}\n!mkdir -p {label_dir}","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:58:24.095275Z","iopub.execute_input":"2021-12-02T14:58:24.096089Z","iopub.status.idle":"2021-12-02T14:58:25.551639Z","shell.execute_reply.started":"2021-12-02T14:58:24.09605Z","shell.execute_reply":"2021-12-02T14:58:25.550696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Getting the path for the image\ndef get_path(row):\n    row['old_image_path'] = f'{gbr_dir}/train_images/video_{row.video_id}/{row.video_frame}.jpg'\n    row['image_path'] = f'{image_dir}/video_{row.video_id}_{row.video_frame}.jpg'\n    row['label_path'] = f'{label_dir}/video_{row.video_id}_{row.video_frame}.txt'\n    return row","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:58:25.555727Z","iopub.execute_input":"2021-12-02T14:58:25.55597Z","iopub.status.idle":"2021-12-02T14:58:25.560753Z","shell.execute_reply.started":"2021-12-02T14:58:25.555941Z","shell.execute_reply":"2021-12-02T14:58:25.559966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Reading Train Data","metadata":{}},{"cell_type":"code","source":"# Train Data\ndf_train = pd.read_csv(f'{gbr_dir}/train.csv')\n\ndf_train = df_train.progress_apply(get_path, axis=1)\ndf_train['annotations'] = df_train['annotations'].progress_apply(lambda x: ast.literal_eval(x))","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:58:25.562359Z","iopub.execute_input":"2021-12-02T14:58:25.562622Z","iopub.status.idle":"2021-12-02T14:59:06.389428Z","shell.execute_reply.started":"2021-12-02T14:58:25.562589Z","shell.execute_reply":"2021-12-02T14:59:06.388633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:06.390915Z","iopub.execute_input":"2021-12-02T14:59:06.39138Z","iopub.status.idle":"2021-12-02T14:59:06.411242Z","shell.execute_reply.started":"2021-12-02T14:59:06.391341Z","shell.execute_reply":"2021-12-02T14:59:06.410585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Feature Engineering\nFor the standrad implementatoion YOLO algorithm all the input data needs to be labelized. So, in our case we initially we use only the data having proper labels for each box in images.","metadata":{}},{"cell_type":"code","source":"# Check the details of the boxes not having lables.\ndf_train['num_bbox'] = df_train['annotations'].progress_apply(lambda x: len(x))\ndata = (df_train.num_bbox>0).value_counts()/len(df_train)*100\n\n# Print the details\nprint(f\"No BBox: {data[0]:0.2f}% | With BBox: {data[1]:0.2f}%\")\n\ndel data","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:06.412784Z","iopub.execute_input":"2021-12-02T14:59:06.413224Z","iopub.status.idle":"2021-12-02T14:59:06.508662Z","shell.execute_reply.started":"2021-12-02T14:59:06.413187Z","shell.execute_reply":"2021-12-02T14:59:06.507863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**As from the above calculations we only use the boxes having labels.**","metadata":{}},{"cell_type":"code","source":"# Leave all the un-labeled data\nif True:\n    df_train = df_train.query(\"num_bbox>0\")","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:06.509914Z","iopub.execute_input":"2021-12-02T14:59:06.510272Z","iopub.status.idle":"2021-12-02T14:59:06.527383Z","shell.execute_reply.started":"2021-12-02T14:59:06.510239Z","shell.execute_reply":"2021-12-02T14:59:06.526739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Write Access Modifications\nFor write access on images, we need to copy the images to current working directory `/kaggle/working` as `/kaggle/input` doesn't have the same. This access is required for YOLO implementation.","metadata":{}},{"cell_type":"code","source":"# High-Level file operations library\nimport shutil\n# Library for Mutiprocessing\nfrom multiprocessing import Pool\nfrom joblib import Parallel, delayed\n\n# Function for copying the data.\ndef make_copy(path):\n    data = path.split('/')\n    filename = data[-1]\n    video_id = data[-2]\n    new_path = os.path.join('/kaggle/working/images',f'{video_id}_{filename}')\n    shutil.copy(path, new_path)\n    \n    return\n\n# Copying the data using Multi-processing\nimage_paths = df_train.old_image_path.tolist()\n_ = Parallel(n_jobs=-1, backend='threading')(delayed(make_copy)(path) for path in tqdm(image_paths))","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:06.528764Z","iopub.execute_input":"2021-12-02T14:59:06.52911Z","iopub.status.idle":"2021-12-02T14:59:37.510604Z","shell.execute_reply.started":"2021-12-02T14:59:06.529021Z","shell.execute_reply":"2021-12-02T14:59:37.509847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Helper Functionalities for YOLOv5","metadata":{}},{"cell_type":"code","source":"# Extra libs: Install and call them\nimport imagesize\n\n# VOC2YOLO function for getting box details in VOC_pascal data type\ndef voc2yolo(bboxes, image_height=720, image_width=1280):\n    \n     # otherwise all value will be 0 as voc_pascal dtype is np.int\n    bboxes = bboxes.copy().astype(float)\n    \n    bboxes[..., [0, 2]] = bboxes[..., [0, 2]]/ image_width\n    bboxes[..., [1, 3]] = bboxes[..., [1, 3]]/ image_height\n    \n    w = bboxes[..., 2] - bboxes[..., 0]\n    h = bboxes[..., 3] - bboxes[..., 1]\n    \n    bboxes[..., 0] = bboxes[..., 0] + w/2\n    bboxes[..., 1] = bboxes[..., 1] + h/2\n    bboxes[..., 2] = w\n    bboxes[..., 3] = h\n    \n    return bboxes\n\n\n# YOLO2VOC inverse function with VOC_pascal data type\ndef yolo2voc(bboxes, image_height=720, image_width=1280):\n\n    # otherwise all value will be 0 as voc_pascal dtype is np.int\n    bboxes = bboxes.copy().astype(float) \n    \n    bboxes[..., [0, 2]] = bboxes[..., [0, 2]]* image_width\n    bboxes[..., [1, 3]] = bboxes[..., [1, 3]]* image_height\n    \n    bboxes[..., [0, 1]] = bboxes[..., [0, 1]] - bboxes[..., [2, 3]]/2\n    bboxes[..., [2, 3]] = bboxes[..., [0, 1]] + bboxes[..., [2, 3]]\n    \n    return bboxes\n\n# COCO format to YOLO implementation\ndef coco2yolo(bboxes, image_height=720, image_width=1280):\n\n    # otherwise all value will be 0 as voc_pascal dtype is np.int\n    bboxes = bboxes.copy().astype(float) \n    \n    # normolizinig\n    bboxes[..., [0, 2]]= bboxes[..., [0, 2]]/ image_width\n    bboxes[..., [1, 3]]= bboxes[..., [1, 3]]/ image_height\n    \n    # converstion (xmin, ymin) => (xmid, ymid)\n    bboxes[..., [0, 1]] = bboxes[..., [0, 1]] + bboxes[..., [2, 3]]/2\n    \n    return bboxes\n\ndef coco2yolo_spl(image_height, image_width, bboxes):\n    \n    bboxes = bboxes.copy().astype(float)\n    \n    # normolizinig\n    bboxes[..., [0, 2]]= bboxes[..., [0, 2]]/ image_width\n    bboxes[..., [1, 3]]= bboxes[..., [1, 3]]/ image_height\n    \n    # converstion (xmin, ymin) => (xmid, ymid)\n    bboxes[..., [0, 1]] = bboxes[..., [0, 1]] + bboxes[..., [2, 3]]/2\n    \n    return bboxes\n\n# YOLO format to COCO implementation\ndef yolo2coco(bboxes, image_height=720, image_width=1280):\n    \n    # otherwise all value will be 0 as voc_pascal dtype is np.int\n    bboxes = bboxes.copy().astype(float) \n    \n    # denormalizing\n    bboxes[..., [0, 2]]= bboxes[..., [0, 2]]* image_width\n    bboxes[..., [1, 3]]= bboxes[..., [1, 3]]* image_height\n    \n    # converstion (xmid, ymid) => (xmin, ymin) \n    bboxes[..., [0, 1]] = bboxes[..., [0, 1]] - bboxes[..., [2, 3]]/2\n    \n    return bboxes\n\n# VOC_PASCAL format to COCO implementation\ndef voc2coco(bboxes, image_height=720, image_width=1280):\n    bboxes  = voc2yolo(bboxes, image_height, image_width)\n    bboxes  = yolo2coco(bboxes, image_height, image_width)\n    return bboxes\n\n# Load the image with OpenCV\ndef load_image(image_path):\n    return cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)\n\n# Plot one box for COTS Starfish\ndef plot_one_box(x, img, color=None, label=None, line_thickness=None):\n    # Plots one bounding box on image img\n    tl = line_thickness or round(0.002 * (img.shape[0] + img.shape[1]) / 2) + 1  # line/font thickness\n    color = color or [random.randint(0, 255) for _ in range(3)]\n    c1, c2 = (int(x[0]), int(x[1])), (int(x[2]), int(x[3]))\n    cv2.rectangle(img, c1, c2, color, thickness=tl, lineType=cv2.LINE_AA)\n    if label:\n        tf = max(tl - 1, 1)  # font thickness\n        t_size = cv2.getTextSize(label, 0, fontScale=tl / 3, thickness=tf)[0]\n        c2 = c1[0] + t_size[0], c1[1] - t_size[1] - 3\n        cv2.rectangle(img, c1, c2, color, -1, cv2.LINE_AA)  # filled\n        cv2.putText(img, label, (c1[0], c1[1] - 2), 0, tl / 3, [225, 255, 255], thickness=tf, lineType=cv2.LINE_AA)\n\n# Draw multipe boxes in images for COTS Starfish\ndef draw_bboxes(img, bboxes, classes, class_ids, colors = None, show_classes = None, bbox_format = 'yolo', class_name = False, line_thickness = 2):  \n     \n    image = img.copy()\n    show_classes = classes if show_classes is None else show_classes\n    colors = (0, 255 ,0) if colors is None else colors\n    \n    if bbox_format == 'yolo':\n        \n        for idx in range(len(bboxes)):  \n            \n            bbox  = bboxes[idx]\n            cls   = classes[idx]\n            cls_id = class_ids[idx]\n            color = colors[cls_id] if type(colors) is list else colors\n            \n            if cls in show_classes:\n            \n                x1 = round(float(bbox[0])*image.shape[1])\n                y1 = round(float(bbox[1])*image.shape[0])\n                w  = round(float(bbox[2])*image.shape[1]/2) #w/2 \n                h  = round(float(bbox[3])*image.shape[0]/2)\n\n                voc_bbox = (x1-w, y1-h, x1+w, y1+h)\n                plot_one_box(voc_bbox, \n                             image,\n                             color = color,\n                             label = cls if class_name else str(get_label(cls)),\n                             line_thickness = line_thickness)\n            \n    elif bbox_format == 'coco':\n        \n        for idx in range(len(bboxes)):  \n            \n            bbox  = bboxes[idx]\n            cls   = classes[idx]\n            cls_id = class_ids[idx]\n            color = colors[cls_id] if type(colors) is list else colors\n            \n            if cls in show_classes:            \n                x1 = int(round(bbox[0]))\n                y1 = int(round(bbox[1]))\n                w  = int(round(bbox[2]))\n                h  = int(round(bbox[3]))\n\n                voc_bbox = (x1, y1, x1+w, y1+h)\n                plot_one_box(voc_bbox, \n                             image,\n                             color = color,\n                             label = cls if class_name else str(cls_id),\n                             line_thickness = line_thickness)\n\n    elif bbox_format == 'voc_pascal':\n        \n        for idx in range(len(bboxes)):  \n            \n            bbox  = bboxes[idx]\n            cls   = classes[idx]\n            cls_id = class_ids[idx]\n            color = colors[cls_id] if type(colors) is list else colors\n            \n            if cls in show_classes: \n                x1 = int(round(bbox[0]))\n                y1 = int(round(bbox[1]))\n                x2 = int(round(bbox[2]))\n                y2 = int(round(bbox[3]))\n                voc_bbox = (x1, y1, x2, y2)\n                plot_one_box(voc_bbox, \n                             image,\n                             color = color,\n                             label = cls if class_name else str(cls_id),\n                             line_thickness = line_thickness)\n    else:\n        raise ValueError('wrong bbox format')\n\n    return image\n\n# Get the box details\ndef get_bbox(annots):\n    bboxes = [list(annot.values()) for annot in annots]\n    return bboxes\n\n# Get the image size details\ndef get_imgsize(row):\n    row['width'], row['height'] = imagesize.get(row['image_path'])\n    return row","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:37.51205Z","iopub.execute_input":"2021-12-02T14:59:37.512442Z","iopub.status.idle":"2021-12-02T14:59:37.553907Z","shell.execute_reply.started":"2021-12-02T14:59:37.512401Z","shell.execute_reply":"2021-12-02T14:59:37.553094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sample create a box \ndf_train['bboxes'] = df_train.annotations.progress_apply(get_bbox)\n\n# Get the Image size 1280x720\ndf_train = df_train.progress_apply(get_imgsize,axis=1)","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:37.556513Z","iopub.execute_input":"2021-12-02T14:59:37.557301Z","iopub.status.idle":"2021-12-02T14:59:44.224186Z","shell.execute_reply.started":"2021-12-02T14:59:37.557261Z","shell.execute_reply":"2021-12-02T14:59:44.222463Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Display the details\ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:44.22569Z","iopub.execute_input":"2021-12-02T14:59:44.226299Z","iopub.status.idle":"2021-12-02T14:59:44.260736Z","shell.execute_reply.started":"2021-12-02T14:59:44.226258Z","shell.execute_reply":"2021-12-02T14:59:44.260076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Create & Export Labels\nExport our labels to YOLO format, with one `*.txt` file per image. Details from `*.txt` file following:\n\n* One row per object\n* Each row is class `x_center, y_center, width, height` format.\n* Box coordinates must be in normalized `xywh` format (from 0 - 1). If your boxes are in pixels, divide `x_center` and `width` by image width, and `y_center` and `height` by image height.\n* Class numbers are zero-indexed (start from 0).","metadata":{}},{"cell_type":"code","source":"count = 0\nbboxes_cnt = []\nfor row_idx in tqdm(range(df_train.shape[0])):\n    row = df_train.iloc[row_idx]\n    image_height = row.height\n    image_width  = row.width\n    bboxes_coco  = np.array(row.bboxes).astype(np.float32).copy()\n    num_bbox     = len(bboxes_coco)\n    names        = ['starfish']*num_bbox\n    labels       = [0]*num_bbox\n    \n    ## Create Annotation(YOLO)\n    f = open(row.label_path, 'w')\n    if num_bbox<1:\n        annot = ''\n        f.write(annot)\n        f.close()\n        count+=1\n        continue\n    \n    # Converting box details in COCO format\n    bboxes_yolo  = coco2yolo_spl(image_height, image_width, bboxes_coco)\n    bboxes_cnt.extend(bboxes_yolo)\n    for bbox_idx in range(len(bboxes_yolo)):\n        annot = [str(labels[bbox_idx])]+ list(bboxes_yolo[bbox_idx].astype(str))+(['\\n'] if num_bbox!=(bbox_idx+1) else [''])\n        annot = ' '.join(annot)\n        annot = annot.strip(' ')\n        f.write(annot)\n    f.close()\nprint('Missing:',count)","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:44.266521Z","iopub.execute_input":"2021-12-02T14:59:44.268467Z","iopub.status.idle":"2021-12-02T14:59:46.564441Z","shell.execute_reply.started":"2021-12-02T14:59:44.268429Z","shell.execute_reply":"2021-12-02T14:59:46.563616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualization of Boxes\nWith the use of helper functionalities we are trying to visulaize the boxes on multiple images","metadata":{}},{"cell_type":"code","source":"# Randomize and get color details\nnp.random.seed(32)\ncolors = [(np.random.randint(255), np.random.randint(255), np.random.randint(255))\\\n          for idx in range(1)]\n\n# Visualize only 10 images\ndf_train_mod = df_train[(df_train.num_bbox>0)].sample(100)\nfor idx in range(10):\n    row = df_train_mod.iloc[idx]\n    box_img       = load_image(row.image_path)\n    image_height  = row.height\n    image_width   = row.width\n    bboxes_coco   = np.array(row.bboxes)\n    \n    # Call the COCO2YOLO for creating COCO format boxes\n    bboxes_yolo   = coco2yolo_spl(image_height, image_width, bboxes_coco)\n    names         = ['starfish']*len(bboxes_coco)\n    labels        = [0]*len(bboxes_coco)\n\n    plt.figure(figsize = (10, 8))\n    plt.imshow(draw_bboxes(img = box_img,\n                           bboxes = bboxes_yolo, \n                           classes = names,\n                           class_ids = labels,\n                           class_name = True, \n                           colors = colors, \n                           bbox_format = 'yolo',\n                           line_thickness = 2))\n    plt.axis('OFF')\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:46.565958Z","iopub.execute_input":"2021-12-02T14:59:46.566225Z","iopub.status.idle":"2021-12-02T14:59:50.525314Z","shell.execute_reply.started":"2021-12-02T14:59:46.56619Z","shell.execute_reply":"2021-12-02T14:59:50.524569Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Folds, Dataset & Configuration","metadata":{}},{"cell_type":"code","source":"# Get the Fold details\nfrom sklearn.model_selection import StratifiedKFold\n\nk_Fold = StratifiedKFold(n_splits = 20)\n\n# Train dataset \ndf_train = df_train.reset_index(drop=True)\ndf_train['fold'] = -1\n\n# Get the folds\nfor fold, (train_idx, val_idx) in enumerate(k_Fold.split(df_train, y = df_train.video_id.tolist())):\n    df_train.loc[val_idx, 'fold'] = fold\n\n# Display the fold count\ndf_train.fold.value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:50.526867Z","iopub.execute_input":"2021-12-02T14:59:50.527342Z","iopub.status.idle":"2021-12-02T14:59:51.206297Z","shell.execute_reply.started":"2021-12-02T14:59:50.527308Z","shell.execute_reply":"2021-12-02T14:59:51.20557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the Dataset Enhancements\ntrn_fold = []\nval_fold = []\n\n# Apply the query\ndf_train_upd = df_train.query(\"fold!=@fold\")\ndf_value_upd = df_train.query(\"fold==@fold\")\n\ntrn_fold += list(df_train_upd.image_path.unique())\nval_fold += list(df_value_upd.image_path.unique())\n\n# Display details\nlen(trn_fold), len(val_fold)","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:51.207418Z","iopub.execute_input":"2021-12-02T14:59:51.207755Z","iopub.status.idle":"2021-12-02T14:59:51.2264Z","shell.execute_reply.started":"2021-12-02T14:59:51.207717Z","shell.execute_reply":"2021-12-02T14:59:51.225625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**For YOLO implemntation:**\n\n* Root directory path for dataset and relative paths to `train/val/test` image directories or files with path.\n* Number of classes `nc` and class details are `names`:`starfish`","metadata":{}},{"cell_type":"code","source":"import yaml\n\ncwd_path = '/kaggle/working/'\n\nwith open(os.path.join( cwd_path , 'train.txt'), 'w') as f:\n    for path in df_train_upd.image_path.tolist():\n        f.write(path+'\\n')\n            \nwith open(os.path.join(cwd_path , 'value.txt'), 'w') as f:\n    for path in df_value_upd.image_path.tolist():\n        f.write(path+'\\n')\n\ndata = dict(\n    path  = '/kaggle/working',\n    train =  os.path.join(cwd_path , 'train.txt'),\n    val =  os.path.join(cwd_path , 'value.txt'),\n    nc    = 1,\n    names = ['starfish'],\n    )\n\nwith open(os.path.join( cwd_path , 'tf_gbr.yaml'), 'w') as outfile:\n    yaml.dump(data, outfile, default_flow_style=False)\n\nf = open(os.path.join( cwd_path , 'tf_gbr.yaml'), 'r')\nprint('\\nyaml:')\nprint(f.read())","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:51.227634Z","iopub.execute_input":"2021-12-02T14:59:51.227961Z","iopub.status.idle":"2021-12-02T14:59:51.242062Z","shell.execute_reply.started":"2021-12-02T14:59:51.227925Z","shell.execute_reply":"2021-12-02T14:59:51.241213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Implmentation YOLOv5","metadata":{}},{"cell_type":"code","source":"# Enter the Working directory\n%cd /kaggle/working\n\n# Remove the previous installed yolov5 repo\n!rm -rf /kaggle/working/yolov5\n\n!git clone https://github.com/ultralytics/yolov5 # clone\n\n%cd yolov5\n%pip install -qr requirements.txt\n\nfrom yolov5 import utils\ndisplay = utils.notebook_init()\n\nimport wandb\nwandb.login(anonymous='must')","metadata":{"execution":{"iopub.status.busy":"2021-12-02T14:59:51.243769Z","iopub.execute_input":"2021-12-02T14:59:51.24402Z","iopub.status.idle":"2021-12-02T15:00:03.401638Z","shell.execute_reply.started":"2021-12-02T14:59:51.243988Z","shell.execute_reply":"2021-12-02T15:00:03.400769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Model","metadata":{}},{"cell_type":"code","source":"# Train YOLOv5s on COCO128 for 3 epochs\n!python train.py --img 1280\\\n--batch 16\\\n--epochs 3\\\n--data /kaggle/working/tf_gbr.yaml\\\n--weights yolov5s.pt","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:00:03.403238Z","iopub.execute_input":"2021-12-02T15:00:03.404899Z","iopub.status.idle":"2021-12-02T15:31:43.090928Z","shell.execute_reply.started":"2021-12-02T15:00:03.404854Z","shell.execute_reply":"2021-12-02T15:31:43.089904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Deleting the training dataset.\ndel df_train","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:31:43.092614Z","iopub.execute_input":"2021-12-02T15:31:43.092926Z","iopub.status.idle":"2021-12-02T15:31:43.101513Z","shell.execute_reply.started":"2021-12-02T15:31:43.092889Z","shell.execute_reply":"2021-12-02T15:31:43.099775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Rediness for Prediction","metadata":{}},{"cell_type":"code","source":"# Configuration data\nROOT_DIR  = '/kaggle/input/tensorflow-great-barrier-reef/'\nCKPT_PATH = '/kaggle/working/yolov5/runs/train/exp/weights/best.pt'\nIMG_SIZE  = 1280\nCONF      = 0.21\nIOU       = 0.50\nAUGMENT   = False","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:00.729791Z","iopub.execute_input":"2021-12-02T15:33:00.730264Z","iopub.status.idle":"2021-12-02T15:33:00.734972Z","shell.execute_reply.started":"2021-12-02T15:33:00.730229Z","shell.execute_reply":"2021-12-02T15:33:00.734081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Inference\nGetting all the training data once again for the inference purpose.","metadata":{}},{"cell_type":"code","source":"def get_path_infer(row):\n    row['image_path'] = f'{ROOT_DIR}/train_images/video_{row.video_id}/{row.video_frame}.jpg'\n    return row","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:01.81844Z","iopub.execute_input":"2021-12-02T15:33:01.819324Z","iopub.status.idle":"2021-12-02T15:33:01.823447Z","shell.execute_reply.started":"2021-12-02T15:33:01.819276Z","shell.execute_reply":"2021-12-02T15:33:01.822748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train Data\ndf_train = pd.read_csv(f'{ROOT_DIR}/train.csv')\ndf_train = df_train.progress_apply(get_path_infer, axis=1)\ndf_train['annotations'] = df_train['annotations'].progress_apply(lambda x: ast.literal_eval(x))\ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:02.228314Z","iopub.execute_input":"2021-12-02T15:33:02.229139Z","iopub.status.idle":"2021-12-02T15:33:18.156965Z","shell.execute_reply.started":"2021-12-02T15:33:02.229087Z","shell.execute_reply":"2021-12-02T15:33:18.156157Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['num_bbox'] = df_train['annotations'].progress_apply(lambda x: len(x))\ndata = (df_train.num_bbox>0).value_counts()/len(df_train)*100\nprint(f\"No BBox: {data[0]:0.2f}% | With BBox: {data[1]:0.2f}%\")","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:23.097913Z","iopub.execute_input":"2021-12-02T15:33:23.098374Z","iopub.status.idle":"2021-12-02T15:33:23.195319Z","shell.execute_reply.started":"2021-12-02T15:33:23.098337Z","shell.execute_reply":"2021-12-02T15:33:23.194628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to Load Model","metadata":{}},{"cell_type":"code","source":"from pathlib import Path\n\ndef load_model(ckpt_path, conf=0.25, iou=0.50):\n    \n    my_file = Path(ckpt_path)\n    if my_file.is_file():\n        print (\"Yolo Model Exist\")\n    else:\n        print (\"No Yolo Model!!!\")\n        return\n    \n    model = torch.hub.load('/kaggle/working/yolov5',\n                           'custom',\n                           path=ckpt_path,\n                           source='local',\n                           force_reload=True)  # local repo\n    model.conf = conf  # NMS confidence threshold\n    model.iou  = iou  # NMS IoU threshold\n    model.classes = None   # (optional list) filter by class, i.e. = [0, 15, 16] for persons, cats and dogs\n    model.multi_label = False  # NMS multiple labels per box\n    model.max_det = 1000  # maximum number of detections per image\n    return model","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:26.787343Z","iopub.execute_input":"2021-12-02T15:33:26.787603Z","iopub.status.idle":"2021-12-02T15:33:26.79401Z","shell.execute_reply.started":"2021-12-02T15:33:26.787571Z","shell.execute_reply":"2021-12-02T15:33:26.793288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Helper Function for Prediction","metadata":{}},{"cell_type":"code","source":"def predict(model, img, size=768, augment=False):\n    height, width = img.shape[:2]\n    results = model(img, size=size, augment=augment)  # custom inference size\n    preds   = results.pandas().xyxy[0]\n    bboxes  = preds[['xmin','ymin','xmax','ymax']].values\n    if len(bboxes):\n        bboxes  = voc2coco(bboxes,height,width).astype(int)\n        confs   = preds.confidence.values\n        return bboxes, confs\n    else:\n        return [],[]\n    \ndef format_prediction(bboxes, confs):\n    annot = ''\n    if len(bboxes)>0:\n        for idx in range(len(bboxes)):\n            xmin, ymin, w, h = bboxes[idx]\n            conf             = confs[idx]\n            annot += f'{conf} {xmin} {ymin} {w} {h}'\n            annot +=' '\n        annot = annot.strip(' ')\n    return annot\n\ndef show_img(img, bboxes, bbox_format='yolo'):\n    names  = ['starfish']*len(bboxes)\n    labels = [0]*len(bboxes)\n    img    = draw_bboxes(img = img,\n                           bboxes = bboxes, \n                           classes = names,\n                           class_ids = labels,\n                           class_name = True, \n                           colors = colors, \n                           bbox_format = bbox_format,\n                           line_thickness = 2)\n    return Image.fromarray(img).resize((800, 400))","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:28.935944Z","iopub.execute_input":"2021-12-02T15:33:28.936497Z","iopub.status.idle":"2021-12-02T15:33:28.948521Z","shell.execute_reply.started":"2021-12-02T15:33:28.936458Z","shell.execute_reply":"2021-12-02T15:33:28.947738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training for Model","metadata":{}},{"cell_type":"code","source":"model = load_model(CKPT_PATH, conf=CONF, iou=IOU)\nimage_paths = df_train[df_train.num_bbox>1].sample(100).image_path.tolist()\nfor idx, path in enumerate(image_paths):\n    img = cv2.imread(path)[...,::-1]\n    bboxes, confis = predict(model, img, size=IMG_SIZE, augment=AUGMENT)\n    \n    print(bboxes)\n    show_img(img, bboxes)\n    if idx>5:\n        break","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:30.985812Z","iopub.execute_input":"2021-12-02T15:33:30.986382Z","iopub.status.idle":"2021-12-02T15:33:34.877322Z","shell.execute_reply.started":"2021-12-02T15:33:30.986342Z","shell.execute_reply":"2021-12-02T15:33:34.876462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction for Model","metadata":{}},{"cell_type":"code","source":"import greatbarrierreef\nenv = greatbarrierreef.make_env()# initialize the environment\niter_test = env.iter_test()      # an iterator which loops over the test set and sample submission","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:46.956262Z","iopub.execute_input":"2021-12-02T15:33:46.957025Z","iopub.status.idle":"2021-12-02T15:33:46.962221Z","shell.execute_reply.started":"2021-12-02T15:33:46.956981Z","shell.execute_reply":"2021-12-02T15:33:46.961258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df = []\n\nmodel = load_model(CKPT_PATH, conf=CONF, iou=IOU)\nfor idx, (img, pred_df) in enumerate(tqdm(iter_test)):\n    bboxes, confs  = predict(model, img, size=IMG_SIZE, augment=True)\n    annot          = format_prediction(bboxes, confs)\n    pred_df['annotations'] = annot\n    env.predict(pred_df)\n    if idx<3:\n        display(show_img(img, bboxes, bbox_format='coco'))","metadata":{"execution":{"iopub.status.busy":"2021-12-02T15:33:47.469997Z","iopub.execute_input":"2021-12-02T15:33:47.470226Z","iopub.status.idle":"2021-12-02T15:33:47.987406Z","shell.execute_reply.started":"2021-12-02T15:33:47.4702Z","shell.execute_reply":"2021-12-02T15:33:47.985257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Tried to combine both notebooks designed for training & inference separately. If you like the effort please upvote!!**","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}